ci: detect outbound internet traffic generated while running tests

vasild commented at 1:58 pm on November 22, 2024: contributor

Prevent generating outbound traffic on a non-loopback interface during tests.

Fix all tests, including ones that generate DNS traffic
Change CI to catch new regressions, including DNS traffic. DNS traffic is detected because some VMs have configured a non-loopback DNS server, [1111:1111::1]:53:
- previous releases, depends DEBUG
- TSan, depends, gui
- multiprocess, i686, DEBUG
- no wallet, libbitcoinkernel
Required capabilities within the VM are explicit in 02_run_container.sh: --cap-add NET_RAW
False positives either from non-test generated outbound traffic or responses to outside-originated traffic will fail the CI. I think there is a good chance that this does not happen. If it happens then this can be revisited then and can be easily worked around by removing the exit 1 line added to 03_test_script.sh in this PR.

Summary of each VM wrt the new check:

0ASan + LSan + UBSan + integer, no depends, USDT (GitHub)
132-bit CentOS, dash, gui (Cirrus CI)
2TSan, depends, gui (Cirrus CI)
3fuzzer,address,undefined,integer, no depends (Cirrus CI)
4multiprocess, i686, DEBUG (Cirrus CI)

:heavy_check_mark:

0ARM, unit tests, no functional tests (Cirrus CI)
1MSan, depends (Cirrus CI)
2Win64, unit tests, no gui tests, no functional tests (Cirrus CI)
3no wallet, libbitcoinkernel (Cirrus CI)
4previous releases, depends DEBUG (Cirrus CI)

:warning: can’t stop tcpdump, traffic data is not flushed to the file

0+ kill -SIGTERM 6594 7342
1/ci_container_base/ci/test/03_test_script.sh: line 168: kill: (6594) - Permission denied
2/ci_container_base/ci/test/03_test_script.sh: line 168: kill: (7342) - Permission denied
3...
4++ tcpdump -n -r /tmp/tcpdump_eth0 --direction=out tcp or udp
5tcpdump: truncated dump file; tried to read 4 file header bytes, only got 0

0macOS 14 native, arm64, no depends, sqlite only, gui (GitHub)
1macOS 14 native, arm64, fuzz (GitHub)

:no_entry:

0Running on host system without docker wrapper
1...
2+ tcpdump -n -i utun3 -w /tmp/tcpdump_utun3
3tcpdump: utun3: You don't have permission to capture on that device
4((cannot open BPF device) /dev/bpf0: Permission denied)

0Win64 native, VS 2022 (GitHub)
1Win64 native fuzz, VS 2022 (GitHub)

:no_entry:

0test each commit (GitHub)
1lint (Cirrus CI)
2macOS-cross, gui, no tests (Cirrus CI)
3tidy (Cirrus CI)

Resolves #31339

DrahtBot commented at 1:58 pm on November 22, 2024: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31349.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	maflcko, laanwj, 1440000bytes, jonatack, BrandonOdiwuor

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#31212 (util: Improve documentation and negation of args by hodlinator)
#30997 (build: Switch to Qt 6 by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot added the label Tests on Nov 22, 2024

vasild force-pushed on Nov 22, 2024

DrahtBot commented at 2:12 pm on November 22, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33384128561

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

DrahtBot added the label CI failed on Nov 22, 2024

maflcko commented at 2:17 pm on November 22, 2024: member

Nice. Conecpt ACK!

laanwj commented at 8:33 am on November 25, 2024: member

Concept ACK

i’m slightly worried this may generate false positive. As is, this detects traffic on the entire (virtual) machine while running the tests. Are there no other daemons running on the CI instance that could interfere with this?

vasild force-pushed on Nov 25, 2024

vasild commented at 9:08 am on November 25, 2024: contributor

@laanwj, Right! And ps ax in the VM looks suspiciously scarce: #31339 (comment) showing just bash and 03_test_script.sh.

Another source of false positive could be if somebody from the outside initiates communication to the VM to which it responds. E.g. an outsider tries to connect to the VM to which it responds with an outbound packet e.g. TCP RST. At least that should be obvious from the error log, showing the incoming packet first (I just pushed a slight change for that). Maybe also the traffic-from-another-daemon could be obvious - e.g. if there is traffic to apt.update.ubuntu.com:443…

vasild force-pushed on Nov 25, 2024

DrahtBot removed the label CI failed on Nov 25, 2024

vasild force-pushed on Nov 25, 2024

DrahtBot added the label CI failed on Nov 25, 2024

DrahtBot commented at 12:51 pm on November 25, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33474555794

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

DrahtBot removed the label CI failed on Nov 25, 2024

laanwj commented at 3:07 pm on November 25, 2024: member

Another source of false positive could be if somebody from the outside initiates communication to the VM to which it responds. E.g. an outsider tries to connect to the VM to which it responds with an outbound packet e.g. TCP RST.

Exactly. For all we know, the CI VM is firewalled off sufficiently that this can’t happen, but we don’t know.

At least that should be obvious from the error log, showing the incoming packet first (I just pushed a slight change for that).

Ah yes, as long as it’s only some extra logging, having a manual factor in this is fine. It only becomes critical if network traffic would cause a CI failure.

i’m not aware of a straightforward way to “log network traffic of this process and subproceses only”. Yes, it could be done with a linux network namespace, but that’s a lot of hassle.

And ps ax in the VM looks suspiciously scarce: #31339 (comment) showing just bash and 03_test_script.sh.

Seeing this, it might already be namespaced. Though a process namespace doesn’t necessarily mean the network namespace is isolated.

vasild force-pushed on Nov 25, 2024

vasild commented at 4:12 pm on November 25, 2024: contributor

1592a7dad4...071e43ffae: fix feature_config_args.py and p2p_seednode.py to not generate non-loopback traffic.

It only becomes critical if network traffic would cause a CI failure.

My intention here is to fail the CI because otherwise the log will be buried in the CI output and nobody will notice it. It follows that if this fails randomly with false positives when one would have to investigate it manually for arbitrary PRs which is highly highly highly undesirable.

in ci/test/03_test_script.sh:160 in 071e43ffae outdated

155+}
156+
157+function traffic_monitor_begin()
158+{
159+  for ifname in $(get_interfaces) ; do
160+    tcpdump -n -i "$ifname" -w "$(tcpdump_file_for_interface "$ifname")" &

maflcko commented at 4:25 pm on November 25, 2024:

The return code is ignored, which is why CI passed, despite the tests calling out

vasild commented at 8:54 am on November 26, 2024:

On some of the VMs this produces:

0[09:47:01.270] + tcpdump -n -i eth0 -w /tmp/tcpdump_eth0
1[09:47:01.334] tcpdump: eth0: You don't have permission to perform this capture on that device
2[09:47:01.335] (socket: Operation not permitted)

and then the CI passes because the return code is ignored. I think better not fail the CI when tcpdump does not work in that environment. It is ok as long as tcpdump works on at least one VM to catch problems.

maflcko commented at 9:04 am on November 26, 2024:

The problem is that no one will notice if this isn’t run on any machine, because it will silently pass even if there is an error.

vasild commented at 9:11 am on November 26, 2024:

True, if it stops working on all VMs, then nobody will notice. Any ideas how to approach this?

maflcko commented at 9:24 am on November 26, 2024:

I’d say it is fine to ignore it by default (if you want). However, there should be one machine in the CI matrix to run the check (and fail on any error).

The cirrus workers are running in a user account, so they may not have the permissions (unless they are switched to @0xB10C’s workers, which are running as root?). Alternatively, you could try with --cap-add=.../--privileged, but I haven’t tried this. I guess the only task that has the required permissions right now is the ASan GHA task?

0xB10C commented at 11:22 am on November 27, 2024:

(unless they are switched to @0xB10C’s workers, which are running as root?)

the runner setup I’m working on explicitly doesn’t run as root and is far from finished :)

maflcko commented at 11:52 am on November 27, 2024:

the runner setup I’m working on explicitly doesn’t run as root

Are you sure, because the current CI (in this run) is run in a user account (not root), and gives a permission error. The same CI in your run does not give a permission error, so there seems to be a difference.

The only thing I see is that you are using docker, which IIRC is running rootful by default.

0xB10C commented at 1:30 pm on November 27, 2024:

I’m using rootless-docker which runs dockerd as a user account. Inside the container, you’re root and can tcpdump on the containers eth0 interface, but you can’t* e.g. mount and edit the hosts /etc/passwd like you can’t with the user account.

*until someone finds a vuln in rootless-docker

maflcko commented at 1:59 pm on November 27, 2024:

It may be that docker rootless has a different capabilities set, compared to podman. (Can be checked with capsh --print).

In any case, my preference would be to explicitly list the required caps, instead of relying on a vendor default.

vasild commented at 3:43 pm on November 27, 2024:

The problem is that no one will notice if this isn’t run on any machine

I changed it to insist that the tcpdump file was created on the ASAN env. So the ASAN job will be red if this stops working on it.

maflcko commented at 7:59 am on November 28, 2024:

I don’t think this is sufficient. The CI failure (https://github.com/bitcoin/bitcoin/pull/31349#issuecomment-2499335672) will remain, depending on a vendor default.

Again, my preference would be to explicitly list the required (or removed) caps, instead of relying on a vendor default. Otherwise, it will become harder to run the CI locally, or lead to vendor-lock-in.

vasild commented at 8:41 am on December 3, 2024:

… explicitly list the required (or removed) caps …

Done, NET_RAW is required to run tcpdump: https://www.tcpdump.org/manpages/pcap.3pcap.html

Plus, the ASAN job requires that tcpdump -w runs and creates the file, otherwise it will be red.

I think that resolves the concerns from this thread, so I am closing it. Feel free to comment / reopen if there is more to this.

0xB10C commented at 0:55 am on November 26, 2024: contributor

Ran this on my CI runner which has 8.8.8.8 configured as DNS server for docker.

https://cirrus-ci.com/task/5500763260059648?logs=ci#L1137

0[00:46:26.215] + tcpdump -n -r /tmp/tcpdump_eth0 tcp or udp
1[00:46:26.219] 00:42:50.052764 IP 172.18.0.2.46566 > 8.8.8.8.53: 39301+ A? x9.dummySeed.invalid. (38)
2[00:46:26.219] 00:42:50.053181 IP 172.18.0.2.58686 > 8.8.8.8.53: 36487+ AAAA? x9.dummySeed.invalid. (38)
3[00:46:26.219] 00:42:50.059038 IP 8.8.8.8.53 > 172.18.0.2.46566: 39301 NXDomain 0/1/0 (113)
4[00:46:26.219] 00:42:50.060121 IP 8.8.8.8.53 > 172.18.0.2.58686: 36487 NXDomain 0/1/0 (113)
5[00:46:26.219] 00:42:50.060574 IP 172.18.0.2.34312 > 8.8.8.8.53: 25243+ A? x9.dummySeed.invalid. (38)
6[00:46:26.219] 00:42:50.060939 IP 172.18.0.2.47040 > 8.8.8.8.53: 63641+ AAAA? x9.dummySeed.invalid. (38)
7...

Edit: My understanding is as follows: The DNS requests normally go to a local DNS resolver which then asks an upstream resolver. The upstream resolver (possibly your ISP) indirectly learns that you are running Bitcoin Core tests, even if there was no direct communication over a non-loopback interface.

  0[00:46:26.209] ++ tcpdump -n -r /tmp/tcpdump_eth0 --direction=out tcp or udp
  1[00:46:26.213] reading from file /tmp/tcpdump_eth0, link-type EN10MB (Ethernet), snapshot length 262144
  2[00:46:26.215] + '[' -n '00:42:50.052764 IP 172.18.0.2.46566 > 8.8.8.8.53: 39301+ A? x9.dummySeed.invalid. (38)
  3[00:46:26.215] 00:42:50.053181 IP 172.18.0.2.58686 > 8.8.8.8.53: 36487+ AAAA? x9.dummySeed.invalid. (38)
  4[00:46:26.215] 00:42:50.059038 IP 8.8.8.8.53 > 172.18.0.2.46566: 39301 NXDomain 0/1/0 (113)
  5[00:46:26.215] 00:42:50.060121 IP 8.8.8.8.53 > 172.18.0.2.58686: 36487 NXDomain 0/1/0 (113)
  6[00:46:26.215] 00:42:50.060574 IP 172.18.0.2.34312 > 8.8.8.8.53: 25243+ A? x9.dummySeed.invalid. (38)
  7[00:46:26.215] 00:42:50.060939 IP 172.18.0.2.47040 > 8.8.8.8.53: 63641+ AAAA? x9.dummySeed.invalid. (38)
  8[00:46:26.215] 00:42:50.066767 IP 8.8.8.8.53 > 172.18.0.2.34312: 25243 NXDomain 0/1/0 (113)
  9[00:46:26.215] 00:42:50.068273 IP 8.8.8.8.53 > 172.18.0.2.47040: 63641 NXDomain 0/1/0 (113)
 10[00:46:26.215] 00:42:50.420185 IP 172.18.0.2.55135 > 8.8.8.8.53: 9419+ A? fakenodeaddr.fakedomain.invalid. (49)
 11[00:46:26.215] 00:42:50.420589 IP 172.18.0.2.42709 > 8.8.8.8.53: 57544+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
 12[00:46:26.215] 00:42:50.426139 IP 8.8.8.8.53 > 172.18.0.2.55135: 9419 NXDomain 0/1/0 (124)
 13[00:46:26.215] 00:42:50.426488 IP 8.8.8.8.53 > 172.18.0.2.42709: 57544 NXDomain 0/1/0 (124)
 14[00:46:26.215] 00:42:50.426928 IP 172.18.0.2.42643 > 8.8.8.8.53: 12212+ A? fakenodeaddr.fakedomain.invalid. (49)
 15[00:46:26.215] 00:42:50.427362 IP 172.18.0.2.33528 > 8.8.8.8.53: 41906+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
 16[00:46:26.215] 00:42:50.432905 IP 8.8.8.8.53 > 172.18.0.2.42643: 12212 NXDomain 0/1/0 (124)
 17[00:46:26.215] 00:42:50.433389 IP 8.8.8.8.53 > 172.18.0.2.33528: 41906 NXDomain 0/1/0 (124)
 18[00:46:26.215] 00:42:50.785947 IP 172.18.0.2.40413 > 8.8.8.8.53: 23723+ A? x9.dummySeed.invalid. (38)
 19[00:46:26.215] 00:42:50.786307 IP 172.18.0.2.33152 > 8.8.8.8.53: 51880+ AAAA? x9.dummySeed.invalid. (38)
 20[00:46:26.215] 00:42:50.786916 IP 172.18.0.2.60591 > 8.8.8.8.53: 57214+ A? fakenodeaddr.fakedomain.invalid. (49)
 21[00:46:26.215] 00:42:50.787237 IP 172.18.0.2.51085 > 8.8.8.8.53: 45180+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
 22[00:46:26.215] 00:42:50.792174 IP 8.8.8.8.53 > 172.18.0.2.33152: 51880 NXDomain 0/1/0 (113)
 23[00:46:26.215] 00:42:50.792196 IP 8.8.8.8.53 > 172.18.0.2.40413: 23723 NXDomain 0/1/0 (113)
 24[00:46:26.215] 00:42:50.794262 IP 8.8.8.8.53 > 172.18.0.2.51085: 45180 NXDomain 0/1/0 (124)
 25[00:46:26.215] 00:42:50.794281 IP 8.8.8.8.53 > 172.18.0.2.60591: 57214 NXDomain 0/1/0 (124)
 26[00:46:26.215] 00:42:50.794701 IP 172.18.0.2.53594 > 8.8.8.8.53: 7814+ AAAA? x9.dummySeed.invalid. (38)
 27[00:46:26.215] 00:42:50.794819 IP 172.18.0.2.33826 > 8.8.8.8.53: 50053+ A? x9.dummySeed.invalid. (38)
 28[00:46:26.215] 00:42:50.795297 IP 172.18.0.2.54482 > 8.8.8.8.53: 30981+ A? fakenodeaddr.fakedomain.invalid. (49)
 29[00:46:26.215] 00:42:50.795592 IP 172.18.0.2.48225 > 8.8.8.8.53: 65050+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
 30[00:46:26.215] 00:42:50.800988 IP 8.8.8.8.53 > 172.18.0.2.33826: 50053 NXDomain 0/1/0 (113)
 31[00:46:26.215] 00:42:50.801160 IP 8.8.8.8.53 > 172.18.0.2.53594: 7814 NXDomain 0/1/0 (113)
 32[00:46:26.215] 00:42:50.801189 IP 8.8.8.8.53 > 172.18.0.2.54482: 30981 NXDomain 0/1/0 (124)
 33[00:46:26.215] 00:42:50.801282 IP 8.8.8.8.53 > 172.18.0.2.48225: 65050 NXDomain 0/1/0 (124)
 34[00:46:26.215] 00:42:51.151925 IP 172.18.0.2.45895 > 8.8.8.8.53: 32540+ A? x9.dummySeed.invalid. (38)
 35[00:46:26.215] 00:42:51.154708 IP 172.18.0.2.39532 > 8.8.8.8.53: 3102+ AAAA? x9.dummySeed.invalid. (38)
 36[00:46:26.215] 00:42:51.160435 IP 8.8.8.8.53 > 172.18.0.2.45895: 32540 NXDomain 0/1/0 (113)
 37[00:46:26.215] 00:42:51.160623 IP 8.8.8.8.53 > 172.18.0.2.39532: 3102 NXDomain 0/1/0 (113)
 38[00:46:26.215] 00:42:51.161037 IP 172.18.0.2.54958 > 8.8.8.8.53: 64262+ A? x9.dummySeed.invalid. (38)
 39[00:46:26.215] 00:42:51.161501 IP 172.18.0.2.55394 > 8.8.8.8.53: 38663+ AAAA? x9.dummySeed.invalid. (38)
 40[00:46:26.215] 00:42:51.167068 IP 8.8.8.8.53 > 172.18.0.2.54958: 64262 NXDomain 0/1/0 (113)
 41[00:46:26.215] 00:42:51.167562 IP 8.8.8.8.53 > 172.18.0.2.55394: 38663 NXDomain 0/1/0 (113)
 42[00:46:26.215] 00:43:13.896684 IP 172.18.0.2.38577 > 8.8.8.8.53: 37859+ A? x9.dummySeed.invalid. (38)
 43[00:46:26.215] 00:43:13.898035 IP 172.18.0.2.57872 > 8.8.8.8.53: 26852+ AAAA? x9.dummySeed.invalid. (38)
 44[00:46:26.215] 00:43:13.904071 IP 8.8.8.8.53 > 172.18.0.2.38577: 37859 NXDomain 0/1/0 (113)
 45[00:46:26.215] 00:43:13.904515 IP 8.8.8.8.53 > 172.18.0.2.57872: 26852 NXDomain 0/1/0 (113)
 46[00:46:26.215] 00:43:13.905051 IP 172.18.0.2.41442 > 8.8.8.8.53: 3978+ A? x9.dummySeed.invalid. (38)
 47[00:46:26.215] 00:43:13.905744 IP 172.18.0.2.51126 > 8.8.8.8.53: 44169+ AAAA? x9.dummySeed.invalid. (38)
 48[00:46:26.215] 00:43:13.910763 IP 8.8.8.8.53 > 172.18.0.2.41442: 3978 NXDomain 0/1/0 (113)
 49[00:46:26.215] 00:43:13.912478 IP 8.8.8.8.53 > 172.18.0.2.51126: 44169 NXDomain 0/1/0 (113)
 50[00:46:26.215] 00:43:14.345922 IP 172.18.0.2.46373 > 8.8.8.8.53: 23556+ AAAA? x9.dummySeed.invalid. (38)
 51[00:46:26.215] 00:43:14.346104 IP 172.18.0.2.55199 > 8.8.8.8.53: 59658+ A? x9.dummySeed.invalid. (38)
 52[00:46:26.215] 00:43:14.352039 IP 8.8.8.8.53 > 172.18.0.2.55199: 59658 NXDomain 0/1/0 (113)
 53[00:46:26.215] 00:43:14.352108 IP 8.8.8.8.53 > 172.18.0.2.46373: 23556 NXDomain 0/1/0 (113)
 54[00:46:26.215] 00:43:14.355678 IP 172.18.0.2.40940 > 8.8.8.8.53: 56364+ AAAA? x9.dummySeed.invalid. (38)
 55[00:46:26.215] 00:43:14.356531 IP 172.18.0.2.51128 > 8.8.8.8.53: 29229+ A? x9.dummySeed.invalid. (38)
 56[00:46:26.215] 00:43:14.361956 IP 8.8.8.8.53 > 172.18.0.2.40940: 56364 NXDomain 0/1/0 (113)
 57[00:46:26.215] 00:43:14.362506 IP 8.8.8.8.53 > 172.18.0.2.51128: 29229 NXDomain 0/1/0 (113)
 58[00:46:26.215] 00:44:27.054154 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043001621 ecr 0,nop,wscale 7], length 0
 59[00:46:26.215] 00:44:27.945389 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043002513 ecr 0,nop,wscale 7], length 0
 60[00:46:26.215] 00:44:28.096227 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043002664 ecr 0,nop,wscale 7], length 0
 61[00:46:26.215] 00:44:28.992218 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043003560 ecr 0,nop,wscale 7], length 0
 62[00:46:26.215] 00:44:29.120219 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043003688 ecr 0,nop,wscale 7], length 0
 63[00:46:26.215] 00:44:30.016217 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043004584 ecr 0,nop,wscale 7], length 0
 64[00:46:26.215] 00:44:30.144232 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043004712 ecr 0,nop,wscale 7], length 0
 65[00:46:26.215] 00:44:31.040220 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043005608 ecr 0,nop,wscale 7], length 0
 66[00:46:26.215] 00:44:31.168218 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043005736 ecr 0,nop,wscale 7], length 0
 67[00:46:26.215] 00:44:32.064218 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043006632 ecr 0,nop,wscale 7], length 0
 68[00:46:26.215] 00:45:32.199353 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531031147 ecr 0,nop,wscale 7], length 0
 69[00:46:26.215] 00:45:33.248224 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531032196 ecr 0,nop,wscale 7], length 0
 70[00:46:26.215] 00:45:34.272224 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531033220 ecr 0,nop,wscale 7], length 0
 71[00:46:26.215] 00:45:35.296253 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531034244 ecr 0,nop,wscale 7], length 0
 72[00:46:26.215] 00:45:36.320281 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531035268 ecr 0,nop,wscale 7], length 0
 73[00:46:26.215] 00:45:37.442439 IP 172.18.0.2.52570 > 8.8.8.8.53: 3832+ AAAA? some.node. (27)
 74[00:46:26.215] 00:45:37.442487 IP 172.18.0.2.60704 > 8.8.8.8.53: 1023+ A? some.node. (27)
 75[00:46:26.215] 00:45:37.448551 IP 8.8.8.8.53 > 172.18.0.2.60704: 1023 NXDomain 0/1/0 (102)
 76[00:46:26.215] 00:45:37.448911 IP 8.8.8.8.53 > 172.18.0.2.52570: 3832 NXDomain 0/1/0 (102)
 77[00:46:26.215] 00:45:37.449395 IP 172.18.0.2.53889 > 8.8.8.8.53: 63359+ AAAA? some.node. (27)
 78[00:46:26.215] 00:45:37.449419 IP 172.18.0.2.51133 > 8.8.8.8.53: 6268+ A? some.node. (27)
 79[00:46:26.215] 00:45:37.455701 IP 8.8.8.8.53 > 172.18.0.2.51133: 6268 NXDomain 0/1/0 (102)
 80[00:46:26.215] 00:45:37.455743 IP 8.8.8.8.53 > 172.18.0.2.53889: 63359 NXDomain 0/1/0 (102)
 81[00:46:26.215] 00:45:37.734057 IP 172.18.0.2.50765 > 8.8.8.8.53: 34600+ AAAA? x9.dummySeed.invalid. (38)
 82[00:46:26.215] 00:45:37.734277 IP 172.18.0.2.47474 > 8.8.8.8.53: 53797+ A? x9.dummySeed.invalid. (38)
 83[00:46:26.215] 00:45:37.740114 IP 8.8.8.8.53 > 172.18.0.2.50765: 34600 NXDomain 0/1/0 (113)
 84[00:46:26.215] 00:45:37.740144 IP 8.8.8.8.53 > 172.18.0.2.47474: 53797 NXDomain 0/1/0 (113)
 85[00:46:26.215] 00:45:37.740754 IP 172.18.0.2.33593 > 8.8.8.8.53: 4491+ A? x9.dummySeed.invalid. (38)
 86[00:46:26.215] 00:45:37.740756 IP 172.18.0.2.34759 > 8.8.8.8.53: 34698+ AAAA? x9.dummySeed.invalid. (38)
 87[00:46:26.215] 00:45:37.746602 IP 8.8.8.8.53 > 172.18.0.2.34759: 34698 NXDomain 0/1/0 (113)
 88[00:46:26.215] 00:45:37.746641 IP 8.8.8.8.53 > 172.18.0.2.33593: 4491 NXDomain 0/1/0 (113)
 89[00:46:26.215] 00:45:38.234283 IP 172.18.0.2.33343 > 8.8.8.8.53: 22856+ A? dummySeed.invalid. (35)
 90[00:46:26.215] 00:45:38.234286 IP 172.18.0.2.43752 > 8.8.8.8.53: 55883+ AAAA? dummySeed.invalid. (35)
 91[00:46:26.215] 00:45:38.240530 IP 8.8.8.8.53 > 172.18.0.2.43752: 55883 NXDomain 0/1/0 (110)
 92[00:46:26.215] 00:45:38.240809 IP 8.8.8.8.53 > 172.18.0.2.33343: 22856 NXDomain 0/1/0 (110)
 93[00:46:26.215] 00:45:38.241237 IP 172.18.0.2.35861 > 8.8.8.8.53: 14859+ A? dummySeed.invalid. (35)
 94[00:46:26.215] 00:45:38.241238 IP 172.18.0.2.51487 > 8.8.8.8.53: 46093+ AAAA? dummySeed.invalid. (35)
 95[00:46:26.215] 00:45:38.247292 IP 8.8.8.8.53 > 172.18.0.2.35861: 14859 NXDomain 0/1/0 (110)
 96[00:46:26.215] 00:45:38.247320 IP 8.8.8.8.53 > 172.18.0.2.51487: 46093 NXDomain 0/1/0 (110)
 97[00:46:26.215] 00:45:39.779784 IP 172.18.0.2.34493 > 8.8.8.8.53: 26799+ A? fakenodeaddr. (30)
 98[00:46:26.215] 00:45:39.779791 IP 172.18.0.2.56878 > 8.8.8.8.53: 61600+ AAAA? fakenodeaddr. (30)
 99[00:46:26.215] 00:45:39.785890 IP 8.8.8.8.53 > 172.18.0.2.56878: 61600 NXDomain 0/1/0 (105)
100[00:46:26.215] 00:45:39.785928 IP 8.8.8.8.53 > 172.18.0.2.34493: 26799 NXDomain 0/1/0 (105)
101[00:46:26.215] 00:45:39.786515 IP 172.18.0.2.54375 > 8.8.8.8.53: 52196+ A? fakenodeaddr. (30)
102[00:46:26.215] 00:45:39.786693 IP 172.18.0.2.53292 > 8.8.8.8.53: 56037+ AAAA? fakenodeaddr. (30)
103[00:46:26.215] 00:45:39.792298 IP 8.8.8.8.53 > 172.18.0.2.54375: 52196 NXDomain 0/1/0 (105)
104[00:46:26.215] 00:45:39.792592 IP 8.8.8.8.53 > 172.18.0.2.53292: 56037 NXDomain 0/1/0 (105)
105[00:46:26.215] 00:45:42.174906 IP 11.22.33.44.18444 > 172.18.0.2.51812: Flags [R.], seq 0, ack 3432062660, win 65535, length 0
106[00:46:26.215] 00:45:42.722951 IP 172.18.0.2.33938 > 8.8.8.8.53: 23646+ AAAA? fakeaddress1. (30)
107[00:46:26.215] 00:45:42.723043 IP 172.18.0.2.38954 > 8.8.8.8.53: 24914+ A? fakeaddress1. (30)
108[00:46:26.215] 00:45:42.728907 IP 8.8.8.8.53 > 172.18.0.2.38954: 24914 NXDomain 0/1/0 (105)
109[00:46:26.215] 00:45:42.729111 IP 8.8.8.8.53 > 172.18.0.2.33938: 23646 NXDomain 0/1/0 (105)
110[00:46:26.215] 00:45:42.729684 IP 172.18.0.2.52893 > 8.8.8.8.53: 3801+ AAAA? fakeaddress1. (30)
111[00:46:26.215] 00:45:42.729715 IP 172.18.0.2.57497 > 8.8.8.8.53: 62431+ A? fakeaddress1. (30)
112[00:46:26.215] 00:45:42.735746 IP 8.8.8.8.53 > 172.18.0.2.57497: 62431 NXDomain 0/1/0 (105)
113[00:46:26.215] 00:45:42.735771 IP 8.8.8.8.53 > 172.18.0.2.52893: 3801 NXDomain 0/1/0 (105)
114[00:46:26.215] 00:45:43.175350 IP 11.22.33.44.18444 > 172.18.0.2.51828: Flags [R.], seq 0, ack 3743466588, win 65535, length 0' ']'
115[00:46:26.215] + echo 'Outbound TCP or UDP packets on the non loopback interface generated during tests:'
116[00:46:26.215] Outbound TCP or UDP packets on the non loopback interface generated during tests:
117[00:46:26.215] + tcpdump -n -r /tmp/tcpdump_eth0 tcp or udp
118[00:46:26.218] reading from file /tmp/tcpdump_eth0, link-type EN10MB (Ethernet), snapshot length 262144
119[00:46:26.219] 00:42:50.052764 IP 172.18.0.2.46566 > 8.8.8.8.53: 39301+ A? x9.dummySeed.invalid. (38)
120[00:46:26.219] 00:42:50.053181 IP 172.18.0.2.58686 > 8.8.8.8.53: 36487+ AAAA? x9.dummySeed.invalid. (38)
121[00:46:26.219] 00:42:50.059038 IP 8.8.8.8.53 > 172.18.0.2.46566: 39301 NXDomain 0/1/0 (113)
122[00:46:26.219] 00:42:50.060121 IP 8.8.8.8.53 > 172.18.0.2.58686: 36487 NXDomain 0/1/0 (113)
123[00:46:26.219] 00:42:50.060574 IP 172.18.0.2.34312 > 8.8.8.8.53: 25243+ A? x9.dummySeed.invalid. (38)
124[00:46:26.219] 00:42:50.060939 IP 172.18.0.2.47040 > 8.8.8.8.53: 63641+ AAAA? x9.dummySeed.invalid. (38)
125[00:46:26.219] 00:42:50.066767 IP 8.8.8.8.53 > 172.18.0.2.34312: 25243 NXDomain 0/1/0 (113)
126[00:46:26.219] 00:42:50.068273 IP 8.8.8.8.53 > 172.18.0.2.47040: 63641 NXDomain 0/1/0 (113)
127[00:46:26.219] 00:42:50.420185 IP 172.18.0.2.55135 > 8.8.8.8.53: 9419+ A? fakenodeaddr.fakedomain.invalid. (49)
128[00:46:26.219] 00:42:50.420589 IP 172.18.0.2.42709 > 8.8.8.8.53: 57544+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
129[00:46:26.219] 00:42:50.426139 IP 8.8.8.8.53 > 172.18.0.2.55135: 9419 NXDomain 0/1/0 (124)
130[00:46:26.219] 00:42:50.426488 IP 8.8.8.8.53 > 172.18.0.2.42709: 57544 NXDomain 0/1/0 (124)
131[00:46:26.219] 00:42:50.426928 IP 172.18.0.2.42643 > 8.8.8.8.53: 12212+ A? fakenodeaddr.fakedomain.invalid. (49)
132[00:46:26.219] 00:42:50.427362 IP 172.18.0.2.33528 > 8.8.8.8.53: 41906+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
133[00:46:26.219] 00:42:50.432905 IP 8.8.8.8.53 > 172.18.0.2.42643: 12212 NXDomain 0/1/0 (124)
134[00:46:26.219] 00:42:50.433389 IP 8.8.8.8.53 > 172.18.0.2.33528: 41906 NXDomain 0/1/0 (124)
135[00:46:26.219] 00:42:50.785947 IP 172.18.0.2.40413 > 8.8.8.8.53: 23723+ A? x9.dummySeed.invalid. (38)
136[00:46:26.219] 00:42:50.786307 IP 172.18.0.2.33152 > 8.8.8.8.53: 51880+ AAAA? x9.dummySeed.invalid. (38)
137[00:46:26.219] 00:42:50.786916 IP 172.18.0.2.60591 > 8.8.8.8.53: 57214+ A? fakenodeaddr.fakedomain.invalid. (49)
138[00:46:26.219] 00:42:50.787237 IP 172.18.0.2.51085 > 8.8.8.8.53: 45180+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
139[00:46:26.219] 00:42:50.792174 IP 8.8.8.8.53 > 172.18.0.2.33152: 51880 NXDomain 0/1/0 (113)
140[00:46:26.219] 00:42:50.792196 IP 8.8.8.8.53 > 172.18.0.2.40413: 23723 NXDomain 0/1/0 (113)
141[00:46:26.219] 00:42:50.794262 IP 8.8.8.8.53 > 172.18.0.2.51085: 45180 NXDomain 0/1/0 (124)
142[00:46:26.219] 00:42:50.794281 IP 8.8.8.8.53 > 172.18.0.2.60591: 57214 NXDomain 0/1/0 (124)
143[00:46:26.219] 00:42:50.794701 IP 172.18.0.2.53594 > 8.8.8.8.53: 7814+ AAAA? x9.dummySeed.invalid. (38)
144[00:46:26.219] 00:42:50.794819 IP 172.18.0.2.33826 > 8.8.8.8.53: 50053+ A? x9.dummySeed.invalid. (38)
145[00:46:26.219] 00:42:50.795297 IP 172.18.0.2.54482 > 8.8.8.8.53: 30981+ A? fakenodeaddr.fakedomain.invalid. (49)
146[00:46:26.219] 00:42:50.795592 IP 172.18.0.2.48225 > 8.8.8.8.53: 65050+ AAAA? fakenodeaddr.fakedomain.invalid. (49)
147[00:46:26.219] 00:42:50.800988 IP 8.8.8.8.53 > 172.18.0.2.33826: 50053 NXDomain 0/1/0 (113)
148[00:46:26.219] 00:42:50.801160 IP 8.8.8.8.53 > 172.18.0.2.53594: 7814 NXDomain 0/1/0 (113)
149[00:46:26.219] 00:42:50.801189 IP 8.8.8.8.53 > 172.18.0.2.54482: 30981 NXDomain 0/1/0 (124)
150[00:46:26.219] 00:42:50.801282 IP 8.8.8.8.53 > 172.18.0.2.48225: 65050 NXDomain 0/1/0 (124)
151[00:46:26.219] 00:42:51.151925 IP 172.18.0.2.45895 > 8.8.8.8.53: 32540+ A? x9.dummySeed.invalid. (38)
152[00:46:26.219] 00:42:51.154708 IP 172.18.0.2.39532 > 8.8.8.8.53: 3102+ AAAA? x9.dummySeed.invalid. (38)
153[00:46:26.219] 00:42:51.160435 IP 8.8.8.8.53 > 172.18.0.2.45895: 32540 NXDomain 0/1/0 (113)
154[00:46:26.219] 00:42:51.160623 IP 8.8.8.8.53 > 172.18.0.2.39532: 3102 NXDomain 0/1/0 (113)
155[00:46:26.219] 00:42:51.161037 IP 172.18.0.2.54958 > 8.8.8.8.53: 64262+ A? x9.dummySeed.invalid. (38)
156[00:46:26.219] 00:42:51.161501 IP 172.18.0.2.55394 > 8.8.8.8.53: 38663+ AAAA? x9.dummySeed.invalid. (38)
157[00:46:26.219] 00:42:51.167068 IP 8.8.8.8.53 > 172.18.0.2.54958: 64262 NXDomain 0/1/0 (113)
158[00:46:26.219] 00:42:51.167562 IP 8.8.8.8.53 > 172.18.0.2.55394: 38663 NXDomain 0/1/0 (113)
159[00:46:26.219] 00:43:13.896684 IP 172.18.0.2.38577 > 8.8.8.8.53: 37859+ A? x9.dummySeed.invalid. (38)
160[00:46:26.219] 00:43:13.898035 IP 172.18.0.2.57872 > 8.8.8.8.53: 26852+ AAAA? x9.dummySeed.invalid. (38)
161[00:46:26.219] 00:43:13.904071 IP 8.8.8.8.53 > 172.18.0.2.38577: 37859 NXDomain 0/1/0 (113)
162[00:46:26.219] 00:43:13.904515 IP 8.8.8.8.53 > 172.18.0.2.57872: 26852 NXDomain 0/1/0 (113)
163[00:46:26.219] 00:43:13.905051 IP 172.18.0.2.41442 > 8.8.8.8.53: 3978+ A? x9.dummySeed.invalid. (38)
164[00:46:26.219] 00:43:13.905744 IP 172.18.0.2.51126 > 8.8.8.8.53: 44169+ AAAA? x9.dummySeed.invalid. (38)
165[00:46:26.219] 00:43:13.910763 IP 8.8.8.8.53 > 172.18.0.2.41442: 3978 NXDomain 0/1/0 (113)
166[00:46:26.219] 00:43:13.912478 IP 8.8.8.8.53 > 172.18.0.2.51126: 44169 NXDomain 0/1/0 (113)
167[00:46:26.219] 00:43:14.345922 IP 172.18.0.2.46373 > 8.8.8.8.53: 23556+ AAAA? x9.dummySeed.invalid. (38)
168[00:46:26.219] 00:43:14.346104 IP 172.18.0.2.55199 > 8.8.8.8.53: 59658+ A? x9.dummySeed.invalid. (38)
169[00:46:26.219] 00:43:14.352039 IP 8.8.8.8.53 > 172.18.0.2.55199: 59658 NXDomain 0/1/0 (113)
170[00:46:26.219] 00:43:14.352108 IP 8.8.8.8.53 > 172.18.0.2.46373: 23556 NXDomain 0/1/0 (113)
171[00:46:26.219] 00:43:14.355678 IP 172.18.0.2.40940 > 8.8.8.8.53: 56364+ AAAA? x9.dummySeed.invalid. (38)
172[00:46:26.219] 00:43:14.356531 IP 172.18.0.2.51128 > 8.8.8.8.53: 29229+ A? x9.dummySeed.invalid. (38)
173[00:46:26.219] 00:43:14.361956 IP 8.8.8.8.53 > 172.18.0.2.40940: 56364 NXDomain 0/1/0 (113)
174[00:46:26.219] 00:43:14.362506 IP 8.8.8.8.53 > 172.18.0.2.51128: 29229 NXDomain 0/1/0 (113)
175[00:46:26.219] 00:44:27.054154 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043001621 ecr 0,nop,wscale 7], length 0
176[00:46:26.219] 00:44:27.945389 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043002513 ecr 0,nop,wscale 7], length 0
177[00:46:26.219] 00:44:28.096227 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043002664 ecr 0,nop,wscale 7], length 0
178[00:46:26.219] 00:44:28.992218 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043003560 ecr 0,nop,wscale 7], length 0
179[00:46:26.219] 00:44:29.120219 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043003688 ecr 0,nop,wscale 7], length 0
180[00:46:26.219] 00:44:30.016217 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043004584 ecr 0,nop,wscale 7], length 0
181[00:46:26.219] 00:44:30.144232 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043004712 ecr 0,nop,wscale 7], length 0
182[00:46:26.219] 00:44:31.040220 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043005608 ecr 0,nop,wscale 7], length 0
183[00:46:26.219] 00:44:31.168218 IP 172.18.0.2.51812 > 11.22.33.44.18444: Flags [S], seq 3432062659, win 64240, options [mss 1460,sackOK,TS val 1043005736 ecr 0,nop,wscale 7], length 0
184[00:46:26.219] 00:44:32.064218 IP 172.18.0.2.51828 > 11.22.33.44.18444: Flags [S], seq 3743466587, win 64240, options [mss 1460,sackOK,TS val 1043006632 ecr 0,nop,wscale 7], length 0
185[00:46:26.219] 00:45:32.199353 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531031147 ecr 0,nop,wscale 7], length 0
186[00:46:26.219] 00:45:33.248224 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531032196 ecr 0,nop,wscale 7], length 0
187[00:46:26.219] 00:45:34.272224 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531033220 ecr 0,nop,wscale 7], length 0
188[00:46:26.219] 00:45:35.296253 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531034244 ecr 0,nop,wscale 7], length 0
189[00:46:26.219] 00:45:36.320281 IP 172.18.0.2.34256 > 0.0.0.1.18444: Flags [S], seq 2197884795, win 64240, options [mss 1460,sackOK,TS val 2531035268 ecr 0,nop,wscale 7], length 0
190[00:46:26.219] 00:45:37.442439 IP 172.18.0.2.52570 > 8.8.8.8.53: 3832+ AAAA? some.node. (27)
191[00:46:26.219] 00:45:37.442487 IP 172.18.0.2.60704 > 8.8.8.8.53: 1023+ A? some.node. (27)
192[00:46:26.219] 00:45:37.448551 IP 8.8.8.8.53 > 172.18.0.2.60704: 1023 NXDomain 0/1/0 (102)
193[00:46:26.219] 00:45:37.448911 IP 8.8.8.8.53 > 172.18.0.2.52570: 3832 NXDomain 0/1/0 (102)
194[00:46:26.219] 00:45:37.449395 IP 172.18.0.2.53889 > 8.8.8.8.53: 63359+ AAAA? some.node. (27)
195[00:46:26.219] 00:45:37.449419 IP 172.18.0.2.51133 > 8.8.8.8.53: 6268+ A? some.node. (27)
196[00:46:26.219] 00:45:37.455701 IP 8.8.8.8.53 > 172.18.0.2.51133: 6268 NXDomain 0/1/0 (102)
197[00:46:26.219] 00:45:37.455743 IP 8.8.8.8.53 > 172.18.0.2.53889: 63359 NXDomain 0/1/0 (102)
198[00:46:26.219] 00:45:37.734057 IP 172.18.0.2.50765 > 8.8.8.8.53: 34600+ AAAA? x9.dummySeed.invalid. (38)
199[00:46:26.219] 00:45:37.734277 IP 172.18.0.2.47474 > 8.8.8.8.53: 53797+ A? x9.dummySeed.invalid. (38)
200[00:46:26.219] 00:45:37.740114 IP 8.8.8.8.53 > 172.18.0.2.50765: 34600 NXDomain 0/1/0 (113)
201[00:46:26.219] 00:45:37.740144 IP 8.8.8.8.53 > 172.18.0.2.47474: 53797 NXDomain 0/1/0 (113)
202[00:46:26.219] 00:45:37.740754 IP 172.18.0.2.33593 > 8.8.8.8.53: 4491+ A? x9.dummySeed.invalid. (38)
203[00:46:26.219] 00:45:37.740756 IP 172.18.0.2.34759 > 8.8.8.8.53: 34698+ AAAA? x9.dummySeed.invalid. (38)
204[00:46:26.219] 00:45:37.746602 IP 8.8.8.8.53 > 172.18.0.2.34759: 34698 NXDomain 0/1/0 (113)
205[00:46:26.219] 00:45:37.746641 IP 8.8.8.8.53 > 172.18.0.2.33593: 4491 NXDomain 0/1/0 (113)
206[00:46:26.219] 00:45:38.234283 IP 172.18.0.2.33343 > 8.8.8.8.53: 22856+ A? dummySeed.invalid. (35)
207[00:46:26.219] 00:45:38.234286 IP 172.18.0.2.43752 > 8.8.8.8.53: 55883+ AAAA? dummySeed.invalid. (35)
208[00:46:26.219] 00:45:38.240530 IP 8.8.8.8.53 > 172.18.0.2.43752: 55883 NXDomain 0/1/0 (110)
209[00:46:26.219] 00:45:38.240809 IP 8.8.8.8.53 > 172.18.0.2.33343: 22856 NXDomain 0/1/0 (110)
210[00:46:26.219] 00:45:38.241237 IP 172.18.0.2.35861 > 8.8.8.8.53: 14859+ A? dummySeed.invalid. (35)
211[00:46:26.219] 00:45:38.241238 IP 172.18.0.2.51487 > 8.8.8.8.53: 46093+ AAAA? dummySeed.invalid. (35)
212[00:46:26.219] 00:45:38.247292 IP 8.8.8.8.53 > 172.18.0.2.35861: 14859 NXDomain 0/1/0 (110)
213[00:46:26.219] 00:45:38.247320 IP 8.8.8.8.53 > 172.18.0.2.51487: 46093 NXDomain 0/1/0 (110)
214[00:46:26.219] 00:45:39.779784 IP 172.18.0.2.34493 > 8.8.8.8.53: 26799+ A? fakenodeaddr. (30)
215[00:46:26.219] 00:45:39.779791 IP 172.18.0.2.56878 > 8.8.8.8.53: 61600+ AAAA? fakenodeaddr. (30)
216[00:46:26.219] 00:45:39.785890 IP 8.8.8.8.53 > 172.18.0.2.56878: 61600 NXDomain 0/1/0 (105)
217[00:46:26.219] 00:45:39.785928 IP 8.8.8.8.53 > 172.18.0.2.34493: 26799 NXDomain 0/1/0 (105)
218[00:46:26.219] 00:45:39.786515 IP 172.18.0.2.54375 > 8.8.8.8.53: 52196+ A? fakenodeaddr. (30)
219[00:46:26.219] 00:45:39.786693 IP 172.18.0.2.53292 > 8.8.8.8.53: 56037+ AAAA? fakenodeaddr. (30)
220[00:46:26.219] 00:45:39.792298 IP 8.8.8.8.53 > 172.18.0.2.54375: 52196 NXDomain 0/1/0 (105)
221[00:46:26.219] 00:45:39.792592 IP 8.8.8.8.53 > 172.18.0.2.53292: 56037 NXDomain 0/1/0 (105)
222[00:46:26.219] 00:45:42.174906 IP 11.22.33.44.18444 > 172.18.0.2.51812: Flags [R.], seq 0, ack 3432062660, win 65535, length 0
223[00:46:26.219] 00:45:42.722951 IP 172.18.0.2.33938 > 8.8.8.8.53: 23646+ AAAA? fakeaddress1. (30)
224[00:46:26.219] 00:45:42.723043 IP 172.18.0.2.38954 > 8.8.8.8.53: 24914+ A? fakeaddress1. (30)
225[00:46:26.219] 00:45:42.728907 IP 8.8.8.8.53 > 172.18.0.2.38954: 24914 NXDomain 0/1/0 (105)
226[00:46:26.219] 00:45:42.729111 IP 8.8.8.8.53 > 172.18.0.2.33938: 23646 NXDomain 0/1/0 (105)
227[00:46:26.219] 00:45:42.729684 IP 172.18.0.2.52893 > 8.8.8.8.53: 3801+ AAAA? fakeaddress1. (30)
228[00:46:26.219] 00:45:42.729715 IP 172.18.0.2.57497 > 8.8.8.8.53: 62431+ A? fakeaddress1. (30)
229[00:46:26.219] 00:45:42.735746 IP 8.8.8.8.53 > 172.18.0.2.57497: 62431 NXDomain 0/1/0 (105)
230[00:46:26.219] 00:45:42.735771 IP 8.8.8.8.53 > 172.18.0.2.52893: 3801 NXDomain 0/1/0 (105)
231[00:46:26.219] 00:45:43.175350 IP 11.22.33.44.18444 > 172.18.0.2.51828: Flags [R.], seq 0, ack 3743466588, win 65535, length 0

1440000bytes commented at 1:09 pm on November 26, 2024: none

Concept ACK

A simple solution to avoid leaking IP address when running tests locally would be to disconnect internet while running tests.

jonatack commented at 1:57 pm on November 26, 2024: member

Concept ACK. Per https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2024-11-26#1069602: “it turns out the owners of 1.2.3.4, 11.22.33.44 and 8.8.8.8, if they would bother, would know the IP address of every dev who runs the functional tests at home.”

vasild force-pushed on Nov 27, 2024

DrahtBot added the label CI failed on Nov 27, 2024

DrahtBot commented at 4:42 pm on November 27, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33609854840

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

vasild force-pushed on Nov 27, 2024

vasild commented at 5:36 pm on November 27, 2024: contributor

8799018bd5...803ed4638b: include #31343 into this PR to demonstrate that #31343 works as intended and also to turn the CI here green.

About the false positives - I think it is worth trying this in its current mode where any detected traffic is assumed to have originated from the tests and fails the CI. If this ever fails the CI for another reason (false positive), then it would be easy to turn this into a “report in the logs only but don’t fail” by removing the exit 1 line.

About the DNS traffic - I did cat /etc/resolv.conf in the CI and the name server is indeed configured as a local one: nameserver 127.0.0.11.

BrandonOdiwuor commented at 5:44 pm on November 27, 2024: contributor

Concept ACK

vasild marked this as a draft on Nov 28, 2024

vasild force-pushed on Nov 28, 2024

vasild commented at 5:31 pm on November 28, 2024: contributor

Converted to draft for a while, testing docker with full privileges (need cirrus which does not run in my personal fork).

vasild force-pushed on Nov 29, 2024

DrahtBot removed the label CI failed on Nov 29, 2024

vasild force-pushed on Nov 29, 2024

vasild force-pushed on Dec 2, 2024

test: avoid generating non-loopback traffic from p2p_seednode.py

`p2p_seednode.py` would try to connect to `0.0.0.1` and `0.0.0.2` as
seed nodes. This sends outbound TCP packets on a non-loopback interface
to the default router.

Configure an unavailable proxy for all executions of `bitcoind` during
this test. Also change `0.0.0.1` and `0.0.0.2` because connecting to
them would skip the `-proxy=` setting because for such an address:
* `CNetAddr::IsLocal()` is true, thus
* `CNetAddr::IsRoutable()` is false, thus
* `CNetAddr::GetNetwork()` is `NET_UNROUTABLE`, even though
  `CNetAddr::m_net` is `NET_IPV4`.

This speeds up the execution time of `p2p_seednode.py`
from 12.5s to 2.5s.

66358f73e3

test: avoid generating non-loopback traffic from feature_config_args.py

`feature_config_args.py` uses a proxy address of `1.2.3.4`. This results
in actually trying to open TCP connections over the internet to
`1.2.3.4:9050`.

The test does not need those to succeed so use `127.0.0.1:1` instead.

Also avoid `-noconnect=0` because that is interpreted as `-connect=1`
which is interpreted as `-connect=0.0.0.1` and a connection to
`0.0.0.1:18444` is attempted.

134b7a2ff2

test: avoid generating non-loopback traffic from p2p_dns_seeds.py

`p2p_dns_seeds.py` would try to connect to the DNS server configured on
the machine and resolve `dummySeed.invalid`.

To block that configure an unavailable proxy which will be used also to
connect to the name server. The test needs 2 successful connections to
other peers (two Python `P2PInterface`s) and they work in spite of the
unavailable proxy because they are on `127.0.0.1` (`NET_UNROUTABLE`) and
the proxy is not used for that.

b291a632b5

test: avoid internet traffic in rpc_net.py

Can be tested by running

```
$ sudo tcpdump -i eth0 host 11.22.33.44
```

and verifying that no packets appear in the tcpdump output.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>

a824236a3c

ci: detect outbound internet traffic generated while running tests

Resolves https://github.com/bitcoin/bitcoin/issues/31339

c88464d754

vasild force-pushed on Dec 3, 2024

vasild commented at 10:48 am on December 3, 2024: contributor

Ready for review. I updated the OP with some details.

vasild marked this as ready for review on Dec 3, 2024

in ci/test/03_test_script.sh:160 in c88464d754

155+}
156+
157+function traffic_monitor_begin()
158+{
159+  for ifname in $(get_interfaces) ; do
160+    tcpdump -n -i "$ifname" -w "$(tcpdump_file_for_interface "$ifname")" &

maflcko commented at 10:59 am on December 3, 2024:

I don’t think this works reliably? See:

0              This output will be buffered if written to a file or pipe, so a program reading from the file or pipe may not see packets for an arbitrary amount of time after they are received.  Use the -U flag to cause packets to be
1              written as soon as they are received.

in ci/test/03_test_script.sh:170 in c88464d754

165+{
166+  # Stop all tcpdump instances (we want the word splitting if "jobs -p" returns more than one PID).
167+  # shellcheck disable=SC2046
168+  while kill -SIGTERM $(jobs -p) ; do
169+    sleep 1
170+  done

maflcko commented at 11:01 am on December 3, 2024:

Why is this needed? The file isn’t cleared anyway (and exited if it isn’t clear), so might as well just run the processes and never kill them?

Also, it doesn’t seem to be working anyway on some of the tasks?

ci: detect outbound internet traffic generated while running tests #31349

Code Coverage & Benchmarks

Reviews

Conflicts