ci: isolate container test networks #35417

willcl-ark commented at 7:54 PM on May 29, 2026: member

Local CI jobs currently share Docker networks while requesting fixed routable addresses. That prevents two jobs from running at the same time because the second container cannot claim the same static IPs. See 33362.

Name the networks "{container_name}-ipv[4|6]", using a subnet based on the hash of their name, so each CI job gets a separate network while the tests continue to see the addresses they expect.

Increment them to 11.x.x.x and 1111:2222:x to avoid hitting (existing) dangling networks on machines.

On successful runs, the networks are cleaned up. On failed runs they will remain (as they are being used by the failing container).

DrahtBot added the label Tests on May 29, 2026

DrahtBot commented at 7:54 PM on May 29, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35417.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	fanquake

If your review is incorrectly listed, please copy-paste <code></code> into the comment that the bot should ignore.

willcl-ark commented at 7:56 PM on May 29, 2026: member

Implements one apprach from #35416 (comment)

I tried other ways of generating deterministic networks here (we have prcedent for using 'os.getpid()`), but I think in the end this is probably the most robust idea.

IMO cleaning up the networks is nicer, but I can drop that commit if it's not wanted for some reason unknown to me currently (or i can squash it).

Also open to other solutions here.

fanquake commented at 8:01 PM on May 29, 2026: member

Concept ACK - thanks for following up here.

in ci/test/02_run_container.py:149 in d23a74aa1b

 142 | @@ -114,8 +143,10 @@ def main():
 143 |                  sys.exit(1)
 144 |              CI_CCACHE_MOUNT = f"type=bind,src={os.environ['CCACHE_DIR']},dst={os.environ['CCACHE_DIR']}"
 145 |  
 146 | -        run(["docker", "network", "create", "--ipv6", "--subnet", "1111:1111::/112", "ci-ip6net"], check=False)
 147 | -        run(["docker", "network", "create", "--subnet", "1.1.1.0/24", "ci-ip4net"], check=False)
 148 | +        ci_ip6net = f"{os.environ['CONTAINER_NAME']}-ipv6"
 149 | +        ci_ip4net = f"{os.environ['CONTAINER_NAME']}-ipv4"
 150 | +        run(["docker", "network", "create", "--ipv6", "--subnet", str(ip6_subnet), ci_ip6net], check=False)
 151 | +        run(["docker", "network", "create", "--subnet", str(ip4_subnet), ci_ip4net], check=False)

maflcko commented at 1:28 PM on June 2, 2026:

nit: Would it not be easier to spin up a single network with both subnets?

willcl-ark commented at 7:55 PM on June 2, 2026:

neat. done.

in ci/test/02_run_container.py:179 in d23a74aa1b

 174 | @@ -144,8 +175,8 @@ def main():
 175 |              *CI_BUILD_MOUNT,
 176 |              f"--env-file={env_file}",
 177 |              f"--name={os.environ['CONTAINER_NAME']}",
 178 | -            "--network=ci-ip6net",
 179 | -            "--ip6=1111:1111::5", # Used by some of the tests, don't change it just here (keep them in sync).
 180 | +            f"--network={ci_ip6net}",
 181 | +            f"--ip6={os.environ['CI_ROUTABLE_IPV6']}",

maflcko commented at 1:29 PM on June 2, 2026:

nit: Instead of assigning one ip here, and the other below, why not assign both below?

willcl-ark commented at 7:56 PM on June 2, 2026:

both now assigned together

in ci/test/02_run_container.py:44 in d23a74aa1b

  39 |  def main():
  40 | +    container_id = None
  41 | +    ci_ip4net = None
  42 | +    ci_ip6net = None
  43 | +    ip4_subnet = None
  44 | +    ip6_subnet = None

maflcko commented at 1:35 PM on June 2, 2026:

Why this hunk? This is verbose and only turns a well-understood python run-time error into a harder-to-debug other runtime error?

willcl-ark commented at 7:56 PM on June 2, 2026:

At first I had something in cleanup using it, but I dropped it, so have now dropped this too.

in test/functional/feature_bind_port_discover.py:34 in d23a74aa1b

  30 | @@ -29,8 +31,8 @@
  31 |  # FreeBSD and MacOS:
  32 |  # ifconfig INTERFACE_NAME 1.1.1.5/32 alias && ifconfig INTERFACE_NAME inet6 1111:1111::5/128 alias  # to set up
  33 |  # ifconfig INTERFACE_NAME 1.1.1.5 -alias && ifconfig INTERFACE_NAME inet6 1111:1111::5 -alias       # to remove it, after the test
  34 | -ADDR1 = '1.1.1.5' # This and the address below are set in the CI environment, don't change it just here (keep them in sync).
  35 | -ADDR2 = '1111:1111::5'
  36 | +ADDR1 = os.getenv('CI_ROUTABLE_IPV4', '1.1.1.5') # This and the address below are set in the CI environment, don't change it just here (keep them in sync).

maflcko commented at 1:44 PM on June 2, 2026:

ADDR1 = os.getenv('BIND_TEST_ROUTABLE_IPV4', '1.1.1.5') # This and the address below are set in the CI environment, don't change it just here (keep them in sync).

Maybe change the prefix, so that it isn't tied to the CI?

willcl-ark commented at 7:56 PM on June 2, 2026:

Done

maflcko approved

maflcko commented at 1:44 PM on June 2, 2026: member

lgtm, seems fine. Left some nits

ci: isolate local test networks

Local CI jobs currently share Docker networks while requesting fixed
routable addresses. That prevents two jobs from running at the same time
because the second container cannot claim the same static IPs. See
33362.

Name the networks "{container_name}-ipv[4|6]", using a subnet based on
the hash of their name, so each CI job gets a separate network while the
tests continue to see the addresses they expect.

ee81a469c8

ci: clean up local test networks

Previously the (single, shared) networks were left in docker after runs,
but with a network container this may get messy.

Remove the IPv4 and IPv6 networks when the run is successful. On a
failed run, networks will remain dangling and need manual pruning (same
behaviour as previously).

We can't remove networks automagically form failed runs without first
killing the container (docker doesn't let you remove networks which are
active), which we want to avoid so we can jump in to failures.

3fefa9c120

willcl-ark force-pushed on Jun 2, 2026

DrahtBot added the label CI failed on Jun 2, 2026

DrahtBot commented at 3:19 PM on June 10, 2026: contributor

CI failure:

+ docker network create --ipv6 --subnet 11.51.44.16/28 --subnet 1111:2222::3:32c1:0/112 ci_arm_linux-net
cd1c43f84c4b72cb079c930b48765e36ea8e79d751d336e90365917a9f5a72d9
Prune all dangling bitcoin-ci-test images
+ docker image prune --force --filter label=bitcoin-ci-test
Total reclaimed space: 0B
+ docker run --rm --interactive --detach --tty --cap-add=LINUX_IMMUTABLE --mount=type=bind,src=/home/runner/work/bitcoin/bitcoin,dst=/home/runner/work/bitcoin/bitcoin,readonly --mount=type=bind,src=/home/runner/work/_temp/ccache_dir,dst=/home/runner/work/_temp/ccache_dir --mount=type=bind,src=/home/runner/work/_temp/depends/built,dst=/home/runner/work/_temp/depends/built --mount=type=bind,src=/home/runner/work/_temp/depends/sources,dst=/home/runner/work/_temp/depends/sources --mount=type=bind,src=/home/runner/work/_temp/previous_releases,dst=/home/runner/work/_temp/previous_releases --mount=type=bind,src=/home/runner/work/_temp/build,dst=/home/runner/work/_temp/build --env-file=/tmp/env-runner-ci_arm_linux --name=ci_arm_linux --network=none --platform=linux/arm64 ci_arm_linux
+ docker network connect --ip=11.51.44.21 --ip6=1111:2222::3:32c1:5 ci_arm_linux-net b9b658b4df7a1d872d5e588104b7ff98d97f76a0c19fac54310aeef8bd2f23eb
Error response from daemon: container cannot be connected to multiple networks with one of the networks in private (none) mode
Command '['docker', 'network', 'connect', '--ip=11.51.44.21', '--ip6=1111:2222::3:32c1:5', 'ci_arm_linux-net', 'b9b658b4df7a1d872d5e588104b7ff98d97f76a0c19fac54310aeef8bd2f23eb']' returned non-zero exit status 1.

DrahtBot marked this as a draft on Jun 19, 2026

DrahtBot commented at 8:46 AM on June 19, 2026: contributor

Converted to draft, due to failing CI