test: use IP_PORTRANGE_HIGH on FreeBSD for dynamic port allocation #34346

pull w0xlt wants to merge 2 commits into bitcoin:master from w0xlt:freebsd_high_port_range-again changing 3 files +46 −3
  1. w0xlt commented at 5:57 pm on January 19, 2026: contributor

    Reopening #34336. I’ve now tested it on FreeBSD and confirmed it works.

    On FreeBSD, the default ephemeral port range (10000-65535) overlaps with the test framework’s static port range (11000-26000), possibly causing intermittent “address already in use” failures when tests use dynamic port allocation (port=0).

    This PR adds a helper that sets IP_PORTRANGE_HIGH via setsockopt() before binding, requesting ports from 49152-65535 instead, which avoids the overlap, as suggested in #34331 (comment) by @maflcko .

    From FreeBSD’s sys/netinet/in.h:

    0#define IP_PORTRANGE         19
    1#define IP_PORTRANGE_HIGH    1
    2#define IPPORT_EPHEMERALFIRST 10000  /* default range start */
    3#define IPPORT_HIFIRSTAUTO   49152   /* high range start */
    

    See also: FreeBSD https://man.freebsd.org/cgi/man.cgi?query=ip&sektion=4 man page.

    Fixes #34331

  2. DrahtBot added the label Tests on Jan 19, 2026
  3. DrahtBot commented at 5:57 pm on January 19, 2026: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/34346.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK vasild

    If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

    Conflicts

    No conflicts as of last run.

    LLM Linter (✨ experimental)

    Possible places where named args for integral literals may be used (e.g. func(x, /*named_arg=*/0) in C++, and func(x, named_arg=0) in Python):

    • self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) in test/functional/test_framework/socks5.py

    2026-01-21

  4. w0xlt marked this as a draft on Jan 19, 2026
  5. w0xlt force-pushed on Jan 19, 2026
  6. in test/functional/test_framework/netutil.py:190 in 01875b1122
    185+
    186+def set_freebsd_high_port_range(sock):
    187+    '''On FreeBSD, set socket to use the high ephemeral port range (49152-65535).
    188+
    189+    FreeBSD's default ephemeral port range (10000-65535) overlaps with the test
    190+    framework's static port range (11000-26000). Using IP_PORTRANGE_HIGH avoids
    


    maflcko commented at 6:14 pm on January 19, 2026:
    0    framework's static port range starting at TEST_RUNNER_PORT_MIN (default=11000). Using IP_PORTRANGE_HIGH avoids
    

    nit: Could avoid hard-coding the upper range, which may change in the future.


    w0xlt commented at 6:26 pm on January 20, 2026:
    Done. Thanks.
  7. DrahtBot added the label CI failed on Jan 19, 2026
  8. in test/functional/test_framework/netutil.py:186 in 01875b1122
    180@@ -181,3 +181,21 @@ def format_addr_port(addr, port):
    181         return f"[{addr}]:{port}"
    182     else:
    183         return f"{addr}:{port}"
    184+
    185+
    186+def set_freebsd_high_port_range(sock):
    


    vasild commented at 10:22 am on January 20, 2026:

    Maybe better to use more generic name for the function. It contains if sys.platform.startswith('freebsd') and would accommodate other OSes well if at some point it would be necessary to do tweaks for others. E.g. if freebsd tweak1, else if openbsd tweak2, if linuxcool tweak3, if windoz ugh.

    What about set_ephemeral_port_range()?


    w0xlt commented at 6:27 pm on January 20, 2026:
    Done. Thanks.
  9. in test/functional/test_framework/socks5.py:209 in 01875b1122
    202@@ -202,6 +203,10 @@ def __init__(self, conf):
    203         self.conf = conf
    204         self.s = socket.socket(conf.af)
    205         self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    206+        # When using dynamic port allocation (port=0), ensure we don't get a
    207+        # port that conflicts with the test framework's static port range.
    208+        if conf.addr[1] == 0:
    209+            set_freebsd_high_port_range(self.s)
    


    vasild commented at 10:30 am on January 20, 2026:

    I checked that this works as expected - without it some ports are below 49152 and with it the selected ports are above (for a couple of dozen tries).

    This covers only the port for the Socks5 server itself, but not ports used by the P2P listeners, created with network_event_loop.create_server(). Here is an extension to this patch that would cover the later as well:

      0diff --git i/test/functional/test_framework/netutil.py w/test/functional/test_framework/netutil.py
      1index 792afabff6..8fda0baa31 100644
      2--- i/test/functional/test_framework/netutil.py
      3+++ w/test/functional/test_framework/netutil.py
      4@@ -180,13 +180,13 @@ def format_addr_port(addr, port):
      5     if ":" in addr:
      6         return f"[{addr}]:{port}"
      7     else:
      8         return f"{addr}:{port}"
      9 
     10 
     11-def set_freebsd_high_port_range(sock):
     12+def set_ephemeral_port_range(sock):
     13     '''On FreeBSD, set socket to use the high ephemeral port range (49152-65535).
     14 
     15     FreeBSD's default ephemeral port range (10000-65535) overlaps with the test
     16     framework's static port range (11000-26000). Using IP_PORTRANGE_HIGH avoids
     17     this overlap when binding to port 0 for dynamic port allocation.
     18     '''
     19diff --git i/test/functional/test_framework/p2p.py w/test/functional/test_framework/p2p.py
     20index 986eaf1e88..b6903a6ba2 100755
     21--- i/test/functional/test_framework/p2p.py
     22+++ w/test/functional/test_framework/p2p.py
     23@@ -19,16 +19,18 @@ P2PDataStore: A p2p interface class that keeps a store of transactions and block
     24               and can respond correctly to getdata and getheaders messages
     25 P2PTxInvStore: A p2p interface class that inherits from P2PDataStore, and keeps
     26               a count of how many times each txid has been announced."""
     27 
     28 import asyncio
     29 from collections import defaultdict
     30+import ipaddress
     31 from io import BytesIO
     32 import logging
     33 import platform
     34 import struct
     35+import socket
     36 import sys
     37 import threading
     38 
     39 from test_framework.messages import (
     40     CBlockHeader,
     41     MAX_HEADERS_RESULTS,
     42@@ -73,12 +75,15 @@ from test_framework.messages import (
     43     msg_wtxidrelay,
     44     NODE_NETWORK,
     45     NODE_WITNESS,
     46     MAGIC_BYTES,
     47     sha256,
     48 )
     49+from test_framework.netutil import (
     50+    set_ephemeral_port_range,
     51+)
     52 from test_framework.util import (
     53     assert_not_equal,
     54     MAX_NODES,
     55     p2p_port,
     56     wait_until_helper_internal,
     57 )
     58@@ -744,12 +749,14 @@ class NetworkThread(threading.Thread):
     59         self.network_event_loop.run_forever()
     60 
     61     def close(self, *, timeout=10):
     62         """Close the connections and network event loop."""
     63         self.network_event_loop.call_soon_threadsafe(self.network_event_loop.stop)
     64         wait_until_helper_internal(lambda: not self.network_event_loop.is_running(), timeout=timeout)
     65+        for listener in NetworkThread.listeners.values():
     66+            listener.close()
     67         self.network_event_loop.close()
     68         self.join(timeout)
     69         # Safe to remove event loop.
     70         NetworkThread.network_event_loop = None [@classmethod](/bitcoin-bitcoin/contributor/classmethod/)
     71@@ -790,14 +797,28 @@ class NetworkThread(threading.Thread):
     72         if port == 0 or (addr, port) not in cls.listeners:
     73             # When creating a listener on a given (addr, port) we only need to
     74             # do it once. If we want different behaviors for different
     75             # connections, we can accomplish this by providing different
     76             # `proto` functions
     77 
     78-            listener = await cls.network_event_loop.create_server(peer_protocol, addr, port)
     79-            port = listener.sockets[0].getsockname()[1]
     80+            if port == 0:
     81+                # Manually create the socket in order to set the range to be
     82+                # used for the port before the bind() call.
     83+                if ipaddress.ip_address(addr).version == 4:
     84+                    address_family = socket.AF_INET
     85+                else:
     86+                    address_family = socket.AF_INET6
     87+                s = socket.socket(address_family)
     88+                set_ephemeral_port_range(s)
     89+                s.bind((addr, 0))
     90+                s.listen()
     91+                listener = await cls.network_event_loop.create_server(peer_protocol, sock=s)
     92+                port = listener.sockets[0].getsockname()[1]
     93+            else:
     94+                listener = await cls.network_event_loop.create_server(peer_protocol, addr, port)
     95+
     96             logger.debug("Listening server on %s:%d should be started" % (addr, port))
     97             cls.listeners[(addr, port)] = listener
     98 
     99         cls.protos[(addr, port)] = proto
    100         callback(addr, port)
    101 
    102diff --git i/test/functional/test_framework/socks5.py w/test/functional/test_framework/socks5.py
    103index 4c77a6ff25..085c5a2e32 100644
    104--- i/test/functional/test_framework/socks5.py
    105+++ w/test/functional/test_framework/socks5.py
    106@@ -9,13 +9,13 @@ import socket
    107 import threading
    108 import queue
    109 import logging
    110 
    111 from .netutil import (
    112     format_addr_port,
    113-    set_freebsd_high_port_range,
    114+    set_ephemeral_port_range,
    115 )
    116 
    117 logger = logging.getLogger("TestFramework.socks5")
    118 
    119 # Protocol constants
    120 class Command:
    121@@ -203,13 +203,13 @@ class Socks5Server():
    122         self.conf = conf
    123         self.s = socket.socket(conf.af)
    124         self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    125         # When using dynamic port allocation (port=0), ensure we don't get a
    126         # port that conflicts with the test framework's static port range.
    127         if conf.addr[1] == 0:
    128-            set_freebsd_high_port_range(self.s)
    129+            set_ephemeral_port_range(self.s)
    130         self.s.bind(conf.addr)
    131         # When port=0, the OS assigns an available port. Update conf.addr
    132         # to reflect the actual bound address so callers can use it.
    133         self.conf.addr = self.s.getsockname()
    134         self.s.listen(5)
    135         self.running = False
    

    w0xlt commented at 6:27 pm on January 20, 2026:
    Done. Added the patch. Thanks.

    w0xlt commented at 8:06 pm on January 20, 2026:
    With this patch, p2p_add_connections.py fails on macOS, ARM, i686, and Windows with: TypeError: 'NoneType' object is not iterable. Looking into this.

    w0xlt commented at 11:43 pm on January 20, 2026:

    The issue in this patch appears to stem from the listener.close() calls in NetworkThread.close(). Removing these calls allows the test to pass.

    It seems that these calls can trigger asyncio cleanup errors on macOS (and likely other platforms), because they may be executed after the event loop has already stopped. This results in a TypeError: 'NoneType' object is not iterable originating from asyncio’s _SelectorTransport.__del__.

    Given that the event loop’s close() method apparently already performs resource cleanup—and that all tests pass without the explicit listener.close() calls, are these calls still necessary?


    w0xlt commented at 2:44 am on January 21, 2026:
    Re-added that patch in a separate commit https://github.com/bitcoin/bitcoin/pull/34346/commits/23f613acf94b1cefdc7d2e339cdd3c1bb0d33d0e . Removed the listener.close().

    vasild commented at 2:50 pm on January 21, 2026:

    I was not sure if the added close() calls are necessary. The python docs say:

    https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_server

    sock can optionally be specified in order to use a preexisting socket object. If specified, host and port must not be specified.

    Note

    The sock argument transfers ownership of the socket to the server created. To close the socket, call the server’s close() method.

    We only put the listeners in the NetworkThread.listeners list, but never read from that, so it is probably recognized and used by the Python system code itself? NetworkThread.close() would call close() on each member?


    w0xlt commented at 7:31 pm on January 21, 2026:

    Technically, per the Python docs, Server.close() (i.e., listener.close() in the code) should be called explicitly.

    However, we would need to await wait_closed() after close(), before closing the event loop, which adds complexity and can introduce cross-platform timing issues (I tested a run_coroutine_threadsafe approach with wait_closed() and it deadlocked on macOS).

    Given that:

    • The test framework processes are short-lived
    • The OS reclaims all sockets when the process exits
    • Explicit cleanup introduces complexity without practical benefit here

    I think it’s fine to skip the explicit listener.close() calls.


    vasild commented at 7:47 am on January 22, 2026:

    I think it is fine too.

    Just curious, where did you find that: “per the Python docs, Server.close() (i.e., listener.close() in the code) should be called explicitly”?


    w0xlt commented at 6:17 pm on January 22, 2026:

    This was my (possibly an over-interpretation) of: “To close the socket, call the server’s close() method.https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_server

    This error isn’t related to your patch itself. If the same change is applied to master, it fails too.

     0@@ -744,12 +749,14 @@ class NetworkThread(threading.Thread):
     1         self.network_event_loop.run_forever()
     2 
     3     def close(self, *, timeout=10):
     4         """Close the connections and network event loop."""
     5         self.network_event_loop.call_soon_threadsafe(self.network_event_loop.stop)
     6         wait_until_helper_internal(lambda: not self.network_event_loop.is_running(), timeout=timeout)
     7+        for listener in NetworkThread.listeners.values():
     8+            listener.close()
     9         self.network_event_loop.close()
    10         self.join(timeout)
    11         # Safe to remove event loop.
    12         NetworkThread.network_event_loop = None
    

    Even if the listeners are closed before stopping the event loop, the error persists:

    0     def close(self, *, timeout=10):
    1         """Close the connections and network event loop."""
    2+        for listener in NetworkThread.listeners.values():
    3+            listener.close()
    4         self.network_event_loop.call_soon_threadsafe(self.network_event_loop.stop)
    5         wait_until_helper_internal(lambda: not self.network_event_loop.is_running(), timeout=timeout)
    

    The error only happens on macOS, ARM, i686, and Windows. On Linux, it proceeds as expected.

    Should an issue be opened for it ?


    vasild commented at 10:01 am on January 23, 2026:

    Should an issue be opened for it ?

    I think no.

    “To close the socket, call the server’s close() method.”

    Ok, that probably is intended to hint the user that now the socket is owned by the server and the proper way to close it is to use the server’s close() method instead of closing the socket directly.

    I assume self.network_event_loop.close() will call those close() methods on the servers, but couldn’t find an explicit docs about it.

    I think no further changes are needed here.

  10. vasild approved
  11. vasild commented at 10:31 am on January 20, 2026: contributor
    ACK 01875b1122d17b1d760550f87babd2ce3dc24c13
  12. DrahtBot removed the label CI failed on Jan 20, 2026
  13. bitcoin deleted a comment on Jan 20, 2026
  14. w0xlt force-pushed on Jan 20, 2026
  15. w0xlt marked this as ready for review on Jan 20, 2026
  16. w0xlt force-pushed on Jan 20, 2026
  17. DrahtBot added the label CI failed on Jan 20, 2026
  18. w0xlt force-pushed on Jan 20, 2026
  19. test: use IP_PORTRANGE_HIGH on FreeBSD for dynamic port allocation
    On FreeBSD, the default ephemeral port range (10000-65535) overlaps
    with the test framework's static port range (11000-26000), causing
    intermittent "address already in use" failures when tests use dynamic
    port allocation (port=0).
    
    Add a helper function that sets the IP_PORTRANGE/IPV6_PORTRANGE socket
    option to IP_PORTRANGE_HIGH before binding, which requests ports from
    the high range (49152-65535) instead. This range does not overlap with
    the test framework's static ports.
    
    Constants from FreeBSD's netinet/in.h and netinet6/in6.h:
    - IP_PORTRANGE = 19 (for IPv4 sockets)
    - IPV6_PORTRANGE = 14 (for IPv6 sockets)
    - IP_PORTRANGE_HIGH = 1
    
    Fixes: bitcoin/bitcoin#34331
    
    Co-Authored-By: Vasil Dimov <vd@FreeBSD.org>
    Co-Authored-By: MarcoFalke <*~=\`'#}+{/-|&$^_@721217.xyz>
    34bed0ed8c
  20. w0xlt force-pushed on Jan 20, 2026
  21. DrahtBot removed the label CI failed on Jan 20, 2026
  22. hebasto commented at 1:45 pm on January 21, 2026: member

    Running CI jobs on:

  23. in test/functional/test_framework/netutil.py:184 in 23f613acf9 outdated
    180@@ -181,3 +181,22 @@ def format_addr_port(addr, port):
    181         return f"[{addr}]:{port}"
    182     else:
    183         return f"{addr}:{port}"
    184+
    


    vasild commented at 3:19 pm on January 21, 2026:

    In the commit message of 23f613acf94b1cefdc7d2e339cdd3c1bb0d33d0e test: use IP_PORTRANGE_HIGH on FreeBSD for dynamic port allocation:

    s/commit added/commit added/


    w0xlt commented at 7:34 pm on January 21, 2026:
    Done. Thanks.
  24. vasild approved
  25. vasild commented at 3:21 pm on January 21, 2026: contributor
    ACK 23f613acf94b1cefdc7d2e339cdd3c1bb0d33d0e
  26. test: extend FreeBSD ephemeral port range fix to P2P listeners
    The previous commit added set_ephemeral_port_range() to
    avoid port conflicts on FreeBSD by requesting ports from the high
    ephemeral range (49152-65535) instead of the default range
    which overlaps with the test framework's static port range.
    
    That fix was applied to the SOCKS5 server but not to P2P listeners
    created via NetworkThread.create_listen_server(). This commit extends
    the fix to cover P2P listeners as well.
    
    When port=0 is requested (dynamic allocation), we now:
    1. Manually create a socket with the appropriate address family
    2. Call set_ephemeral_port_range() to configure the port range
    3. Bind and listen on the socket
    4. Pass the pre-configured socket to asyncio's create_server()
    
    This ensures that dynamically allocated ports for P2P listeners also
    come from the high range on FreeBSD, avoiding conflicts with the test
    framework's static port assignments.
    
    Co-Authored-By: Vasil Dimov <vd@FreeBSD.org>
    2845f10a2b
  27. w0xlt force-pushed on Jan 21, 2026
  28. vasild approved
  29. vasild commented at 7:48 am on January 22, 2026: contributor
    ACK 2845f10a2be0fee13b2772d24e948052243782b8

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-01-27 06:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me