test: close the listeners before terminating the event loop #35583

pull vasild wants to merge 2 commits into bitcoin:master from vasild:test_close_listeners changing 1 files +13 −1
  1. vasild commented at 4:47 PM on June 22, 2026: contributor

    Whenever a test creates a new P2PInterface object a new listener is created inside NetworkThread.create_listen_server() by calling cls.network_event_loop.create_server().

    These listeners are never closed which might result in:

    2026-06-10T22:13:35.3934880Z Task was destroyed but it is pending!
    2026-06-10T22:13:35.3936020Z task: <Task pending name='Task-54' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/asyncio/selector_events.py:217> wait_for=<Future finished result=None>>
    

    when the event loop is closed.

    Fix that by closing the listeners.

    Fixes: #35508

  2. DrahtBot added the label Tests on Jun 22, 2026
  3. DrahtBot commented at 4:47 PM on June 22, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35583.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. vasild commented at 4:54 PM on June 22, 2026: contributor

    I can't reproduce the problem from #35508 locally, so I can't confirm 100% that this fixes it.

    The reasoning is this - the error is coming from a Python asyncio listening server. We do not use those for the SOCKS5 proxy, but use them for the P2PInterface objects:

    bitcoind --> SOCKS5 proxy --> P2PInterface
    

    we collect the listeners in NetworkThread.listeners but only to check for duplicate listening addr:port and never close them. So, extend NetworkThread.close() to close all of them. This seems like a good hygiene anyway.

  5. andrewtoth commented at 8:38 PM on June 22, 2026: contributor

    I managed to get a diff that reproduces this locally by monkey-patching _accept_connection2:

    diff --git a/test/functional/test_framework/test_framework.py b/test/functional/test_framework/test_framework.py
    index 64dcbfd7ec..16a0805c4b 100755
    --- a/test/functional/test_framework/test_framework.py
    +++ b/test/functional/test_framework/test_framework.py
    @@ -7,6 +7,7 @@
     import configparser
     from enum import Enum
     import argparse
    +import asyncio
     from datetime import datetime, timezone
     from importlib.util import find_spec
     import logging
    @@ -18,6 +19,7 @@ import random
     import re
     import shutil
     import subprocess
    +import socket
     import sys
     import tempfile
     import time
    @@ -281,6 +283,21 @@ class BitcoinTestFramework(metaclass=BitcoinTestMetaClass):
                 pdb.set_trace()
     
             self.log.debug('Closing down network thread')
    +
    +        loop = self.network_thread.network_event_loop
    +        make_transport = type(loop)._make_socket_transport
    +
    +        async def slow_accept(self, protocol_factory, conn, extra, sslcontext=None, server=None, *a, **k):
    +            transport = make_transport(self, conn, asyncio.Protocol(), extra=extra, server=server)
    +            try:
    +                await asyncio.sleep(2)
    +            finally:
    +                transport.close()
    +        type(loop)._accept_connection2 = slow_accept
    +
    +        addr, port = next(iter(NetworkThread.listeners))
    +        self._repro_sock = socket.create_connection((addr, port))
    +        time.sleep(0.3)
             self.network_thread.close(timeout=self.options.timeout_factor * 10)
             if self.success == TestStatus.FAILED:
                 self.log.info("Not stopping nodes as test failed. The dangling processes will be cleaned up later.")
    

    Confirmed this triggers the error every run on master, and does not trigger on this PR. Note: this fix only works on python > 3.12.1, so will still break on previous releases. (but wait, do we run the private broadcast tests on that CI job?)

  6. DrahtBot added the label CI failed on Jun 22, 2026
  7. vasild commented at 7:11 AM on June 24, 2026: contributor

    this fix only works on python > 3.12.1

    Why?

    so will still break on previous releases. (but wait, do we run the private broadcast tests on that CI job?)

    test_runner.py is ran, so "yes". Note that the change in this PR is not private broadcast specific.

  8. vasild marked this as a draft on Jun 24, 2026
  9. vasild commented at 8:25 AM on June 24, 2026: contributor

    Converted to draft because this needs some more fiddling with.

  10. test: close the listeners before terminating the event loop
    Whenever a test creates a new `P2PInterface` object a new listener is
    created inside `NetworkThread.create_listen_server()` by calling
    `cls.network_event_loop.create_server()`.
    
    These listeners are never closed which might result in:
    
    ```
    2026-06-10T22:13:35.3934880Z Task was destroyed but it is pending!
    2026-06-10T22:13:35.3936020Z task: <Task pending name='Task-54' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/asyncio/selector_events.py:217> wait_for=<Future finished result=None>>
    ```
    
    when the event loop is closed.
    
    Fix that by closing the listeners.
    
    Fixes: https://github.com/bitcoin/bitcoin/issues/35508
    f7a68e308b
  11. test: close the loop after the network thread has completed
    https://docs.python.org/3.15/library/asyncio-eventloop.html#asyncio.loop.close
    reads "The loop must not be running when this function is called". It
    seems safer to call `close()` after the thread has exited.
    7a64f92589
  12. vasild force-pushed on Jun 24, 2026
  13. andrewtoth commented at 2:30 PM on June 24, 2026: contributor

    this fix only works on python > 3.12.1

    Why?

    https://github.com/python/cpython/blob/3.12/Lib/asyncio/base_events.py#L390-L395

    Sorry, should be >= 3.12.1.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-06-25 10:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me