test: Avoid shutdown race in NetworkThread #33140

1. 
   maflcko commented at 7:25 am on August 6, 2025: member

   Locally, I am seeing rare intermittent exceptions in the network thread:

    0stderr:
    1Exception in thread NetworkThread:
    2Traceback (most recent call last):
    3File "/python3.10/threading.py", line 1016, in _bootstrap_inner
    4self.run()
    5File "./test/functional/test_framework/p2p.py", line 744, in run
    6self.network_event_loop.run_forever()
    7File "/python3.10/asyncio/base_events.py", line 603, in run_forever
    8self._run_once()
    9File "/python3.10/asyncio/base_events.py", line 1871, in _run_once
   10event_list = self._selector.select(timeout)
   11AttributeError: 'NoneType' object has no attribute 'select'
   

   I can reproduce this intermittently via while ./bld-cmake/test/functional/test_runner.py $(for i in {1..400}; do echo -n "tool_rpcauth "; done) -j 400 ; do true ; done.

   I suspect this is a race where the shutdown starts the close of the network thread while it is starting.

   A different exception showing this race can be reproduced via:

    0diff --git a/test/functional/test_framework/p2p.py b/test/functional/test_framework/p2p.py
    1index 610aa4ccca..64561e157c 100755
    2--- a/test/functional/test_framework/p2p.py
    3+++ b/test/functional/test_framework/p2p.py
    4@@ -741,6 +741,7 @@ class NetworkThread(threading.Thread):
    5 
    6     def run(self):
    7         """Start the network thread."""
    8+        import time;time.sleep(.1)
    9         self.network_event_loop.run_forever()
   10 
   11     def close(self, *, timeout=10):
   

   It is trivial to reproduce via any test (e.g. ./bld-cmake/test/functional/tool_rpcauth.py) and shows a similar traceback to the one above:

    0Exception in thread NetworkThread:
    1Traceback (most recent call last):
    2  File "/python3.10/threading.py", line 1016, in _bootstrap_inner
    3    self.run()
    4  File "./test/functional/test_framework/p2p.py", line 745, in run
    5    self.network_event_loop.run_forever()
    6  File "/python3.10/asyncio/base_events.py", line 591, in run_forever
    7    self._check_closed()
    8  File "/python3.10/asyncio/base_events.py", line 515, in _check_closed
    9    raise RuntimeError('Event loop is closed')
   10RuntimeError: Event loop is closed
   

   So fix the second runtime error in hope of fixing the first one as well.

2. DrahtBot added the label Tests on Aug 6, 2025
3. 
   DrahtBot commented at 7:25 am on August 6, 2025: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage & Benchmarks

   For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33140.

   Reviews

   See the guideline for information on the review process.

   Type	Reviewers
   ACK	brunoerg

   If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

4. maflcko force-pushed on Sep 4, 2025
5. test: Avoid shutdown race in NetworkThread fa6db79302
6. maflcko force-pushed on Sep 29, 2025
7. brunoerg approved
8. 
   brunoerg commented at 2:55 pm on October 28, 2025: contributor

   code review ACK fa6db79302d28e6b31b07b24580430614ffcd0e8

   I could not reproduce the issue intermittently via the provided script, but I could see the race when adding a sleep before calling run_forever(). Anyway, regardless of it, waiting until the thread is actually running is a best practice.

9. in test/functional/test_framework/test_framework.py:346 in fa6db79302

   344@@ -345,6 +345,7 @@ def setup(self):
   345         self.log.debug('Setting up network thread')
   346         self.network_thread = NetworkThread()
   

   ---

   ---

   

   brunoerg commented at 2:59 pm on October 28, 2025:

   Not related but why we don’t call set_event_loop when running it?

    0class NetworkThread(threading.Thread):
    1    network_event_loop = None
    2
    3    def __init__(self):
    4        super().__init__(name="NetworkThread")
    5        # There is only one event loop and no more than one thread must be created
    6        assert not self.network_event_loop
    7
    8        NetworkThread.listeners = {}
    9        NetworkThread.protos = {}
   10        if platform.system() == 'Windows':
   11            asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
   12        NetworkThread.network_event_loop = asyncio.new_event_loop()
   13
   14    def run(self):
   15        """Start the network thread."""
   16        self.network_event_loop.run_forever()
   

10. fanquake merged this on Dec 3, 2025
11. fanquake closed this on Dec 3, 2025

    ---

Contributors
maflcko DrahtBot brunoerg

Labels
Tests