test: Avoid shutdown race in NetworkThread #33140

Locally, I am seeing rare intermittent exceptions in the network thread:

stderr:
Exception in thread NetworkThread:
Traceback (most recent call last):
File "/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "./test/functional/test_framework/p2p.py", line 744, in run
self.network_event_loop.run_forever()
File "/python3.10/asyncio/base_events.py", line 603, in run_forever
self._run_once()
File "/python3.10/asyncio/base_events.py", line 1871, in _run_once
event_list = self._selector.select(timeout)
AttributeError: 'NoneType' object has no attribute 'select'

I can reproduce this intermittently via while ./bld-cmake/test/functional/test_runner.py $(for i in {1..400}; do echo -n "tool_rpcauth "; done) -j 400 ; do true ; done.

I suspect this is a race where the shutdown starts the close of the network thread while it is starting.

A different exception showing this race can be reproduced via:

diff --git a/test/functional/test_framework/p2p.py b/test/functional/test_framework/p2p.py
index 610aa4ccca..64561e157c 100755
--- a/test/functional/test_framework/p2p.py
+++ b/test/functional/test_framework/p2p.py
@@ -741,6 +741,7 @@ class NetworkThread(threading.Thread):
 
     def run(self):
         """Start the network thread."""
+        import time;time.sleep(.1)
         self.network_event_loop.run_forever()
 
     def close(self, *, timeout=10):

It is trivial to reproduce via any test (e.g. ./bld-cmake/test/functional/tool_rpcauth.py) and shows a similar traceback to the one above:

Exception in thread NetworkThread:
Traceback (most recent call last):
  File "/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "./test/functional/test_framework/p2p.py", line 745, in run
    self.network_event_loop.run_forever()
  File "/python3.10/asyncio/base_events.py", line 591, in run_forever
    self._check_closed()
  File "/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

So fix the second runtime error in hope of fixing the first one as well.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33140.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process.

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

code review ACK fa6db79302d28e6b31b07b24580430614ffcd0e8

I could not reproduce the issue intermittently via the provided script, but I could see the race when adding a sleep before calling run_forever(). Anyway, regardless of it, waiting until the thread is actually running is a best practice.