qa: `tool_bench_sanity_check.py --bench=MempoolCheck` failure

hebasto commented at 1:16 pm on January 10, 2026: member

On NetBSD 10.1, https://github.com/hebasto/bitcoin-core-nightly/actions/runs/20872235098/job/59975888965:

 090/455 - tool_bench_sanity_check.py --bench=MempoolCheck failed, Duration: 26 s
 1
 2stdout:
 32026-01-10T04:19:37.691064Z TestFramework (INFO): PRNG seed is: 4615966168349338210
 42026-01-10T04:19:37.692250Z TestFramework (INFO): Initializing test directory /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/test_runner_₿_🏃_20260110_041921/tool_bench_sanity_check_363
 52026-01-10T04:19:37.692513Z TestFramework (INFO): Starting: /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/build/bin/bench_bitcoin -filter=MempoolCheck -sanity-check
 6Running with -sanity-check option, output is being suppressed as benchmark results will be useless.
 72026-01-10T04:19:38.567933Z TestFramework (INFO): Success!
 82026-01-10T04:20:03.637759Z TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
 9        wait_until_helper_internal(lambda: not self.network_event_loop.is_running(), timeout=timeout)
10'''
11
12
13stderr:
14Traceback (most recent call last):
15  File "/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/build/test/functional/tool_bench_sanity_check.py", line 41, in <module>
16    BenchSanityCheck(__file__).main()
17    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
18  File "/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/test_framework/test_framework.py", line 154, in main
19    exit_code = self.shutdown()
20  File "/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/test_framework/test_framework.py", line 281, in shutdown
21    self.network_thread.close()
22    ~~~~~~~~~~~~~~~~~~~~~~~~~^^
23  File "/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/test_framework/p2p.py", line 749, in close
24    wait_until_helper_internal(lambda: not self.network_event_loop.is_running(), timeout=timeout)
25    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
26  File "/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/test_framework/util.py", line 420, in wait_until_helper_internal
27    raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout))
28AssertionError: Predicate ''''
29        wait_until_helper_internal(lambda: not self.network_event_loop.is_running(), timeout=timeout)
30''' not true after 10.0 seconds

hebasto added the label Tests on Jan 10, 2026

maflcko commented at 3:29 pm on January 10, 2026: member

Not sure how this could happen. After https://github.com/bitcoin/bitcoin/commit/39ca01525977e5aab078198874f6bf035b552b3b , the test waits for the loop to be running, so a stop should always be called on a running loop and result in it eventually not running.

vasild commented at 3:03 pm on January 21, 2026: contributor

@maflcko, I think yes, after https://github.com/bitcoin/bitcoin/commit/39ca01525977e5aab078198874f6bf035b552b3b a stop() will always be called on a running loop. I think what is happening here is:

0self.network_event_loop.call_soon_threadsafe(self.network_event_loop.stop)
1wait_until_helper_internal(lambda: not self.network_event_loop.is_running(), timeout=timeout)

the second line times out, “not true after 10.0 seconds”. So it just did not stop in 10 seconds? In other words, that “soon” in call_soon_threadsafe() is not soon enough?

maflcko commented at 3:23 pm on January 21, 2026: member

Heh, the simplest cause could be the correct one. A 10 second delay due to resource contention seems a bit high, but this is running a VM (bsd) inside another VM (GHA), so it seems plausible.

I guess scaling here with the timeout factor may be correct anyway. Done in #34369

If this happens the next time in the nightly repo, the timeout should be 80 seconds here. If that happens, we can start looking again.

hebasto closed this on Jan 21, 2026

hebasto referenced this in commit 3ea2b6fe18 on Jan 21, 2026

qa: tool_bench_sanity_check.py --bench=MempoolCheck failure #34248

qa: `tool_bench_sanity_check.py --bench=MempoolCheck` failure #34248