(This picks up my prior attempt from #13384)
Currently, the test_runner is using a time.sleep before polling to check if any tests have completed. This is largely fine when running a few tests, or when the tests take a long time.
However, when running many fast tests, this can accumulate and leave the CPU idle for no reason.
A trivial improvement would be to only sleep when really needed:
0diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py
1index 7c8c15f391..1d9f28cee4 100755
2--- a/test/functional/test_runner.py
3+++ b/test/functional/test_runner.py
4@@ -747,7 +747,6 @@ class TestHandler:
5 dot_count = 0
6 while True:
7 # Return all procs that have finished, if any. Otherwise sleep until there is one.
8- time.sleep(.5)
9 ret = []
10 for job in self.jobs:
11 (name, start_time, proc, testdir, log_out, log_err) = job
12@@ -771,6 +770,7 @@ class TestHandler:
13 ret.append((TestResult(name, status, int(time.time() - start_time)), testdir, stdout, stderr, skip_reason))
14 if ret:
15 return ret
16+ time.sleep(.5)
17 if self.use_term_control:
18 print('.', end='', flush=True)
19 dot_count += 1
However, ideally there is no sleep at all. So do that by using a ThreadPoolExecutor.
This can be tested via something like:
0time ./bld-cmake/test/functional/test_runner.py $(for i in {1..200}; do echo -n "tool_rpcauth "; done) -j 200
The result should show:
- Current
masteris the slowest - The “sleep patch” from above is a bit faster (1.5x improvement)
- This pull request is the fastest (2x improvement)
