ci: Run unit tests parallel with functional tests #33000

1. 
   maflcko commented at 10:36 am on July 17, 2025: member

   Fixes #32770
2. maflcko force-pushed on Jul 17, 2025
3. maflcko force-pushed on Jul 17, 2025
4. ci: Run unit test parallel with functional tests fab25c6f58
5. maflcko force-pushed on Jul 17, 2025
6. 
   DrahtBot commented at 2:17 pm on July 17, 2025: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage & Benchmarks

   For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33000.

   Reviews

   See the guideline for information on the review process.

   Type	Reviewers
   ACK	jonatack
   Concept ACK	l0rinc, fanquake

   If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #33136 (ci: Remove redundant RUN_UNIT_TESTS_SEQUENTIAL by maflcko)
   • #32989 (ci: Migrate CI to hosted Cirrus Runners by willcl-ark)
   • #31679 (cmake: Install internal binaries to /libexec/ by ryanofsky)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

7. maflcko added the label Tests on Jul 17, 2025
8. 
   maflcko commented at 3:45 pm on July 17, 2025: member

   Looks like this gives a 20% faster Msan, Tsan, and previous releases task with N=1, comparing with the fastest master run in the last week:

   • https://cirrus-ci.com/build/4989581210157056
   • https://cirrus-ci.com/build/5602428818554880
9. 
   jonatack commented at 5:16 pm on July 25, 2025: member

   I don’t know how much variance we typically see in CI run times, but light lgtm ACK fab25c6f5803ae8699f068d089f1c6e4f1387743

   0$ /test_runner.py -h
   1  --valgrind            run nodes under the valgrind memory error detector: expect at least a ~10x slowdown. valgrind 3.14 or later required. Does not apply to previous release
   2                        binaries.
   3  --bash-cmd-extra-tests BASH_CMD_EXTRA_TESTS
   4                        Run an arbitrary string as Bash command in parallel to the functional tests, for example to run unit tests. See tool_extra_cmd.py
   5  --randomseed RANDOMSEED
   6                        set a random seed for deterministically reproducing a previous test run
   

   0$ ./build/test/functional/tool_extra_cmd.py
   12025-07-25T17:15:46.612122Z TestFramework (INFO): PRNG seed is: 5551363100202563085
   22025-07-25T17:15:46.612554Z TestFramework (INFO): Initializing test directory /var/folders/bz/mn3hr6td37bczwp7j89frs4w0000gn/T/bitcoin_func_test_c89awace
   32025-07-25T17:15:46.612626Z TestFramework (INFO): No command to run.
   42025-07-25T17:15:46.667850Z TestFramework (INFO): Stopping nodes
   52025-07-25T17:15:46.667957Z TestFramework (INFO): Cleaning up /var/folders/bz/mn3hr6td37bczwp7j89frs4w0000gn/T/bitcoin_func_test_c89awace on exit
   62025-07-25T17:15:46.667990Z TestFramework (INFO): Tests successful
   

10. 
    l0rinc commented at 0:44 am on July 26, 2025: contributor

    Concept ACK
11. 
    fanquake commented at 9:30 am on July 26, 2025: member

    Concept ACK. Not sure about the approach of more Bash, to call new Python, that wraps more Bash.
12. maflcko marked this as a draft on Jul 28, 2025
13. 
    maflcko commented at 7:36 am on July 28, 2025: member

    The implementation can be adjusted, if needed. I guess the main questions are:

    • Do people want to use this locally, if yes, then there could be a more user friendly way of calling it?
    • Does it help with the new CI? With the old CI system planned to be replaced, it makes little sense to fine-tune it this late, so I’ll turn this into a draft for now.
14. 
    fanquake commented at 9:00 am on July 28, 2025: member

    Does it help with the new CI? With the old CI system planned to be replaced, it makes little sense to fine-tune it this late, so I’ll turn this into a draft for now.

    cc @m3dwards @willcl-ark

15. 
    maflcko commented at 9:07 am on July 28, 2025: member

    I’d say it should be done after the move. No need to delay the move or block it on this.
16. maflcko renamed this:
    ci: Run unit test parallel with functional tests
    ci: Run unit tests parallel with functional tests
    on Aug 5, 2025
17. 
    maflcko commented at 8:34 am on August 6, 2025: member

    Concept ACK. Not sure about the approach of more Bash, to call new Python, that wraps more Bash.

    Yeah, it is ugly. I think I’ll drop some of the (actually redundant) Bash in #33136 first. Then, this can be done in pure Python, possibly.

18. DrahtBot added the label Needs rebase on Aug 7, 2025
19. 
    DrahtBot commented at 3:32 pm on August 7, 2025: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

Contributors
maflcko DrahtBot jonatack l0rinc fanquake

Labels
Tests Needs rebase