test: Turn util/test_runner into functional test #32697

1. 
   maflcko commented at 8:55 am on June 7, 2025: member

   The test/util/test_runner.py has many issues:

   • The boilerplate for the test runner is duplicate or inconsistent with the other (functional) tests. For example, logging options, ConfigParser handling, Binaries handling …
   • The cmake/ci behavior is brittle and can silently fail, as explained in #31476
   • corecheck (and likely other places that manually run the tests) completely forget to run it
   • If the test is manually called, it runs single threaded, when it could just run in parallel with the other functional tests

   Fix all issues by removing the util test_runner and moving the test logic into a new functional test file.

2. test: Remove duplicate ConfigParser

   It is sufficient to parse once.

   fad27aadfe
3. test: Use lowercase env var as attribute name

   This is a refactor.

   fafc18127a
4. 
   DrahtBot commented at 8:55 am on June 7, 2025: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage & Benchmarks

   For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32697.

   Reviews

   See the guideline for information on the review process.

   Type	Reviewers
   Concept ACK	Prabhat1308, hebasto, BrandonOdiwuor, brunoerg

   If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #32324 (test: Treat executable paths in tests consistently by hebasto)
   • #32297 (bitcoin-cli: Add -ipcconnect option by ryanofsky)
   • #31669 (cmake: Introduce WITH_PYTHON build option by hebasto)
   • #30437 (ipc: add bitcoin-mine test program by ryanofsky)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

5. DrahtBot renamed this:
   test: Turn util/test_runner into functional test
   test: Turn util/test_runner into functional test
   on Jun 7, 2025
6. DrahtBot added the label Tests on Jun 7, 2025
7. maflcko force-pushed on Jun 7, 2025
8. DrahtBot added the label CI failed on Jun 7, 2025
9. 
   DrahtBot commented at 9:03 am on June 7, 2025: contributor

   🚧 At least one of the CI tasks failed. Task lint: https://github.com/bitcoin/bitcoin/runs/43661503873 LLM reason (✨ experimental): Python linting failed due to an unused import error identified by ruff.

   Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

   • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

   • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

   • An intermittent issue.

   Leave a comment here, if you need help tracking down a confusing failure.

10. maflcko force-pushed on Jun 7, 2025
11. maflcko force-pushed on Jun 7, 2025
12. 
    maflcko commented at 9:57 am on June 7, 2025: member

    force pushed to fix a python type related error after an error. Stuff like this wouldn’t happen in a type-safe language.

    0                                       logging.error("OSError, Failed to execute " + execprog)
    1                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
    2                                   TypeError: can only concatenate str (not "list") to str
    

13. DrahtBot removed the label CI failed on Jun 7, 2025
14. test: Add missing tx util to Binaries fac9c4dafe
15. test: Turn util/test_runner into functional test

    The moved portion can be reviewed via:
    
    --color-moved=dimmed-zebra  --color-moved-ws=ignore-all-space

    fae1c12296
16. maflcko force-pushed on Jun 7, 2025
17. 
    Prabhat1308 commented at 10:07 am on June 8, 2025: contributor

    fae1c12 tACK
18. 
    hebasto commented at 8:05 am on June 9, 2025: member

    Concept ACK.
19. 
    BrandonOdiwuor commented at 6:31 pm on June 9, 2025: contributor

    Concept ACK
20. 
    brunoerg commented at 4:26 pm on June 11, 2025: contributor

    Concept ACK

Contributors
maflcko DrahtBot Prabhat1308 hebasto BrandonOdiwuor brunoerg

Labels
Tests