test: add logging to mock external signers #32928

1. 
   Sjors commented at 2:44 pm on July 9, 2025: member

   This will hopefully aid in debugging #32855.

   Because stdout and stderr are consumed by the node, the mock external signers can’t use them for logging.

   Instead have them print directly into test_framework.log, which can then be be retrieved via combine_logs.py.

   In preparation (and addtion) this PR starts with a few refactors:

   • move mock_signer_path helpers to util and reuse them between rpc_signer.py and wallet_signer.py
   • move the mocks from test/functional/mocks to test/functional: this is needed to avoid Python import hell in the next commits
   • have the mocks take advantage of the test_framework by extracting (and reusing) perform_pre_checks and mock_signer_psbt_path into util
   • extract the debug.log style Formatter into util so we can use it in the final commit
2. test: move mock signer path helpers to util b38e0578b7
3. DrahtBot added the label Tests on Jul 9, 2025
4. 
   DrahtBot commented at 2:44 pm on July 9, 2025: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage & Benchmarks

   For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32928.

   Reviews

   See the guideline for information on the review process.

   Type	Reviewers
   Concept ACK	naiyoma

   If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #33230 (cli: Handle arguments that can be either JSON or string by achow101)
   • #33142 (test: Run bench sanity checks in parallel with functional tests by maflcko)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

5. test: move mock external signers to parent dir d619f7d580
6. test: have signer mocks use test framework

   - reuse perform_pre_checks helper
   - add mock_psbt_path helper

   b65ea68aaf
7. test: move log formatter to util 58ac12331a
8. test: add logging to mock signers

   Because stdout and stderr are consumed by the node, we can't use them for logging. Instead have the mocks print directly into test_framework.log, which can then be be retrieved via combine_logs.py.

   3ff8e6643e
9. Sjors force-pushed on Jul 9, 2025
10. DrahtBot added the label CI failed on Jul 9, 2025
11. Sjors renamed this:
    test: add logging to mock signers
    test: add logging to mock external signers
    on Jul 9, 2025
12. DrahtBot removed the label CI failed on Jul 9, 2025
13. 
    Sammie05 commented at 3:30 am on July 13, 2025: none

    Adding mock_signer_log and logging “Started” / “Finished” makes debugging much clearer and great improvement, especially since stdout/stderr can’t be used here. Thanks for cleaning this up and making it easier for future test debugging
14. 
    naiyoma commented at 8:04 pm on July 19, 2025: contributor

    Concept ACK -> I attempted to debug -> #32855 (comment) after pulling this branch, and so far I can see some useful logs in combined.log

    grep “mock_external_signer” combined.log node1 2025-07-19T19:35:11.455260Z [init] [common/args.cpp:850] [logArgsPrefix] Command-line arg: signer="/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py" test 2025-07-19T19:35:11.823136Z TestFramework (DEBUG): -signer=/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py node1 2025-07-19T19:35:11.846121Z [httpworker.1] [common/run_command.cpp:29] [RunCommandParseJSON] RunCommandParseJSON: command=’/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py enumerate’, stdin_length=0, stdin_empty=true test 2025-07-19T19:35:11.963244Z TestFramework.mock_external_signer (DEBUG): Started test 2025-07-19T19:35:11.969415Z TestFramework.mock_external_signer (DEBUG): Finished node1 2025-07-19T19:35:11.986061Z [httpworker.1] [common/run_command.cpp:29] [RunCommandParseJSON] RunCommandParseJSON: command=’/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py –fingerprint 00000001 –chain regtest getdescriptors –account 0’, stdin_length=0, stdin_empty=true test 2025-07-19T19:35:12.078454Z TestFramework.mock_external_signer (DEBUG): Started test 2025-07-19T19:35:12.084684Z TestFramework.mock_external_signer (DEBUG): Naiyoma HANGING: stdin hang for getdescriptors

Contributors
Sjors DrahtBot Sammie05 naiyoma

Labels
Tests