test: add logging to mock external signers #32928

1. 
   Sjors commented at 2:44 pm on July 9, 2025: member

   Because stdout and stderr are consumed by the node, the mock external signers can’t use them for logging.

   Instead have them print directly into test_framework.log, which can then be be retrieved via combine_logs.py.

   In preparation (and addtion) this PR starts with a few refactors:

   • move mock_signer_path helpers to util and reuse them between rpc_signer.py and wallet_signer.py
   • move the mocks from test/functional/mocks to test/functional: this is needed to avoid Python import hell in the next commits
   • have the mocks take advantage of the test_framework by extracting (and reusing) perform_pre_checks and mock_signer_psbt_path into util
   • extract the debug.log style Formatter into util so we can use it in the final commit

   Originally this PR was intended to aid in debugging #32855, which has been solved since.

2. DrahtBot added the label Tests on Jul 9, 2025
3. 
   DrahtBot commented at 2:44 pm on July 9, 2025: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage & Benchmarks

   For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32928.

   Reviews

   See the guideline for information on the review process.

   Type	Reviewers
   Concept ACK	naiyoma

   If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #33702 (contrib: Remove brittle, confusing and redundant UTF8 encoding from Python IO by maflcko)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

4. Sjors force-pushed on Jul 9, 2025
5. DrahtBot added the label CI failed on Jul 9, 2025
6. Sjors renamed this:
   test: add logging to mock signers
   test: add logging to mock external signers
   on Jul 9, 2025
7. DrahtBot removed the label CI failed on Jul 9, 2025
8. 
   Sammie05 commented at 3:30 am on July 13, 2025: none

   Adding mock_signer_log and logging “Started” / “Finished” makes debugging much clearer and great improvement, especially since stdout/stderr can’t be used here. Thanks for cleaning this up and making it easier for future test debugging
9. 
   naiyoma commented at 8:04 pm on July 19, 2025: contributor

   Concept ACK -> I attempted to debug -> #32855 (comment) after pulling this branch, and so far I can see some useful logs in combined.log

   grep “mock_external_signer” combined.log node1 2025-07-19T19:35:11.455260Z [init] [common/args.cpp:850] [logArgsPrefix] Command-line arg: signer="/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py" test 2025-07-19T19:35:11.823136Z TestFramework (DEBUG): -signer=/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py node1 2025-07-19T19:35:11.846121Z [httpworker.1] [common/run_command.cpp:29] [RunCommandParseJSON] RunCommandParseJSON: command=’/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py enumerate’, stdin_length=0, stdin_empty=true test 2025-07-19T19:35:11.963244Z TestFramework.mock_external_signer (DEBUG): Started test 2025-07-19T19:35:11.969415Z TestFramework.mock_external_signer (DEBUG): Finished node1 2025-07-19T19:35:11.986061Z [httpworker.1] [common/run_command.cpp:29] [RunCommandParseJSON] RunCommandParseJSON: command=’/usr/bin/python3 /home/ubuntu/Projects/bitcoin/test/functional/test_framework/../mock_external_signer.py –fingerprint 00000001 –chain regtest getdescriptors –account 0’, stdin_length=0, stdin_empty=true test 2025-07-19T19:35:12.078454Z TestFramework.mock_external_signer (DEBUG): Started test 2025-07-19T19:35:12.084684Z TestFramework.mock_external_signer (DEBUG): Naiyoma HANGING: stdin hang for getdescriptors

10. Sjors force-pushed on Sep 1, 2025
11. 
    maflcko commented at 7:30 am on September 1, 2025: member

    This will hopefully aid in debugging #32855.

    For reference, the issue was something else, and has been fixed now, in the meantime.

12. 
    Sjors commented at 8:04 am on September 1, 2025: member

    @maflcko updated the PR description. I assume more logging is still useful though.
13. DrahtBot added the label Needs rebase on Sep 25, 2025
14. Sjors force-pushed on Sep 26, 2025
15. DrahtBot removed the label Needs rebase on Sep 26, 2025
16. DrahtBot added the label Needs rebase on Oct 21, 2025
17. test: move mock signer path helpers to util 95b67a0683
18. test: move mock external signers to parent dir 4da75192b1
19. test: have signer mocks use test framework

    - reuse perform_pre_checks helper
    - add mock_psbt_path helper

    aee218920d
20. test: move log formatter to util 3ae3533fe3
21. test: add logging to mock signers

    Because stdout and stderr are consumed by the node, we can't use them for logging. Instead have the mocks print directly into test_framework.log, which can then be be retrieved via combine_logs.py.

    be09b4393d
22. Sjors force-pushed on Oct 24, 2025
23. DrahtBot removed the label Needs rebase on Oct 24, 2025

Contributors
Sjors DrahtBot Sammie05 naiyoma maflcko

Labels
Tests