test, refactor: Add TestNode.binaries to hold binary paths #31866

pull ryanofsky wants to merge 2 commits into bitcoin:master from ryanofsky:pr/wrapp changing 6 files +82 −38
  1. ryanofsky commented at 3:45 pm on February 14, 2025: contributor

    Add new TestNode.binaries object to manage paths to bitcoin binaries.

    The binaries object makes it possible for the test framework to exercise the bitcoin wrapper executable introduced in #31375 and also makes it easier in general to add new binaries, and new options and environment variables controlling how they are invoked, because logic for invoking them that was previously spread out is now consolidated in one place.

    These changes were originally part of #31375 but made that PR harder to review because they were unrelated to the other changes there. If this PR can get merged first, python changes in #31375 will be simple, and the test framework changes here should also get a higher quality review.

  2. test, refactor: Add TestNode.binaries to hold binary paths
    Add new TestNode.binaries object to manage paths to bitcoin binaries.
    
    Having this object makes it possible for the test framework to exercise the
    bitcoin wrapper executable introduced in
    https://github.com/bitcoin/bitcoin/pull/31375 and also makes it easier to add
    new binaries and options and environment variables controlling how they are
    invoked, because logic for invoking them that was previously spread out is now
    consolidated in one place.
    
    Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
    e60ff180c2
  3. test, contrib: Fix signer/miner command line escaping
    Pass bitcoin binary command lines from test framework to signet/miner utility
    using shell escaping so they are unambigous and don't get mangled if they
    contain spaces.
    
    This change is not needed for tests to pass currently, but is a useful change
    to avoid CI failures in followup PR
    https://github.com/bitcoin/bitcoin/pull/31375 and to avoid other bugs.
    8f5228d923
  4. DrahtBot commented at 3:45 pm on February 14, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31866.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK maflcko
    Concept ACK Sjors

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31723 (qa debug: Add –debug_runs/-waitfordebugger [DRAFT] by hodlinator)
    • #31375 (multiprocess: Add bitcoin wrapper executable by ryanofsky)
    • #31250 (wallet: Disable creating and loading legacy wallets by achow101)
    • #29838 (Feature: Use different datadirs for different signets by BrandonOdiwuor)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  5. in test/functional/test_framework/test_framework.py:79 in e58f5e0e61 outdated
    74+
    75+    def daemon_argv(self):
    76+        "Return argv array that should be used to invoke bitcoind"
    77+        return self._argv("bitcoind")
    78+
    79+    def rpc_argv(self):
    


    maflcko commented at 9:02 am on February 18, 2025:
    nit in e58f5e0e6180e0f0f39d4fb52d1ed822cbde1078: Could name this cli_argv to keep inline with the name of the exe and to avoid confusion with the existing TestNode::rpc and TestNode::cli naming?

    ryanofsky commented at 7:00 pm on February 18, 2025:

    re: #31866 (review)

    nit in e58f5e0: Could name this cli_argv to keep inline with the name of the exe and to avoid confusion with the existing TestNode::rpc and TestNode::cli naming?

    Followup commit 60b89ab4897e175c8d73a5dd21cd52a0c0c790ef from #31375 might provide context because these names just match names of bitcoin subcommands.

    To answer this more directly, all of this classes’s methods return command lines, so calling the one that returns a command line for call an rpc “cli” instead of “rpc” seems less descriptive, not more descriptive.

    If these were TestNode methods, I would agree with your point about the rpc_argv() method looking too much like TestNode.rpc. But the Binaries class is a separate class that doesn’t reference the TestNode object and can be used independently.

    If there are places you see this method being called that are confusing, it would be good to address, but I wouldn’t want to call this one method “cli” when all the methods here are returning command lines.

  6. in test/functional/test_framework/test_framework.py:21 in e58f5e0e61 outdated
    17@@ -18,6 +18,7 @@
    18 import sys
    19 import tempfile
    20 import time
    21+import types
    


    maflcko commented at 9:43 am on February 18, 2025:
    nit in https://github.com/bitcoin/bitcoin/commit/e58f5e0e6180e0f0f39d4fb52d1ed822cbde1078: Is there a reason to use types, when a simple dict seems sufficient to store key-value pairs?

    ryanofsky commented at 6:49 pm on February 18, 2025:

    re: #31866 (review)

    nit in e58f5e0: Is there a reason to use types, when a simple dict seems sufficient to store key-value pairs?

    Imo, plain objects are simpler than dicts, more readable (o.prop vs d["prop"]) and more compatible with useful features like static type checking.

    Right now the paths object is just a collection of paths to executables, but in followup 60b89ab4897e175c8d73a5dd21cd52a0c0c790ef from #31375 it gets a new bitcoin_cmd member that can be set to a more generic command line, and in the future if we add other environment variables or configuration values that influence how binaries are located, it could contain other information, which a dict would only make more awkward to access.

    However, if you are not a fan of types.SimpleNamespacefor some reason, I could drop it and declare a custom class BinaryPaths. Would just be a little more verbose.

  7. in test/functional/wallet_encryption.py:118 in e58f5e0e61 outdated
    114@@ -112,7 +115,7 @@ def run_test(self):
    115 
    116             def do_wallet_tool(*args):
    117                 proc = subprocess.Popen(
    118-                    [self.options.bitcoinwallet, f"-datadir={self.nodes[0].datadir_path}", f"-chain={self.chain}"] + list(args),
    119+                    Binaries(self.paths).wallet_argv() + [f"-datadir={self.nodes[0].datadir_path}", f"-chain={self.chain}"] + list(args),
    


    maflcko commented at 10:04 am on February 18, 2025:

    nit in https://github.com/bitcoin/bitcoin/commit/e58f5e0e6180e0f0f39d4fb52d1ed822cbde1078: I’d say it would be clearer to not have self.paths at all, and also remove the need to somehow import and handle Binaries in individual tests.

    It would be better to fully move all touched code of this pull into TestNode and only allow getting binaries through the TestNode interface. The reasons are:

    • All functional tests revolve around nodes (and their datadirs) and it is hard to imagine a test that calls the wallet util (or another executable) without also depending on a node. Even if there were a few of such tests, I still think it would be clearer to access the util via the TestNode like self.nodes[0].wallet_argv().
    • In the context of previous releases, it is confusing to have Binaries(self.paths) for the “global master” binaries/paths and a second set of binaries in TestNode. It would be clearer to just have the TestNode ones, so that previous_node.wallet_argv() calls a previous release binary and current_node.wallet_argv() calls the current release binary.

    ryanofsky commented at 8:04 pm on February 18, 2025:

    re: #31866 (review)

    nit in e58f5e0: I’d say it would be clearer to not have self.paths at all, and also remove the need to somehow import and handle Binaries in individual tests.

    That’s a good idea. Added a TestFramework.get_binaries() method to make this more convenient. There are only 3 tests that use it, but it is also convenient to use internally.

    I am not sure why you would want to get rid of the TestFramework.paths variable entirely. As an alternative I guess we could keep the existing TestFramework.bitcoind TestFramework.bitcoincli TestFramework.bitcoinutil and TestFramework.bitcoinwallet variables and add even more variables like these in the future. But I thought would be a clear improvement to move these variables into a TestFramework.paths object so TestFramework class would be more comprehensible and so interaction between the TestFramework class and the Binaries class would not require passing individual options around or adding a circular dependency.

    It would be better to fully move all touched code of this pull into TestNode and only allow getting binaries through the TestNode interface.

    This doesn’t make much sense to me. I wouldn’t want tests to be using self.nodes[0].binaries if they aren’t actually interacting with node 0, and when a node 0 object might not even exist. If tests just need to call some binaries, they should be able to use a standalone binaries object without creating a fake node or going through an unrelated node.

    I also don’t understand what you are saying about “global master” binaries in the previous releases code. The latest version should be a little simpler now with get_binaries method and I’m open to any suggestions to simplify it more.


    maflcko commented at 11:46 am on February 19, 2025:

    It would be better to fully move all touched code of this pull into TestNode and only allow getting binaries through the TestNode interface.

    This doesn’t make much sense to me. I wouldn’t want tests to be using self.nodes[0].binaries if they aren’t actually interacting with node 0, and when a node 0 object might not even exist. If tests just need to call some binaries, they should be able to use a standalone binaries object without creating a fake node or going through an unrelated node.

    I understand your point, but the whole test framework is built around the concept of “nodes”, where the version of the nodes is given in a versions array. Normally, it is not supported to use bitcoincli of one version with bitcoind of another version. (The wallet tool may be an exception, because it doesn’t speak with another binary, but only interacts with the datadir.) So normally, I expect binaries interacting only with binaries of their own version. So I think it would be simpler to mirror this expectation by default. When a test violates this expectation, it could be worked around for the specific test only, and it would be obvious.

    I see your point about not wanting to use self.nodes[0].binaries, so an alternative might be to remove TestFramework::binary_paths and instead add an array of the same size as the versions array to store the result of a call to [Binaries(self.get_binary_paths(), bin_dir_from_version(v)) for v in versions].

    With that suggestion, I expect it would be easier to call the right binary (if several versions are used in the same test). But no strong opinion.


    ryanofsky commented at 2:39 pm on February 19, 2025:

    re: #31866 (review)

    I see your point about not wanting to use self.nodes[0].binaries, so an alternative might be to remove TestFramework::binary_paths and instead add an array of the same size as the versions array to store the result of a call to [Binaries(self.get_binary_paths(), bin_dir_from_version(v)) for v in versions].

    I think I don’t fully understand the suggestion and the problem this is trying to solve. If the goal is to drop the TestFramework.binary_paths variable, that could be done pretty easily with:

     0--- a/test/functional/test_framework/test_framework.py
     1+++ b/test/functional/test_framework/test_framework.py
     2@@ -263,7 +263,6 @@ class BitcoinTestFramework(metaclass=BitcoinTestMetaClass):
     3         config = configparser.ConfigParser()
     4         config.read_file(open(self.options.configfile))
     5         self.config = config
     6-        self.binary_paths = self.get_binary_paths()
     7         if self.options.v1transport:
     8             self.options.v2transport=False
     9 
    10@@ -307,7 +306,7 @@ class BitcoinTestFramework(metaclass=BitcoinTestMetaClass):
    11         return paths
    12 
    13     def get_binaries(self, bin_dir=None):
    14-        return Binaries(self.binary_paths, bin_dir)
    15+        return Binaries(self.get_binary_paths(), bin_dir)
    16 
    17     def setup(self):
    18         """Call this method to start up the test framework object with options set."""
    

    Or if the problem is that having a BitcoinTestFramework.get_binaries() method is bad because developers might call this method to access binaries in cases where it would be better to use the TestNode.binaries object instead, then maybe better documentation might be helpful:

     0--- a/test/functional/test_framework/test_framework.py
     1+++ b/test/functional/test_framework/test_framework.py
     2@@ -307,6 +307,10 @@ class BitcoinTestFramework(metaclass=BitcoinTestMetaClass):
     3         return paths
     4 
     5     def get_binaries(self, bin_dir=None):
     6+        """Return new Binaries object that can be used to call binaries.
     7+
     8+        Typically tests will not need to call this method themselves and can use
     9+        the TestNode.binaries objects assigned to each test node."""
    10         return Binaries(self.binary_paths, bin_dir)
    11 
    12     def setup(self):
    

    I’m sure other changes might be good too, you can let me know what you think might be useful, or if there is another problem with the current approach that I’m not seeing.


    maflcko commented at 4:17 pm on February 19, 2025:
    Just a nit, anything is fine here. Further clean-ups can trivially done as a follow-up, if there is need to.
  8. maflcko commented at 10:04 am on February 18, 2025: member
    left a question
  9. ryanofsky force-pushed on Feb 18, 2025
  10. ryanofsky commented at 8:36 pm on February 18, 2025: contributor

    Thanks for the review! Updated with some simplifications and cleanups based on your suggestions.

    Updated bfadc7d241a6288e70fb0c58b6ad5fa5a305af51 -> 11615b1025a09e62f12224ae55c92fe85c42b24c (pr/wrapp.1 -> pr/wrapp.2, compare) adding get_binaries method, renaming some variables, and avoiding getattr.

  11. ryanofsky force-pushed on Feb 18, 2025
  12. DrahtBot added the label CI failed on Feb 18, 2025
  13. DrahtBot commented at 8:39 pm on February 18, 2025: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/37426971220

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  14. ryanofsky commented at 8:39 pm on February 18, 2025: contributor
    Updated 11615b1025a09e62f12224ae55c92fe85c42b24c -> 8f5228d9239464ece06f3a56372aeec29685801c (pr/wrapp.2 -> pr/wrapp.3, compare) to fix lint errors https://cirrus-ci.com/task/6024617585803264
  15. DrahtBot removed the label CI failed on Feb 18, 2025
  16. maflcko commented at 4:17 pm on February 19, 2025: member

    review ACK 8f5228d9239464ece06f3a56372aeec29685801c 😸

    Signature:

    0untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
    1RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
    2trusted comment: review ACK 8f5228d9239464ece06f3a56372aeec29685801c 😸
    3et5ca1pe7unBgpCKoxG/XauMAz7SfwmVFTEe7+IgI/62K1awvsuMRPfCjwxGCyBqSTmRCIRRmrzoZ3Czy4lkCQ==
    
  17. Sjors commented at 4:33 pm on February 19, 2025: member
    Concept ACK. Reviewed as part of #31375, but did not check if they’re identical.

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-02-22 06:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me