util: add RunCommandParseJSON #15382

pull Sjors wants to merge 7 commits into bitcoin:master from Sjors:2019/02/runCommandParseJSON changing 24 files +295 −15
  1. Sjors commented at 3:09 pm on February 11, 2019: member

    Prerequisite for external signer support in #16546. Big picture overview in this gist.

    This adds a new dependency boost process. This is part of Boost since 1.64 which is part of depends. Because the minimum Boost version is 1.47, this functionality is skipped for older versions of Boost.

    Use ./configure --with-boost-process to opt in, which checks for the presence of Boost::Process.

    We add UniValue runCommandParseJSON(const std::string& strCommand) to system.{h,cpp} which calls an arbitrary command and processes the JSON returned by it. This is currently only called by the test suite.

    For testing purposes this adds a new regtest-only RPC method runcommand, as well as test/mocks/command.py used by functional tests. (this is no longer the case)

    TODO:

    • review boost process in #15440
  2. Sjors force-pushed on Feb 11, 2019
  3. Sjors commented at 3:13 pm on February 11, 2019: member

    See earlier discussion in #14912#pullrequestreview-186165666 for different approaches like _popen or boost process.

    cc @practicalswift @promag @ken2812221 @ryanofsky

  4. jgarzik commented at 3:32 pm on February 11, 2019: contributor
    This C++11 python-inspired subprocess module may or may not be of relevant interest, as it avoids Boost in favor of straight C++11: http://templated-thoughts.blogspot.com/2016/03/sub-processing-with-modern-c.html
  5. fanquake added the label RPC/REST/ZMQ on Feb 11, 2019
  6. fanquake added the label Utils/log/libs on Feb 11, 2019
  7. Sjors commented at 8:16 am on February 12, 2019: member

    @jgarzik thanks! 1600 lines of code seems a bit overkill, but on the other hand it looks very clean and presumably better than anything I would come up with.

    It currently doesn’t have Windows support, but they’re open to patches.

    Speaking of UniValue :-) I think I need to reorder the Makefile to prevent some Travis instances from failing with undefined reference to UniValue::read(char const*, unsigned int).

  8. Sjors force-pushed on Feb 12, 2019
  9. Sjors force-pushed on Feb 12, 2019
  10. Sjors force-pushed on Feb 12, 2019
  11. Sjors force-pushed on Feb 12, 2019
  12. practicalswift commented at 10:14 am on February 12, 2019: contributor
    The -regtest requirement is very important: could add a test case triggering the “runcommand for regression testing (-regtest mode) only” code path?
  13. Sjors commented at 10:24 am on February 12, 2019: member
    @practicalswift agreed, but I don’t know if the functional test framework can launch a Testnet node without triggering a real IBD. I suppose I can configure it to not connect to any peers.
  14. Sjors force-pushed on Feb 12, 2019
  15. Sjors force-pushed on Feb 12, 2019
  16. Sjors force-pushed on Feb 12, 2019
  17. in src/rpc/misc.cpp:375 in e2f681b215 outdated
    363@@ -364,6 +364,35 @@ static UniValue setmocktime(const JSONRPCRequest& request)
    364     return NullUniValue;
    365 }
    366 
    367+static UniValue runcommand(const JSONRPCRequest& request)
    368+{
    369+    if (request.fHelp || request.params.size() != 1)
    370+        throw std::runtime_error(
    371+            RPCHelpMan{"runcommand",
    372+                "\nRun command and parse JSON from stdout (-regtest only)\n",
    


    MarcoFalke commented at 4:20 pm on February 12, 2019:
    Why is this RPC exposed? For testing purposes? It should be a unit test in that case.

    Sjors commented at 4:28 pm on February 12, 2019:

    It’s restricted to regtest (similar to setmocktime). Main reasons I didn’t build a unit test:

    1. no idea how
    2. potentially creates even more cross-platform headaches
    3. runCommand is also only tested via functional tests (via blocknotify, etc)

    promag commented at 8:34 am on September 28, 2019:

    potentially creates even more cross-platform headaches

    What you mean? Because of the command itself?


    promag commented at 3:42 pm on October 10, 2019:
    @Sjors how about supporting -exec "cmd" in bitcoin_test binary (or in any other test binary), then that could be used in the functional test (instead of this RPC).

    ryanofsky commented at 12:04 pm on October 31, 2019:

    I think it’s ok to have this RPC only enabled in regtest. If you wanted to add more protection against misuse, you could disable it without gArgs.GetBoolArg("-enable-unsafe-runcommand-rpc", false) or similar.

    If you did want to drop the RPC and just write a unit test for runCommandParseJSON, I don’t think it would be difficult to do by just having the unit test write a temporary json file and then calling runCommandParseJSON with cat path_to_file on unix and type path_to_file on windows.

  18. Sjors force-pushed on Feb 12, 2019
  19. Sjors force-pushed on Feb 12, 2019
  20. Sjors force-pushed on Feb 12, 2019
  21. Sjors force-pushed on Feb 12, 2019
  22. Sjors commented at 8:19 am on February 13, 2019: member

    AppVeyor seems unhappy about the RPCHelpMan syntax:

    0[C:\projects\bitcoin\build_msvc\libbitcoin_util\libbitcoin_util.vcxproj]
    125c:\projects\bitcoin\src\leveldb\db\skiplist.h(360): warning C4312: 'reinterpret_cast': conversion from 'int' to 'void *' of greater size [C:\projects\bitcoin\build_msvc\libleveldb\libleveldb.vcxproj]
    226c:\projects\bitcoin\src\rpc\misc.cpp(371): error C2440: 'initializing': cannot convert from 'initializer list' to 'RPCHelpMan' [C:\projects\bitcoin\build_msvc\libbitcoin_server\libbitcoin_server.vcxproj]
    327c:\projects\bitcoin\src\rpc\misc.cpp(379): error C2512: 'std::runtime_error': no appropriate default constructor available [C:\projects\bitcoin\build_msvc\libbitcoin_server\libbitcoin_server.vcxproj]
    428Command exited with code 1
    529
    
  23. in test/functional/mocks/command.py:28 in 35cf444c86 outdated
    23+parser.add_argument('--fail', action='store_true', help='Respond with {success: false, error: "reason"}')
    24+parser.add_argument('--invalid', action='store_true', help='Return malformed JSON')
    25+parser.set_defaults(func=command)
    26+
    27+if len(sys.argv) == 1:
    28+   args = parser.parse_args(['-h'])
    


    practicalswift commented at 10:55 am on February 13, 2019:
    Wrong indentation :-)
  24. in test/functional/mocks/command.py:10 in 35cf444c86 outdated
     5+
     6+import argparse
     7+import json
     8+import sys
     9+
    10+def command(args):
    


    practicalswift commented at 10:56 am on February 13, 2019:
    This is redefining args from the outer scope.
  25. in src/rpc/misc.cpp:374 in 35cf444c86 outdated
    369+    if (request.fHelp || request.params.size() != 1)
    370+        throw std::runtime_error(
    371+            RPCHelpMan{"runcommand",
    372+                "\nRun command and parse JSON from stdout (-regtest only)\n",
    373+                {
    374+                    {"command", RPCArg::Type::STR, /* opt */ false, /* default_val */ "", "Command must return JSON\n"},
    


    MarcoFalke commented at 1:51 pm on February 13, 2019:
    0                    {"command", RPCArg::Type::STR, RPCArg::Optional::NO, "Command must return JSON\n"},
    
  26. Sjors force-pushed on Feb 13, 2019
  27. Sjors force-pushed on Feb 13, 2019
  28. Sjors force-pushed on Feb 13, 2019
  29. Sjors force-pushed on Feb 13, 2019
  30. Sjors force-pushed on Feb 13, 2019
  31. Sjors commented at 6:03 pm on February 13, 2019: member
    I switched to boost::process, which should give us Windows support. It’s included with the depends Boost 1.64. Because the minimum Boost version is 1.47, this functionality is just not compiled for older versions of Boost.
  32. Sjors force-pushed on Feb 13, 2019
  33. Sjors force-pushed on Feb 13, 2019
  34. Sjors commented at 8:25 pm on February 13, 2019: member

    @DrahtBot AppVeyor vcpkg install fails (I tried restarting), mostly likely as a result of me adding the boost-process package:

    I add AppVeyor to my own Github account and it finished vcpkg install successfully (it’s still building though). @MarcoFalke maybe it needs a cache purge?

  35. MarcoFalke commented at 8:34 pm on February 13, 2019: member
    Not sure how to purge the cache, maybe @ken2812221 knows
  36. Sjors commented at 8:59 pm on February 13, 2019: member
    On the bright side, the Gitian build does work:
  37. Sjors commented at 9:15 pm on February 13, 2019: member
    Trying something based on the docs
  38. DrahtBot commented at 9:18 pm on February 13, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #19504 (Bump minimum python version to 3.6 by ajtowns)
    • #19077 (wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets by achow101)
    • #19064 (refactor: Cleanup thread ctor calls by hebasto)
    • #19013 (test: add v0.20.0 to backwards compatibility test by Sjors)
    • #18750 (build: optionally skip external warnings by vasild)
    • #15719 (Wallet passive startup by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  39. Sjors force-pushed on Feb 13, 2019
  40. Sjors commented at 9:21 pm on February 13, 2019: member

    I can’t build master either on my own instance (logs).

    0...
    1libbitcoin_server.lib(index_txindex.obj) : error LNK2001: unresolved external symbol "public: void __cdecl leveldb::WriteBatch::Clear(void)" (?Clear@WriteBatch@leveldb@@QEAAXXZ) [C:\projects\bitcoin\build_msvc\bitcoin-wallet\bitcoin-wallet.vcxproj]
    253C:\projects\bitcoin\build_msvc\x64\Release\bitcoin-wallet.exe : fatal error LNK1120: 14 unresolved externals [C:\projects\bitcoin\build_msvc\bitcoin-wallet\bitcoin-wallet.vcxproj]
    

    Are there any non-default settings I (and the docs) should know about?

  41. jonasschnelli commented at 2:03 am on February 14, 2019: contributor
    @Sjors: I miss some context. Can you elaborate why this is necessary for external signer support?
  42. Sjors commented at 8:31 am on February 14, 2019: member
    @jonasschnelli #14912 allows Bitcoin Core to communicate with HWI which in turns communicates with the device. That way we don’t have to implement device specific support, but instead communicate via a (to be) standardised protocol to a driver. That driver could be HWI, but it could be any other command line utility that understands the same commands and gives the same responses. The protocol is described in more detail here.
  43. Sjors force-pushed on Feb 14, 2019
  44. Sjors renamed this:
    WIP [util] add runCommandParseJSON
    [util] add runCommandParseJSON
    on Feb 14, 2019
  45. Sjors force-pushed on Feb 14, 2019
  46. Sjors force-pushed on Feb 15, 2019
  47. ken2812221 commented at 8:59 am on February 15, 2019: contributor

    Looks like appveyor has not updated their boost-process package. Maybe you could try to add

    0- cmd: git -C C:\tools\vcpkg pull -q
    

    before

    0- cmd: vcpkg remove --outdated --recurse
    

    Edit: It doesn’t work. But it works on my PC.

  48. Sjors commented at 10:05 am on February 15, 2019: member
    @ken2812221 AppVeyor actually passed before my rebase. I think the problem is that it needs to bust cache before running, so that it adds boost-process.
  49. MarcoFalke referenced this in commit bf3677a6bb on Feb 15, 2019
  50. luke-jr commented at 2:14 pm on February 16, 2019: member
    Most distros seem to split boost modules, so just because 1.64 is used doesn’t guarantee boost::process is installed I think? We probably need a configure check…
  51. DrahtBot added the label Needs rebase on Feb 17, 2019
  52. Sjors commented at 8:40 am on February 18, 2019: member
    @luke-jr good point, I actually noticed that with AppVeyor which aborted much too late in the build process before I added the boost-process package.
  53. Sjors force-pushed on Feb 19, 2019
  54. DrahtBot removed the label Needs rebase on Feb 19, 2019
  55. Sjors commented at 12:56 pm on February 19, 2019: member

    Rebased. I cherry-picked Luke’s commit to explicitly detect boost-process with #ifdef HAVE_BOOST_PROCESS.

    I started reviewing boost-process in #15440

  56. in src/Makefile.test.include:524 in 2ca22a2b69 outdated
    517@@ -518,10 +518,10 @@ test_fuzz_blocktransactionsrequest_deserialize_CPPFLAGS = $(AM_CPPFLAGS) $(BITCO
    518 test_fuzz_blocktransactionsrequest_deserialize_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
    519 test_fuzz_blocktransactionsrequest_deserialize_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS)
    520 test_fuzz_blocktransactionsrequest_deserialize_LDADD = \
    521- $(LIBUNIVALUE) \
    522  $(LIBBITCOIN_SERVER) \
    523  $(LIBBITCOIN_COMMON) \
    524  $(LIBBITCOIN_UTIL) \
    525+ $(LIBUNIVALUE) \
    


    MarcoFalke commented at 8:18 pm on March 6, 2019:
    Needs rebase and all those ugly hunks dropped in this file

    Sjors commented at 6:59 pm on March 7, 2019:
    Hooray, I was able to drop 829171fa3182db545d9a07834fa1c2006fa01b6e
  57. DrahtBot added the label Needs rebase on Mar 6, 2019
  58. Sjors force-pushed on Mar 7, 2019
  59. DrahtBot removed the label Needs rebase on Mar 7, 2019
  60. Sjors force-pushed on Mar 15, 2019
  61. Sjors commented at 1:08 pm on March 15, 2019: member
    Added support for stdin.
  62. Sjors commented at 2:11 pm on March 15, 2019: member

    Not sure what I’m doing wrong for the windows cross compile build

  63. Sjors force-pushed on Mar 15, 2019
  64. jgarzik commented at 2:23 pm on March 15, 2019: contributor

    Not sure what I’m doing wrong for the windows cross compile [build](https://travis-

    The string stdin is a symbol from stdio.h of type FILE *. The compiler thinks you are passing the well-known FILE* to runCommandParseJSON()

    Recommend avoiding the stdin string altogether, since that’s a symbol from the earliest days of ANSI C. Call it stdinData or something.

  65. Sjors force-pushed on Mar 15, 2019
  66. in test/functional/test_framework/test_framework.py:583 in ed88f5b38a outdated
    575@@ -571,3 +576,7 @@ def is_zmq_compiled(self):
    576         config.read_file(open(self.options.configfile))
    577 
    578         return config["components"].getboolean("ENABLE_ZMQ")
    579+
    580+    def is_runcommand_compiled(self, node):
    


    practicalswift commented at 8:46 am on March 25, 2019:
    This does not have to be a method, right? Could be a function instead?
  67. Sjors force-pushed on Apr 15, 2019
  68. Sjors force-pushed on Apr 16, 2019
  69. Sjors force-pushed on Apr 24, 2019
  70. Sjors commented at 5:31 pm on April 24, 2019: member

    Note that on macOS with Homebrew boost 1.69.0_2 won’t find boost::process, due to similar errors as described here. It works fine with depends though.

    Update 2019-06-11: Homebrew boost 1.70 is out and works

  71. Sjors force-pushed on Apr 25, 2019
  72. DrahtBot added the label Needs rebase on Apr 26, 2019
  73. Sjors force-pushed on Apr 27, 2019
  74. DrahtBot removed the label Needs rebase on Apr 29, 2019
  75. DrahtBot added the label Needs rebase on Apr 30, 2019
  76. Sjors force-pushed on May 3, 2019
  77. DrahtBot removed the label Needs rebase on May 3, 2019
  78. Sjors force-pushed on May 12, 2019
  79. Sjors force-pushed on Jun 5, 2019
  80. DrahtBot added the label Needs rebase on Jun 6, 2019
  81. Sjors force-pushed on Jun 7, 2019
  82. DrahtBot removed the label Needs rebase on Jun 7, 2019
  83. DrahtBot added the label Needs rebase on Jun 19, 2019
  84. Sjors force-pushed on Jul 5, 2019
  85. Sjors force-pushed on Aug 2, 2019
  86. DrahtBot removed the label Needs rebase on Aug 2, 2019
  87. Sjors force-pushed on Aug 2, 2019
  88. Sjors force-pushed on Sep 4, 2019
  89. DrahtBot added the label Needs rebase on Sep 11, 2019
  90. Sjors force-pushed on Sep 16, 2019
  91. DrahtBot removed the label Needs rebase on Sep 16, 2019
  92. fanquake renamed this:
    [util] add runCommandParseJSON
    util: add runCommandParseJSON
    on Sep 25, 2019
  93. in src/util/system.cpp:1157 in b5b04db35c outdated
    1144@@ -1140,6 +1145,38 @@ void runCommand(const std::string& strCommand)
    1145 }
    1146 #endif
    1147 
    1148+#ifdef HAVE_BOOST_PROCESS
    1149+UniValue runCommandParseJSON(const std::string& strCommand, const std::string& strStdIn)
    


    promag commented at 8:37 am on September 28, 2019:
    Will this be used out of RPC? Otherwise could be moved to src/rpc/util.h?

    Sjors commented at 2:00 pm on October 10, 2019:
    Yes, this be used by the wallet (including from the GUI).
  94. in test/functional/mocks/command.py:26 in b5b04db35c outdated
    21+        sys.stdout.write(json.dumps({"success": False, "error": "reason"}))
    22+    else:
    23+        raise RuntimeError("Missing arguments")
    24+
    25+
    26+parser = argparse.ArgumentParser(prog='./mock_command.py', description='Test runCommandParseJSON() via runcommand RPC')
    


    promag commented at 8:39 am on September 28, 2019:
    nit, wrong prog=.
  95. promag commented at 8:42 am on September 28, 2019: member
    Tested locally on macos and looks good. I’ll keep #13339 rebase with this to have boost process support - will keep testing on my branch. Nice work @Sjors and @luke-jr.
  96. Sjors force-pushed on Oct 10, 2019
  97. in src/util/system.h:102 in d4ed9f6c25 outdated
    83@@ -82,6 +84,16 @@ fs::path GetSpecialFolderPath(int nFolder, bool fCreate = true);
    84 #if HAVE_SYSTEM
    85 void runCommand(const std::string& strCommand);
    86 #endif
    87+#ifdef HAVE_BOOST_PROCESS
    88+/**
    89+ * Execute a command with returns JSON, and parse the result.
    


    promag commented at 3:38 pm on October 10, 2019:

    Typo, witch.

    Also, could say what happens if the output can’t be parsed.


    practicalswift commented at 3:58 pm on October 10, 2019:

    Typo, witch.

    Which, seems like like a typo witch hunt.


    promag commented at 4:05 pm on October 10, 2019:
    Ah! I wanted to type “with”.

    hebasto commented at 5:11 pm on July 19, 2020:

    A typo seems remain :)

    0 * Execute a command which returns JSON, and parse the result.
    
  98. Sjors force-pushed on Oct 10, 2019
  99. promag commented at 8:30 pm on October 22, 2019: member

    While testing #13339 rebased with this one I’ve come to the conclusion that boost::process::system is way slower than standard system. Looks like there is some internal synchronization when multiple concurrent calls are made - can be observed with bitcoin-cli -regtest generatetoaddress 10 ... and -blocknotify=....

    I’ll push the above to a different branch.

  100. in src/rpc/misc.cpp:385 in 9ba024da60 outdated
    380+                RPCResults{},
    381+                RPCExamples{""},
    382+            }.ToString()
    383+        );
    384+
    385+    if (!Params().MineBlocksOnDemand())
    


    practicalswift commented at 8:47 pm on October 22, 2019:
    Can we check for -regtest explicitly instead of implicitly via fPowNoRetargeting? Explicit feels better than implicit when it comes to enabling remote execution :)

    Sjors commented at 8:18 am on October 23, 2019:
    Done
  101. Sjors force-pushed on Oct 23, 2019
  102. Sjors force-pushed on Oct 30, 2019
  103. Sjors commented at 7:23 pm on October 30, 2019: member

    I added USE_EXTERNAL_SIGNER to test/config.ini so that the test suite can check for it directly. Previously the test suite would call runcommand to find out. I also added --disable-external-signer to allow opting out.

    0Options used to compile and link:
    1  with wallet     = yes
    2  with gui / qt   = yes
    3    with qr       = yes
    4  external signer = no
    5  with zmq        = yes
    
  104. Sjors force-pushed on Oct 30, 2019
  105. Sjors force-pushed on Oct 31, 2019
  106. practicalswift commented at 8:23 am on October 31, 2019: contributor
    @Sjors Since runcommand support increases the attack surface perhaps it should be opt-in (--enable-external-signer) instead of opt-out (--disable-external-signer)?
  107. Sjors commented at 8:45 am on October 31, 2019: member
    @practicalswift if this were a standalone PR I would agree, but this PR only makes sense if we decide to go ahead with external signer support. Disabling that by default seems overkill.
  108. in build-aux/m4/ax_boost_process.m4:21 in 0e019caf5f outdated
    16+#
    17+#     HAVE_BOOST_PROCESS
    18+#
    19+# LICENSE
    20+#
    21+#   Copyright (c) 2012 Xiyue Deng <manphiz@gmail.com>
    


    ryanofsky commented at 11:33 am on October 31, 2019:

    In commit “configure: Clone ax_boost_chrono to ax_boost_process” (0e019caf5fb5b86bbbf7c23edc43be5a4313ae85)

    Where did this file come from? Is it copied from some upstream source? If there are bugfixes or updates, how will we incorporate them here? If there are bugfixes here, will we send them upstream? This seems similar to files at https://github.com/autoconf-archive/autoconf-archive/tree/master/m4 but I don’t see this file there.

    If copying external code to out gir repository, I think there should at least be a git commit message saying where the code comes from and what local modifications were made, if not more documentation saying what relationship to upstream should be and how code should be maintained over time.


    Sjors commented at 3:30 pm on October 31, 2019:

    Sjors commented at 6:15 pm on November 1, 2019:
    I swapped it out in favor of a more recent boost::system based example, listed the changes, and made an upstream PR: https://github.com/autoconf-archive/autoconf-archive/pull/202
  109. in configure.ac:1464 in ec6af5d1c9 outdated
    1456@@ -1451,6 +1457,26 @@ else
    1457   fi
    1458 fi
    1459 
    1460+dnl enable external signer support
    1461+AC_MSG_CHECKING([whether to build with external signer support])
    1462+if test x$ax_cv_boost_process = xno; then
    1463+  if test x$use_external_signer = xyes; then
    1464+     AC_MSG_ERROR("External signer support requested but cannot be built.")
    


    ryanofsky commented at 11:37 am on October 31, 2019:

    In commit “configure: add –enable-external-signer” (ec6af5d1c993d584861be4ce6a0f5188f1af73af)

    “but cannot be built” is really vague, could say “but requires Boost.Pocess”

  110. in configure.ac:1460 in ec6af5d1c9 outdated
    1456@@ -1451,6 +1457,26 @@ else
    1457   fi
    1458 fi
    1459 
    1460+dnl enable external signer support
    


    ryanofsky commented at 11:46 am on October 31, 2019:

    In commit “configure: add –enable-external-signer” (ec6af5d1c993d584861be4ce6a0f5188f1af73af)

    At first glance there seems to be little reason for a build flag --enable-external-signer to exist when the user will have to explicitly enable and configure the feature at runtime anyway. But the reason for the build flag seems to be that older versions of boost that we support don’t include Boost.Process. Would suggest a comment here like dnl External signer support is optional. It requires Boost.Process which is only present in newer boost versions (>=1.64)


    Sjors commented at 3:32 pm on October 31, 2019:
    I should also rename the commit message to --disable-external-signer given the default is enable when a recent enough Boost version is found. I’ll update the version info as well, although I believe it will throw if you try --enable-external-signer on a old version of Boost.

    Sjors commented at 6:22 pm on November 1, 2019:
    Added, and clarified the commit to indicate external signer is an upcoming feature.
  111. in configure.ac:1472 in ec6af5d1c9 outdated
    1467+  use_external_signer=no
    1468+else
    1469+  if test x$use_external_signer != xno; then
    1470+    AC_MSG_RESULT(yes)
    1471+    use_external_signer=yes
    1472+    AC_DEFINE_UNQUOTED([USE_EXTERNAL_SIGNER],[$use_external_signer],[External signer support not compiled if undefined, otherwise value (0 or 1) determines default state])
    


    ryanofsky commented at 11:52 am on October 31, 2019:

    In commit “configure: add –enable-external-signer” (ec6af5d1c993d584861be4ce6a0f5188f1af73af)

    Should change USE_EXTERNAL_SIGNER to ENABLE_EXTERNAL_SIGNER. USE_ is appropriate if the flag toggles use of an external library (configure --with- option). ENABLE_ is appropriate if flag toggles a feature our application (configure --enable- option, https://autotools.io/autoconf/arguments.html)


    Sjors commented at 3:33 pm on October 31, 2019:
    Any tips for efficiently renaming a constant accross multiple commits?

    Sjors commented at 8:42 pm on November 1, 2019:

    To rename in each commit:

    0git rebase -i HEAD~5 --exec 'git diff --name-only --diff-filter=AMC HEAD~ | xargs sed -i "s/USE_EXTERNAL_SIGNER/ENABLE_EXTERNAL_SIGNER/g"; git commit -a --amend --no-edit'
    

    To rename messages:

    0git filter-branch -f --msg-filter 'sed -e "s/USE_EXTERNAL_SIGNER/ENABLE_EXTERNAL_SIGNER/g"'  HEAD~5..HEAD
    

    I wasn’t able to get filter-branch to work with --tree-filter like here: https://stackoverflow.com/questions/39608191/rename-variable-in-all-commits-on-a-given-branch (e.g. this leads to followup commits undoing earlier ones: git filter-branch -f --tree-filter 'git diff --name-only --diff-filter=AMC $GIT_COMMIT~ | xargs gsed -i "s/USE_EXTERNAL_SIGNER/ENABLE_EXTERNAL_SIGNER/g"' HEAD~4^..HEAD HEAD)

    (use gsed on macOS)

  112. in src/util/system.cpp:1157 in 82079a04e6 outdated
    1152@@ -1148,6 +1153,38 @@ void runCommand(const std::string& strCommand)
    1153 }
    1154 #endif
    1155 
    1156+#ifdef USE_EXTERNAL_SIGNER
    1157+UniValue runCommandParseJSON(const std::string& strCommand, const std::string& strStdIn)
    


    ryanofsky commented at 12:10 pm on October 31, 2019:

    In commit “[util] add runCommandParseJSON” (82079a04e6505424bbcb4b030a81d368933d24a7)

    Naming convention for argument and variables in this function could use snake_case instead of camelCase and drop strHungarianNotationPrefixes

  113. in configure.ac:129 in ec6af5d1c9 outdated
    124@@ -125,6 +125,11 @@ AC_ARG_ENABLE([upnp-default],
    125   [use_upnp_default=$enableval],
    126   [use_upnp_default=no])
    127 
    128+AC_ARG_ENABLE([external-signer],
    129+    [AS_HELP_STRING([--disable-external-signer],[do not compile external signer support (default is to compile if Boost::Process is found)])],
    


    ryanofsky commented at 12:28 pm on October 31, 2019:

    In commit “configure: add –enable-external-signer” (ec6af5d1c993d584861be4ce6a0f5188f1af73af)

    Before we actually check in code making this flag useful, the ./configure --help text should say (incomplete, experimental) or similar to avoid suggesting the build will support a feature that isn’t actually usable.

  114. in configure.ac:1681 in ec6af5d1c9 outdated
    1675-    echo "    with qr     = $use_qr"
    1676+    echo "    with qr       = $use_qr"
    1677 fi
    1678-echo "  with zmq      = $use_zmq"
    1679-echo "  with test     = $use_tests"
    1680+echo "  external signer = $use_external_signer"
    


    ryanofsky commented at 12:30 pm on October 31, 2019:

    In commit “configure: add –enable-external-signer” (ec6af5d1c993d584861be4ce6a0f5188f1af73af)

    This is pretty misleading to print before the feature actually implemented. Would suggest commenting this out by default to avoid confusion.

    EDIT: Alternately you could drop --enable-external-signer replacing it with --with-boost-process instead and printing with Boost.Process = $use_boost_process here. But that is probably overkill just to fix a temporarily misleading print.


    Sjors commented at 7:35 pm on November 2, 2019:
    I commented this line out, but leaving --enable-external-signer in place (and keeping the indentation change).
  115. ryanofsky approved
  116. ryanofsky commented at 12:33 pm on October 31, 2019: member

    Code review ACK 82079a04e6505424bbcb4b030a81d368933d24a7

    Looks great! Really nicely implemented updating everything from the msvc build to the test framework to the doxygen configuration. I left review suggestions that I think would be nice to address but are not critical.

  117. ryanofsky commented at 12:59 pm on October 31, 2019: member

    TODO:

    • review boost process in #15440

    Just saw this TODO. Is it still valid? If so, could you clarify who is supposed to do what? Is this blocking? Or do you just want to encourage people to look at the boost process implementation?(Could include a link to https://github.com/boostorg/process in that case)

    If Boost.Process does need more scrutiny, that might be a reason to disable its use by default. It might also be a reason to replace or augment the --enable-external-signer option with a --with-boost-process flag for clarity. A --with-boost-process flag might also be nice because Boost.Process could be used for other things not related to external signing, like wallet notifications (#13339).

    My previous review ack / approval still stands, but I am wondering about this TODO in the PR description.

  118. ryanofsky commented at 1:16 pm on October 31, 2019: member

    If Boost.Process does need more scrutiny, that might be a reason to disable its use by default.

    Note that to disable boost process by default but still get test coverage on travis you would need to add a depends option and enable the option it in one or more travis configurations, passing it to configure through depends/config.site.in. #16367 has an example of this with the MULTIPROCESS depends flag and --enable-multiprocess autoconf option.

  119. Sjors commented at 3:43 pm on October 31, 2019: member

    There is a --with-boost-process option already as part of the first commit (Clone ax_boost_chrono to ax_boost_process). I believe this can be used to turn Boost::Process off entirely, but I haven’t tested it.

    Just saw this TODO. Is it still valid? If so, could you clarify who is supposed to do what? Is this blocking?

    We haven’t included Boost::Process code before. If anyone feels uncomfortable turning this feature flag on by default, then hopefully someone else can review Boost::Process to alleviate those concerns.

    On the other hand, perhaps this can be merged faster if it’s off by default. In that case I’ll open a separate PR to switch the default; which is necessary to make external signer functionality actually useful, rather than something only for users who self compile). But I would rather just get it over with.

  120. Sjors force-pushed on Nov 1, 2019
  121. Sjors commented at 8:59 pm on November 1, 2019: member
    Rebased, implemented some of @ryanofsky’s suggestions. I might look into ditching the runcommand test RPC in favor of a unit test later.
  122. in build-aux/m4/ax_boost_process.m4:77 in 39a681958f outdated
    81+
    82+            AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/process.hpp>]],
    83+                [[boost::process::child* child = new boost::process::child; delete child;]])],
    84+                ax_cv_boost_process=yes, ax_cv_boost_process=no)
    85+			 CXXFLAGS=$CXXFLAGS_SAVE
    86+             AC_LANG_POP([C++])
    


    ryanofsky commented at 10:15 pm on November 1, 2019:
    Extra space indenting the line? And also the one above assuming tabs are supposed to be 4 spaces.

    Sjors commented at 9:16 am on August 4, 2020:
    Any lasting changes to this file would have to made upstream: https://www.gnu.org/software/autoconf-archive/ax_boost_process.html

    ryanofsky commented at 3:19 pm on August 4, 2020:

    re: #15382 (review)

    Any lasting changes to this file would have to made upstream: gnu.org/software/autoconf-archive/ax_boost_process.html

    Oops, sorry, that was a stale review comment. But good to see changes made are upstream in https://github.com/autoconf-archive/autoconf-archive/blob/master/m4/ax_boost_process.m4. If PR needs to be force pushed again, could mention the new file is external in the commit description.

  123. laanwj commented at 10:56 am on November 2, 2019: member

    I haven’t dived into details of why it’s needed for external signers, but from a security perspective I really dislike an RPC call that can invoke arbitrary commands.

    Edit: could we restrict this to a set of commands specified in bitcoin.conf?

  124. Sjors commented at 5:03 pm on November 2, 2019: member

    I haven’t dived into details of why it’s needed for external signers, but from a security perspective I really dislike an RPC call that can invoke arbitrary commands.

    Edit: could we restrict this to a set of commands specified in bitcoin.conf?

    External signer support has precisely that restriction (the only command that’s called is specified by -signer).

  125. Sjors force-pushed on Nov 2, 2019
  126. Sjors commented at 7:30 pm on November 2, 2019: member

    I dropped the runcommand regtest RPC call and added test/system_tests.cpp instead. This test file creates the command.py test script programatically, because test_bitcoin is a standalone executable.

    I think --disable-external-signer is easier to understand than --without-boost-process and more future proof. We might add different communication methods later to this feature, e.g. web based requests, file handlers or QR code scanning. For people who are worried about the security implications of those things, this offers one easy switch to turn it off.

  127. Sjors force-pushed on Nov 2, 2019
  128. Sjors force-pushed on Nov 2, 2019
  129. Sjors force-pushed on Nov 2, 2019
  130. Sjors force-pushed on Nov 3, 2019
  131. Sjors commented at 9:34 am on November 3, 2019: member

    I simplified the tests by just using echo instead of calling a Python script.

    The test is skipped on the Windows Travis machine (boost::process::process_error: CreateProcess failed: No such device or address), but does run on AppVeyor. I also skip the stdin test on all Windows platforms, but this will be tested via functional tests here: https://github.com/bitcoin/bitcoin/pull/16546/commits/1902c3a2f16705e6ba48b17cc5f3dc8d82e9ff2a#diff-792894ce3a8d8f19311b7e763b294fc3R76-R78

    I moved the external signer specific commits to #16546. runCommandParseJSON now checks for HAVE_BOOST_PROCESS.

    Update: this approach seems to cause random Travis failures of the test, without error message. I set --without-boost-process for the ARM and macOS builds, but I also noticed the failure on Bionic (machine 5, which often fails for others reasons too). Maybe echo doesn’t survive the many layers of virtualization on Travis.

  132. Sjors force-pushed on Nov 3, 2019
  133. Sjors force-pushed on Nov 3, 2019
  134. Sjors force-pushed on Nov 3, 2019
  135. Sjors force-pushed on Nov 3, 2019
  136. Sjors force-pushed on Nov 3, 2019
  137. Sjors force-pushed on Nov 3, 2019
  138. Sjors force-pushed on Nov 3, 2019
  139. Sjors force-pushed on Nov 3, 2019
  140. Sjors force-pushed on Nov 4, 2019
  141. Sjors force-pushed on Nov 4, 2019
  142. Sjors commented at 4:12 pm on November 4, 2019: member
    Rebased after native ARM support for Travis was added in #17233. I re-enabled Boost::Process for the ARM Travis machine (replacing 23b653aad with 8f811db).
  143. DrahtBot added the label Needs rebase on Nov 7, 2019
  144. Sjors force-pushed on Nov 7, 2019
  145. DrahtBot removed the label Needs rebase on Nov 7, 2019
  146. Sjors commented at 8:16 am on November 8, 2019: member
    @sipsorcery it looks like AppVeyor is not picking up the boost-process package I added (after I rebased on #17364
  147. Sjors referenced this in commit d5faf21e72 on Nov 8, 2019
  148. laanwj commented at 9:20 am on November 8, 2019: member
    Concept ACK, recent changes (test in a different way than adding a (potential security hole) RPC call, --disable-external-signer instead of --without-boost-process) definitely seem like improvements to me.
  149. sipsorcery commented at 9:21 am on November 8, 2019: member

    @Sjors the mechanism I added to save the 2 minutes updating the vcpkg source each build assumes if any installed packages are cached then all required packages are installed. As you’ve highlighted that’s not a great assumption.

    I’ll come up with a fix pronto.

  150. Sjors referenced this in commit 3c5e30b959 on Nov 8, 2019
  151. MarcoFalke referenced this in commit a6f6333ba2 on Nov 11, 2019
  152. DrahtBot added the label Needs rebase on Nov 11, 2019
  153. Sjors force-pushed on Nov 11, 2019
  154. Sjors commented at 4:49 pm on November 11, 2019: member
    Rebased after AppVeyor improvement in #17416. My ax_boost_process PR has been merged upstream, but I missed a spot: https://github.com/autoconf-archive/autoconf-archive/pull/203. That doesn’t have to hold this PR back though.
  155. Sjors force-pushed on Nov 11, 2019
  156. DrahtBot removed the label Needs rebase on Nov 11, 2019
  157. DrahtBot added the label Needs rebase on Nov 19, 2019
  158. Sjors force-pushed on Nov 19, 2019
  159. DrahtBot removed the label Needs rebase on Nov 19, 2019
  160. Sjors commented at 6:15 pm on November 19, 2019: member
    I cherry-picked #17520 to ignore the compiler error due to an unused variable in Boost 1.70. Alternatively we can upgrade to Boost 1.71.
  161. Sjors force-pushed on Nov 23, 2019
  162. DrahtBot added the label Needs rebase on Nov 25, 2019
  163. Sjors force-pushed on Nov 25, 2019
  164. DrahtBot removed the label Needs rebase on Nov 25, 2019
  165. Sjors force-pushed on Dec 9, 2019
  166. DrahtBot added the label Needs rebase on Jan 8, 2020
  167. Sjors force-pushed on Jan 9, 2020
  168. DrahtBot removed the label Needs rebase on Jan 9, 2020
  169. Sjors commented at 2:53 pm on January 16, 2020: member
    I made a separate PR to bump Boost, as that seems to be causing some CI issues: #17941
  170. DrahtBot added the label Needs rebase on Jan 24, 2020
  171. Sjors force-pushed on Jan 24, 2020
  172. DrahtBot removed the label Needs rebase on Jan 24, 2020
  173. Sjors commented at 4:34 pm on January 24, 2020: member

    Travis fails: https://travis-ci.org/bitcoin/bitcoin/jobs/641302549

     0Running 2 test cases...
     12956Test cases order is shuffled using seed: 899780953
     22957Entering test module "Bitcoin Core Test Suite"
     32958test/system_tests.cpp(11): Entering test suite "system_tests"
     42959test/system_tests.cpp(20): Entering test case "run_command"
     529602020-01-24T11:25:50Z Seed: Setting random seed for current tests to RANDOM_CTX_SEED=1dbce6f9493a59dd60c8c2615905dc30901e172c156ddf6727a37f3fe178f46b
     629612020-01-24T11:25:50Z Bitcoin Core version v0.19.99.0-5bc5674a (release build)
     729622020-01-24T11:25:52Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
     829632020-01-24T11:25:52Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
     92964Makefile:15875: recipe for target 'test/system_tests.cpp.test' failed
    102965make[3]: *** [test/system_tests.cpp.test] Error 1
    

    I reproduced this once on an Ubuntu machine, but after that even git clean -dfx doesn’t reproduce.

  174. Sjors commented at 10:14 am on January 28, 2020: member

    Looks like valgrind finds an issue, but I’m not sure how to interpret the result:

    0valgrind --suppressions=contrib/valgrind.supp --leak-check=full src/test/test_bitcoin --run_test=system_tests
    
     0==1639== Memcheck, a memory error detector
     1==1639== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
     2==1639== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
     3==1639== Command: src/test/test_bitcoin --run_test=system_tests
     4==1639== 
     5Running 2 test cases...
     6==1639== 
     7==1639== Process terminating with default action of signal 13 (SIGPIPE)
     8==1639==    at 0x5492281: write (write.c:27)
     9==1639==    by 0x848C6E: write (basic_pipe.hpp:85)
    10==1639==    by 0x848C6E: _write_impl (pipe.hpp:226)
    11==1639==    by 0x848C6E: boost::process::basic_pipebuf<char, std::char_traits<char> >::sync() (pipe.hpp:179)
    12==1639==    by 0x64A4D7D: std::ostream::flush() (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
    13==1639==    by 0x846A26: flush<char, std::char_traits<char> > (ostream:613)
    14==1639==    by 0x846A26: endl<char, std::char_traits<char> > (ostream:591)
    15==1639==    by 0x846A26: operator<< (ostream:113)
    16==1639==    by 0x846A26: runCommandParseJSON(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (system.cpp:1063)
    17==1639==    by 0x3C4393: system_tests::run_command::test_method() (system_tests.cpp:32)
    18==1639==    by 0x3C7460: system_tests::run_command_invoker() (system_tests.cpp:20)
    19==1639==    by 0x58F02CD: boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) (in /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1)
    20==1639==    by 0x58EF77C: boost::execution_monitor::catch_signals(boost::function<int ()> const&) (in /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1)
    21==1639==    by 0x58EF860: boost::execution_monitor::execute(boost::function<int ()> const&) (in /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1)
    22==1639==    by 0x58EFFDC: boost::execution_monitor::vexecute(boost::function<void ()> const&) (in /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1)
    23==1639==    by 0x591E8D0: boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned int) (in /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1)
    24==1639==    by 0x58FAC6A: boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) (in /usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.65.1)
    

    Detailed log.

    I can narrow it down a bit. The errors go away if I disable the following two tests:

    0const UniValue result = runCommandParseJSON("echo \"{\"success\": true}\"");
    
    0BOOST_REQUIRE_THROW(runCommandParseJSON("echo \"{\""), std::runtime_error); // Unable to parse JSON
    

    The remaining two don’t cause problems:

    0const UniValue result = runCommandParseJSON("");
    
    0const UniValue result = runCommandParseJSON("cat", "{\"success\": true}");
    

    My guess is that the use of echo is causing problems.

  175. Sjors force-pushed on Jan 28, 2020
  176. in src/util/system.cpp:1063 in 3fc0c5e6a8 outdated
    1058+        bp::std_out > stdout_stream,
    1059+        bp::std_in < stdin_stream
    1060+    );
    1061+    if (!str_std_in.empty()) {
    1062+        stdin_stream << str_std_in << std::endl;
    1063+        stdin_stream.pipe().close();
    


    Sjors commented at 2:10 pm on January 28, 2020:
    Valgrind warns here: ==22385== Warning: invalid file descriptor -1 in syscall close() during the test with std::in.
  177. Sjors commented at 3:37 pm on January 28, 2020: member
    I added some additional checks and now Travis (with valgrind) is happy: https://travis-ci.org/bitcoin/bitcoin/builds/642905548. It looks like valgrind was actually helpful here, because the mysterious failures I was seeing previously in #15382 (comment) are also gone. I no longer need to skip Windows, macOS and ARM Travis builds.
  178. Sjors force-pushed on Jan 28, 2020
  179. Sjors force-pushed on Jan 28, 2020
  180. Sjors force-pushed on Jan 28, 2020
  181. Sjors force-pushed on Jan 30, 2020
  182. Sjors force-pushed on Feb 19, 2020
  183. DrahtBot added the label Needs rebase on Mar 10, 2020
  184. Sjors force-pushed on Mar 10, 2020
  185. DrahtBot removed the label Needs rebase on Mar 10, 2020
  186. DrahtBot added the label Needs rebase on Apr 6, 2020
  187. Sjors force-pushed on Apr 10, 2020
  188. DrahtBot removed the label Needs rebase on Apr 10, 2020
  189. DrahtBot added the label Needs rebase on Apr 10, 2020
  190. DrahtBot removed the label Needs rebase on Apr 11, 2020
  191. Sjors force-pushed on Apr 27, 2020
  192. Sjors force-pushed on Apr 27, 2020
  193. Sjors commented at 2:03 pm on April 27, 2020: member
    Instead of bumping Boost to 1.72 I added a small patch to get rid of the unused variable in Boost Process (needed for ./configure --enable-werror). Now that descriptor wallets are merged, this should be ready for review, as one of the prerequisites for HWI support.
  194. Sjors force-pushed on Apr 27, 2020
  195. in src/util/system.cpp:1112 in 5e1f378d75 outdated
    1106+    std::string result;
    1107+    std::getline(stdout_stream, result);
    1108+
    1109+    c.wait();
    1110+    const int n_error = c.exit_code();
    1111+    if (n_error) throw std::runtime_error(strprintf("runCommandParseJSON error: process(%s) returned %d\n", str_command, n_error));
    


    fjahr commented at 1:05 pm on April 28, 2020:
    I was thinking that throwing the function name at the user might not be helpful for most of them. Similar below, it might be confusing to give a JSON error when the workflow is automatic and the user is not handling JSON themselves. But I guess there will be better, more specific error messages used when this is utilized in external signer support.

    Sjors commented at 4:30 pm on April 28, 2020:
    Indeed, consumers should catch this error and do something more useful. There’s different ways to do that, e.g. subclassing std::runtime_error or using a return enum like TransactionError, but we can figure that out later.
  196. fjahr approved
  197. fjahr commented at 2:02 pm on April 28, 2020: member

    Concept ACK

    Also built and ran tests with and without --without-boost-process on macOS.

  198. jb55 commented at 4:53 pm on April 28, 2020: member
    Concept ACK, weak Boost NACK but in the absence of a better alternative perhaps its fine for now. but surely it wouldn’t be difficult to make a few small, well tested functions that execute external commands on each OS? It wouldn’t need to be fancy or full featured like #15382 (comment).
  199. Sjors commented at 6:19 pm on April 28, 2020: member

    surely it wouldn’t be difficult to make a few small, well tested functions that execute external commands on each OS

    Maybe merging this use of Boost will motivate more people to actually try and do that :-)

  200. Sjors force-pushed on May 22, 2020
  201. instagibbs commented at 7:41 pm on May 22, 2020: member
    concept ACK
  202. laanwj added this to the "Blockers" column in a project

  203. in src/test/system_tests.cpp:13 in b0d22226cd outdated
     8+
     9+#include <boost/test/unit_test.hpp>
    10+
    11+BOOST_FIXTURE_TEST_SUITE(system_tests, BasicTestingSetup)
    12+
    13+// At least one test is required (in case HAVE_BOOST_PROCESS is not defined)
    


    MarcoFalke commented at 10:01 pm on May 31, 2020:
    Could mention that this is a workaround for #19128 ?

    Sjors commented at 12:31 pm on June 1, 2020:
    Done and rebased.
  204. Sjors force-pushed on Jun 1, 2020
  205. MarcoFalke commented at 12:58 pm on June 1, 2020: member
    Is it required that the signer can run arbitrary system commands in the name of bitcoind when instead some well defined interface could be defined? Or more generally, to better understand the motivation behind this pull request, I’d like to see which other options have been explored and discarded (and why).
  206. Sjors commented at 1:57 pm on June 1, 2020: member

    runCommandParseJSON is not exposed anywhere until #16546. That PR is indeed limited to very specific commands. It lets you specify a script location, but the commands themselves (e.g. signtx) and expected response format are hard coded. An earlier version of this PR did allow arbitrary commands, though only on regtest, through a runcommand RPC. This was dropped because of security concerns (and because I could work around it): #15382 (comment)

    Two different approaches I considered (also discussed in various places, but not sure where to find logs):

    1. bitcoind could call an RPC instead of executing a command. That requires HWI (or similar tool) to run a server continuously, which seems problematic. It’s also hard to have an RPC on localhost that can’t be reached from a browser by a malicious website, so you’d have to add cookie / password authentication.
    2. URI handlers. HWI could register under a URI like bitcoin-signer:sign?transaction=PSBT and when complete open a URI like bitcoin-core:here_you_go?transaction=PSBT. URI handlers might work for GUI, but they’re not very suitable for RPC. They’re also a pain.

    If HWI (or similar tool) was malicious, it really doesn’t matter if it’s called through a script invocation, or if it runs as a server. It also doesn’t matter what parameters you’d call it with.

  207. MarcoFalke commented at 2:03 pm on June 1, 2020: member
    Fair enough. And the command is static and supplied at startup in the bitcoin.conf file (just like walletnotify), right?
  208. Sjors commented at 2:37 pm on June 1, 2020: member
    @MarcoFalke yes, at least in #16546 it’s set through -signer. #16549 lets you configure it from the GUI without a restart, but that PR is still draft.
  209. MarcoFalke commented at 4:25 pm on June 1, 2020: member
    Concept ACK
  210. Sjors force-pushed on Jun 10, 2020
  211. Sjors commented at 3:49 pm on June 10, 2020: member
    It seems Travis decided not to run this time. I rebased #16546 on top of this, and Travis does run there…
  212. promag commented at 11:36 pm on June 11, 2020: member
    Tested ACK 6878673311d1081d804e2ac888473ddca3baa3a0 on macos. Are you considering supporting parsing multiline json?
  213. fanquake commented at 6:14 am on June 12, 2020: member

    Reading through the discussion here, it’s still unclear to me why this is opt-out rather than opt-in. I don’t understand the comment here:

    On the other hand, perhaps this can be merged faster if it’s off by default. In that case I’ll open a separate PR to switch the default; which is necessary to make external signer functionality actually useful, rather than something only for users who self compile). But I would rather just get it over with.

    We can ship releases with this on (if wanted) by adding --enable-external-signer to the gitian / Guix builds, and developers or users who are compiling locally can opt in when configuring. I’m not sure why having it compiled-in-by-default if Boost Process is available is a requirement for it to be “actually useful”?

    Even if it has not necessarily been the case in the past, I’d much rather have users and developers intentionally opt-in to these kinds of new features, rather than us just bundling all sorts of things into bitcoind, based on whatever you happen to have installed on the system where you are compiling.

  214. Sjors commented at 9:01 am on June 12, 2020: member

    So there’s 3 scenarios:

    1. binary download
    2. compiled when boost::process is absent
    3. compiled when boost::process is present

    I mainly considered (1) and (2), because I don’t think requiring self compilation for GUI features makes sense.

    If you have a strong preference for turning it off by default for (3) I can do that. However, it’s unusual for features that are shipped in binaries to be turned off by default. That itself could be confusing.

    Another approach could be to turn it off by default, even in the binaries, until GUI support in #16549 lands. Requiring RPC users to self-compile for now seems fine by me.

  215. Sjors force-pushed on Jun 16, 2020
  216. Sjors force-pushed on Jun 16, 2020
  217. Sjors commented at 1:35 pm on June 16, 2020: member

    I made boost::process opt-in, and switched all CI instances to use --with-boost-process (except win64). It’s not listed in the configure summary but you can tell it works from the number of tests run by test_bitcoin --run_test=system_test.

    The next PR in this series (RPC support) will also be opt-in: --enable-external-signer. This will be included in the configure summary and, when enabled, assumes --with-boost-process.

    I’ll wait until the GUI PR to switch this default, e.g. to when GUI is configured, or only for Gitian builds, TBD.

  218. Sjors force-pushed on Jun 16, 2020
  219. Sjors force-pushed on Jun 18, 2020
  220. Sjors force-pushed on Jun 19, 2020
  221. DrahtBot added the label Needs rebase on Jun 19, 2020
  222. Sjors force-pushed on Jun 19, 2020
  223. DrahtBot removed the label Needs rebase on Jun 19, 2020
  224. jonatack commented at 7:12 pm on June 19, 2020: member
    Concept ACK, will review shortly
  225. meshcollider added this to the "PRs" column in a project

  226. in src/Makefile.test.include:243 in bfed9cdd3b outdated
    238@@ -239,6 +239,7 @@ BITCOIN_TESTS =\
    239   test/ref_tests.cpp \
    240   test/reverselock_tests.cpp \
    241   test/rpc_tests.cpp \
    242+  test/system_tests.cpp \
    


    jonatack commented at 2:56 pm on June 22, 2020:
    bfed9cdd nit: sort
  227. in src/util/system.h:108 in bfed9cdd3b outdated
    103+ *
    104+ * @param str_command The command to execute, including any arguments
    105+ * @param str_std_in string to pass to stdin
    106+ * @return parsed JSON
    107+ */
    108+UniValue runCommandParseJSON(const std::string& str_command, const std::string& str_std_in="");
    


    jonatack commented at 3:02 pm on June 22, 2020:

    bfed9cdd

    0UniValue RunCommandParseJSON(const std::string& str_command, const std::string& str_std_in="");
    
  228. jonatack commented at 3:17 pm on June 22, 2020: member
    Looked over the last commit bfed9cdd, the only one I felt semi-competent to review. Built with Clang on Debian with and without --with-boost-process and ran unit tests and just src/test/test_bitcoin -t system_tests -l all. I’m not sure how else to test; I did not read the middle 137 comments hidden by GitHub.
  229. Sjors force-pushed on Jun 25, 2020
  230. Sjors force-pushed on Jun 25, 2020
  231. Sjors renamed this:
    util: add runCommandParseJSON
    util: add RunCommandParseJSON
    on Jun 25, 2020
  232. Sjors commented at 5:00 pm on June 25, 2020: member

    Thanks, rebased and addressed nits.

    I did not read the middle 137 comments hidden by GitHub

    You don’t know what you’re missing :-)

  233. DrahtBot added the label Needs rebase on Jun 29, 2020
  234. Sjors force-pushed on Jun 29, 2020
  235. Sjors commented at 5:21 pm on June 29, 2020: member
    A day not spent rebasing, is a day not lived… (ok, this one was trivial)
  236. DrahtBot removed the label Needs rebase on Jun 29, 2020
  237. in src/util/system.cpp:1184 in 140716eab1 outdated
    1095+    if (str_command.empty()) return UniValue::VNULL;
    1096+
    1097+    bp::child c(
    1098+        str_command,
    1099+        bp::std_out > stdout_stream,
    1100+        bp::std_in < stdin_stream
    


    achow101 commented at 8:03 pm on June 29, 2020:

    In commit [util] add RunCommandParseJSON

    We should be piping stderr too. In HWI, this is used for some info to the user that we can discard here. But on failure, it can provide useful debugging information that we would want to see in the exception.

  238. in src/test/system_tests.cpp:41 in 140716eab1 outdated
    36+        const UniValue& success = find_value(result, "success");
    37+        BOOST_CHECK(!success.isNull());
    38+        BOOST_CHECK_EQUAL(success.getBool(), true);
    39+    }
    40+    {
    41+        BOOST_REQUIRE_THROW(RunCommandParseJSON("invalid_command"), std::runtime_error); // Command failed
    


    achow101 commented at 8:07 pm on June 29, 2020:

    In commit [util] add RunCommandParseJSON

    Maybe use the false command which always fails? Or is the intent to try a command that probably won’t exist?


    Sjors commented at 6:03 pm on July 2, 2020:

    Keeping it for now, but I suppose false could work too, assuming it exists on Windows.

    When a command doesn’t exist, Boost::Process intercepts this and throws its own exception.

  239. achow101 commented at 8:10 pm on June 29, 2020: member

    Some comments on configure:

    • It would be nice to indicate whether we have boost process in the summary at the end of configure’s output.
    • Instead of explicitly opting in, I would prefer boost process was used if it is available, like we do for miniupnp and zmq.
  240. DrahtBot added the label Needs rebase on Jul 2, 2020
  241. Sjors commented at 6:02 pm on July 2, 2020: member

    Did I mention I love rebasing? :-)

    I added it to the configure summary. Also added std::err content to the exception message. @achow101 wrote:

    Instead of explicitly opting in, I would prefer boost process was used if it is available, like we do for miniupnp and zmq.

    So do I, but @fanquake didn’t agree:

    Even if it has not necessarily been the case in the past, I’d much rather have users and developers intentionally opt-in to these kinds of new features, rather than us just bundling all sorts of things into bitcoind, based on whatever you happen to have installed on the system where you are compiling.

    So for now my aim is to switch to opt-out once the GUI feature lands. In either case, I’d rather get this merged as opt-in than not at all.

  242. Sjors force-pushed on Jul 2, 2020
  243. DrahtBot removed the label Needs rebase on Jul 2, 2020
  244. Sjors force-pushed on Jul 6, 2020
  245. DrahtBot added the label Needs rebase on Jul 14, 2020
  246. Sjors force-pushed on Jul 14, 2020
  247. DrahtBot removed the label Needs rebase on Jul 14, 2020
  248. promag commented at 11:01 pm on July 16, 2020: member

    appveyor failed with

    0
    1C:/projects/bitcoin/src/test/system_tests.cpp(28): error: in "system_tests/run_command": check ex.what() == std::string("execve failed: No such file or directory") has failed [ CreateProcess failed: The system cannot find the file specified. != execve failed: No such file or directory]
    

    So for now my aim is to switch to opt-out once the GUI feature lands. In either case, I’d rather get this merged as opt-in than not at all.

    Let’s not put @Sjors’s love to rebase at stake! I agree to merge this too.

  249. MarcoFalke commented at 3:01 pm on July 17, 2020: member
  250. DrahtBot added the label Needs rebase on Jul 17, 2020
  251. Sjors force-pushed on Jul 17, 2020
  252. DrahtBot removed the label Needs rebase on Jul 17, 2020
  253. Sjors force-pushed on Jul 17, 2020
  254. Sjors commented at 7:40 pm on July 17, 2020: member
    Rebased and hopefully made peace with AppVeyor
  255. hebasto commented at 4:06 pm on July 19, 2020: member
    Concept ACK.
  256. in configure.ac:1194 in 7588341b8a outdated
    1187@@ -1188,7 +1188,11 @@ fi
    1188 AX_BOOST_SYSTEM
    1189 AX_BOOST_FILESYSTEM
    1190 AX_BOOST_THREAD
    1191-AX_BOOST_PROCESS
    1192+
    1193+dnl Opt-in to boost-process
    


    hebasto commented at 4:40 pm on July 19, 2020:

    This part of the OP

    … use ./configure --without-boost-process to opt out…

    should be inverted, right?


    Sjors commented at 11:44 am on July 21, 2020:
    Fixed
  257. in configure.ac:1195 in 7588341b8a outdated
    1187@@ -1188,7 +1188,11 @@ fi
    1188 AX_BOOST_SYSTEM
    1189 AX_BOOST_FILESYSTEM
    1190 AX_BOOST_THREAD
    1191-AX_BOOST_PROCESS
    1192+
    1193+dnl Opt-in to boost-process
    1194+if test x$with_boost_process != x; then
    1195+  AX_BOOST_PROCESS
    1196+fi
    


    hebasto commented at 4:54 pm on July 19, 2020:
    7588341b8a88370ce3382799d739063965376301 May I suggest to use AS_IF macro for those reasons?
  258. hebasto approved
  259. hebasto commented at 5:21 pm on July 19, 2020: member

    ACK fe9a6b71f47f80d0a77d239948341ea168d600d0, tested on Linux Mint 20 (x86_64).

    nits:

  260. Sjors commented at 11:45 am on July 21, 2020: member
    Thanks @hebasto, I’ll apply those suggestions if I need to rebase again.
  261. DrahtBot added the label Needs rebase on Jul 28, 2020
  262. Sjors force-pushed on Jul 30, 2020
  263. Sjors commented at 11:02 am on July 30, 2020: member
    Rebased and addressed @hebasto’s comments.
  264. DrahtBot removed the label Needs rebase on Jul 30, 2020
  265. in depends/packages/boost.mk:36 in 344c53c22e outdated
    30@@ -31,7 +31,9 @@ $(package)_cxxflags_linux=-fPIC
    31 $(package)_cxxflags_android=-fPIC
    32 endef
    33 
    34+# Fix unused variable in boost_process, can be removed after upgrading to 1.72
    35 define $(package)_preprocess_cmds
    36+	sed -i.old "s/int ret_sig = 0;//" boost/process/detail/posix/wait_group.hpp && \
    


    hebasto commented at 12:54 pm on July 30, 2020:
    Is a tab character placed here intentionally?

    Sjors commented at 11:38 am on July 31, 2020:
    Just copy-pasted that line… Fixed, since I had to rebase again.
  266. hebasto approved
  267. hebasto commented at 12:56 pm on July 30, 2020: member
    re-ACK 7d88848793b10aafd530e6eb95fbdb19570f6819, only rebased (verified with git range-diff) and suggested changes implemented since the previous review.
  268. [depends] boost: patch unused variable in boost_process
    Co-Authored-By: fanquake <fanquake@gmail.com>
    8314c23d7b
  269. configure: add ax_boost_process
    Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org>
    929cda5470
  270. [build] make boost-process opt-in c47e4bbf0b
  271. [build] msvc: add boost::process
    * AppVeyor boost-process vcpkg package.
    * Tell Boost linter to ignore it
    * Add HAVE_BOOST_PROCESS for MSVC build (bitcoin_config.h)
    3c84d85f7d
  272. [doc] include Doxygen comments for HAVE_BOOST_PROCESS 32128ba682
  273. [ci] use boost::process
    Explictly opt-out on win64, in case the default changes.
    c17f54ee53
  274. [util] add RunCommandParseJSON 31cf68a3ad
  275. Sjors force-pushed on Jul 31, 2020
  276. hebasto approved
  277. hebasto commented at 12:00 pm on July 31, 2020: member
    re-ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, only rebased (verified with git range-diff) and removed an unintentional tab character since the previous review.
  278. Sjors commented at 1:34 pm on July 31, 2020: member
    I think the sanatizer failure is unrelated. Restarted. Log for reference: https://cirrus-ci.com/task/5039271566376960
  279. Sjors requested review from ryanofsky on Jul 31, 2020
  280. Sjors requested review from promag on Jul 31, 2020
  281. achow101 commented at 8:20 pm on July 31, 2020: member
    ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0
  282. meshcollider commented at 0:43 am on August 2, 2020: contributor
    Very light utACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, although I am not very confident with build stuff.
  283. in src/util/system.h:107 in 31cf68a3ad
    102+/**
    103+ * Execute a command which returns JSON, and parse the result.
    104+ *
    105+ * @param str_command The command to execute, including any arguments
    106+ * @param str_std_in string to pass to stdin
    107+ * @return parsed JSON
    


    promag commented at 0:51 am on August 2, 2020:
    nit, could mention it throws if command fails (failure return code) or JSON parser fails.
  284. promag commented at 1:05 am on August 2, 2020: member
    Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, don’t mind the nit.
  285. in src/test/system_tests.cpp:60 in 31cf68a3ad
    55+#ifdef WIN32
    56+        // Windows requires single quotes to prevent escaping double quotes from the JSON...
    57+        const UniValue result = RunCommandParseJSON("echo '{\"success\": true}'");
    58+#else
    59+        // ... but Linux and macOS echo a single quote if it's used
    60+        const UniValue result = RunCommandParseJSON("echo \"{\"success\": true}\"");
    


    ryanofsky commented at 3:48 pm on August 3, 2020:

    In commit “[util] add RunCommandParseJSON” (31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0)

    This code seems nonsensical, and I spent like an hour trying to figure out how it might be working. I think what is happening is that boost’s build_arg function is building this up into a vector of three strings:

    • echo
    • "{"success":
    • true}"

    and passing these as argv to execve, which I guess gets the job done but is not the most straightforward thing. Might be better to pass an argument vector, or a normal bash command line instead of a fake command line string that uses boost’s weird command line splitting code and escaping rules. This would just be for clarity though, since I think the current code probably should work.

    On #ifdef WIN32 code, I don’t see how that code is working, and I wonder it is actually running and passing on windows with appveyor, or is just broken and should be removed. Since we don’t seem to be enabling the shell (cmd.exe) as a boost process option, and since single quotes have no meaning on windows, and since windows doesn’t have an echo.exe binary, it seems impossible that this could be working.


    Sjors commented at 9:21 am on August 4, 2020:

    “boost’s weird command line splitting code and escaping rule” are what the user has to deal with when using -signer= in the followup PR, so it seems reasonable to have to test use that too.

    I fixed a failed AppVeyor tests with this #ifdef WIN32 line, so the code does run (and system_tests is in the log). Maybe it’s running in the power shell?

    The external signer Python tests used to run on AppVeyor in the followup PR, but it looks like we dropped functional tests from AppVeyor in April #18631 (#15382 proposes using a native Windows Travis instance).


    ryanofsky commented at 5:27 pm on August 4, 2020:

    re: #15382 (review)

    I think the current PR is ok, and any followup can happen later. This is kind of a mess, but just read details below if interested in what I think is wrong here.

    I think it is bad if an external signer option isn’t going to execute commands as straight shell commands the way -alertnotify -blocknotify and -walletnotify do and instead is going to use a strange, undocumented boost-shell splitting syntax, that doesn’t even try to try be posix compatible like python’s shlex.split().

    I was going to suggest the following change to stop using boost’s argument splitter and instead pass commands straight to the system shell the same way -alertnotify -blocknotify and -walletnotify do:

     0diff --git a/src/util/system.cpp b/src/util/system.cpp
     1index 066cfe88925..24f6869a871 100644
     2--- a/src/util/system.cpp
     3+++ b/src/util/system.cpp
     4@@ -1178,6 +1178,7 @@ UniValue RunCommandParseJSON(const std::string& str_command, const std::string&
     5     if (str_command.empty()) return UniValue::VNULL;
     6 
     7     bp::child c(
     8+        bp::shell,
     9         str_command,
    10         bp::std_out > stdout_stream,
    11         bp::std_err > stderr_stream,
    12diff --git a/src/test/system_tests.cpp b/src/test/system_tests.cpp
    13index a55145c7383..529fd74e472 100644
    14--- a/src/test/system_tests.cpp
    15+++ b/src/test/system_tests.cpp
    16@@ -57,7 +57,7 @@ BOOST_AUTO_TEST_CASE(run_command)
    17         const UniValue result = RunCommandParseJSON("echo '{\"success\": true}'");
    18 #else
    19         // ... but Linux and macOS echo a single quote if it's used
    20-        const UniValue result = RunCommandParseJSON("echo \"{\"success\": true}\"");
    21+        const UniValue result = RunCommandParseJSON("echo '{\"success\": true}'");
    22 #endif
    23         BOOST_CHECK(result.isObject());
    24         const UniValue& success = find_value(result, "success");
    

    But it turns out there is a bug in boost so this doesn’t work: https://github.com/boostorg/process/issues/95. The bug is caused by commit https://github.com/boostorg/process/commit/c31f69725c3160a7a0adce744ed9d4c0a1a4afbe which seems kind of insane and more alarming than the shell splitting code above. The joining code added in this commit is wrong and vulnerable to shell code injection. And the extra quotes added in the cmd_shell() function there just flat-out break boost’s shell option on posix.

    Separately, why the test above works on windows (thank you for the appveyor link) is still kind of a mystery, but I bet appveyor has some nonstandard echo.exe on the windows PATH, maybe part of git or mingw or msys or wsl, since current code is just going to invoke CreateProcess without a shell.

    In practice, I’m sure none of this matters at all, and not sure what I would suggest here. I think the ideal thing would be to use the bp::shell like the diff above and have bug https://github.com/boostorg/process/issues/95 fixed upstream. But the workaround used in https://github.com/boostorg/process/issues/95 is also pretty reasonable, and something we could adopt.

    I’m a little disappointed boost::process has these shell issues, since they seem like bad misunderstandings, and boost::process otherwise looks like a flexible and nicely-designed library.

  286. in configure.ac:1195 in c47e4bbf0b outdated
    1189@@ -1190,7 +1190,9 @@ fi
    1190 AX_BOOST_SYSTEM
    1191 AX_BOOST_FILESYSTEM
    1192 AX_BOOST_THREAD
    1193-AX_BOOST_PROCESS
    1194+
    1195+dnl Opt-in to boost-process
    1196+AS_IF([ test x$with_boost_process != x ], [ AX_BOOST_PROCESS ], [ ax_cv_boost_process=no ] )
    


    ryanofsky commented at 5:47 pm on August 4, 2020:

    In commit “[build] make boost-process opt-in” (c47e4bbf0b44f2de1278f9538124ec98ee0815bb)

    This seems ok, and I’m assuming it’s a temporary thing, but I think ideally you wouldn’t need to mess with library detection to build bitcoin with a feature turned off. I.e. instead of #if HAVE_BOOST_PROCESS in the code it would be better to have #if ENABLE_EXTERNAL_SIGNER. So if external signing wasn’t enabled, the boost process code would always be stripped out and not built, regardless of whether autoconf detected the library.

  287. in src/util/system.cpp:1194 in 31cf68a3ad
    1189+    stdin_stream.pipe().close();
    1190+
    1191+    std::string result;
    1192+    std::string error;
    1193+    std::getline(stdout_stream, result);
    1194+    std::getline(stderr_stream, error);
    


    ryanofsky commented at 5:59 pm on August 4, 2020:

    In commit “[util] add RunCommandParseJSON” (31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0)

    I think it would be better not to require output to be formatted on a single line. Also better to trigger an error if these is unexpected data after the first line:

     0--- a/src/util/system.cpp
     1+++ b/src/util/system.cpp
     2@@ -1188,10 +1188,8 @@ UniValue RunCommandParseJSON(const std::string& str_command, const std::string&
     3     }
     4     stdin_stream.pipe().close();
     5 
     6-    std::string result;
     7-    std::string error;
     8-    std::getline(stdout_stream, result);
     9-    std::getline(stderr_stream, error);
    10+    std::string result{std::istreambuf_iterator<char>(stdout_stream), {}};
    11+    std::string error{std::istreambuf_iterator<char>(stderr_stream), {}};
    12 
    13     c.wait();
    14     const int n_error = c.exit_code();
    
  288. ryanofsky approved
  289. ryanofsky commented at 6:17 pm on August 4, 2020: member
    Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0. I left some comments below that could be ignored or followed up later. The current change is clean and comprehensive.
  290. meshcollider merged this on Aug 5, 2020
  291. meshcollider closed this on Aug 5, 2020

  292. meshcollider removed this from the "Blockers" column in a project

  293. meshcollider moved this from the "PRs" to the "Done" column in a project

  294. sidhujag referenced this in commit 06bceb01d7 on Aug 5, 2020
  295. fanquake referenced this in commit 82127d27c9 on Aug 6, 2020
  296. sidhujag referenced this in commit d1dcb3c0ba on Aug 6, 2020
  297. Sjors deleted the branch on Aug 6, 2020
  298. Sjors referenced this in commit a5c05dfe11 on Dec 10, 2020
  299. fanquake referenced this in commit 1e4b1973a0 on Feb 15, 2021
  300. fanquake referenced this in commit 1b960f0780 on Feb 15, 2021
  301. fanquake referenced this in commit ffe3d65aaa on Feb 15, 2021
  302. fanquake referenced this in commit 7bf04e358a on Feb 17, 2021
  303. laanwj referenced this in commit 04336086d3 on Feb 17, 2021
  304. sidhujag referenced this in commit 5d5698c0ae on Feb 17, 2021
  305. kittywhiskers referenced this in commit 5038ef1ae0 on Aug 24, 2021
  306. kittywhiskers referenced this in commit d5cb98e763 on Aug 24, 2021
  307. kittywhiskers referenced this in commit bb610a6574 on Aug 24, 2021
  308. kittywhiskers referenced this in commit 175d271a8b on Aug 24, 2021
  309. kittywhiskers referenced this in commit a72edd4ccc on Aug 24, 2021
  310. kittywhiskers referenced this in commit 9ccab34c89 on Aug 24, 2021
  311. kittywhiskers referenced this in commit 27a5a7a787 on Aug 25, 2021
  312. kittywhiskers referenced this in commit b79ec3b7d4 on Aug 25, 2021
  313. kittywhiskers referenced this in commit 3e1a723ab5 on Aug 26, 2021
  314. kittywhiskers referenced this in commit 04f64088a0 on Aug 26, 2021
  315. kittywhiskers referenced this in commit 22003fa27d on Aug 26, 2021
  316. kittywhiskers referenced this in commit 309b7f54dc on Aug 27, 2021
  317. kittywhiskers referenced this in commit 03c291e31c on Aug 30, 2021
  318. kittywhiskers referenced this in commit e7b607bcf8 on Sep 1, 2021
  319. kittywhiskers referenced this in commit fe6007e70b on Sep 1, 2021
  320. kittywhiskers referenced this in commit 382580bf35 on Sep 2, 2021
  321. kittywhiskers referenced this in commit 9807be830f on Sep 3, 2021
  322. kittywhiskers referenced this in commit f001a564f6 on Sep 3, 2021
  323. kittywhiskers referenced this in commit f7ad75fabc on Sep 3, 2021
  324. kittywhiskers referenced this in commit ad7045047b on Sep 4, 2021
  325. linuxsh2 referenced this in commit 15e3ccfe5b on Sep 16, 2021
  326. linuxsh2 referenced this in commit 9a52ea9ee4 on Sep 16, 2021
  327. gades referenced this in commit 826d3db209 on May 2, 2022
  328. DrahtBot locked this on Aug 18, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-22 12:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me