cli: add -usefile option #32399

1. 
   mzumsande commented at 10:42 pm on May 1, 2025: contributor

   Running large commands via bitcoin-cli (such as submitting a large tx or block) runs into the MAX_ARG_STRLEN limit, which is 128KB on many systems. (see e.g. https://trofi.github.io/posts/299-maximum-argument-count-on-linux-and-in-gcc.html for more info).

   This PR suggests to make it possible to bypass this limit by allowing to put the command and all of its args into a file. The functional test framework is then adjusted to automatically use this option (using a temporary file) if --usecli is specified and there is an arg that is too large. This might also be helpful for users outside of the functional tests (although there is always the option to use the rpc interface directly instead of bitcoin-cli).

   This is something many of our functional tests run into when executed with --usecli on master: (OSError: [Errno 7] Argument list too long: 'bitcoin-cli') The third commit enables several of them - there are a few more tests than those mentioned there, that could be enabled in a follow-up, but need some more adjustments beyond that.

2. 
   DrahtBot commented at 10:42 pm on May 1, 2025: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage & Benchmarks

   For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32399.

   Reviews

   See the guideline for information on the review process. A summary of reviews will appear here.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #32297 (bitcoin-cli: Add -ipcconnect option by ryanofsky)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

3. DrahtBot added the label Scripts and tools on May 1, 2025
4. cli: add -usefile option

   This allows to do rpc requests via cli when they would otherwise not
   be possible due to execve limits.

   71952f933f
5. mzumsande force-pushed on May 2, 2025
6. test: use -usefile for large rpc commands ec67dc328d
7. test: enable some functional tests with cli

   Also, the following tests (for which self.supports_cli = False was not
   set) will now work with --usecli:
   
   feature_fastprune.py
   feature_fee_estimation.py
   feature_reindex_readonly.py
   feature_taproot.py
   mempool_package_rbf.py
   p2p_net_deadlock.py
   p2p_tx_download.py
   rpc_packages.py

   991eb005c2
8. mzumsande force-pushed on May 2, 2025
9. 
   0xB10C commented at 9:59 am on May 2, 2025: contributor

   This might also be helpful for users outside of the functional tests (although there is always the option to use the rpc interface directly instead of bitcoin-cli).

   I’ve been using -stdin (e.g. cat block.hex | bitcoin-cli -stdin submitblock) for this. Would this work here too?

10. 
    mzumsande commented at 3:21 pm on May 2, 2025: contributor

    I’ve been using -stdin (e.g. cat block.hex | bitcoin-cli -stdin submitblock) for this. Would this work here too?

    Oh, I didn’t think of that - I tried it out and I think that would work too - that would make it simpler, no changes to bitcoin-cli necessary, will change to that approach.

11. 
    mzumsande commented at 3:26 pm on May 2, 2025: contributor

    Since this will only change the test framework now, I’ll just add it to #32290 - no extra PR needed anymore.
12. mzumsande closed this on May 2, 2025

    ---

Contributors
mzumsande DrahtBot 0xB10C

Labels
Scripts and tools