cmake: Move internal binaries from bin/ to libexec/ #31679

pull ryanofsky wants to merge 3 commits into bitcoin:master from ryanofsky:pr/libexec changing 9 files +59 −10
  1. ryanofsky commented at 5:01 pm on January 17, 2025: contributor

    This moves binaries not intended to be called directly by users from the bin/ directory to the libexec/ directory in cmake installs and binary releases. After this change, the binaries remaining in bin/ are:

    • bitcoin
    • bitcoin-cli
    • bitcoind
    • bitcoin-qt
    • bitcoin-tx
    • bitcoin-util
    • bitcoin-wallet

    And the binaries that are moved to libexec/ are:

    • bench_bitcoin
    • bitcoin-chainstate
    • bitcoin-gui
    • bitcoin-node
    • test_bitcoin
    • test_bitcoin-qt

    Removing these esoteric and not generally useful binaries from bin/ and from the system PATH should make it easier for typical users to find and identify the binaries which are actually useful.

    Advanced users can still run all the binaries in libexec/ and the bitcoin wrapper executable introduced in #31375 is also able to locate binaries regardless of whether they are installed in bin/ or libexec/.


    This PR is part of the process separation project.

  2. DrahtBot commented at 5:01 pm on January 17, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31679.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK Sjors
    Concept ACK TheCharlatan, jonatack, ismaelsadeeq

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31349 (ci: detect outbound internet traffic generated while running tests by vasild)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • noticable -> noticeable [correct spelling]

    drahtbot_id_4_m

  3. DrahtBot added the label Build system on Jan 17, 2025
  4. DrahtBot added the label CI failed on Jan 17, 2025
  5. DrahtBot commented at 7:52 am on January 18, 2025: contributor
    CI failed with [17:52:38.033] /ci_container_base/ci/test/03_test_script.sh: line 154: /ci_container_base/ci/scratch/out/bin/test_bitcoin: No such file or directory
  6. luke-jr commented at 3:25 pm on January 23, 2025: member
    libexec is usually used for binaries run by programs rather than people. I don’t think most of the ones you’re moving fit that, and should remain in bin.
  7. Sjors commented at 7:46 am on January 24, 2025: member

    Run by people is a higher bar than “not typically useful”, but could make sense.

    If there are no binaries for which it currently makes sense to put them in libexec, since they’re all called by people, then it seems this PR has be based on #31375.

  8. ryanofsky commented at 5:57 pm on January 28, 2025: contributor

    The idea that an executable placed in libexec/ should be called by other executables is a good rule of thumb but not a requirement of some kind. (And if it were a requirement, it would be satisfied by #31375.)

    The technical difference between binaries installed to bin/ and binaries installed to libexec/ is that binaries in bin/ get exposed on the PATH while binaries in libexec/ don’t.

    There is nothing wrong with binaries in libexec/ being documented and called used by users and packagers and other tools and scripts. Not every executable that a user could conceivably call needs to be present in PATH.

    Packages like postfix have manpages for internal executables installed in libexec/ (anvil, bounce, smtpd, etc). Git subcommands can be called manually from libexec and have manpages of their own, too. I would say postfix and git are pretty much archetypes of packages designed around unix principles and there is nothing novel about installing low-level executables in libexec/.

    With this PR, two sets of binaries are installed to libexec/ if they have been enabled in cmake:

    1. multiprocess binaries bitcoin-node and bitcoin-gui
    2. testing binaries bench_bitcoin test_bitcoin test_bitcoin-qt.

    I can’t think of a reason why multiprocess binaries should be installed on PATH, but if someone wants to make that case I’m ready to hear it. As an alternative to this PR, they could be installed in bin/ with different names like bitoin-node-multiprocess or to a completely new folder.

    For the test and bench binaries I just think it’s confusing and not useful to install these on the PATH. I think this is true today and will be especially true after #31375.

  9. ryanofsky force-pushed on Jan 30, 2025
  10. ryanofsky commented at 3:05 am on January 30, 2025: contributor
    Updated 2a952710661a1990ed1c24b0ebd16de8ac0df87c -> 47a872236e070814ad74922d3f8a653e1c6af968 (pr/libexec.1 -> pr/libexec.2, compare) to fix CI error
  11. DrahtBot removed the label CI failed on Jan 30, 2025
  12. DrahtBot added the label Needs rebase on Feb 12, 2025
  13. ryanofsky force-pushed on Feb 13, 2025
  14. ryanofsky commented at 6:48 pm on February 13, 2025: contributor
    Rebased 47a872236e070814ad74922d3f8a653e1c6af968 -> d2ceb2e0735a2c8343f8316b55fac55323aba62c (pr/libexec.2 -> pr/libexec.3, compare) due to conflict with #31834
  15. DrahtBot removed the label Needs rebase on Feb 13, 2025
  16. DrahtBot added the label Needs rebase on Feb 14, 2025
  17. ryanofsky force-pushed on Feb 14, 2025
  18. ryanofsky commented at 3:19 pm on February 14, 2025: contributor
    Rebased d2ceb2e0735a2c8343f8316b55fac55323aba62c -> 45bfd97ec7c9991f41673d79b01277bfd940e64e (pr/libexec.3 -> pr/libexec.4, compare) due to conflict with #31844
  19. DrahtBot removed the label Needs rebase on Feb 14, 2025
  20. TheCharlatan commented at 7:14 pm on April 8, 2025: contributor
    Concept ACK
  21. jonatack commented at 4:37 am on April 9, 2025: member

    Concept ACK

    Would need a release note?

  22. ryanofsky commented at 1:34 pm on April 14, 2025: contributor

    Would need a release note?

    Thanks! Added release note and files.md documention in new commits.


    Added 2 commits 45bfd97ec7c9991f41673d79b01277bfd940e64e -> bda7626e987ce3d7287f72e6989dedde2c4ab666 (pr/libexec.4 -> pr/libexec.5, compare) with release notes and documentation

  23. TheCharlatan commented at 9:16 pm on April 15, 2025: contributor

    I think some installation packagers run test_bitcoin as part of their installation recipes, or at least I found one such example: https://github.com/epiccurious/bitcoin-core-node-builder/pull/164/files. @hebasto also polled usage of test_bitcoin on social media: https://x.com/hhebasto/status/1729879307768738070. Clearly there aren’t many people doing this, but are we ok with breaking the workflows of the few people that are? You mention that:

    For the test and bench binaries I just think it’s confusing and not useful to install these on the PATH. I think this is true today and will be especially true after #31375.

    What is confusing about keeping them where they are for now? Maybe there is some utility in testing your platform before actually running bitcoin on it.

  24. DrahtBot added the label Needs rebase on May 7, 2025
  25. cmake: Move internal binaries from bin/ to libexec/
    This moves binaries not intended to be called directly by users from the `bin/`
    directory to the `libexec/` directory in cmake installs and binary releases.
    After this change, the binaries remaining in `bin/` are:
    
    - bitcoin
    - bitcoin-cli
    - bitcoind
    - bitcoin-qt
    - bitcoin-tx
    - bitcoin-util
    - bitcoin-wallet
    
    And the binaries that are moved to `libexec/` are:
    
    - bench_bitcoin
    - bitcoin-chainstate
    - bitcoin-gui
    - bitcoin-node
    - test_bitcoin
    - test_bitcoin-qt
    
    Removing these esoteric and not generally useful binaries from `bin/` and from
    the system PATH should make it easier for typical users to find and identify the
    binaries which are actually useful.
    
    Advanced users can still run all the binaries in `libexec/` and the `bitcoin`
    wrapper executable introduced in #31375 is also able to locate binaries
    regardless of whether they are installed in `bin/` or `libexec/`.
    eded935be7
  26. doc: Add release notes describing libexec/ binaries 5e89320949
  27. doc: Add description of installed files to files.md d253fa9ebe
  28. ryanofsky force-pushed on May 29, 2025
  29. ryanofsky commented at 9:45 pm on May 29, 2025: contributor
    Rebased bda7626e987ce3d7287f72e6989dedde2c4ab666 -> d253fa9ebe4305118417a02aa72cc06810662ac6 (pr/libexec.5 -> pr/libexec.6, compare) due to conflict with #28710
  30. DrahtBot removed the label Needs rebase on May 29, 2025
  31. Sjors commented at 5:49 am on May 30, 2025: member

    Just #28710, no conflict with #31375?

    Concept ACK

  32. ismaelsadeeq commented at 11:17 am on May 30, 2025: member
    Concept ACK
  33. Sjors commented at 4:04 pm on May 30, 2025: member

    tACK d253fa9ebe4305118417a02aa72cc06810662ac6

    I tested on top of af227a0e7d9754623d36925c56599f9b921f5cb3 from #31802 on macOS 15.5 and 13.7.6, as well as Windows 11 (not impacted).

    The bitcoin test command is perhaps a bit too well hidden. Maybe it’s better to drop the help command and put a bit more emphasis on the --all option:

    0Commands:
    1  gui [ARGS]     Start GUI, equivalent to running 'bitcoin-qt [ARGS]' or 'bitcoin-gui [ARGS]'
    2  ...
    3  tx ...
    4
    5Use 'bitcoin --help --all' for additional commands, e.g. for running tests.
    

    That last “e.g.” might address @TheCharlatan’s concern:

    Maybe there is some utility in testing your platform before actually running bitcoin on it.

  34. DrahtBot requested review from TheCharlatan on May 30, 2025
  35. DrahtBot requested review from jonatack on May 30, 2025
  36. DrahtBot requested review from ismaelsadeeq on May 30, 2025

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-05-31 06:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me