Add bitcoin-{node,gui} to release binaries for IPC #31802

pull Sjors wants to merge 17 commits into bitcoin:master from Sjors:2025/02/ipc-yea changing 101 files +6126 −98
  1. Sjors commented at 4:04 pm on February 5, 2025: member

    Have depends make libmultiprocess by default, which has the effect of including it in the release binaries. Except for Windows and OpenBSD which are not supported yet, the latter due to a fairly trivial upstream issue.

    The initial main use case is to enable experimental support for the Mining IPC interface. A working example of a Stratum v2 Template Provider client using this interface can be found here: https://github.com/Sjors/bitcoin/pull/48.

    Additionally the bitcoin-node and bitcoin-gui are added to Maintenance.cmake.

    Builds on #30975 (which originally tried to do this) and #31741.

    See #31756 for discussion of when this should happen. See also #31098 for any remaining issues specific to the Mining interface. Will keep this draft for now.

    Guix hashes:

     0f084fccde78e2fa19c2a357cda441abe7db29cc7abaf5927a2939a7a01f49724  guix-build-7d448ee28aed/output/aarch64-linux-gnu/SHA256SUMS.part
 1425e9118ce2adc401b1ca708c8e88eb7157c57c2888899f45104cceb6f3977b5  guix-build-7d448ee28aed/output/aarch64-linux-gnu/bitcoin-7d448ee28aed-aarch64-linux-gnu-debug.tar.gz
 275b4700e2421d8957acd2d618e4bc444894e592a33692cde6b474baeb3f7ed86  guix-build-7d448ee28aed/output/aarch64-linux-gnu/bitcoin-7d448ee28aed-aarch64-linux-gnu.tar.gz
 38ca6fe3b56ae17ba293b1b6c4f3c37cd81d8f9790e16b5ec8391a8c65a74f711  guix-build-7d448ee28aed/output/arm-linux-gnueabihf/SHA256SUMS.part
 45cc328841280fef5c95ec2c43a42bd947c6754527126af25b376b6d8f1cd116d  guix-build-7d448ee28aed/output/arm-linux-gnueabihf/bitcoin-7d448ee28aed-arm-linux-gnueabihf-debug.tar.gz
 554f41cd079cefe8f24f752efb2e70fae046de73a8ca7af0fae6bb315e089ce17  guix-build-7d448ee28aed/output/arm-linux-gnueabihf/bitcoin-7d448ee28aed-arm-linux-gnueabihf.tar.gz
 64504530e00830ef2ea2b317e89da1d6755117a38fe77a26fc2dbf3fc496aa3d1  guix-build-7d448ee28aed/output/arm64-apple-darwin/SHA256SUMS.part
 718ee44f02a65952cb026b9b3d8e3579e43e2746a0c3ccc6e68be2ad78cb7a307  guix-build-7d448ee28aed/output/arm64-apple-darwin/bitcoin-7d448ee28aed-arm64-apple-darwin-codesigning.tar.gz
 86086b14ffb06e34766d6e74521a098c1764a930af003ae22f83e65871de661ba  guix-build-7d448ee28aed/output/arm64-apple-darwin/bitcoin-7d448ee28aed-arm64-apple-darwin-unsigned.tar.gz
 94831118c0216ceaad53b2b81cec825429f4b6213e12d6cadbecf98166dbe24d9  guix-build-7d448ee28aed/output/arm64-apple-darwin/bitcoin-7d448ee28aed-arm64-apple-darwin-unsigned.zip
1083cff34cdf4393398abfcf9dffd81b4ee8b0687208c07eaf27b6b1a661085104  guix-build-7d448ee28aed/output/dist-archive/bitcoin-7d448ee28aed.tar.gz
11ff243e9bffc49f0ef5a5edd5023e9b201fb83fb06e2d26e4d0463e1f7bb63e62  guix-build-7d448ee28aed/output/powerpc64-linux-gnu/SHA256SUMS.part
1268c4e7ce5363656f6f5e55bd5487cb4d6b693d65443270e4b76e227537d7428e  guix-build-7d448ee28aed/output/powerpc64-linux-gnu/bitcoin-7d448ee28aed-powerpc64-linux-gnu-debug.tar.gz
136801afdd86d9a1be8e71e2dfd62623480f34d516daad53e20b21eea46714a762  guix-build-7d448ee28aed/output/powerpc64-linux-gnu/bitcoin-7d448ee28aed-powerpc64-linux-gnu.tar.gz
14690e721e5bcdc6f612dfa8f6e0a09f69e414c842a6df3d7cec6b71e1f8b0dd63  guix-build-7d448ee28aed/output/riscv64-linux-gnu/SHA256SUMS.part
15131b0ea8654fc86c1cf288b23b94ba2e40dc975d43725a262c5cb1fb85b69a46  guix-build-7d448ee28aed/output/riscv64-linux-gnu/bitcoin-7d448ee28aed-riscv64-linux-gnu-debug.tar.gz
1617e724b9d1d7dcdfe55034002de0eab4ddd1112413879c675365fd2b928b9c91  guix-build-7d448ee28aed/output/riscv64-linux-gnu/bitcoin-7d448ee28aed-riscv64-linux-gnu.tar.gz
17e4269993afcd7696df02b4b34ea49efbc736f193511d8473b64f12bc7dbee49e  guix-build-7d448ee28aed/output/x86_64-apple-darwin/SHA256SUMS.part
1854ce620b8c8b29f725f4855630ed4eb5cb49bf496c3cf0f5c79b3e3c901fd496  guix-build-7d448ee28aed/output/x86_64-apple-darwin/bitcoin-7d448ee28aed-x86_64-apple-darwin-codesigning.tar.gz
19569788fd9fd33feca26a59235a3287980a9a807b95d389c9b536d374a596b431  guix-build-7d448ee28aed/output/x86_64-apple-darwin/bitcoin-7d448ee28aed-x86_64-apple-darwin-unsigned.tar.gz
20e064baa33d4b10f68b1ebb84865c7c45626658888739f19f35342e368d6c67a0  guix-build-7d448ee28aed/output/x86_64-apple-darwin/bitcoin-7d448ee28aed-x86_64-apple-darwin-unsigned.zip
21ad4c8eb7ed3342279d2955bdc8672e9225cf094396793b1128c925e7395f15ce  guix-build-7d448ee28aed/output/x86_64-linux-gnu/SHA256SUMS.part
22230baf7d889b6180e2be299823c6fc8c93f01e33bb5b2165874e12c1c2de58ea  guix-build-7d448ee28aed/output/x86_64-linux-gnu/bitcoin-7d448ee28aed-x86_64-linux-gnu-debug.tar.gz
23d4474d05f743b619583988446ae2a376f5881f87e1f2f44f89acc504fe91d7f2  guix-build-7d448ee28aed/output/x86_64-linux-gnu/bitcoin-7d448ee28aed-x86_64-linux-gnu.tar.gz
248a9ba0886e948045c19670371f211194f187ae6f3d0b7a9783a6a5f4f5debf17  guix-build-7d448ee28aed/output/x86_64-w64-mingw32/SHA256SUMS.part
251e1bfa1abd2b9040209723eff3dab3be75c7bfd38785911e16130fcdecd10936  guix-build-7d448ee28aed/output/x86_64-w64-mingw32/bitcoin-7d448ee28aed-win64-codesigning.tar.gz
26da5460252827e2822774a4dd38169bb96b5ac16b306390abcbbaf15bb16c49de  guix-build-7d448ee28aed/output/x86_64-w64-mingw32/bitcoin-7d448ee28aed-win64-debug.zip
278e3de209aa412352156ff51bdd59d07b335827dee1ccd7235d8add2ca7f6e008  guix-build-7d448ee28aed/output/x86_64-w64-mingw32/bitcoin-7d448ee28aed-win64-setup-unsigned.exe
2813455905b884ad61d685d8099a042c7eac0658d50a2b133fc6335aa5dc43aba3  guix-build-7d448ee28aed/output/x86_64-w64-mingw32/bitcoin-7d448ee28aed-win64-unsigned.zip
  2. DrahtBot commented at 4:04 pm on February 5, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31802.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #32086 (Shuffle depends instructions and recommend modern make for macOS by Sjors)
    • #32054 (cmake, guix: Skip building tests in subtrees for releases by hebasto)
    • #31375 (multiprocess: Add bitcoin wrapper executable by ryanofsky)
    • #31349 (ci: detect outbound internet traffic generated while running tests by vasild)
    • #31282 (refactor: Make node_id a const& in RemoveBlockRequest by maflcko)
    • #30997 (build: Switch to Qt 6 by hebasto)
    • #30595 (kernel: Introduce initial C header API by TheCharlatan)
    • #28792 (Embed default ASMap as binary dump header file by fjahr)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Needs rebase on Feb 6, 2025
  4. Sjors force-pushed on Feb 7, 2025
  5. Sjors force-pushed on Feb 7, 2025
  6. sipa commented at 9:03 pm on February 7, 2025: member

    Some chatter from IRC:

     017:21:41 < darosior> It might be confusing to release both a bitcoin-wallet utility and a bitcoin-wallet binary as part of multiprocess?
 117:24:08 < darosior> We could rename the utility, but then it would be nice to at least have one deprecation cycle. Given recent momentum i estimate it's possible we might release multiprocess in 
 2                     30.0, which means if we want to deprecate the bitcoin-wallet utility name we should do it.. now?
 317:33:31 < sipa> bitcoin-wallet-util ?
 4...
 523:38:57 < _aj_> darosior: bitcoin-wallet-process, bitcoin-gui-process, etc? it's multi *-process!
 603:04:34 < Sjors[m]> darosior: at the moment there is no wallet binary if were to enable multiprocess. That won't happen until [#19460](/bitcoin-bitcoin/19460/).
 7...
 803:05:18 < Sjors[m]> Or maybe already in [#10102](/bitcoin-bitcoin/10102/)
 9...
1003:05:50 < Sjors[m]> In any case [#31802](/bitcoin-bitcoin/31802/) only adds bitcoin-node and bitcoin-gui.
11...
1203:07:19 < Sjors[m]> Though if we want to rename the utility eventually, it's always better to do it early.
  7. Sjors commented at 10:58 am on February 8, 2025: member
    @sipa I opened #31827
  8. onlinesipahimithu commented at 1:26 am on February 9, 2025: none
    GOOD lesson from
  9. bitcoin deleted a comment on Feb 9, 2025
  10. Sjors force-pushed on Feb 10, 2025
  11. Sjors force-pushed on Feb 13, 2025
  12. DrahtBot removed the label Needs rebase on Feb 13, 2025
  13. Sjors force-pushed on Feb 17, 2025
  14. Sjors force-pushed on Feb 17, 2025
  15. DrahtBot added the label Needs rebase on Feb 20, 2025
  16. ryanofsky referenced this in commit 83e40d3b52 on Feb 24, 2025
  17. ryanofsky referenced this in commit 8619f03ec2 on Feb 24, 2025
  18. ryanofsky referenced this in commit cbb7b41c20 on Feb 24, 2025
  19. fanquake referenced this in commit 01f7715766 on Feb 25, 2025
  20. fanquake referenced this in commit ba0a4391ff on Feb 25, 2025
  21. pablomartin4btc referenced this in commit bd2453d134 on Feb 26, 2025
  22. pablomartin4btc referenced this in commit 75d5d235a6 on Feb 26, 2025
  23. Sjors force-pushed on Feb 27, 2025
  24. DrahtBot removed the label Needs rebase on Feb 27, 2025
  25. ngn999 referenced this in commit fd12bfb763 on Feb 27, 2025
  26. DrahtBot added the label Needs rebase on Mar 5, 2025
  27. Sjors force-pushed on Mar 11, 2025
  28. DrahtBot removed the label Needs rebase on Mar 11, 2025
  29. DrahtBot added the label Needs rebase on Mar 12, 2025
  30. Sjors force-pushed on Mar 12, 2025
  31. DrahtBot removed the label Needs rebase on Mar 12, 2025
  32. DrahtBot added the label Needs rebase on Mar 14, 2025
  33. Sjors force-pushed on Mar 17, 2025
  34. DrahtBot removed the label Needs rebase on Mar 17, 2025
  35. DrahtBot added the label Needs rebase on Mar 18, 2025
  36. Squashed 'src/ipc/libmultiprocess/' content from commit f35df6bdc536 
    git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: f35df6bdc536b068597559d4ab470dab9cff7cfc
    05b153349d
  37. Merge commit '05b153349dcccdc4eafa2a2a578eccec21faad9e' as 'src/ipc/libmultiprocess' 6a13467772
  38. scripted-diff: s/WITH_MULTIPROCESS/ENABLE_IPC/ in cmake 
    Rename WITH_MULTIPROCESS to ENABLE_IPC, because ENABLE_IPC is a more accurate
name for the feature. It controls whether the src/ipc/ directory is built and
whether IPC features like -ipcbind, -ipcconnect, and -ipcfd are available. It
does NOT currently enable multiprocess features which are implemented in #10102
building on top of the IPC features. It will also no longer (as of the next
commit), control whether a find_package call is made so the "WITH_" prefix is
also inappropriate.

-BEGIN VERIFY SCRIPT-
git grep -l WITH_MULTIPROCESS | xargs sed -i s/WITH_MULTIPROCESS/ENABLE_IPC/g
-END VERIFY SCRIPT-
    002088280f
  39. cmake: Support building with libmultiprocess subtree 
    When ENABLE_IPC option is on, build with libmultiprocess subtree and
`add_subdirectory(src/ipc/libmultiprocess)` instead of external package
and `find_package(Libmultiprocess)` by default.

Behavior can be toggled with `WITH_EXTERNAL_LIBMULTIPROCESS` option. Using a
subtree should be more convenient for most bitcoin developers, but using an
external package is more convenient for developing in the libmultiprocess
repository.

The `WITH_EXTERNAL_LIBMULTIPROCESS` option is also used to avoid needing to
changing the depends build here. But in later commits, the depends build is
switched to use the add_subdirectory build as well.

Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
    bc6ca56720
  40. cmake: Fix ctest mptest "Unable to find executable" errors 
    This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR #30975 which enables multiprocess build
option in more CI jobs. In that PR, several jobs fail due to the mptest
executable not being built by default, as reported
https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2623801480
    9dc41f9a86
  41. cmake: Fix warnings from boost headers 
    This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR #30975 which enables multiprocess build
option in more CI jobs. In that PR, the "macOS 14 native no depends job" fails
due to warnings in boost headers treated as errors, reported in
https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2623801480 and
https://github.com/chaincodelabs/libmultiprocess/issues/138
    b5813cf553
  42. cmake: Fix clang-tidy "no input files" errors 
    This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR #30975 which enables multiprocess build
option in more CI jobs. In that PR, clang-tidy job fails due to missing
generated example files as reported
https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2627116832

Different fixes were suggested
https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2627152623 and
https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2627403382

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
    c813d3c643
  43. cmake: Fix fuzzer "multiple definition of `main'" errors 
    This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR #30975 which enables multiprocess build
option in more CI jobs. Specifically this makes it possible to use the
BUILD_FOR_FUZZING and ENABLE_IPC options together easily.

The change just updates the cmake build so when -DSANITIZERS=fuzzer is
specified, the fuzz test binary is built with -fsanitize=fuzzer (so it can use
libFuzzer's main function), and libraries are built with
-fsanitize=fuzzer-no-link (so they can be linked into other executables with
their own main functions).

Previously when -DSANITIZERS=fuzzer was specified, -fsanitize=fuzzer was
applied to ALL libraries and executables.  This was inappropriate because it
made it impossible to build any executables other than the fuzz test executable
without triggering link errors:

- "multiple definition of `main'"
- "undefined reference to `LLVMFuzzerTestOneInput'"

if they depended on any libraries instrumented for fuzzing.

This was especially a problem when the ENABLE_IPC option was set because the
mpgen code generator could not be built, so nothing else that depended on
generated sources including the fuzz test binary could be built either.
    e3d940dbc6
  44. doc: Update documentation to explain libmultiprocess subtree 
    Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
    77c13c8a58
  45. lint: Add exclusions for libmultiprocess subtree 
    Without this change linter produces errors about:

- Use of std::filesystem the libmultiprocess example program.
- Use of locale-dependent functions in example program, in the build time code
  generator, and in the runtime library for debug logging.
- Include guards not beginning with BITCOIN_
    91d7112281
  46. depends, moveonly: split up int_get_build_id function 
    Move parts of the int_get_build_id into a new int_get_build_properties
function. There is no change in behavior. This just organizes assignments
better so some build properties can be used to help compute build ids in the
next commit.
    6cb7f7c8cd
  47. depends: Switch libmultiprocess packages to use local git subtree 
    With newly introduced libmultiprocess subtree, there's no need for depends
system to download and track changes to the upstream repository.

Note that adding the libmultiprocess subtree does not allow dropping
libmultiprocess packages from the depends build, because libmultiprocess
includes a code generation tool called mpgen, and in cross-compiled builds,
bitcoin core's cmake build system doesn't have access to a native toolchain and
can't build mpgen itself, so the depends system (or the native environment if
not using depends) needs to supply it.
    61dfe2d925
  48. depends: remove non-native libmultiprocess build 
    Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
    4e265debdc
  49. ci: build multiprocess for all depends jobs 
    Except:
1. i686, DEBUG (changed to "no multiprocess")
2. Windows due to lack of support
    041f78c128
  50. ci: enable multiprocess for non-depends jobs 
    Install capnp where needed.

The bitcoin-node binary is built on all platforms which have
multiprocess enabled, but for functional tests it's only used in
the macOS native (no depends) and CentOS native (depends) jobs.
    f50f6e33b9
  51. build: add bitcoin-{node,gui} to Maintenance.cmake bdb04cdfd0
  52. build: depends makes libmultiprocess by default 
    Except for Windows and OpenBSD.
    7d448ee28a
  53. Sjors force-pushed on Mar 18, 2025
  54. DrahtBot removed the label Needs rebase on Mar 18, 2025


Sjors DrahtBot sipa onlinesipahimithu

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-03-23 21:13 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me