Issue for discussion about ways multiprocess functionality could be packaged and released. Trying to figure out which of the 3 release options described below makes the most sense.
Binaries
One goal of the multiprocess project has been to provide minimal binaries that only include node, wallet, or gui code, which spawn or connect to other processes as needed to provide other functionality.
The idea implemented in #10102 is to have 3 binaries:
bitcoin-node- contains node code and libraries (leveldb), and no wallet or gui codebitcoin-wallet- contains wallet code and libraries (sqlite), and no node or gui codebitcoin-gui- contains gui code and libraries (qt), and no node or wallet code
Which are built differently than current binaries:
bitcoind- contains node code+libraries (leveldb) and wallet code+libraries (sqlite)bitcoin-qt- contains node code+libraries (leveldb), wallet code+libraries (sqlite), and gui code+libraries (qt)
Original release plan
My original idea implementing this was for there to be a separate multiprocess bitcoin release with the multiprocess binaries.
The main release would be unchanged, including existing bitcoind and bitcoin-qt binaries, but there would be a separate release containing bitcoin-node, bitcoin-wallet and bitcoin-gui binaries. The main release would be available in the normal place like bitcoincore.org/bin/bitcoin-core-27.1/, and the multiprocess release would be available alongside, at bitcoincore.org/bin/bitcoin-core-multiprocess-27.1/ or someplace similar. Then, after a release cycle or two, the multiprocess release could replace the main release if it was working well.
But this approach has some drawbacks, namely that it would be a burden for maintainers to create separate releases and make them accessible, and that it could be confusing for users to encounter two releases with different binaries. So it doesn’t seem like this approach is optimal (at least not for general users, maybe it would be ok for miners using the IPC mining interface).
Release options overview
Taking a step back, it seems like there are 3 possible options for releasing multiprocess functionality:
- Side-release: create a separate multiprocess release for multiprocess binaries, as described above.
- Side-binaries: add multiprocess binaries (
bitcoin-node,bitcoin-wallet,bitcoin-gui) to existing releases alongside existing binaries (bitcoindandbitcoin-qt). - Combined binaries: add multiprocess functionality to existing
bitcoindandbitcoin-qtbinaries that can be enabled/disabled with runtime arguments.
Release options comparison
From discussion in recent PR’s and offline, it seems like approach (3) might be favored, but it is worth considering pros and cons of the three approaches.
-
Side-release
- Pros:
- Opt-in: Keeps original binaries unchanged for users who don’t want multiprocess functionality.
- Minimal binaries: Node binary contains only node code not wallet code, gui binary contains no node or wallet code.
- Cons:
- Extra work: It’s more work for maintainers and signers to create an extra set of packages and make them accessible.
- Complexity: Posting a separate release could be confusing for users.
- Pros:
-
Side-binaries:
- Pros:
- Opt-in and Minimal binaries: same as previous
- Easy to implement: Implemented in #30975 and basically requires no changes other than flipping cmake options.
- Cons:
- Confusing: This would probably be the most confusing approach for users. They would see one release with
bitcoind,bitcoin-qt,bitcoin-node,bitcoin-wallet,bitcoin-guibinaries and not be able to easily understand how they are intended to be used.
- Confusing: This would probably be the most confusing approach for users. They would see one release with
- Potential mitigations:
-
We would probably put multiprocess binaries in another folder (like
multiprocess/bin/instead ofbin/) so they are clearly labeled and aren’t confused with the default binaries. -
We might want to rethink our approach to packaging binaries to begin with. We are already shipping a substantial collection of binaries:
bitcoin-cli,bitcoind,bitcoin-qt,bitcoin-tx,bitcoin-util,bitcoin-wallet, andtest_bitcoinwhich users are expected to call individually, and are probably already somewhat confusing. By contrast, git ships dozens of binaries and nobody is confused by them because they live in a libexec directory and are wrapped with singlegitcommand that finds and executes the right ones. Maybe takinggitas inspiration we could have add a unifiedbitcoincommand that looks like:0bitcoin gui [OPTIONS] # Start a gui 1bitcoin daemon [OPTIONS] # Start a daemon 2bitcoin rpc [OPTIONS] COMMAND # Call an rpc 3bitcoin wallet [OPTIONS] COMMAND # Access a wallet 4bitcoin test [OPTIONS] # Run tests 5bitcoin help 6# ... and more for bitcoin-util, bitcoin-tx ...
-
- Pros:
-
Combined binaries:
- Pros:
- Compatibility: Users could download and invoke
bitcoindandbitcoin-qtbinaries as before, not know they support IPC, not use IPC unless it’s switched on by default, and there would be runtime options to turn it on or off.
- Compatibility: Users could download and invoke
- Cons:
- Bigger binaries and more dependencies: Binaries will both contain node, wallet, and IPC code. There will be no non-IPC binaries users can deploy and run separately if they don’t use IPC, and no node binary users can deploy and run separately if they don’t use the wallet, or want to run the wallet and node on different hosts.
- Requires code changes: Will require some thinking about how to add runtime options to #10102 to control whether
bitcoindshould spawn separate wallet process or run wallet code internally, and similarly whetherbitcoin-guishould use a separate node process or run node code in the same process. Will also require updating the build system.
- Pros:
As mentioned, approach (3) seems favored so far, though personally I like approach (2), especially if we could have a unified bitcoin command.
Creating this issue to get feedback and document any decisions we make.
