doc: Add multiprocess design doc #28978

  28 | +Alternately, you can install [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) packages on your system, and just run `./configure --enable-multiprocess` without using the depends system. The configure script will be able to locate the installed packages via [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/). See [Installation](https://github.com/chaincodelabs/libmultiprocess/blob/master/doc/install.md) section of the libmultiprocess readme for install steps. See [build-unix.md](build-unix.md) and [build-osx.md](build-osx.md) for information about installing dependencies in general.
  29 | +
  30 | +## Usage
  31 | +
  32 | +`bitcoin-node` is a drop-in replacement for `bitcoind`, and `bitcoin-gui` is a drop-in replacement for `bitcoin-qt`, and there are no differences in use or external behavior between the new and old executables. But internally after [#10102](https://github.com/bitcoin/bitcoin/pull/10102), `bitcoin-gui` will spawn a `bitcoin-node` process to run P2P and RPC code, communicating with it across a socket pair, and `bitcoin-node` will spawn `bitcoin-wallet` to run wallet code, also communicating over a socket pair. This will let node, wallet, and GUI code run in separate address spaces for better isolation, and allow future improvements like being able to start and stop components independently on different machines and environments.
  33 | +[#19160](https://github.com/bitcoin/bitcoin/pull/19160) adds a new `bitcoin-node` `-ipcbind` option and an `bitcoind-wallet` `-ipcconnect` option to allow new wallet processes to connect to an existing node process.

---

  29 | +
  30 | +## Usage
  31 | +
  32 | +`bitcoin-node` is a drop-in replacement for `bitcoind`, and `bitcoin-gui` is a drop-in replacement for `bitcoin-qt`, and there are no differences in use or external behavior between the new and old executables. But internally after [#10102](https://github.com/bitcoin/bitcoin/pull/10102), `bitcoin-gui` will spawn a `bitcoin-node` process to run P2P and RPC code, communicating with it across a socket pair, and `bitcoin-node` will spawn `bitcoin-wallet` to run wallet code, also communicating over a socket pair. This will let node, wallet, and GUI code run in separate address spaces for better isolation, and allow future improvements like being able to start and stop components independently on different machines and environments.
  33 | +[#19160](https://github.com/bitcoin/bitcoin/pull/19160) adds a new `bitcoin-node` `-ipcbind` option and an `bitcoind-wallet` `-ipcconnect` option to allow new wallet processes to connect to an existing node process.
  34 | +[#19161](https://github.com/bitcoin/bitcoin/pull/19161) adds a new `bitcoin-gui` `-ipcconnect` option to allow new GUI processes to connect to an existing node process.

---

 217 | +
 218 | +- Separating indexes from `bitcoin-node`, and running indexing code in separate processes.
 219 | +- Enabling wallet processes to listen for JSON-RPC requests on their own ports instead of needing the node process to listen and forward requests to them.
 220 | +- Automatically generating `.capnp` files from C++ interface definitions (see [Interface Definition Maintenance](#interface-definition-maintenance))
 221 | +- Simplifying and stabilizing interfaces (see [Interface Stability](#interface-stability))
 222 | +- Adding sandbox features, restricting subprocess access to resources and data (see [https://eklitzke.org/multiprocess-bitcoin](https://eklitzke.org/multiprocess-bitcoin)).

---

 163 | +## Design Considerations
 164 | +
 165 | +### Selection of Cap’n Proto
 166 | +The choice to use [Cap’n Proto](https://capnproto.org/) for IPC was primarily influenced by its support for passing object references and managing object lifetimes, which would have to be implemented manually with a framework that only supported plain requests and responses like [gRPC](https://grpc.io/). The support is especially helpful for passing callback objects like `std::function` and supporting bidirectional communication.
 167 | +
 168 | +The choice to use an RPC framework at all instead of a custom protocol was neccesitated by the size of Bitcoin Core internal interfaces which consist of around 150 methods that pass complex data structures and are called in complicated ways (in parallel, from callbacks that can be nested and stored). Writing a custom protocol to wrap these complicated interfaces would be too much work, akin to writing a new RPC framework.

---

 195 | +
 196 | +1. **Initiation in bitcoin-wallet**
 197 | +   - The wallet process calls the `getBlockHash` method on a `Chain` object. This method is defined as a virtual function in [`src/interfaces/chain.h`](../../src/interfaces/chain.h).
 198 | +
 199 | +2. **Translation to Cap’n Proto RPC**
 200 | +   - The call to `getBlockHash` is translated into a Cap’n Proto RPC call. This translation is handled by code automatically generated by the `mpgen` tool, based on the [`chain.capnp`](../../src/ipc/capnp/chain.capnp)` file in [`src/ipc/capnp/`](../../src/ipc/capnp/).

---

 203 | +   - The generated `Chain` subclass in `bitcoin-wallet` populates a Cap’n Proto request with the `height` parameter.
 204 | +   - The request is sent to the `bitcoin-node` process.
 205 | +
 206 | +4. **Handling in bitcoin-node**
 207 | +   - Upon receiving the request, the `bitcoin-node` process reads the `height` value from the request.
 208 | +   - It then calls the `getBlockHash` method on its local `Chain` object with the provided `height`.

---

 106 | +
 107 | +## Proposed Architecture
 108 | +
 109 | +The new architecture divides the existing code into three specialized executables:
 110 | +
 111 | +- `bitcoin-node`: Manages the node, indexes, and JSON-RPC server.

---

  96 | +  - [References](#references)
  97 | +- [Acknowledgements](#acknowledgements)
  98 | +
  99 | +## Introduction
 100 | +
 101 | +The Bitcoin Core software, central to the Bitcoin network, has historically employed a monolithic architecture. The existing design has integrated functionality like P2P network operations, wallet management, and a GUI into a single executable. While effective, it has limitations in flexibility, security, and scalability. This project introduces changes that transition Bitcoin Core to a more modular architecture. It aims to enhance security, improve usability, and facilitate maintenance and development of the software.

---

 220 | +- Automatically generating `.capnp` files from C++ interface definitions (see [Interface Definition Maintenance](#interface-definition-maintenance)).
 221 | +- Simplifying and stabilizing interfaces (see [Interface Stability](#interface-stability)).
 222 | +- Adding sandbox features, restricting subprocess access to resources and data (see [https://eklitzke.org/multiprocess-bitcoin](https://eklitzke.org/multiprocess-bitcoin)).
 223 | +- Using Cap'n Proto's support for [other languages](https://capnproto.org/otherlang.html), such as [Rust](https://github.com/capnproto/capnproto-rust), to allow code written in other languages to call Bitcoin Core C++ code, and vice versa (see [How to rustify libmultiprocess? #56](https://github.com/chaincodelabs/libmultiprocess/issues/56)).
 224 | +
 225 | +## Conclusion

---

 151 | +- **Bootstrapping IPC Connections**: It provides functions for starting new IPC connections, specifically binding generated client and server classes for an initial `interfaces::Init` interface (defined in [`src/interfaces/init.h`](../../src/interfaces/init.h)) to a UNIX socket. This initial interface has methods returning other interfaces that different Bitcoin-Core modules use to communicate after the bootstrapping phase.
 152 | +- **Asynchronous I/O and Thread Management**: The library is also responsible for managing I/O and threading. Particularly, it ensures that IPC requests never block each other and that new threads on either side of a connection can always make client calls. It also manages worker threads on the server side of calls, ensuring that calls from the same client thread always execute on the same server thread (to avoid locking issues and support nested callbacks).
 153 | +
 154 | +### Type Hooks in [`src/ipc/capnp/*-types.h`](../../src/ipc/capnp/)
 155 | +- **Custom Type Conversions**: In [`src/ipc/capnp/*-types.h`](../../src/ipc/capnp/), multiple function overloads of two `libmultiprocess` C++ functions, `mp::CustomReadField` and `mp::CustomBuildFields`, are defined. These functions are used for customizing the conversion of specific C++ types to and from Cap’n Proto types.
 156 | +- **Handling Special Cases**: The `mpgen` tool and `libmultiprocess` library can convert most C++ types to and from Cap’n Proto types automatically, including interface types, primitive C++ types, standard C++ types like `std::vector`, `std::set`, `std::map`, `std::tuple`, and `std::function`, as well as simple C++ structs that consist of afforementioned types and whose fields correspond 1:1 with Cap’n Proto struct fields. For other types, `*-types.h` files provide custom code to convert between C++ and Cap’n Proto data representations.

---

 171 | +
 172 | +The IPC mechanism is deliberately isolated from the rest of the codebase so less code has to be concerned with IPC.
 173 | +
 174 | +Building Bitcoin Core with IPC support is optional, and the same code can be compiled to either run in the same process or separate processes. The build system also ensures Cap’n Proto library headers can only be used within the [`src/ipc/capnp/`](../../src/ipc/capnp/) directory, not in other parts of the codebase.
 175 | +
 176 | +The libmultiprocess runtime is designed to place as few constraints as possible on IPC interfaces and make IPC calls act like normal function calls. Method arguments, return values, and exceptions are automatically serialized and sent between processes. Object references and `std::function` arguments are automatically tracked and mapped to allow invoked code to call back into invoking code at any time. And there is a 1:1 threading model where any every client thread has a corresponding server thread responsible for executing incoming calls from that thread (there can be multiple calls from the same thread due to callbacks), without blocking, and holding the same thread-local variables and locks so behavior is the same whether IPC is used or not.

---

 177 | +
 178 | +### Interface Definition Maintenance
 179 | +
 180 | +The choice to maintain interface definitions and C++ type mappings as `.capnp` files in the [`src/ipc/capnp/`](../../src/ipc/capnp/) was mostly done for convenience, and probably something that could be improved in the future.
 181 | +
 182 | +In the current design, class names, method names, and parameter names are duplicated between C++ interfaces in [`src/interfaces/`](../../src/interfaces/) and Cap’n Proto files in [`src/ipc/capnp/`](../../src/ipc/capnp/). While this keeps C++ interface headers simple and free of references to IPC, it is a maintence burden because it means inconsistencies between C++ declarations and Cap’n Proto declarations will result in compile errors. (Static type checking ensures these are not runtime errors.)

---

  10 | +
  11 | +The `-debug=ipc` command line option can be used to see requests and responses between processes.
  12 | +
  13 | +## Installation
  14 | +
  15 | +The multiprocess feature requires [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) as dependencies. A simple way to get starting using it without installing these dependencies manually is to use the [depends system](../depends) with the `MULTIPROCESS=1` [dependency option](../depends#dependency-options) passed to make:

---

 189 | +
 190 | +## Example Use Cases and Flows
 191 | +
 192 | +### Retrieving a Block Hash
 193 | +
 194 | +Let’s walk through an example where the `bitcoin-wallet` process requests the hash of a block at a specific height from the `bitcoin-node` process. This example demonstrates the practical application of the IPC mechanism, specifically the interplay between C++ method calls and Cap’n Proto-generated RPC calls.

---

 258 | +
 259 | +5. **Response and Return**
 260 | +   - The `getBlockHash` method of the generated `Chain` client subclass in `bitcoin-wallet` which sent the request now receives the response.
 261 | +   - It extracts the block hash value from the response, and returns it to the original caller.
 262 | +
 263 | +<table><tr><td>

---

  15 | +The multiprocess feature requires [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) as dependencies. A simple way to get started using it without installing these dependencies manually is to use the [depends system](../depends) with the `MULTIPROCESS=1` [dependency option](../depends#dependency-options) passed to make:
  16 | +
  17 | +```
  18 | +cd <BITCOIN_SOURCE_DIRECTORY>
  19 | +make -C depends NO_QT=1 MULTIPROCESS=1
  20 | +CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure

---

  92 | +
  93 | +### C++ Client Subclasses in Generated Code
  94 | +- In the generated code, we have C++ client subclasses that inherit from the abstract classes in [`src/interfaces/`](../../src/interfaces/). These subclasses are the workhorses of the IPC mechanism.
  95 | +- They implement all the methods of the interface, marshalling arguments into a structured format, sending them as requests to the IPC server via a UNIX socket, and handling the responses.
  96 | +- These subclasses effectively mask the complexity of IPC, presenting a familiar C++ interface to developers.
  97 | +- Internally, the client subclasses generated by the `mpgen` tool wrap [client classes generated by Cap'n Proto](https://capnproto.org/cxxrpc.html#clients), and use them to send IPC requests.

---

 159 | +
 160 | +In the current design, class names, method names, and parameter names are duplicated between C++ interfaces in [`src/interfaces/`](../../src/interfaces/) and Cap’n Proto files in [`src/ipc/capnp/`](../../src/ipc/capnp/). While this keeps C++ interface headers simple and free of references to IPC, it is a maintenance burden because it means inconsistencies between C++ declarations and Cap’n Proto declarations will result in compile errors. (Static type checking ensures these are not runtime errors.)
 161 | +
 162 | +An alternate approach could use custom [C++ Attributes](https://en.cppreference.com/w/cpp/language/attributes) embedded in interface declarations to automatically generate `.capnp` files from C++ headers. This has not been pursued because parsing C++ headers is more complicated than parsing Cap’n Proto interface definitions, especially portably on multiple platforms.
 163 | +
 164 | +In the meantime, the developer guide [Internal interface guidelines](developer-notes.md#internal-interface-guidelines) can provide guidance on keeping interfaces consistent and functional and avoiding compile errors.

---

 254 | +
 255 | +4. **Handling in bitcoin-node**
 256 | +   - Upon receiving the request, the Cap'n Proto dispatching code in the `bitcoin-node` process calls the `getBlockHash` method of the `Chain` [server class](#c-server-classes-in-generated-code).
 257 | +   - The server class is automatically generated by the `mpgen` tool from the [`chain.capnp`](https://github.com/ryanofsky/bitcoin/blob/pr/ipc/src/ipc/capnp/chain.capnp) file in [`src/ipc/capnp/`](../../src/ipc/capnp/).
 258 | +   - The `getBlockHash` method of the generated `Chain` server subclass in `bitcoin-wallet` receives a Cap’n Proto request object with the `height` parameter, and calls the `getBlockHash` method on its local `Chain` object with the provided `height`.
 259 | +   - When the call returns, it encapsulates the return value in a Cap’n Proto response, which it sends back to the `bitcoin-wallet` process,

---

 289 | +
 290 | +- **Cap’n Proto interface**: A set of methods defined in Cap’n Proto to facilitate structured communication between different software components.
 291 | +
 292 | +- **Cap’n Proto struct**: A structured data format used in Cap’n Proto, similar to structs in C++, for organizing and transporting data across different processes.
 293 | +
 294 | +- **client class (in generated code)**: A C++ class generated from a Cap’n Proto interface which inherits from a Bitcoin core abstract class, and implements each virtual method to send IPC requests to another process. (see also [components section](#c-client-subclasses-in-generated-code))

---