The Multiprocess Bitcoin Design Document represents the long running ambition of splitting our existing single process application into separate node, GUI and wallet processes.
Since this document was written, we’ve introduced an entirely different approach: designing a new interface from scratch for a specific use case, in this case (Stratum v2) mining IPC clients.
Should we continue to pursue both approaches, or switch to the latter?
This ties into a larger discussion around whether we want to support, or even encourage, independent wallet and GUI implementations. If we do, then I think we should give them an interface that’s as ergonomic as possible. Based on my experience working on the Mining interface, that takes an iterative and incremental approach.
Making external clients consume our existing node, wallet and gui interfaces, does not seem like a good idea. Improving these interfaces to make them more ergonomic, requires simultaneously refactoring our GUI and wallet. This is a tedious process that’s hard to get reviewers for, and without user-visible improvements. Another problem with this approach is that it will be hard to get good feedback from actual third party implementations. They’d have to test something that’s complete, but hard to use. By the time they figure it out and get their application working, they’re not going to want to change it.
It’s much better to start with an interface that’s incomplete, but simple, and then expand it based on what clients actually need. Client implementers can fall back to RPC to work around the gaps, as we’ve seen SRI do with the Mining interface. Conversely, existing RPC users can gradually switch over to IPC and notice the performance gains.
I’m assuming our own wallet and GUI implementations aren’t going anywhere (I use them both). It remains a good idea to run them as separate processes. But there’s a trade-off: since we can’t stop people from building clients anyway, even if just for testing, we could get flooded by bug reports in the best case, or have a permanent maintenance burden in the worst case.
I think it’s worth considering a change of course: keep our wallet and our GUI in the same process. We present them as a convenience, and continue to take bugs seriously, while at the same time recommending power users with 24/7 servers to not build them, out of an abundance of caution.
Meanwhile we incrementally craft new interfaces and ship capnp files those. We start from narrow use cases (e.g. indexers, scriptPubKey watchers, some node control functions), eventually reaching the point where a full fledged wallet and GUI can be built.
Our own wallet and GUI can use these new interfaces too (internally). Crucially, these two efforts don’t block each other: interfaces can be designed without waiting for GUI refactors and the GUI can be refactored anytime after a new interface show up. As these refactors take place, “legacy” interface methods can be deleted incrementally, without having to worry about breaking external software (since we don’t ship canpnp files).
I suspect this refactoring process will be easier for the wallet than for the GUI, so eventually we might end up shipping a bitcoin-wallet that connects to our bitcoin-node, exactly as originally envisioned, but using an interface that was designed for external clients first.
It may turn out that some, or many, existing interface methods are already in great shape for external use. We could start with blank capnp files for existing interfaces and incrementally expose those methods. We also developed a good versioning convention while working on the Mining interface, so breaking changes are certainly possible. But it’s harder to change a method that’s used both by our internal code and by external clients, so we should carefully select these.
