[RFC] Post-0.9 modularization #3440

1. 
   laanwj commented at 1:42 pm on December 18, 2013: member

   I would like to propose modularization of Bitcoin Core. This will make it easier for different people to to experiment with and implement different features without over-burdening the basic code.

   The main code will only consist of a full P2P node implementation with RPC server to control it (similar to –disable-wallet build now), which can be extended with optional modules.

   Example modules:

   • wallets: the current wallet, but could also be a simple read-only wallet, super-HD wallet with alternative storage, …
   • internalminer: the current internal miner will be an optional module
   • notifications: launch a script (current), ZeroMQ (#2415), websocket (#3388), …
   • unauthenticated HTTP REST interface (#2844), …

   Modules exist in their own directory and have one entry point that is called at startup that can register RPC calls, HTTP path handlers, as well as hook into certain events at startup, shutdown, or node events (through CWalletInterface).

   They can do this through the ‘module interface’ which will replace and extend the current UI interface (in ui_interface.h). Like the current UI interface this will be a global structure with boost::signals.

   Initially all modules will be built in the bitcoin/bitcoin repository, and which ones are built can be configured through configure flags. Which ones are initialized can be specified through bitcoind command line or bitcoin.conf.

   Later on, building modules externally as well as dynamic loading of modules could be added but this requires a stabilized API first (and maybe security measures like code signing).

2. 
   sipa commented at 2:23 pm on December 18, 2013: member

   ACK ACK ACK
3. 
   jgarzik commented at 4:00 pm on December 18, 2013: contributor

   The first step should be a lib-ification of the core. There is plenty of code in there that, e.g., ancillary utilities can use but is not relevant to a node implementation.
4. 
   laanwj commented at 4:50 pm on December 18, 2013: member

   @jgarzik I’m not sure I completely understand what you mean with lib-ification of the core. Maybe to make a library from the block chain / utxo / consensus stuff, so that it can be used by other software that is not a node? That is a different project from what I’m proposing here, but could be useful too.
5. 
   sipa commented at 4:53 pm on December 18, 2013: member

   One complication is modularizing both notification systems and wallets, as the former requires an instance of the second typically. There may need to be a second signals set for events emitted by wallets.
6. 
   luke-jr commented at 4:55 pm on December 18, 2013: member

   This is essentially what I’ve been pushing for since 2011, so ACK… I’d make the RPC server a module too, though (it already is now, I think?)
7. 
   laanwj commented at 5:24 pm on December 18, 2013: member

   @sipa Agreed. The wallets/keystores already have a few signals (currently used by the GUI), so moving these to some common interface class and letting modules know about registered/unregistered wallets would solve this @luke-jr That would be possible. A module could provide ‘a way to submit commands’, such as JSON-over-zeromq as alternative to the current JSON RPC HTTP server. Only the RPC command table really needs to stay in the main part, to make it possible for modules to register commands in the first place.
8. 
   esbullington commented at 6:23 pm on April 29, 2014: contributor

   Not sure what the status of this is, but I think it’s a very good idea. I’m very eager to see the much-discussed ZeroMQ API integrated into the code base, and it looks like its best shot would be through this type of modular interface.

   To me, having the module system entirely decoupled from the rest of the code, with JSON (or protobuf or whatever serialization) over ZeroMQ providing the interface, would offer a lot of possibilities, particularly with respect to scaling. And it’s relatively simple to extend ZeroMQ with websockets, rpc, and/or http.

   So is modularization still on the table?

9. 
   gmaxwell commented at 6:34 pm on April 29, 2014: contributor

   Take care not to split too far, building with and without is a full configuration which really needs to be tested— maybe there is a nasty crash built with or without some feature or combination. And so it might not be worth it for something like the internal miner, but the general idea is obviously the right thing to do.
10. laanwj added this to the milestone 0.11.0 on Nov 17, 2014
11. laanwj removed this from the milestone 0.10.0 on Nov 17, 2014
12. jonasschnelli referenced this in commit 6828277076 on Feb 4, 2015
13. jonasschnelli referenced this in commit 7bd633c39a on Feb 4, 2015
14. laanwj removed this from the milestone 0.11.0 on May 1, 2015
15. laanwj closed this on Feb 9, 2016

    ---

16. MarcoFalke referenced this in commit c2bcb99c1d on Jun 18, 2020
17. sidhujag referenced this in commit e593102316 on Jul 7, 2020
18. DrahtBot locked this on Feb 15, 2022

Contributors
laanwj sipa jgarzik luke-jr esbullington gmaxwell

Labels
Brainstorming Refactoring