[design] Pluggable message handlers and asynchronous operation #4382

Right now, there is a single ProcessMessages handler (which is somewhat pluggable through the use of boost::signals2), but it still implies:

• A single method needing to deal with everything.
• Only a single network handler thread (because it assumes everything needs cs_main anyway).
• No way to process messages asynchronously (we want to process everything from a single peer in-order, but there is no need why a pong reply to peer B's ping needs to wait while we're waiting for the hard disk to find a block in response to peer A's getdata).

So, what I would like is pluggable modules that can independently install handlers for specific network messages, that can have independent state, and can run independently and asynchronously, while still maintaining the guarantee that a single peer's messages are always processed in-order.

I'll try to post an idea for a design soon.

Seems like we get most of this for free, simply by using boost::asio's M-T socket handling? a la https://github.com/jgarzik/rpcsrv

It sounds like a more complicated design than the more common design pattern of simply servicing multiple sockets in parallel threads using boost::asio dispatch.

I'm not a fan of asio. Perhaps unnecessarily so, but all I've seen of it is wasteful resource usage and hard to read code.

And simply processing multiple sockets in parallel is not enough. If two sockets both have a 'block' message to be processed, having two threads will still only allow only one to work (as it needs cs_main, or whatever its successor is), even though there may be other threads that do have messages that can be processed without it.

Also, that's mostly an implementation issue - what I really want is a nice API so handling of P2P messages can be broken out into separate modules with their own state. The implementation of the base networking code that implements the API is still open.

After discussion with @theuni I'm closing this. There seems too much state shared between multiple messages handlers to separate them.

I believe this can still be revisited as a concept after the current network refactors, but as a layer on top instead.