Substantial overhead caused by encoding messages for logging #215

1. 
   theuni commented at 8:24 pm on September 23, 2025: contributor

   I’m working on a multiprocess branch that passes a lot of data back and forth between server and client.

   When looking at a flamegraph, it’s clear that >50% of my client’s cpu time is spent on logging (encoding) sends. Likewise for the server with responses.

   The culprit is toString() from the params/results/request/response, which ends up in kj::_::encodeCEscapeImpl(), which is horribly slow.

   Current master: Fixed:

   As you can see, before removing the logging, 52% of my thread’s runtime is spent in serverInvoke (which calls foo.toString(), which calls structString(), which calls kj::_::encodeCEscapeImpl(). With the debug lines removed, serverInvoke is no longer visible in the flamegraph at all.

   I would imagine the sv2 implementation suffers from this as well, though I’m not sure how chatty it is, so I can’t speak to the potential impact.

   I’ll open a RFC PR which fixes the issue rather naively. A more robust solution would introduce logging categories similar to Bitcoin Core’s.

   Edit: See #216. Also, toned down the original title :)

2. 
   theuni commented at 9:08 pm on September 23, 2025: contributor

   Ping @Sjors, you might be seeing this as well.
3. theuni renamed this:
   Extreme overhead caused by encoding messages for logging
   Substantial overhead caused by encoding messages for logging
   on Sep 23, 2025
4. 
   Sjors commented at 8:02 am on September 24, 2025: member

   Do I understand correctly that this “only” impacts Bitcoin Core when running with -debug=ipc?
5. 
   theuni commented at 10:42 am on September 24, 2025: contributor

   @Sjors No, that’s the issue I’m afraid.

   The stringification happens in Core regardless of the selected logging level, and on the client even if no log callback is set.

6. 
   Sjors commented at 10:44 am on September 24, 2025: member

   Ah, ok that sounds more urgent then. I’ll go and review your fix.
7. ryanofsky closed this on Oct 2, 2025

   ---

8. fanquake referenced this in commit ec86e4336e on Oct 2, 2025
9. glozow referenced this in commit db4bde0b03 on Oct 14, 2025

Contributors
theuni Sjors