Crash upon RPC v1 connection in v28.0.0 #31041

1. 
   dr-orlovsky commented at 6:14 pm on October 6, 2024: none

   Is there an existing issue for this?

   • I have searched the existing issues

   Current behaviour

   On RPC v1 connection, after the block sync and mempool read is complete, a crash happens (see logs). I used mempool fork of electrs indexer which did an RPC request.

   Original report can be found here: #31039 (comment)

   Expected behaviour

   No crash is expected

   Steps to reproduce

   Run Bitcoin Core (I used machine with 8 cores, and 4GB or 8GB of memory)

   Relevant log output

    0Oct 06 17:54:18 core bitcoind[16532]: 2024-10-06T17:54:18Z initload thread exit
    1Oct 06 17:55:43 core systemd[1]: bitcoind.service: A process of this unit has been killed by the OOM killer.
    2░░ Subject: A process of bitcoind.service unit has been killed by the OOM killer.
    3░░ Defined-By: systemd
    4░░ Support: https://www.debian.org/support
    5░░ 
    6░░ A process of unit [@UNIT](/bitcoin-bitcoin/contributor/unit/) has been killed by the Linux kernel out-of-memory (OOM)
    7░░ killer logic. This usually indicates that the system is low on memory and that
    8░░ memory needed to be freed. A process associated with bitcoind.service has been determined
    9░░ as the best process to terminate and has been forcibly terminated by the
   10░░ kernel.
   11░░ 
   12░░ Note that the memory pressure might or might not have been caused by bitcoind.service.
   13Oct 06 17:55:43 core systemd[1]: bitcoind.service: Main process exited, code=killed, status=9/KILL
   14░░ Subject: Unit process exited
   15░░ Defined-By: systemd
   16░░ Support: https://www.debian.org/support
   17░░ 
   18░░ An ExecStart= process belonging to unit bitcoind.service has exited.
   19░░ 
   20░░ The process' exit code is 'killed' and its exit status is 9.
   21Oct 06 17:55:43 core systemd[1]: bitcoind.service: Failed with result 'oom-kill'.
   22░░ Subject: Unit failed
   23░░ Defined-By: systemd
   24░░ Support: https://www.debian.org/support
   25░░ 
   26░░ The unit bitcoind.service has entered the 'failed' state with result 'oom-kill'.
   27Oct 06 17:55:43 core systemd[1]: bitcoind.service: Consumed 3min 18.749s CPU time.
   28░░ Subject: Resources consumed by unit runtime
   29░░ Defined-By: systemd
   30░░ Support: https://www.debian.org/support
   31░░ 
   32░░ The unit bitcoind.service completed and consumed the indicated resources.
   33Oct 06 17:55:43 core systemd[1]: bitcoind.service: Scheduled restart job, restart counter is at 1.
   34░░ Subject: Automatic restarting of a unit has been scheduled
   35░░ Defined-By: systemd
   36░░ Support: https://www.debian.org/support
   37░░ 
   38░░ Automatic restarting of the unit bitcoind.service has been scheduled, as the result for
   

   How did you obtain Bitcoin Core

   Compiled from source

   What version of Bitcoin Core are you using?

   v28.0.0

   Operating system and version

   Debian Bookworm

   Machine specifications

   8 CPUs; 4GB RAM

2. dr-orlovsky renamed this:
   Crash due to inadequate memory allocation request upon RPC v1 connection in v28.0.0
   Crash upon RPC v1 connection in v28.0.0
   on Oct 6, 2024
3. 
   dr-orlovsky commented at 6:24 pm on October 6, 2024: none

   With the increase in memory to 8GB the crash has disappeared.
4. 
   maflcko commented at 8:13 am on October 7, 2024: member

   Steps to reproduce

   Run Bitcoin Core

   What are the exact steps to reproduce? What is the config? Which RPCs were called, in what order, and in what timing (parallel, sequential), …?

   Does it still happen after https://github.com/romanz/electrs/pull/1091 ?

5. 
   andrewtoth commented at 1:19 pm on October 7, 2024: contributor

   Does it still happen after https://github.com/romanz/electrs/pull/1091 ?

   That is for a different fork of electrs. For the mempool/electrs, you can see in the original logs in #31039 that it is already waiting for mempool to load before syncing.

6. 
   maflcko commented at 2:32 pm on October 7, 2024: member

   I see. It would be good to know what the batch request is looking like. Also, it would be good to confirm that this works on 27.x and is broken on 28.x. Also, it would be good to confirm that the system has enough memory to fit the whole batch request response (as json) into memory.
7. fanquake added this to the milestone 28.1 on Oct 22, 2024

Contributors
dr-orlovsky maflcko andrewtoth

Milestone
28.1