Pause IBD during AssumeUTXO snapshot load #29993

issue Sjors openend this issue on April 29, 2024
  1. Sjors commented at 11:40 am on April 29, 2024: member

    Please describe the feature you’d like to see added.

    The loadtxoutset RPC call should cause IBD to pause until the snapshot is loaded and the snapshot chain is activated.

    When I start a fresh node and load a mainnet shapshot (#28553) any progress is completely buried in the log.

    More importantly, I get the strong impression, though haven’t properly tested this, that the IBD process slows down the snapshot load. This makes sense given the amount of I/O involved. It’s definitely slower than the process of creating the snapshot.

    On my AMD Ryzen 7950x machine the IBD reaches block 300,000 while the snapshot has only loaded 71.2% (where is triggers a cache flush). By the time 100% is loaded IBD is at 324,000. When the snapshot chainstate is saved IBD reaches 362,000.

    Describe the solution you’d like

    Pause IBD until the snapshot is loaded and the snapshot chain is activated.

    Describe any alternatives you’ve considered

    1. Do nothing
    2. Keep IBD going but at low priority (how???)

    Please leave any additional context

    No response

  2. Sjors added the label Feature on Apr 29, 2024
  3. Sjors commented at 1:30 pm on May 28, 2024: member

    I loaded a mainnet snapshot today with a pruned node with limited –dbcache=10000 on a 2019 MacBook Pro. This time I had the node sync for about an hour before loading the snapshot.

    At some point the log says:

    02024-05-28T12:38:11Z Cache size (10375257872) exceeds total space (10370521497)
    12024-05-28T12:38:11Z FlushSnapshotToDisk: flushing coins cache (10375 MB) started
    

    This flush took 50 minutes, while the chain synced from 392,803 to 417,312. It then continues loading coins from the snapshot.

  4. jamesob commented at 8:49 pm on May 28, 2024: member
    In an earlier version of the code, we would acquire cs_main during the duration of snapshot load/activation. Maybe worth going back to that?
  5. Sjors commented at 4:13 pm on May 29, 2024: member
    IIUC taking cs_main blocks all RPC requests and the GUI, and this process can take half an hour or so on a modernish laptop. So I would prefer a less overkill method of pausing IBD. During my own testing I just turned the network off, but that’s maybe not ideal either. Adding a few seconds delay before requesting new blocks might do the trick, while preventing our connections from timing out.


Sjors jamesob

Labels
Feature


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-22 06:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me