In-memory UTXO cache is flushed too frequently when pruning #19742

issue andronoob openend this issue on August 17, 2020
  1. andronoob commented at 5:31 am on August 17, 2020: none

    Is your feature request related to a problem? Please describe. I’m experimenting assumevalid=0 (full validation) by running two bitcoin-qt on my laptop, one of which is -connecting to 127.0.0.1. In hope that my hard drive would wear as little as possible (also to accelerate the syncing process as much as possible), I set dbcache to 10240 MiB, then pointed blocksdir to a ramdisk. Then I found Bitcoin Core didn’t work as my expectation. The files under chainstate directory were still read/written frequently. I think these lines from debug.log should explain what was going on:

    02020-08-16T17:29:01Z UpdateTip: new best=000000000000066cccef1be790592e559ec4f1cd18f8683554a6bcecb87f8f97 height=154467 version=0x00000001 log2_work=67.223948 tx=1904617 date='2011-11-23T09:49:28Z' progress=0.003424 cache=40.7MiB(276733txo)
    12020-08-16T17:29:01Z UpdateTip: new best=00000000000009879303cae0f79181c45eecf749eecd56976c369a546ba7d78e height=154468 version=0x00000001 log2_work=67.223991 tx=1904624 date='2011-11-23T09:49:41Z' progress=0.003424 cache=40.7MiB(276741txo)
    22020-08-16T17:29:01Z UpdateTip: new best=0000000000000aa936ad49eb0f7f3342fd8f193960c7fac1571b0625d5aed9de height=154469 version=0x00000001 log2_work=67.224033 tx=1904633 date='2011-11-23T09:55:22Z' progress=0.003424 cache=40.7MiB(276730txo)
    32020-08-16T17:29:01Z Pre-allocating up to position 0xd00000 in rev00005.dat
    42020-08-16T17:29:01Z Prune: UnlinkPrunedFiles deleted blk/rev (00002)
    52020-08-16T17:29:01Z FlushStateToDisk: write coins cache to disk (276837 coins, 42709kB) started
    62020-08-16T17:29:02Z FlushStateToDisk: write coins cache to disk (276837 coins, 42709kB) completed (0.59s)
    72020-08-16T17:29:02Z UpdateTip: new best=0000000000000bf0a84c8f47fa804644fab66081f4d1207009e09ffe8b1dd272 height=154470 version=0x00000001 log2_work=67.224076 tx=1904752 date='2011-11-23T10:08:52Z' progress=0.003424 cache=6.9MiB(0txo)
    82020-08-16T17:29:02Z UpdateTip: new best=0000000000000823d28d3053a071d12e57072b93733a2e57047dff4aac495a6e height=154471 version=0x00000001 log2_work=67.224119 tx=1904756 date='2011-11-23T10:10:42Z' progress=0.003424 cache=6.9MiB(15txo)
    92020-08-16T17:29:02Z UpdateTip: new best=0000000000000b8732b07e326bde0a79f91e4808a5683a77c0fc85728a4ca06b height=154472 version=0x00000001 log2_work=67.224162 tx=1904777 date='2011-11-23T10:16:37Z' progress=0.003424 cache=6.9MiB(154txo)
    

    The log shows that in-memory UTXO cache is flushed too frequently.

    I then found nothing to control the cache flushing behavoir.

    Previously I had once done a full sync with similar settings (except that pruning was not enabled, and the blockchain was downloaded from the P2P network, rather than localhost), if I recalled correctly, the cache didn’t flush until I finally shut down the full node.

    Describe the solution you’d like Provide an option to control how often the UTXO cache should be flushed, so that the user can take his/her own responsibility for consequences (loss of syncing progress) of system crash/power loss etc.

    It’s also well known that consumer HDDs are flooded with sh*tty DM-SMR models. I think it’s really worthwhile to trade such “crash tolerance” for performance improvement if there’s plenty of RAM.

    There’s also a relatively high chance for a user to take out his/her old spare laptop to run the full node, so that even if more and more PCs come with SSD only in the future, it’s still meaningful to take care of this issue.

    Describe alternatives you’ve considered

    Additional context

  2. andronoob added the label Bug on Aug 17, 2020
  3. achow101 commented at 5:53 am on August 17, 2020: member
    This is a known issue with pruning. See #11315
  4. MarcoFalke renamed this:
    In-memory UTXO cache is flushed too frequently
    In-memory UTXO cache is flushed too frequently when pruning
    on Aug 17, 2020
  5. andronoob commented at 7:16 am on August 17, 2020: none
    I’m sorry. I’ll close this since it’s duplicate.
  6. andronoob closed this on Aug 17, 2020

  7. DrahtBot locked this on Feb 15, 2022


andronoob achow101

Labels
Bug


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-23 21:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me