Stuck chainstate when utxo_snapshot.sh fails #27841

issue hazeycode openend this issue on June 9, 2023
  1. hazeycode commented at 0:04 am on June 9, 2023: contributor

    Is there an existing issue for this?

    • I have searched the existing issues

    Current behaviour

    IBD/chainstate gets stuck if utxo_snapshot.sh fails. No new blocks are downloaded. Restarting bitcoind does not help. Taking another snapshot at an earlier height or reindex is required to repair.

    utxo_snapshot.sh output:

    0Rewinding chain back to height 788440 (by invalidating 000000000000000000052ac0ce9b8f1a2e7691771e6386f770432c9443dc2af7); this may take a while
    1Generating UTXO snapshot...
    2error code: -8
    3error message:
    4/home/chris/Desktop/assumeutxo/test/utxo-788440.dat already exists. If you are sure this is what you want, move it out of the way first
    

    Note that moving the file or choosing another filename and rerunning does not fix. But taking another snapshot at an earlier height will.

    Expected behaviour

    If utxo_snapshot.sh fails, bitcoind should cancel rewind and recover.

    Also, if a filepath that already exists is passed to utxosnapshot.sh, it should early exit before rewinding. This could perhaps be considered a separate issue.

    Steps to reproduce

    1. Create a dummy file for the purpose of the test e.g. echo "some bytes so file is not empty" > ./utxo-788440.dat
    2. Run bitcoind and wait for it to sync
    3. Run utxosnapshot.sh with some blockheight and a filepath that already exists e.g. ./contrib/devtools/utxo_snapshot.sh 788440 ./utxo-788440.dat ./src/bitcoin-cli
    4. Wait for chain rewind to happen, snapshot to be generated and write to disk fail
    5. Observe bitcoind is now stuck with an invalid block at the rewind height

    Relevant log output

    02023-06-08T23:51:50Z [httpworker.3] UpdateTip: new best=000000000000000000014a9d36f6a19aca593f0c228ae1b1b6fe33361fa67be1 height=788440 version=0x2001e000 log2_work=94.162204 tx=832727241 date='2023-05-06T02:54:13Z' progress=0.987258 cache=790.3MiB(1245160txo)
    12023-06-08T23:51:50Z [httpworker.3] InvalidChainFound: invalid block=000000000000000000052ac0ce9b8f1a2e7691771e6386f770432c9443dc2af7  height=788441  log2_work=94.162217  date=2023-05-06T02:55:43Z
    22023-06-08T23:51:50Z [httpworker.3] InvalidChainFound:  current best=000000000000000000014a9d36f6a19aca593f0c228ae1b1b6fe33361fa67be1  height=788440  log2_work=94.162204  date=2023-05-06T02:54:13Z
    
    0Rewinding chain back to height 788440 (by invalidating 000000000000000000052ac0ce9b8f1a2e7691771e6386f770432c9443dc2af7); this may take a while
    1Generating UTXO snapshot...
    2error code: -8
    3error message:
    4/home/chris/Desktop/assumeutxo/test/utxo-788440.dat already exists. If you are sure this is what you want, move it out of the way first
    

    How did you obtain Bitcoin Core

    Compiled from source

    What version of Bitcoin Core are you using?

    v25.99.0-3dc6fd620e28

    Operating system and version

    Linux 6.3.1-arch2-1

    Machine specifications

    No response

  2. jamesob commented at 2:16 pm on June 9, 2023: contributor
    Thanks for the report. I think to solve this without a reindex, you just need to run reconsiderblock 000000000000000000052ac0ce9b8f1a2e7691771e6386f770432c9443dc2af7. We should probably amend the script to catch failures or interrupts and do this automatically.
  3. hazeycode commented at 3:54 pm on June 9, 2023: contributor
    Thanks @jamesob. I can confirm that reconsiderblock works as you expected. I’ll have a go at updating the script with error handing.
  4. hazeycode referenced this in commit 63a510f40f on Jun 9, 2023
  5. hazeycode referenced this in commit 3a1e3d6093 on Jun 9, 2023
  6. hazeycode referenced this in commit 1026068b26 on Jun 9, 2023
  7. hazeycode referenced this in commit ce10d50583 on Jun 9, 2023
  8. hazeycode referenced this in commit 5db4c8affa on Jun 9, 2023
  9. hazeycode referenced this in commit d7969fd81b on Jun 9, 2023
  10. hazeycode referenced this in commit 10d9bef004 on Jul 3, 2023
  11. ryanofsky closed this on Dec 4, 2023

  12. Julio-Rats referenced this in commit 1c8893bd1c on Jan 26, 2024
  13. PastaPastaPasta referenced this in commit e89c513828 on Oct 24, 2024
  14. PastaPastaPasta referenced this in commit 0aeb0b0be7 on Oct 24, 2024
  15. PastaPastaPasta referenced this in commit 8cd85d311f on Oct 24, 2024
  16. bitcoin locked this on Dec 3, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-19 03:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me