Rebuilding block database after corruption behaves strangely

GSPP commented at 12:57 pm on August 16, 2016: none

After clicking OK I get:

Which appears to be a bug. When I now click OK Bitcoin Core crashes:

After restarting it I get:

At this point nothing changes when I restart again.

The log says:

2016-08-16 12:52:42 Bitcoin version v0.12.1 (2016-04-11 13:01:43 +0200) 2016-08-16 12:52:42 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1 2016-08-16 12:52:43 GUI: “registerShutdownBlockReason: Successfully registered: Bitcoin Core didn’t yet exit safely…” 2016-08-16 12:52:44 Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010) 2016-08-16 12:52:44 Default data directory C:\Users\x\AppData\Roaming\Bitcoin 2016-08-16 12:52:44 Using data directory C:\Users\x\AppData\Roaming\Bitcoin 2016-08-16 12:52:44 Using config file C:\Users\x\AppData\Roaming\Bitcoin\bitcoin.conf 2016-08-16 12:52:44 Using at most 125 connections (2048 file descriptors available) 2016-08-16 12:52:44 Using 8 threads for script verification 2016-08-16 12:52:44 scheduler thread start 2016-08-16 12:52:44 HTTP: creating work queue of depth 16 2016-08-16 12:52:44 Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcuser for rpcauth auth generation. 2016-08-16 12:52:44 HTTP: starting 4 worker threads 2016-08-16 12:52:44 Using wallet wallet.dat 2016-08-16 12:52:44 init message: Verifying wallet… 2016-08-16 12:52:44 CDBEnv::Open: LogDir=C:\Users\x\AppData\Roaming\Bitcoin\database ErrorFile=C:\Users\x\AppData\Roaming\Bitcoin\db.log 2016-08-16 12:52:44 Cache configuration: 2016-08-16 12:52:44 * Using 128.0MiB for block index database 2016-08-16 12:52:44 * Using 232.0MiB for chain state database 2016-08-16 12:52:44 * Using 664.0MiB for in-memory UTXO set 2016-08-16 12:52:44 init message: Loading block index… 2016-08-16 12:52:44 Opening LevelDB in C:\Users\x\AppData\Roaming\Bitcoin\blocks\index 2016-08-16 12:52:44 Opened LevelDB successfully 2016-08-16 12:52:44 Using obfuscation key for C:\Users\x\AppData\Roaming\Bitcoin\blocks\index: 0000000000000000 2016-08-16 12:52:44 Opening LevelDB in C:\Users\x\AppData\Roaming\Bitcoin\chainstate 2016-08-16 12:52:44 Opened LevelDB successfully 2016-08-16 12:52:44 Using obfuscation key for C:\Users\x\AppData\Roaming\Bitcoin\chainstate: ceb7ff593b2a09a4 2016-08-16 12:52:44 LoadBlockIndexDB: last block file = 0 2016-08-16 12:52:44 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=0, size=0, heights=0…0, time=1970-01-01…1970-01-01) 2016-08-16 12:52:44 Checking all blk files are present… 2016-08-16 12:52:44 LoadBlockIndexDB: transaction index enabled 2016-08-16 12:52:44 Initializing databases… 2016-08-16 12:52:44 Pre-allocating up to position 0x1000000 in blk00000.dat

Hope this helps find the problem.

I now delete chainstate and database and db.log. The app can now start and syncs with the network. For some reason it does not find the blocks that are still on disk (I did not touch the blocks folder when deleting files). Something about them must have gotten corrupted previously (bug?). I would have liked if I was warned that the existing blocks could not be used instead of silently ignoring them (potential improvement).

jonasschnelli commented at 7:51 am on August 17, 2016: contributor

@sipa: isn’t this solved with the new reindex / reindex-chainstate change?

jonasschnelli added the label Block storage on Aug 17, 2016

MarcoFalke added the label Windows on Aug 20, 2016

MarcoFalke commented at 5:54 pm on August 21, 2016: member

I could not reproduce on fedora. @GSPP Can you check if this is reproducible always? (You can try to set a different datadir and -testnet and then corrupt the db on purpose)

GSPP commented at 10:50 am on September 10, 2016: none

I’ve got a lead for what the assertion is about. I just restored a saved chainstate folder. The folder should match the contents of the blocks folder. I don’t think I copied it over from a different Bitcoin Core instance because I know that’s not supported. But maybe I created the backup before I learned that… Sorry that I cannot provide more reliable info here.

sipa commented at 10:52 am on September 10, 2016: member

The chainstate and blocks directories are independent, and you are allowed to copy both from different sources.

However, the blocks database cannot be older than the chainstate. This means that the latest block seen by the chainstate-sourcing instance must be present in the blocks directory.

GSPP commented at 12:00 pm on September 10, 2016: none

I’m certain that the blocks were newer.

Ayms commented at 11:46 am on February 6, 2017: none

Windows v0.13.0, this happened twice after two electricity cuts: first message above (corrupted, do you want…), OK, then it started to rebuild the blocks and chainstate from the beginning, another electricity cut during this process after a few days, and again, restarted from the beginning, good to wait for 10 days again…

Of course electricity cuts and no protection against could be seen as abnormal and I could install bitcoin qt on my servers instead, but, still, this does not look normal (unless the drive got damaged but it does not seem to be the case), it should be able to recover

A workaround is probably to backup the blockchain directory and restart from it if something happens, this brings again the #8738 subject

fanquake commented at 8:34 am on August 9, 2022: member

Closing for now. Can be reopened if someone reproduces with a recent version of Core.

fanquake closed this on Aug 9, 2022

bitcoin locked this on Aug 9, 2023

Rebuilding block database after corruption behaves strangely #8523