Fatal LevelDB error: Corruption: block checksum mismatch

kristapsk commented at 5:16 pm on April 18, 2021: contributor

Had this problem with one long running node. Tried to restart bitcoind with -debug=leveldb -reindex, ended up with:

 02021-04-18T15:00:03Z UpdateTip: new best=0000000000000000001153fc4f4aee341eacc92b28450dfff90810b4d2df216e height=616461 version=0x2fffe000 log2_work=91.642167 tx=500946903 date='2020-02-08T04:25:07Z' progress=0.799912 cache=147.0MiB(649296txo)
 12021-04-18T15:00:03Z leveldb: Generated table [#215595](/bitcoin-bitcoin/215595/): 44038 keys, 2177932 bytes
 22021-04-18T15:00:04Z leveldb: Generated table [#215596](/bitcoin-bitcoin/215596/): 44998 keys, 2180094 bytes
 32021-04-18T15:00:04Z leveldb: Generated table [#215597](/bitcoin-bitcoin/215597/): 23369 keys, 1204615 bytes
 42021-04-18T15:00:04Z leveldb: Compacted 1@1 + 2@2 files => 5562641 bytes
 52021-04-18T15:00:04Z leveldb: compacted to: files[ 0 10 124 455 1486 0 0 ]
 62021-04-18T15:00:04Z leveldb: Delete type=2 [#215401](/bitcoin-bitcoin/215401/)
 72021-04-18T15:00:04Z leveldb: Delete type=2 [#215292](/bitcoin-bitcoin/215292/)
 82021-04-18T15:00:04Z leveldb: Delete type=2 [#215579](/bitcoin-bitcoin/215579/)
 92021-04-18T15:00:04Z leveldb: Delete type=2 [#215580](/bitcoin-bitcoin/215580/)
102021-04-18T15:00:04Z leveldb: Compacting 1@2 + 0@3 files
112021-04-18T15:00:04Z LevelDB read failure: Corruption: block checksum mismatch
122021-04-18T15:00:04Z Fatal LevelDB error: Corruption: block checksum mismatch
132021-04-18T15:00:04Z You can use -debug=leveldb to get more complete diagnostic messages
142021-04-18T15:00:05Z leveldb: Generated table [#215598](/bitcoin-bitcoin/215598/): 28652 keys, 1426450 bytes
152021-04-18T15:00:05Z Error: Error reading from database, shutting down.
162021-04-18T15:00:05Z Error reading from database: Fatal LevelDB error: Corruption: block checksum mismatch

Any hints how to debug this?

0# bitcoind -version
1Bitcoin Core version v0.20.1.0-gentoo

kristapsk added the label Bug on Apr 18, 2021

sipa commented at 5:18 pm on April 18, 2021: member

Sounds like a hardware or OS problem.

jarolrod commented at 5:20 pm on April 18, 2021: member

@kristapsk I used to get this on an NVME drive that was overheating. Fixed that and haven’t seen this since.

kristapsk commented at 5:34 pm on April 18, 2021: contributor

It’s a Gigabyte Brix with SATA HDD with nothing strange in a dmesg output…

jarolrod commented at 5:41 pm on April 18, 2021: member

Was bitcoin compiled (i assume so since on Gentoo), try recompiling again? Also, make sure to run diagnostics on your system, there could be a hardware issue somewhere. If not with the HDD then maybe with memory?

This is marcofalke’s template for these type of issues :)

Bitcoin Core makes heavy use of CPU, RAM, and disk IO. Hardware defects might only become visible when running Bitcoin Core. You might want to check your hardware for defects.

memtest86 to check your RAM to check the CPU behavior under load, use linpack or Prime95 to test your storage device use smartctl or CrystalDiskInfo

Source: https://bitcoin.stackexchange.com/a/12206

Edit: Below are some similar issues, the common theme is a hardware issue with HD, memory, or overheating CPU:

maflcko removed the label Bug on Apr 19, 2021

maflcko added the label Data corruption on Apr 19, 2021

maflcko added the label Questions and Help on Apr 19, 2021

kristapsk commented at 7:57 pm on May 2, 2021: contributor

@jarolrod , thanks for the hints / links, will test the hardware.

jarolrod commented at 4:30 am on July 2, 2021: member

@kristapsk any update?

maflcko commented at 5:22 am on July 2, 2021: member

Closing for now

maflcko closed this on Jul 2, 2021

kristapsk commented at 1:11 am on July 3, 2021: contributor

Did some hardware tests on a specific machine back in a May, haven’t found any problems (but I don’t remember details about everything I tested anymore). Was busy with other things, have decided to replace it with something more serious for mainnet. But will try to figure out what’s wrong there at some point probably. It’s ok to close this for now.

ademar111190 commented at 1:39 pm on July 19, 2022: none

I had this checksum LevelDB problem frequently in the last weeks, following this answer #10897 (comment) I managed to solve it, the HD I have been using to keep a copy of bitcoins blockchain is not working properly anymore, copying everything to a new HD and using it completely solve the errors.

tansanDOTeth commented at 4:20 am on June 26, 2023: none

I had this checksum LevelDB problem frequently in the last weeks, following this answer #10897 (comment) I managed to solve it, the HD I have been using to keep a copy of bitcoins blockchain is not working properly anymore, copying everything to a new HD and using it completely solve the errors.

I read that comment, but not sure what I have to do run tests against my external. Do you have more instructions written somewhere?

bitcoin locked this on Jun 25, 2024

Fatal LevelDB error: Corruption: block checksum mismatch #21724