Corruption: block checksum mismatch

jorisbontje commented at 8:03 am on August 6, 2015: none

with bitcoind 0.11.0 on a 64bit Linux VPS (with simfs)

 02015-08-06 05:53:59 Corruption: block checksum mismatch
 12015-08-06 05:53:59 *** System error while flushing: Database corrupted
 22015-08-06 05:53:59 Error: Error: A fatal internal error occured, see debug.log for details
 32015-08-06 05:53:59 ERROR: ProcessNewBlock: ActivateBestChain failed
 42015-08-06 05:53:59 opencon thread interrupt
 52015-08-06 05:53:59 addcon thread interrupt
 62015-08-06 05:53:59 msghand thread interrupt
 72015-08-06 05:53:59 scheduler thread interrupt
 82015-08-06 05:53:59 net thread interrupt
 92015-08-06 05:53:59 Shutdown: In progress...
102015-08-06 05:54:00 StopNode()
112015-08-06 05:54:00 Corruption: block checksum mismatch
122015-08-06 05:54:00 *** System error while flushing: Database corrupted
132015-08-06 05:54:00 Error: Error: A fatal internal error occured, see debug.log for details
142015-08-06 05:54:00 Shutdown: done

This is occasionally happening after a -reindex on 0.11.0, as this was also previously happening on earlier releases. bitcoind.conf includes settings preventing transaction flooding:

0minrelaytxfee=0.00005
1limitfreerelay=5

laanwj commented at 6:18 am on August 7, 2015: member

These are disk corruption issues: leveldb keeps its own checksums of all data it writes, when it notices during (background) compaction that these mismatch it will return the error on the next flush.

02015-08-06 05:53:59 Corruption: block checksum mismatch
12015-08-06 05:53:59 *** System error while flushing: Database corrupted

Weax commented at 7:37 pm on August 11, 2015: none

Same issue here, but with Win10 x64. Checkdisk returned no errors on disk.

jorisbontje commented at 10:11 am on August 13, 2015: none

The support team of the VPS hosting provider claims there are no disk corruption issues:

0There is no disk corruption from our side, because all our nodes use RAID array in which case
1disks is duplicated in the array and in case of any disk failure this disk is automatically removed
2from the array and node is working with other left disk until faulty hard disk replace.
3You database corruption may be related to your program/service crash or errors in it etc.

TheBlueMatt commented at 10:33 am on August 13, 2015: member

Silent corruption in raid arrays is absolutely possible. Its possible this is an issue with leveldb, but it seems more likely there are silent errors somewhere in your disk/raid controller/etc.

rnicoll commented at 12:03 pm on August 13, 2015: contributor

Have seen something similar in dogecoin/dogecoin#1216 - could be specific to our changes, but the version in question is a complete rebuild around Bitcoin Core 0.11. Either way, am looking into this, if we find anything we will of course push fixes into Bitcoin as well.

sipa commented at 12:40 pm on August 13, 2015: member

@jorisbontje Is it possible that you ran out of memory or the disk was full?

jorisbontje commented at 1:39 pm on August 13, 2015: none

@sipa enough disk space (50GB+). I was expecting an out of memory (only 3GB total), but didn’t get an Out of memory error anywhere. I configured the relay settings to reduce the pending tx pool, but still happening.

I can try to reproduce this with a -reindex, is there any kind of additional monitoring / debug logging, etc you’d like me to enable?

sipa commented at 3:06 pm on August 13, 2015: member

Joris: does the crash happen while you are synchronizing/running, or after a stop/start?

If so, could you try running with -par=1 (to reduce risk for CPU problems, but I don’t think that is the problem here, you’d get validation errors instead).

My last guess would be an incompatibility between LevelDB and the filesystem yo’re using.

jorisbontje commented at 7:03 pm on August 20, 2015: none

This happened again while synchronizing/running bitcoind -reindex -par=1

 02015-08-20 16:56:48 UpdateTip: new best=0000000000000000043a35f28c76f951eef172b33a53a49c47a891c759943341  height=370730  log2_work=83.236584  tx=80585198  date=2015-08-20 16:55:41 progress=0.999999  cache=62.6MiB(15705tx)
 12015-08-20 16:56:48 UpdateTip: 1 of last 100 blocks above version 3
 22015-08-20 16:56:48 keypool reserve 3695
 32015-08-20 16:56:48 keypool return 3695
 42015-08-20 16:56:49 Corruption: block checksum mismatch
 52015-08-20 16:56:49 *** System error while flushing: Database corrupted
 62015-08-20 16:56:49 Error: Error: A fatal internal error occured, see debug.log for details
 72015-08-20 16:56:49 ERROR: ProcessNewBlock: ActivateBestChain failed
 82015-08-20 16:56:49 net thread interrupt
 92015-08-20 16:56:49 opencon thread interrupt
102015-08-20 16:56:49 addcon thread interrupt
112015-08-20 16:56:49 scheduler thread interrupt
122015-08-20 16:56:49 msghand thread interrupt
132015-08-20 16:56:49 Shutdown: In progress...
142015-08-20 16:56:49 StopNode()
152015-08-20 16:56:50 Corruption: block checksum mismatch
162015-08-20 16:56:50 *** System error while flushing: Database corrupted
172015-08-20 16:56:50 Error: Error: A fatal internal error occured, see debug.log for details
182015-08-20 16:56:51 Shutdown: done

Mikadily commented at 10:32 am on August 25, 2015: none

I’m experiencing similar problem. Ubuntu 12.04.5 on a physical box, software raid.

miguelmorales85 commented at 1:55 pm on September 10, 2015: none

I have the same problem here. It all started with the following message “ERROR: AcceptToMemoryPool: non-final”. Then after stopping bitcoind I get this message “Corruption: block checksum mismatch *** System error while flushing: Database corrupted”

jorisbontje commented at 2:00 pm on September 10, 2015: none

@miguelmorales85 what kind of filesystem / storage setup do you have? (raid, vps, simfs, etc)

miguelmorales85 commented at 2:06 pm on September 10, 2015: none

@jorisbontje the hd was formated in ext4

chuacw commented at 8:48 am on September 14, 2015: none

I have the same issue. and I don’t have disk corruption. My database was newly imported from bootstrap.dat

This is Bitcoin Core 0.11 (64-bit), and on Windows 2008 R2 (64-bit), NTFS. I have about 1TB free.

2015-09-14 08:42:33 UpdateTip: new best=0000000000000000184d37ea91dc37ad61f2c9eca556020476787bf9916a5a24 height=337433 log2_work=81.923139 tx=55737117 date=2015-01-04 11:54:53 progress=0.635931 cache=62.6MiB(10952tx) 2015-09-14 08:42:34 Corruption: block checksum mismatch 2015-09-14 08:42:34 *** System error while flushing: Database corrupted 2015-09-14 08:44:13 ERROR: ProcessNewBlock: ActivateBestChain failed 2015-09-14 08:44:13 opencon thread interrupt 2015-09-14 08:44:13 scheduler thread interrupt 2015-09-14 08:44:13 addcon thread interrupt 2015-09-14 08:44:13 net thread interrupt 2015-09-14 08:44:14 ERROR: AcceptToMemoryPool: non-final 2015-09-14 08:44:14 msghand thread interrupt 2015-09-14 08:44:14 Shutdown: In progress… 2015-09-14 08:44:14 StopNode() 2015-09-14 08:44:14 UPNP_DeletePortMapping() returned: 0 2015-09-14 08:44:14 upnp thread interrupt 2015-09-14 08:44:14 Corruption: block checksum mismatch 2015-09-14 08:44:14 *** System error while flushing: Database corrupted 2015-09-14 08:44:16 Shutdown: done

miguelmorales85 commented at 3:20 pm on September 14, 2015: none

Again database corruption at block 252.dat I am reindexing.. this takes a while so I will let you know the results…

miguelmorales85 commented at 1:43 pm on September 16, 2015: none

I have just failed to reindex bitcoind.. this is what happened. 2015-09-16 05:15:09 UpdateTip: new best=0000000000000000087d41de97e50f2c63151b0caee4a63f1387dd0eb8ae2153 height=345409 log2_work=82.326044 tx=60898518 date=2015-02-27 14:50:06 progress=0.724568 cache=64.9MiB(14575tx) 2015-09-16 05:15:09 Corruption: block checksum mismatch 2015-09-16 05:15:09 *** System error while flushing: Database corrupted 2015-09-16 05:15:10 Error: Error: A fatal internal error occured, see debug.log for details 2015-09-16 05:15:10 ERROR: ProcessNewBlock: ActivateBestChain failed 2015-09-16 05:15:10 addcon thread interrupt 2015-09-16 05:15:10 opencon thread interrupt 2015-09-16 05:15:10 scheduler thread interrupt 2015-09-16 05:15:10 net thread interrupt 2015-09-16 05:15:10 msghand thread interrupt 2015-09-16 05:15:10 Shutdown: In progress… 2015-09-16 05:15:10 RPCAcceptHandler: Error: Operation canceled 2015-09-16 05:15:10 StopNode() 2015-09-16 05:15:11 Corruption: block checksum mismatch 2015-09-16 05:15:11 *** System error while flushing: Database corrupted 2015-09-16 05:15:11 Error: Error: A fatal internal error occured, see debug.log for details 2015-09-16 05:15:11 Shutdown: done

can somebody help me. Please dont tell me I have to reindex AGAIN…

jgarzik commented at 1:53 pm on September 16, 2015: contributor

@miguelmorales85 it is highly likely you have hardware problems

miguelmorales85 commented at 3:42 pm on September 16, 2015: none

@jgarzik I ran e2fsk on the unmounted partition in a live session with 0 errors :(

laanwj added the label UTXO Db and Indexes on Sep 22, 2015

funyung commented at 2:21 pm on September 27, 2015: none

Firstly, I created my account because of this issue. Maybe this isn’t a hardware issue, as I am also running x64 Linux (ubuntu 14.04) w/ no disk errors. I’ve tried reindexing around 6 times since thursday. 5 out of those times I hit database corruption at random points, but before this last time it hit I had filled my ext4 drive when it hit 2 weeks til completion. So I set up a symlink so the blocks directory is stored on a much larger ntfs drive. Go to reindex on next launch because it’s required and I hit database corruption at 52 weeks. I literally just purchased the ntfs drive a month ago, and I’ve not had any errors reported on either disks.

miguelmorales85 commented at 2:47 pm on September 27, 2015: none

I finally finally did not get the checksum mismatch after several reindexing (like a week of try and error). But now the node keeps getting STALLED, I have can not SYNC the node. The closer I got was like verification progress to 0.9989xxxx and I am slowly going down… Can somebody help me? I have restarted the node, reboot the Ubuntu PC and I just get the node UP and then STALLED. Of course I am accepting conections.

sipa commented at 2:52 pm on September 27, 2015: member

Miguel: anything in debug.log mentioning invalid blocks?

miguelmorales85 commented at 2:56 pm on September 27, 2015: none

@sipa Hello, I have run “cat /home/USER/.bitcoin/debug.log | grep invalid” and found this: 2015-09-26 02:50:28 ERROR: AcceptBlockHeader: block is marked invalid 2015-09-26 02:50:28 ERROR: invalid header received 2015-09-26 04:49:47 ERROR: AcceptBlockHeader: block is marked invalid 2015-09-26 04:49:47 ERROR: invalid header received 2015-09-26 07:25:55 ERROR: AcceptBlockHeader: block is marked invalid 2015-09-26 07:25:55 ERROR: invalid header received

what can I do about it?

sipa commented at 2:59 pm on September 27, 2015: member

That almost certainly means Bitcoin Core incorrectly decided that a signature or transaction was invalid in a block, and subsequently marked the block invalid.

Can you try reindexing with -par=1 or par=1 in bitcoin.conf?

miguelmorales85 commented at 3:04 pm on September 27, 2015: none

oh god… I feared you would say that… Ok .. will do. See you in a few days. Thanks.

miguelmorales85 commented at 8:55 am on October 7, 2015: none

Hello @sipa I have reindexed. I have the same problem, I can never get to SYNC the node. At this moment the node progression is at 0.9983xxxx and going down and down every seccond… it’s going to Stall again. So no par=1 in bitcoin.conf did something to prevent my problem.

santzi commented at 4:44 am on October 8, 2015: none

Hi, I have got this several times. About 1-2 times per week. When I started bitcoind again, it working ok and I don’t need to do reindex it again. So this is not so fatal, but annoying. My Bitcoin core version is quite new. I compiled it from source (master) about 2-3 weeks ago. Now there are over 200 commits more than this version. I think pure 0.11.0 version working better.

Here is the last log on yestersay: 2015-10-07 11:29:45 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=88.193.138.220:8333, peer=593 2015-10-07 11:32:39 receive version message: /Snoopy:0.2.1/: version 70001, blocks=0, us=88.193.138.220:8333, peer=594 2015-10-07 11:32:42 receive version message: /bitcoinj:0.14SNAPSHOT/: version 70001, blocks=377853, us=127.0.0.1:8333, peer=595 2015-10-07 11:33:20 Corruption: block checksum mismatch 2015-10-07 11:33:20 *** System error while flushing: Database corrupted 2015-10-07 11:33:20 Error: Error: A fatal internal error occurred, see debug.log for details 2015-10-07 11:33:20 ERROR: ProcessNewBlock: ActivateBestChain failed 2015-10-07 11:33:21 scheduler thread interrupt 2015-10-07 11:33:21 opencon thread interrupt 2015-10-07 11:33:21 addcon thread interrupt 2015-10-07 11:33:21 msghand thread interrupt 2015-10-07 11:33:21 net thread interrupt 2015-10-07 11:33:21 Shutdown: In progress… 2015-10-07 11:33:21 StopNode() 2015-10-07 11:33:21 Corruption: block checksum mismatch 2015-10-07 11:33:21 *** System error while flushing: Database corrupted 2015-10-07 11:33:21 Error: Error: A fatal internal error occurred, see debug.log for details 2015-10-07 11:33:21 Shutdown: done

When this occurred next time, I’ll pull new sources from master branch and try again.

trevelocity commented at 8:02 pm on October 28, 2015: none

With bitcoind 0.11.1 on a 64-bit Linux PC I got the “Corruption: block checksum mismatch” error. I ran a full system check, which reported single bit data dependent corruption in one of the DIMMs. I think the LevelDB data was corrupted when cached in the DRAM memory and then got written back to disk. I replaced both DIMMs, ran with -reindex=1, and bitcoind has been running without error for a week.

z0rg1nc commented at 11:32 am on December 17, 2015: none

Have the same on Ubuntu 15.04 x64 + Bitcoin core 0.11.1.0 (VM 2GB ram, enough hard drive space, ram check OK). First it happened 2 days ago (24hr client just receiving info most of the time), I deleted last 3 blk filed, reindex started, on -2y30w throws “error reading from database”. Yesterday I cleared btc data directory except bitcoin.conf and wallet.dat files, moved btc directory to different physical drive with zfs, after resync from scratch it throws “error reading from database” on -2y30w again.

 02015-12-17 00:35:19 UpdateTip: new best=000000000000005bb239fc8ff69c2d9fd33d1d23532dcebcb24192758f1baba3  height=237055  log2_work=70.080825  tx=18133567  date=2013-05-20 12:51:09 progress=0.081912  cache=4.8MiB(3525tx)
 12015-12-17 00:35:19 UpdateTip: new best=000000000000015767a84949b0bbf7ba25c24515e6198fd57909b8d3d1d7a7f5  height=237056  log2_work=70.08088  tx=18133568  date=2013-05-20 12:51:48 progress=0.081912  cache=4.8MiB(3526tx)
 22015-12-17 00:35:19 UpdateTip: new best=0000000000000156d33717e7055beb017cd07e0d044fc7a4a364cf459d148aa1  height=237057  log2_work=70.080936  tx=18134049  date=2013-05-20 12:55:01 progress=0.081914  cache=5.0MiB(4225tx)
 32015-12-17 00:35:20 UpdateTip: new best=00000000000000ce70375942e44e5a5f5f2192afecdc178fe0fe240bb8310d1c  height=237058  log2_work=70.080991  tx=18134844  date=2013-05-20 13:18:02 progress=0.081917  cache=5.9MiB(5339tx)
 42015-12-17 00:35:20 UpdateTip: new best=00000000000001679e03844bc5e651ff9394b57f4612a43e555fc5cedab5c3b2  height=237059  log2_work=70.081047  tx=18135625  date=2013-05-20 13:51:19 progress=0.081921  cache=6.2MiB(6234tx)
 52015-12-17 00:35:20 UpdateTip: new best=000000000000001a74e886007d45abf8feae6668516e35462fe207f0bfe48b7e  height=237060  log2_work=70.081102  tx=18135989  date=2013-05-20 14:02:32 progress=0.081923  cache=6.9MiB(7048tx)
 62015-12-17 00:35:20 UpdateTip: new best=0000000000000027162a00ff456c8f1f2a37ac915714c17b9d2c622dab9b94e6  height=237061  log2_work=70.081158  tx=18136400  date=2013-05-20 14:03:09 progress=0.081924  cache=7.0MiB(7490tx)
 72015-12-17 00:35:20 UpdateTip: new best=00000000000000f6f3787321ff929f44a9bfc0e75ac54ac90157e5bf5dd23cb6  height=237062  log2_work=70.081213  tx=18136619  date=2013-05-20 14:14:18 progress=0.081925  cache=7.4MiB(7899tx)
 82015-12-17 00:35:20 UpdateTip: new best=0000000000000158fc5cee0bcac58973ddc158e4891df80eef7c67f78e5cd47e  height=237063  log2_work=70.081269  tx=18136747  date=2013-05-20 14:17:25 progress=0.081926  cache=7.8MiB(8135tx)
 92015-12-17 00:35:20 LevelDB read failure: Corruption: block checksum mismatch
102015-12-17 00:35:20 Corruption: block checksum mismatch

 02015-12-17 09:36:10 init message: Rescanning...
 12015-12-17 09:36:10 Rescanning last 1101 blocks (from block 235950)...
 22015-12-17 09:36:10  rescan                    1ms
 32015-12-17 09:36:10 init message: Activating best chain...
 42015-12-17 09:36:11 UpdateTip: new best=000000000000008bdabb35a3ce5f85b8c3a11ffefd5a29defcd4c786e271f8a9  height=237052  log2_work=70.080658  tx=18132299  date=2013-05-20 11:49:50 progress=0.081864  cache=1.2MiB(809tx)
 52015-12-17 09:36:11 UpdateTip: new best=00000000000000b9f7b4bfaee1287b3e51fd534cdb30f42c482dfff7f714f6d3  height=237053  log2_work=70.080714  tx=18132304  date=2013-05-20 12:10:02 progress=0.081864  cache=1.3MiB(820tx)
 62015-12-17 09:36:11 UpdateTip: new best=00000000000000afe58811aea548dac4d6f4fd8c9c2b88836c84170ab52b03b8  height=237054  log2_work=70.080769  tx=18133326  date=2013-05-20 12:35:09 progress=0.081869  cache=3.6MiB(3003tx)
 72015-12-17 09:36:12 UpdateTip: new best=000000000000005bb239fc8ff69c2d9fd33d1d23532dcebcb24192758f1baba3  height=237055  log2_work=70.080825  tx=18133567  date=2013-05-20 12:51:09 progress=0.081870  cache=4.3MiB(3525tx)
 82015-12-17 09:36:12 UpdateTip: new best=000000000000015767a84949b0bbf7ba25c24515e6198fd57909b8d3d1d7a7f5  height=237056  log2_work=70.08088  tx=18133568  date=2013-05-20 12:51:48 progress=0.081870  cache=4.3MiB(3526tx)
 92015-12-17 09:36:12 UpdateTip: new best=0000000000000156d33717e7055beb017cd07e0d044fc7a4a364cf459d148aa1  height=237057  log2_work=70.080936  tx=18134049  date=2013-05-20 12:55:01 progress=0.081872  cache=4.5MiB(4225tx)
102015-12-17 09:36:12 UpdateTip: new best=00000000000000ce70375942e44e5a5f5f2192afecdc178fe0fe240bb8310d1c  height=237058  log2_work=70.080991  tx=18134844  date=2013-05-20 13:18:02 progress=0.081876  cache=5.4MiB(5339tx)
112015-12-17 09:36:12 UpdateTip: new best=00000000000001679e03844bc5e651ff9394b57f4612a43e555fc5cedab5c3b2  height=237059  log2_work=70.081047  tx=18135625  date=2013-05-20 13:51:19 progress=0.081879  cache=5.8MiB(6234tx)
122015-12-17 09:36:13 UpdateTip: new best=000000000000001a74e886007d45abf8feae6668516e35462fe207f0bfe48b7e  height=237060  log2_work=70.081102  tx=18135989  date=2013-05-20 14:02:32 progress=0.081881  cache=6.5MiB(7048tx)
132015-12-17 09:36:13 UpdateTip: new best=0000000000000027162a00ff456c8f1f2a37ac915714c17b9d2c622dab9b94e6  height=237061  log2_work=70.081158  tx=18136400  date=2013-05-20 14:03:09 progress=0.081883  cache=6.6MiB(7490tx)
142015-12-17 09:36:13 UpdateTip: new best=00000000000000f6f3787321ff929f44a9bfc0e75ac54ac90157e5bf5dd23cb6  height=237062  log2_work=70.081213  tx=18136619  date=2013-05-20 14:14:18 progress=0.081884  cache=7.0MiB(7899tx)
152015-12-17 09:36:13 UpdateTip: new best=0000000000000158fc5cee0bcac58973ddc158e4891df80eef7c67f78e5cd47e  height=237063  log2_work=70.081269  tx=18136747  date=2013-05-20 14:17:25 progress=0.081884  cache=7.3MiB(8135tx)
162015-12-17 09:36:13 LevelDB read failure: Corruption: block checksum mismatch
172015-12-17 09:36:13 Corruption: block checksum mismatch
182015-12-17 09:36:53 Error reading from database: Database corrupted

Several guys from bitcointalk.org faced the same problem too (https://bitcointalk.org/index.php?topic=1288944.0 https://bitcointalk.org/index.php?topic=1281530.0 https://bitcointalk.org/index.php?topic=1152996.0)

UPDATE: Looks like it’s my hardware problem, cancelling bitcoin code claims.

iuriguilherme commented at 0:06 am on January 10, 2016: none

I have this issue on a fresh install of debian jessie/sid.

It’s happening with bitcoin, litecoin and dogecoin.

It happens on btrfs, changing filesystem to ext4 and redownloading the blochain helped.

iuriguilherme commented at 3:30 am on January 11, 2016: none

I have looked at my logs and this probably has to do with changing the libdb version.

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=621425

jloughry commented at 5:12 pm on January 15, 2016: contributor

I had the same problem on a Mac Mini (mid 2011) running OS X 10.11.2 with all updates installed, 8 GB RAM. It was consistent; neither -reindex nor deleting the entire blockchain and re-downloading would get around the corrupted database problem, no matter how many times I tried. However, replacing the 0.11.2 software with 0.10.3 (64 bit) worked, after I ran the 0.10.3 with -reindex. So there is something in 0.11.2 that doesn’t like my machine, but I can work with 0.10.3 for now.

ethought commented at 4:37 pm on January 22, 2016: none

We are having the same problem and it is recurring. All other clients seem fine (Litecoin, Dogecoin, Unobtanium etc) - it seems to only be affecting bitcoin.

2016-01-22 16:30:51 Leaving block file 424: CBlockFileInfo(blocks=0, size=0, heights=0…0, time=1970-01-01…1970-01-01) 2016-01-22 16:30:56 Pre-allocating up to position 0x1000000 in blk00424.dat 2016-01-22 16:30:56 LevelDB read failure: Corruption: block checksum mismatch 2016-01-22 16:30:56 Corruption: block checksum mismatch 2016-01-22 16:30:56 Error: Error reading from database, shutting down. 2016-01-22 16:30:56 Error reading from database: Database corrupted

“version” : 110200, “protocolversion” : 70002, “walletversion” : 60000,

iuriguilherme commented at 1:01 am on January 23, 2016: none

I have tried changing libdb versions on a debian sid machine and only bitcoin is being affected as of now. Dogecoin and every other wallet are fine.

laanwj commented at 11:32 am on January 28, 2016: member

Changing libdb won’t fix this: the backing database for the utxo set and block index is leveldb, not berkeleydb.

cornwarecjp commented at 11:22 pm on January 29, 2016: none

FYI: I had the same issue with v0.11.0 on AMD64 Debian Jessie Linux with ext4 on LUKS file system (Bitcoin-Qt compiled by me).

I ran memtest86 and found a couple of faults in my RAM. I disabled 2 sections of 64kB of misbehaving RAM in the GRUB configuration, restarted, and started re-indexing with Bitcoin. Re-indexing took almost a week on my computer; every 8 hours or so I stopped Bitcoin, made a backup of the databases and restarted(*) Bitcoin, just in case something went wrong halfway or at 99%, to prevent having to start re-indexing again. However, I did not encounter any problems during re-indexing, so for me, the problem is solved.

If you are always encountering issues during re-indexing on the same position in the block chain, could it be that there is something wrong in the block files stored on your hard disk? In that case, would it be possible to resolve the issue by removing a certain range of block files, to force Bitcoin to re-download these blocks? WARNING: this is just guess-work, and certainly not a recommendation! I’m not that much of an expert, and from my POV this might as well make things worse.

(*) On 2nd, 3rd etc. start of Bitcoin, I did not provide the re-index commandline argument, since I did not want to re-start indexing every time. Apparently, Bitcoin automatically continues re-indexing if re-indexing was unfinished on the last shutdown.

giacecco commented at 6:39 am on January 30, 2016: none

Reindexing with -par=1 solved it for me on MacOS running 0.11.2.

The only interesting thing my side is that when I had the error for the first time I had just finished re-downloading the whole chain using 0.11.2, after upgrading from 0.11.1. I did not need to re-download, I just did it (it is not relevant why). This would seem to suggest it is a problem that - at least on Mac - arose from 0.11.2.

From people’s descriptions though it seems to me we may be in front of more than one issue. The range of symptoms is wide and inconsistent.

G.

laanwj added the label Data corruption on Feb 9, 2016

lostcodder commented at 10:54 pm on April 21, 2016: none

Same problem. 0.12.1 Win7 x64

2016-04-21 22:05:08 Corruption: block checksum mismatch 2016-04-21 22:05:08 *** System error while flushing: Database corrupted 2016-04-21 22:19:48 UPnP Port Mapping successful. 2016-04-21 22:22:15 ERROR: ProcessNewBlock: ActivateBestChain failed 2016-04-21 22:22:15 tor: Thread interrupt 2016-04-21 22:22:15 opencon thread interrupt 2016-04-21 22:22:15 net thread interrupt 2016-04-21 22:22:15 torcontrol thread exit 2016-04-21 22:22:15 addcon thread interrupt 2016-04-21 22:22:15 scheduler thread interrupt 2016-04-21 22:22:15 msghand thread interrupt 2016-04-21 22:22:15 Shutdown: In progress… 2016-04-21 22:22:15 StopNode() 2016-04-21 22:22:16 UPNP_DeletePortMapping() returned: 0 2016-04-21 22:22:16 upnp thread interrupt 2016-04-21 22:22:16 Corruption: block checksum mismatch 2016-04-21 22:22:16 *** System error while flushing: Database corrupted 2016-04-21 22:22:18 Shutdown: done

netzstrand commented at 7:49 am on May 16, 2016: none

I have the same problem :( Kubuntu 16.04 64bit bitbitcoin-qt 0.12.1.0-g9779e1e

 02016-05-16 05:40:43 Corruption: block checksum mismatch
 12016-05-16 05:40:43 *** System error while flushing: Database corrupted
 22016-05-16 05:57:02 ping timeout: 1200.046503s
 32016-05-16 05:57:23 ping timeout: 1200.016167s
 42016-05-16 05:59:52 ping timeout: 1200.020610s
 52016-05-16 07:33:18 ERROR: ProcessNewBlock: ActivateBestChain failed
 62016-05-16 07:33:18 tor: Thread interrupt
 72016-05-16 07:33:18 torcontrol thread exit
 82016-05-16 07:33:18 addcon thread interrupt
 92016-05-16 07:33:18 opencon thread interrupt
102016-05-16 07:33:18 msghand thread interrupt
112016-05-16 07:33:18 scheduler thread interrupt
122016-05-16 07:33:18 net thread interrupt
132016-05-16 07:33:18 Shutdown: In progress...
142016-05-16 07:33:18 StopNode()
152016-05-16 07:33:18 Corruption: block checksum mismatch
162016-05-16 07:33:18 *** System error while flushing: Database corrupted
172016-05-16 07:33:30 Shutdown: done

jloughry commented at 7:21 pm on May 16, 2016: contributor

Update: this problem went away when I replaced the memory (RAM) in my computer. The old memory was found to have errors by memtest (data mismatch error). The new memory has no such errors according to memtest.

Bitcoin Core has been running 24/7 for weeks now with no problems on a Mac Mini (Mid 2011, 2.3 GHz Intel Core i5, 16 GB 1600 MHz DDR3 memory, OS X 10.11.4).

whitslack commented at 4:17 am on May 17, 2016: contributor

~~For what it’s worth, I’ve run Memtest86+ overnight for two nights and found no errors in my RAM, so that’s not the only reason this error can arise.~~

Discovered that I was getting some very occasional single-bit errors when reading from disk, so that was likely the cause of this error for me. I underclocked my system by 3.76%, and now I am unable to reproduce the single-bit errors, so hopefully my “block checksum mismatch” issues will be gone now too.

hamiltino commented at 0:56 am on July 10, 2016: none

I had the same error i fixed it by formatting my disk to f2fs from ext4.

Leviathn commented at 1:39 pm on September 5, 2017: none

This appears to be entirely related to hardware failures. Closeable @borisbontje

Leviathn commented at 1:40 pm on September 5, 2017: none

@jorisbontje

fanquake closed this on Oct 7, 2017

DrahtBot locked this on Sep 8, 2021

Corruption: block checksum mismatch #6528