CRC32C does not work well when comipled with GCC4.9 #9386

Describe the issue

ENV: Debian 8.6 Amd64 , GCC4.9.2, bitcoin-v0.13.1

PROBLEM: I cloned V0.13.1 to my computer, and changed some keyword, such as abc to xyz, as my working project. I have done anything, and made it work. the first run bitcoin-dt , it work, but I rerun bitcoin-dt, it reported a CRC32 error.

Opening LevelDB in /home/chengc/.xyzcoin/chainstate Corruption: checksum mismatch

So, I traced source ,and modified crc32c.cc, as skipping all "STEP4" stempments, in Value() function, it work well.

And I checked my team member's working env. he tell me he working with Ubuntu16,04 and GCC++ V5.2, and no such problem.

*** I think, may be some difference between GCC 4,9 and GCC 5.2 for bit operation

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

1. Debian 8.6, gcc 4.9.2
2. make ,
3. run bitcoin-qt twice

Expected behaviour

Opening LevelDB in /home/chengc/.xyzcoin/chainstate Corruption: checksum mismatch

Actual behaviour

"crc32c.cc" [Modified] line 314 of 333 remark all STEP4 as this

#if false // *** DISABLE ALL STEP4 // Process bytes 16 at a time while ((e-p) >= 16) { STEP4; STEP4; STEP4; STEP4; } // Process bytes 4 at a time while ((e-p) >= 4) { STEP4; } #endif // Process the last few bytes while (p != e) { STEP1; }

If you comment out the STEP4 calls, your checksum doesn't do anything anymore. As a result, it will always succeed.

My guess is that you just had a hardware issue, and the checksum correctly detected it.

My laptop has GCC 4.9.3, Bitcoin works fine; and of course many people are running it fine on stock Debian.

I only commented out STEP4, CRC32C always done with STEP1.

// Process the last few bytes while (p != e) { STEP1; // *** process all bytes with STEP1 }

I have try modify code to this: foreach byte in buffer, loop : using Extend(), I got the correct value, same as first run.

I am confused about: in first, run, crc is computed in 2 steps: Value(type[1]..) , then Extend(buffer0...) and second run: crc is check by: Value(buffer, length+1), ...in readLog() module. The result is not the same.

I have try one byte by one byte, and call Extend(), same as init result. So, I modify CRC32C, and disable STEP4(but doing with STEP1) I got the same CRC value.

***Let me try UBUNTU 16.04.1 on same PC, and I will post the result!! and report where is CPU issue or compiler issue.

People have without any issues used databases created with some compiler versions, and transported them to systems with completely different CPU architecture. I would be incredibly surprised if there were an issue with those compiler versions or the implementation of LevelDB's CRC code (which is used by all LevelDB-based systems everywhere - not just Bitcoin).

I assume this is either a hardware issue, or a bug you have introduced (what is xyzcoin ?).

this problem not happen with original code , cloned from GITHUB. without STEP4, also return same value doing with STEP4

XYZ, is clone project from V0.13.1.

1. replace all keyword bitcoin with xyzcoin, Bitcoin with Xyzcoin, BITCOIN with XYZCOIN
2. rename file : bitcoin to xyzcoin then autogen, configurate, make, same step as build bitcoin.

I have modify crc32c.h, and let mask/unmaks do nothing, just return the value from parameter. so I can check the CRC32C with some online tool.

I copy the value from byte0000007 to byte 000056 , 39+1 bytes (value: 01 01 00 00 00 00 00 00 00 01 00 00 00 01 0f 0e 00 6f 62 66 75 73 63 61 74 65 5f 6b 65 79 09 08 2b e6 c7 4d f9 80 d3 78) using http://www.scadacore.com/field-applications/programming-calculators/online-checksum-calculator/ return value as : Reversed 0x82F63B78 D0 F8 BC 97 97 BC F8 D0 F8 D0 97 BC BC 97 D0 F8 same as the value first 4 bytes in logfile.

after that , i print the crc value, the acutal_crc is 2a08b163:

2016-12-20 08:30:20 Opened LevelDB successfully 2016-12-20 08:30:20 Using obfuscation key for .xyzcoin/blocks/index: 0000000000000000 2016-12-20 08:30:20 Opening LevelDB in /home/chengc/.xyzcoin/chainstate FILE:leveldb/db/log_reader.cc, LINE:240, acutal_crc=73f2b1b, expected_crc=73f2b1b FILE:leveldb/db/log_reader.cc, LINE:240, acutal_crc=58e60e04, expected_crc=58e60e04 FILE:leveldb/db/log_reader.cc, LINE:240, acutal_crc=2a08b163, expected_crc=97bcf8d0 2016-12-20 08:30:20 Corruption: checksum mismatch

dump of the log file: ~/.xyzcoin$ od -t x1 chainstate/000003.log 0000000 d0 f8 bc 97 27 00 01 01 00 00 00 00 00 00 00 01 0000020 00 00 00 01 0f 0e 00 6f 62 66 75 73 63 61 74 65 0000040 5f 6b 65 79 09 08 2b e6 c7 4d f9 80 d3 78 e1 35 0000060 6a ee 0c 00 01 02 00 00 00 00 00 00 00 00 00 00 0000100 00 3f 86 41 dd 30 00 01 02 00 00 00 00 00 00 00 0000120 01 00 00 00 01 01 42 20 44 04 4b 47 4f 71 60 0a 0000140 ea 40 65 0b 57 e3 24 37 b8 f8 44 28 18 da db e4 0000160 43 30 de 4d f9 80 d3 78 0000170

==================== After hat, I skip all STEP4 , and xyzcoin-qt works well.

now, I try Ubuntu 16.04, GCC 5.4, produce same issue, Linux TV2730-U1604 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux model name : Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz

./configure CPPFLAGS="-I${BDB_PREFIX}/include/" CXXFLAGS="-O3" LDFLAGS="-L${BDB_PREFIX}/lib/" --with-gui --disable-tests --disable-gui-tests --with-libs=no

Once you have carefully checked that this is a real issue - which I'm somewhat sceptical about because I've compiled this stuff with many different compilers, architectures and toolchains and copy around data directories all the time - you should file this issue upstream: https://github.com/google/leveldb

this problem not happen with original code , cloned from GITHUB.

Without your local patch it is not possible to track down this problem. Please bisect your patch to determine which line(s) cause the failure, then let us know.

after compare the assembly code of CRC32C.cc , generated by GCC -S, without difference between original code and patched code. there are some issue with the module.

Thanks a lot, I will post this issue to https://github.com/google/leveldb.

Many Thanks to who pay attations to issue

after compared the code with leveldb::crc32c.cc and the code I had modified, there are 2 differences found, and I have found what mistake I take!! I have changed the default port from 8333 to 7333, for testting, with sed, this also changed the const value in CRC32C

Thank a lot.

====================

< 0x48f3434f, 0xedb2d131, 0x079c1142, 0xa2dd833c,

0x48f3434f, 0xedb2d131, 0x079c1142, 0xa2dd733c, 250c250 < 0xdfc69b2a, 0x02833192, 0x60a1b8ab, 0xbde41213,

0xdfc69b2a, 0x02733192, 0x60a1b8ab, 0xbde41213,