0.14 not loading mempool.dat?

stefment commented at 2:55 pm on February 20, 2017: none

Describe the issue

Restarting bitcoin, mempool starts filling from scratch.

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

Wait for mempool to reach for example 3000 transactions
Restart bitcoin
Watch mempool filling slowly from scratch

Expected behaviour

Mempool would already contain ~3000 TX on startup

Actual behaviour

It starts at 0 and fills up slowly with a few per second.

Screenshots.

http://imgur.com/a/or82d

What version of bitcoin-core are you using?

Windows x64 0.14rc1 binary from https://bitcoin.org/bin/bitcoin-core-0.14.0/test.rc1/

Any extra information that might be useful in the debugging process.

I may not be aware of how the mempool.dat thing works and there could just be a misunderstanding.

fanquake added the label Mempool on Feb 20, 2017

paveljanik commented at 3:00 pm on February 20, 2017: contributor

How do you “Restart bitcoind”? Is mempool.dat saved after stopping bitcoind? Do you have something in the debug.log?

stefment commented at 3:03 pm on February 20, 2017: none

i found this in the debug.log

2017-02-20 14:52:29 Failed to deserialize mempool data on disk: CAutoFile::read: end of file. Continuing anyway.

the way i close bitcoin is by right clicking on the tray icon and choosing “exit”

it created a mempool.dat when i do that.

paveljanik commented at 3:08 pm on February 20, 2017: contributor

Can you please upload the mempool.dat file somewhere? Please think about its contents first (your transactions etc…).

stefment commented at 3:13 pm on February 20, 2017: none

im not sure which place people recommed for uploading files but i found the following place from a quick google search.

mempool.dat

paveljanik commented at 3:18 pm on February 20, 2017: contributor

I can confirm the files does not load.

Can you please try removing the file and not killing the client from the tray, but correctly via menu?

paveljanik commented at 3:20 pm on February 20, 2017: contributor

Do you have enough disc space? Any antivirus or something like that?

stefment commented at 3:24 pm on February 20, 2017: none

thank you for being so swift with the help.

i deleted mempool.dat, closed via menu, and notice it created a new mempool.dat. but on restart i get the same error in debug.log and the mempool is empty and start filling from scratch again.

i only have windows defender activated, no other antivirus etc. i tried doing the same steps above but with windows defender off, but i got the same results.

stefment commented at 3:26 pm on February 20, 2017: none

I have to add that i installed 0.14rc1 on top of my 0.13.2 install. Is that a problem?

laanwj commented at 3:42 pm on February 20, 2017: member

I’m not 100% sure that 0.14.0 can read 0.13.2’s mempool.dat files.

If that’s not the case then after starting the mempool from scratch once, it will work from there on.

stefment commented at 3:51 pm on February 20, 2017: none

I deleted the mempool.dat before i shut down 0.14rc1 so the mempool.dat should be a 0.14 type. Right? Yet it seems to fail to load when i start up 0.14rc1 again. I wish there was something i can do to help.

paveljanik commented at 4:20 pm on February 20, 2017: contributor

Ok, I have modified validation.cpp to contain:

0            fprintf(stderr, "Loading tx %d\n", num);
1            file >> tx;
2            fprintf(stderr, "tx loaded...\n");

and with the supplied file, “tx loaded” is not printed at all.

Hexdump of the beginning of the original file (as the first tx there can’t be deserialized):

000000000  01 00 00 00 00 00 00 00  21 02 00 00 00 00 00 00  |........!.......|
100000010  01 00 00 00 02 4c 05 4f  6b eb 9b 72 87 22 1b 05  |.....L.Ok..r."..|
200000020  25 7d c1 92 7a fa 13 84  74 59 aa ce d1 7a af c1  |%}..z...tY...z..|
300000030  32 21 a7 53 a5 01 00 00  00 6a 47 30 44 02 20 0f  |2!.S.....jG0D. .|
400000040  84 45 50 1f 17 ac 97 c1  6e 7b fd 16 eb 65 dd 46  |.EP.....n{...e.F|
500000050  22 c8 bc 70 b3 b7 61 8e  4a 6c 8d 28 a7 7a c8 02  |"..p..a.Jl.(.z..|
600000060  20 48 65 b3 f4 aa 83 ee  dc 59 77 73 d4 c2 06 cc  | He......Yws....|
700000070  ec 4d 24 b3 53 dc a2 64  2e 15 66 98 d1 21 44 de  |.M$.S..d..f..!D.|

The TX there is 2713042c4f667e00a70d1d7bc4e4939c294e6272934026ea0bd618ef6e236318.

laanwj added this to the milestone 0.14.0 on Feb 20, 2017

paveljanik commented at 5:01 pm on February 20, 2017: contributor

Hmm, CRLF could be an issue?

Raw TX by blockr here: http://btc.blockr.io/api/v1/tx/raw/2713042c4f667e00a70d1d7bc4e4939c294e6272934026ea0bd618ef6e236318 compared with saved mempool.dat:

001000000024c054f6beb9b7287221b05257dc1927afa13847459aaced17aafc13221a753a5010000006a47304402200f8445501f17ac97c16e7bfd16eb65dd4622c8bc70b3b7618e4a6c8d28a77ac802204865b3f4aa83eedc597773d4c206ccec4d24b353dca2642e156698d12144de04012103ab9330e34b208a76a2472e0dcca391a0626bcfbdccc36c2bc2905a2381417d8cfeffffffa44a01b249e55323d15df1a0ef233a8475bd3e94e69b5c4554e7c39d4bc1e0cc000000006b483045022100c93c1a083d9fea577b43d59f93cf09dceea0dfcadb484c18ef8e3c8e6200464b02202574053ac4485774ccc22a3c9d441375e422e32d0271c8d00ab35e621f3224da012103a8bd590e5ba1e0fb79a39845182a10cce0c289ee38c93c3e3320215e928c2b91feffffff025b110200000000001976a9145ba9f27ea1eeb54d5436d5ed153dda571430243f88ac585f4800000000001976a914e8f5ebe914299f9a7c289b063d2c704f7e5aa40488ac18ed0600
101000000024c054f6beb9b7287221b05257dc1927afa13847459aaced17aafc13221a753a5010000006a47304402200f8445501f17ac97c16e7bfd16eb65dd4622c8bc70b3b7618e4a6c8d28a77ac802204865b3f4aa83eedc597773d4c206ccec4d24b353dca2642e156698d12144de04012103ab9330e34b208a76a2472e0dcca391a0626bcfbdccc36c2bc2905a2381417d8cfeffffffa44a01b249e55323d15df1a0ef233a8475bd3e94e69b5c4554e7c39d4bc1e0cc000000006b483045022100c93c1a083d9fea577b43d59f93cf09dceea0dfcadb484c18ef8e3c8e6200464b02202574053ac4485774ccc22a3c9d441375e422e32d0271c8d00d0ab35e621f3224da012103a8bd590e5ba1e0fb79a39845182a10cce0c289ee38c93c3e3320215e928c2b91feffffff025b110200000000001976a9145ba9f27ea1eeb54d5436d5ed153dda571430243f88ac585f4800000000001976a914e8f5ebe914299f9a7c289b063d2c704f7e5aa40488ac18ed0600

paveljanik referenced this in commit 0a7f37b031 on Feb 20, 2017

paveljanik referenced this in commit 171fc91f06 on Feb 20, 2017

paveljanik commented at 5:20 pm on February 20, 2017: contributor

@stefment Thank you for reporting the problem. We have identified your issue and will provide fix for it in the next release candidate. I hope that someone will provide a test build for you so you can verify the build yourself even before the next RC!

stefment commented at 5:32 pm on February 20, 2017: none

Wow that was fast. Dont worry about making a test build for me. I will ask a friend to help me build my own binaries. I should be able to figure out which pullrequest has the fix. Thank you for your work!

paveljanik commented at 5:35 pm on February 20, 2017: contributor

@stefment That would be great! We need more Windows testers! The PR #9813 fixes this issue. This is #9810. 2 PRs/issues in between. The world is crazy.

laanwj closed this on Feb 21, 2017

laanwj referenced this in commit 3972a8efb2 on Feb 21, 2017

stefment commented at 6:09 pm on February 21, 2017: none

Seems to be working now. Im using the latest development build which i built myself. No error in debug.log and i see mempool with transactions after a restart now :) Just thought id let you know. Have a nice day.

laanwj commented at 6:17 pm on February 21, 2017: member

@stefment Thanks for testing!

HashUnlimited referenced this in commit 8bb79877e9 on Feb 28, 2018

lateminer referenced this in commit f48696e30e on Jan 4, 2019

DrahtBot locked this on Sep 8, 2021

0.14 not loading mempool.dat? #9810