“last wallet synchronisation goes beyond pruned data” on totally new pruned node #7494

1. 
   gmaxwell commented at 1:48 am on February 10, 2016: contributor

   Coping from: https://bitcointalk.org/index.php?topic=1347286.msg13832507#msg13832507

   “I’ve been testing out 0.12.0rc3 in pruned mode, and have run into the following error message when restarting Bitcoin Core: “last wallet synchronisation goes beyond pruned data. You need to -reindex (download the whole blockchain again in case of pruned node)”

   To reproduce: 0.) Start from scratch, deleting the entire Bitcoin data directory if it exists (leaving no wallet file or blockchain data). 1.) Install Bitcoin Core 2.) Create Bitcoin data directory if it doesn’t exist, create bitcoin.conf, set “prune=550” without quotes 3.) Run Bitcoin Core 4.) After you’ve verified, say, 100,000 blocks, close Bitcoin Core. Wait until the warning window to not shutdown/restart computer completely closes. 5.) Run Bitcoin Core again, let it verify some more, close it again. 6.) Rinse and repeat step 5…

   At some point, I get the above error message on starting Bitcoin Core.

   I don’t know what exactly is causing this, but if you’re like me and you’ve verified a large amount of blocks and then you’re told that you have to start completely over, you’ll probably be pissed.

   This error message is not new, as a related fix for this kind of error, when converting non-pruned nodes to pruned nodes was made here: #6345

   While it’s obviously easy to reproduce this error if you try plopping an already existing wallet.dat file into a pruned node data directory and then running Core – that’s not what I’ve been experiencing. I’ve been seeing this error with a brand new wallet.dat that was created from scratch alongside a fresh, pruned node.”

2. 
   gmaxwell commented at 1:48 am on February 10, 2016: contributor

   please tag wallet.
3. sipa added the label Wallet on Feb 10, 2016
4. 
   mmortal03 commented at 3:05 am on February 10, 2016: none

   I’m working to attach debug.log and wallet.dat. If you need further verbosity or any other files, let me know, and I’ll try to reproduce the error again to produce these for you. It’s pretty straightforward to reproduce the issue, but time consuming and doesn’t happen every time. It’s happened to me enough times in the last few days, though, that I think I’ll be able to.

   Machine specs are: AMD E-350 Processor, 6 GB RAM, Windows 10 Pro, 64-bit. Bitcoin Core 0.12.0rc3 64-bit.

   bitcoin.conf: daemon=1 server=1 maxuploadtarget=144 prune=550

5. 
   mmortal03 commented at 3:10 am on February 10, 2016: none

   Here is the debug.log and wallet.dat: https://dde19c9eff451e7e0b7d819897f53dd506692a8f.googledrive.com/host/0B2Wm1faUufb7N3J5QkRFeHpRUkk/Bitcoin_prune_debug.zip
6. 
   laanwj commented at 11:21 am on February 10, 2016: member

   Haven’t looked at the wallet and debug.log yet, but at first glance it appears that when it is blasting though large number of blocks (e.g. during initial sync, or when catching up) it is possible for the wallet (at least the state synced to disk, even on a clean shutdown) to lag behind more than the amount of blocks that is retained.
7. laanwj added the label Priority Medium on Feb 10, 2016
8. laanwj added the label Bug on Feb 10, 2016
9. 
   laanwj commented at 1:34 pm on February 10, 2016: member

   From that wallet.dat:

   0bestblock 0000000000000000006b500f79750a2c5a48da5ad2a13fed6cdaffade6cada25 ...
   

   So block 373499. According to the debug.log the chain was way ahead of that before shutting down for the last time:

   02016-02-10 00:11:27 UpdateTip: new best=000000000000000004fd3098365fb488f31f013ee41d0de63bdb7b4b0aa1dcb2  height=373816  log2_work=83.32589  tx=83020932  date=2015-09-10 04:35:03 progress=0.854354  cache=20.6MiB(12251tx)
   

   (373816 -373499)=317 blocks to be exact.

10. 
    sipa commented at 1:52 pm on February 10, 2016: member

    It seems that the bestblock descriptor in the wallet is not updated when flushing occurs for pruning, which is a bug. It could explain the issue here, but only in the case of an unclean shutdown (as the bestblock descriptor is always updated on clean shutdown).
11. 
    laanwj commented at 2:03 pm on February 10, 2016: member

    As the debug.log does show a clean shutdown, I hope this doesn’t mean that there are problems with clean shutdown not being as clean as we think. But updating the bestblock descriptor when flushing for pruning would probably solve the immediate issue.
12. 
    morcos commented at 5:08 pm on February 10, 2016: member

    @laanwj It looks to me like a clean shutdown still doesn’t update the bestblock descriptor if it has been less than an hour. So I think this problem could happen with clean shutdowns but will be fixed properly by #7502.
13. 
    sipa commented at 5:21 pm on February 10, 2016: member

    @morcos Nice catch. I think that explains it.
14. 
    laanwj commented at 7:58 am on February 11, 2016: member

    @laanwj It looks to me like a clean shutdown still doesn’t update the bestblock descriptor if it has been less than an hour

    Yes that must be the problem then :(

    Solved by #7502 (which is in rc5)

15. laanwj closed this on Feb 11, 2016

    ---

16. MarcoFalke locked this on Sep 8, 2021

Contributors
gmaxwell mmortal03 laanwj sipa morcos

Labels
Bug Wallet