Prune: Support noncontiguous block files #6221

1. 
   ajweiss commented at 7:41 pm on June 2, 2015: contributor

   In some corner cases, it may be possible for recent blocks to end up in the same block file as much older blocks. Previously, the pruning code would stop looking for files to remove upon first encountering a file containing a block that cannot be pruned, now it will keep looking for candidate files until the target is met and all other criteria are satisfied.

   This can result in a noncontiguous set of block files (by number) on disk, which is fine except for during some reindex corner cases, so make reindex preparation smarter such that we keep the data we can actually use and throw away the rest. This allows pruning to work correctly while downloading any blocks needed during the reindex.

2. laanwj added the label Block storage on Jun 3, 2015
3. Prune: Support noncontiguous block files

   In some corner cases, it may be possible for recent blocks to end up in
   the same block file as much older blocks.  Previously, the pruning code
   would stop looking for files to remove upon first encountering a file
   containing a block that cannot be pruned, now it will keep looking for
   candidate files until the target is met and all other criteria are
   satisfied.
   
   This can result in a noncontiguous set of block files (by number) on
   disk, which is fine except for during some reindex corner cases, so
   make reindex preparation smarter such that we keep the data we can
   actually use and throw away the rest.  This allows pruning to work
   correctly while downloading any blocks needed during the reindex.

   c257a8c9a6
4. in src/init.cpp: in 588bbb05cf outdated

   509-            if ((it->path().filename().string().length() == 12) &&
   510-                (it->path().filename().string().substr(8,4) == ".dat") &&
   511-                ((it->path().filename().string().substr(0,3) == "blk") ||
   512-                 (it->path().filename().string().substr(0,3) == "rev")))
   513-                boost::filesystem::remove(it->path());
   514+    int nContigCounter = 0;
   

   ---

   ---

   

   laanwj commented at 9:26 am on June 3, 2015:

   So the goal here is to remove all block files, unless they’re contiguous and starting from 0? Maybe add a comment.
5. ajweiss force-pushed on Jun 3, 2015
6. 
   ajweiss commented at 5:27 pm on June 3, 2015: contributor

   Ok, I’ve reworked the comments to make things far more explicit…
7. 
   laanwj commented at 7:00 am on June 4, 2015: member

   Thanks for adding documentation. utACK.
8. 
   laanwj commented at 4:23 pm on June 11, 2015: member

   Tested by @BitpopCoin in #6184, confirmed to fix the issue
9. laanwj merged this on Jun 11, 2015
10. laanwj closed this on Jun 11, 2015

    ---

11. laanwj referenced this in commit dd8fe8215a on Jun 11, 2015
12. 
    laanwj commented at 4:28 pm on June 11, 2015: member

    Cherry-picked to 0.11 as 6cb70ca4ee353ed7e54a357c08ba41d13752731a
13. laanwj referenced this in commit 6cb70ca4ee on Jun 11, 2015
14. DrahtBot locked this on Sep 8, 2021

Contributors
ajweiss laanwj

Labels
Block storage