Speed up block downloading #964

1. 
   sipa commented at 11:08 PM on March 20, 2012: member

   I did a full block load (from a local file, using the loadblocks patch) on my laptop with these settings in 33 minutes.

2. 
   jgarzik commented at 11:48 PM on March 20, 2012: contributor

   ACK. The one-line checkpointing change probably had the largest impact.

3. 
   sipa commented at 11:55 PM on March 20, 2012: member

   @jgarzik:

   Time to load 133k blocks:

   NF: 30.4 LC: 8.6 LC+NF: 5.1 LC+NF+NS: 5.0

   LC = large cache (100 MiB cache, 100 MiB log, 10 MiB log buffer) NF = no flushing every 500 blocks NS = open database env with DB_TX_WRITE_NOSYNC

   The larger db cache size has the largest impact.

4. Speed up block downloading 89516bd4e0
5. 
   laanwj commented at 6:00 AM on March 21, 2012: member

   ACK. Nice!

6. 
   jgarzik commented at 1:25 PM on March 21, 2012: contributor

   Interesting. I guess the large cache helps with all the lookups needed to validate each block. Because we are looking up random hashes, there is likely zero cache locality from one lookup to the next.

7. 
   gavinandresen commented at 7:54 PM on March 21, 2012: contributor

   ACK

8. 
   sipa commented at 10:46 PM on March 21, 2012: member

   The only thing I'm worried about is that the longer interval between log flushes means more/larger log files, which may causes out of disk space errors.

9. 
   gmaxwell commented at 3:19 AM on March 22, 2012: contributor

   During the initial syncup you end up with ~2 gigs of database logs in any case— we don't clean them up until shutdown (or the next startup? I forget).

   I've tested this on three systems, ACK.

10. gavinandresen referenced this in commit 0e894be626 on Mar 22, 2012
11. gavinandresen merged this on Mar 22, 2012
12. gavinandresen closed this on Mar 22, 2012

    ---

13. coblee referenced this in commit 166e5bc031 on Jul 17, 2012
14. namecoin-qt referenced this in commit 8166101ae2 on Jun 12, 2013
15. davispuh referenced this in commit e44275f949 on Feb 18, 2014
16. suprnurd referenced this in commit 180a81eada on Dec 5, 2017
17. lateminer referenced this in commit a03fa6236d on Oct 30, 2019
18. lateminer referenced this in commit b9ce433bd5 on Oct 30, 2019
19. Losangelosgenetics referenced this in commit df6d08ea78 on Mar 12, 2020
20. DrahtBot locked this on Sep 8, 2021

Milestone
0.6.0