Initial Block Download optimisations

rebroad commented at 0:26 am on October 18, 2014: contributor

I’ve been developing these improvements in parallel with headersfirst8.

It’s now been optimised for use with early to late block chain so adapts to the block sizes and also the bandwidth of each node to make optimal use of the connections available.

The sipa disconnect code has been disabled as it usually disconnected the most important node (with my changes), but it will still display when the logic would have executed.

Please do test and compare download times with before this test and I’m sure you’ll see improvements.

It’s not yet optimised for use with earlier blocks (which were small enough to download in batches of 500 each), but that will be coming shortly. Raising the pull request so that people can view and comment on some of the changes.

laanwj added the label P2P on Oct 22, 2014

rebroad force-pushed on Oct 23, 2014

rebroad commented at 8:29 am on October 23, 2014: contributor

There are probably some commits in here that can be removed. I’ve kept it fairly granular so that some of the changes are easily optional. The big change is the commit called “Track partial download progress…”, and I’ve recently updated this so that it now changes the window size per node based upon the bandwidth each note is achieving. @laanwj Is it worth tagging members of the core development team in a comment to draw their attention to this?

laanwj commented at 10:50 am on October 23, 2014: member

@rebroad Possibly. But it could be that this list of change is too intimidating for one pull, and it may be better to trickle the changes instead of having one umbrella “Headersfirst8 improvements”.

rebroad force-pushed on Oct 25, 2014

rebroad force-pushed on Oct 26, 2014

rebroad force-pushed on Oct 27, 2014

rebroad force-pushed on Oct 28, 2014

rebroad renamed this:
~~Headersfirst8 improvements~~
Initial Block Download optimisations
on Oct 28, 2014

rebroad force-pushed on Oct 29, 2014

rebroad commented at 2:30 am on October 30, 2014: contributor

tumbleweed

rebroad force-pushed on Oct 30, 2014

rebroad force-pushed on Oct 31, 2014

rebroad force-pushed on Nov 1, 2014

rebroad force-pushed on Nov 2, 2014

rebroad force-pushed on Nov 3, 2014

Add nLastDataPos variable for partial message tracking. 307fc14760

Add NodeId to CNodeState (for logging/debugging purposes) 47224d1525

Rearrange ProcessMessages 653be33b95

Track partial block download progress, timings and bandwidth, and add disconnect logic for stalls. 3f11d459ae

Add CaughtUp() function (needed by later commits) 061b287bc1

rebroad force-pushed on Nov 10, 2014

rebroad commented at 11:25 pm on November 10, 2014: contributor

I don’t know what travis is talking about - it seems to be running an old version that isn’t present in the current branch.

rebroad force-pushed on Nov 12, 2014

rebroad closed this on Nov 12, 2014

rebroad reopened this on Nov 12, 2014

rebroad force-pushed on Nov 12, 2014

Add extra debugging messages for download and stall logic. ed393b9948

Stripe downloads (so not a contiguous chunk is requested from one node). 7edb20d836

Disable sipa stall disconnect logic, but still log when it would have been triggered. 3bebc1e4a4

Disconnect no longer useful peers. 49bb701140

Find the runt of the litter (and disconnect it). a837fe4837

Reset variables when network changed. acdbc0e2ff

Allow many blocks to be received in one go from a peer for up to 1 second.

This speeds up IDB.

5744197047

Detect when headers downloading is stalling. e2b98a9473

rebroad force-pushed on Nov 13, 2014

laanwj commented at 9:27 am on March 18, 2015: member

Needs rebase.

laanwj closed this on Jul 10, 2015

DrahtBot locked this on Sep 8, 2021

Initial Block Download optimisations #5099