Allow 2 simultaneous (compact-)block downloads #10984

1. 
   TheBlueMatt commented at 10:48 pm on August 3, 2017: member

   This is a (somewhat) simpler version of #9447 based on #10652.

   It allows for up to 2 simultaneous downloads of the same block at once, as long as the second one is using compact blocks and only has up to 10 transactions to request in a getblocktxn (if applicable).

2. fanquake added the label P2P on Aug 5, 2017
3. fanquake added the label Validation on Aug 5, 2017
4. TheBlueMatt force-pushed on Aug 14, 2017
5. TheBlueMatt force-pushed on Aug 14, 2017
6. 
   TheBlueMatt commented at 4:10 pm on August 14, 2017: member

   Rebased and added a simple test
7. 
   TheBlueMatt commented at 9:06 pm on September 27, 2017: member

   Rebased.
8. TheBlueMatt force-pushed on Sep 27, 2017
9. 
   jonasschnelli commented at 3:40 am on September 28, 2017: contributor

   Nice work! Obviously concept ACK (have not looked closely at the code yet).
10. in src/net_processing.cpp:343 in 458feb36fc outdated

    291+    while (range.first != range.second) {
    292+        BlockDownloadMap::iterator itInFlight = range.first;
    293+        range.first++;
    294+        if (itInFlight->second.first == nodeid) {
    295+            if (clearState) ClearDownloadState(itInFlight);
    296+            mmapBlocksInFlight.erase(itInFlight);
    

    ---

    ---

    

    promag commented at 11:15 pm on October 10, 2017:

    Loose itInFlight:

    0while (range.first != range.second) {
    1    if (range.first->second.first == nodeid) {
    2        if (clearState) ClearDownloadState(range.first);
    3        range.first = mmapBlocksInFlight.erase(range.first);
    4    } else {
    5        range.first++;
    6    }
    7}
    

    ---

    

    TheBlueMatt commented at 6:44 pm on January 11, 2018:

    Heh, thats more lines (and IMO less readable)…
11. in src/net_processing.cpp:353 in 458feb36fc outdated

    322-    }
    323-    return false;
    324+    LOCK(cs_main);
    325+    bool found = false;
    326+    std::pair<BlockDownloadMap::iterator, BlockDownloadMap::iterator> range = mmapBlocksInFlight.equal_range(hash);
    327+    while (range.first != range.second) {
    

    ---

    ---

    

    promag commented at 11:20 pm on October 10, 2017:

    Loose found:

    0if (range.first == range.second) return false;
    1while (range.first != range.second) {
    2    ...
    3}
    4return true;
    

    ---

    

    TheBlueMatt commented at 6:52 pm on January 11, 2018:

    Done.
12. in src/net_processing.cpp:372 in 458feb36fc outdated

    345-    if (itInFlight != mapBlocksInFlight.end() && itInFlight->second.first == nodeid) {
    346-        if (pit) {
    347-            *pit = &itInFlight->second.second;
    348+    std::pair<BlockDownloadMap::iterator, BlockDownloadMap::iterator> range = mmapBlocksInFlight.equal_range(hash);
    349+    while (range.first != range.second) {
    350+        BlockDownloadMap::iterator itInFlight = range.first;
    

    ---

    ---

    

    promag commented at 11:22 pm on October 10, 2017:

    Same as above.

    ---

    

    TheBlueMatt commented at 6:52 pm on January 11, 2018:

    Done.
13. TheBlueMatt force-pushed on Nov 29, 2017
14. 
    TheBlueMatt commented at 5:19 pm on November 29, 2017: member

    Rebased.
15. Turn mapBlocksInFlight into a multimap dc7e41bd6f
16. Call NewPoWValidBlock callbacks for all new blocks, not just !IBD

    This pushes some "is this callback useful" logic down into
    net_processing, which is useful for later changes as it allows for
    more notifications to be used.

    489b1a3926
17. MarkBlockAsReceived on NewPoWValidBlock at receive.

    The received block could be malleated, so this is both simpler, and
    supports parallel downloads.

    c133cfeb9b
18. Only request full blocks from the peer we thought had the block in-flight

    This is a change in behavior so that if for some reason we request a block from a peer, we don't allow an unsolicited CMPCT_BLOCK announcement for that same block to cause a request for a full block from the uninvited peer (as some type of request is already outstanding from the original peer)

    a634745469
19. Allow up to 2 simultaneous downloads of the same block at once

    ...as long as the second one uses compact blocks with no more than
    10 missing transactions

    ee1e8a2507
20. Add simple parallel compact block download test 8acd39abf6
21. TheBlueMatt force-pushed on Jan 11, 2018
22. MarcoFalke added the label Needs rebase on Jun 6, 2018
23. TheBlueMatt closed this on Jun 14, 2018

    ---

24. laanwj removed the label Needs rebase on Oct 24, 2019
25. 
    iemwill commented at 11:51 am on February 24, 2021: none

    @TheBlueMatt

    This is a (somewhat) simpler version of #9447 based on #10652.

    It allows for up to 2 simultaneous downloads of the same block at once, as long as the second one is using compact blocks and only has up to 10 transactions to request in a getblocktxn (if applicable).

    The sentence above is controversial to itself.!

    EDIT: (not applicable)!

26. DrahtBot locked this on Aug 16, 2022

Contributors
TheBlueMatt jonasschnelli promag iemwill

Labels
P2P Validation