Parallel compact block download #25258

1. 
   ajtowns commented at 9:08 pm on May 31, 2022: contributor

   There have been a couple of old attempts to enable parallel compact block downloads, see #9447 and #10984. There are additional notes in #16172.

   The idea is that we currently might request missing transactions for a new block from the first peer to announce the block to us; but if that peer is a miner (or close to a miner), that peer might be busy (eg it might be in ConnectTip), and not able to reply quickly, while some other node has meanwhile obtained the block and is able to quickly give you the few transactions you’re missing.

   Therefore, at some point we should pick up the patches in those earlier PRs, or otherwise solve this :)

2. ajtowns added the label Feature on May 31, 2022
3. 
   TheBlueMatt commented at 6:00 pm on February 6, 2023: contributor

   FWIW, the lack of this feature is (mostly) what made the FIBRE patchset difficult to maintain. This is the only part of that patchset that materially conflicted with new releases or even had much to touch inside Bitcoin Core. It would be great to see this to revive the FIBRE patchset.
4. 
   instagibbs commented at 12:27 pm on May 9, 2023: member

   This also seems to happen with spy(?) nodes which are the first to send a header, but then never end up responding to requests for the full block. Due to recent mempool chop, the chances of requiring a round-trip for slightly-later-but-behaving high bandwidth peer’s cb goes way up, and currently cannot be accommodated.

   couple of questions to think about:

   1. should the 2nd download only work for outbound connections? currently there are 3 hb peers possible, and only one outbound is “protected”. Sybil nodes can take up 2 inbound slots pretty easily. Perhaps if first to announce is outbound, we allow an inbound as second as well?
   2. is 10 transaction diff as per #10984 reasonable with today’s mempool characteristics?
5. 
   Sjors commented at 7:26 pm on May 12, 2023: member

   If we wait a little bit before the second attempt, and if we track which peers have since announced the header, we could pick from a larger set. Randomly and maybe with a preference for outbound. We can rely on high bandwidth mode compact block peer(s) for speed, but then for reliability use a bit more patience. Maybe we could even do one attempt at soliciting a compact block and then a few seconds later ask for a regular block.
6. fanquake referenced this in commit 51c050787f on May 24, 2023
7. fanquake closed this on May 24, 2023

   ---

8. 
   mzumsande commented at 2:15 pm on May 24, 2023: contributor

   should this stay open for a while as per #27626 (comment)?
9. 
   instagibbs commented at 2:16 pm on May 24, 2023: member

   We could open a new issue with fresh context as well, summarizing the discussion to this point?
10. 
    fanquake commented at 2:16 pm on May 24, 2023: member

    should this stay open

    The PR shouldn’t stay open, but we can open a tracking issue if needed.

11. 
    instagibbs commented at 2:30 pm on May 24, 2023: member

    opened a new issue: https://github.com/bitcoin/bitcoin/issues/27740
12. bitcoin locked this on May 23, 2024

Contributors
ajtowns TheBlueMatt instagibbs Sjors mzumsande fanquake

Labels
Feature