already have block - causing bandwidth to be wasted - vicious circle? #1120

1. 
   rebroad commented at 10:43 pm on April 17, 2012: contributor

   It occurs to me that this happens as the node asks several other nodes for the same blocks when it does not see the block appear within a certain time. Due to the speed of the nodes providing blocks, they both respond with the blocks requested, but one will send the block first and usually stay ahead. This isn’t a major issue, but it does mean that bandwidth is wasted, and since it’s probably the bandwidth which is causing the problem in the first place, I expect this will continue to get worse due to being a vicious circle. My proposed solution probably requires a BIP, which would allow a node to request to another node to stop sending blocks (i.e. to the node sending the blocks which it already has). It could do this after one duplicate block, or perhaps after several.

   Sample case:

   04/17/12 22:08:49 askfor block 000000000000096fd80c 0 04/17/12 22:08:50 sending getdata: block 000000000000096fd80c 04/17/12 22:14:12 askfor block 000000000000096fd80c 1334700530000000 04/17/12 22:14:12 sending getdata: block 000000000000096fd80c 04/17/12 22:16:39 received block 000000000000096fd80c 04/17/12 22:16:40 SetBestChain: new best=000000000000096fd80c height=176035 work=29615794588444859740 04/17/12 22:20:07 received block 00000000000009856606 04/17/12 22:20:07 ERROR: ProcessBlock() : already have block 176020 00000000000009856606 04/17/12 22:20:56 received block 00000000000008862df7 04/17/12 22:20:56 ERROR: ProcessBlock() : already have block 176021 00000000000008862df7 04/17/12 22:21:36 received block 000000000000035a84ac 04/17/12 22:21:36 ERROR: ProcessBlock() : already have block 176022 000000000000035a84ac 04/17/12 22:21:59 received block 0000000000000a881c26 04/17/12 22:21:59 ERROR: ProcessBlock() : already have block 176023 0000000000000a881c26 04/17/12 22:22:18 received block 00000000000004ed81e6 04/17/12 22:22:18 ERROR: ProcessBlock() : already have block 176024 00000000000004ed81e6 04/17/12 22:22:52 received block 000000000000068bbe48 04/17/12 22:22:52 ERROR: ProcessBlock() : already have block 176025 000000000000068bbe48 04/17/12 22:23:12 received block 00000000000003425f71 04/17/12 22:23:12 ERROR: ProcessBlock() : already have block 176026 00000000000003425f71 04/17/12 22:23:39 received block 00000000000000f89962 04/17/12 22:23:39 ERROR: ProcessBlock() : already have block 176027 00000000000000f89962 04/17/12 22:23:48 received block 000000000000083ec278 04/17/12 22:23:48 ERROR: ProcessBlock() : already have block 176028 000000000000083ec278 04/17/12 22:24:06 received block 000000000000088984c6 04/17/12 22:24:06 ERROR: ProcessBlock() : already have block 176029 000000000000088984c6 04/17/12 22:24:29 received block 00000000000002857e8b 04/17/12 22:24:29 ERROR: ProcessBlock() : already have block 176030 00000000000002857e8b 04/17/12 22:24:45 received block 000000000000075834ed 04/17/12 22:24:45 ERROR: ProcessBlock() : already have block 176031 000000000000075834ed 04/17/12 22:25:10 received block 000000000000033b7b3e 04/17/12 22:25:10 ERROR: ProcessBlock() : already have block 176032 000000000000033b7b3e 04/17/12 22:25:56 received block 00000000000001f32a95 04/17/12 22:25:56 ERROR: ProcessBlock() : already have block 176033 00000000000001f32a95 04/17/12 22:26:20 received block 00000000000004dffd6d 04/17/12 22:26:20 ERROR: ProcessBlock() : already have block 176034 00000000000004dffd6d 04/17/12 22:26:20 received block 000000000000096fd80c 04/17/12 22:26:20 ERROR: ProcessBlock() : already have block 176035 000000000000096fd80c

   Much of this output has been cut, but if I grep my debug.log for “already have”, I can see that blocks 174591 to 176061 (1470 blocks and counting) were being sent by two nodes.

2. 
   sipa commented at 10:55 pm on April 17, 2012: member

   Is this unmodified source code?
3. 
   rebroad commented at 10:58 pm on April 17, 2012: contributor

   @sipa, yes, stock 0.6.0.6-beta, running on Windows 7, using tor.
4. 
   rebroad commented at 1:55 pm on April 28, 2012: contributor

   My proposal for solving this would be to introduce a “stop sending me blocks” command, or a “I already had those blocks” command, which would cause the sending node to stop sending. For backwards compatibility, old nodes would ignore these commands, so nodes would give them a chance to stop, and if they continued to send duplicate blocks (perhaps 6 already haves in a row) they would simply disconnect from the node in question.
5. 
   rebroad commented at 9:52 pm on April 28, 2012: contributor

   I can also confirm that this problem exists in a recent version of litecoin.
6. 
   rebroad commented at 10:26 pm on April 28, 2012: contributor

   Is it possible for ProcessMessages() to determine which block is being sent before it’s finished downloading it? Perhaps get advance notification of its hash rather than wait for the block to completely download before realising it’s wasted time and bandwidth receiving it?
7. 
   sipa commented at 10:35 pm on April 28, 2012: member

   That would require some structural changes to the code, and I’m not sure it’s necessary. I’d much rather like to know why your node requests blocks twice.
8. 
   rebroad commented at 11:18 pm on April 28, 2012: contributor

   Isn’t it standard behaviour that it requests the same block more than once if the nodes requested do not respond within a certain time period?
9. 
   rebroad commented at 0:58 am on April 29, 2012: contributor

   I’ve added some debugging code to work out what’s going on, and I’ve noticed that in ProcessMessages() that the blocks seem to get stuck downloading. The vRecv.size() sometimes never reaches nMessageSize. i.e. every time ProcessMessages() is called, the vRecv.size() is still the same as last time for the same node. I’m pretty sure my added code for debugging isn’t the cause of this.

   I’ll upload this code as soon as I obtain sufficient intelligence to navigate git….

10. 
    rebroad commented at 3:43 pm on May 7, 2012: contributor

    One solution to this problem is to add the ability to include a checksum/hash of the block in the block header, so the node can reject it without downloading it if it already has it. Only one other solution I can think of, other than putting it with receiving blocks that the node already has, but which took so long to arrive that the node ended up requesting it elsewhere.

    The other solution I can think of is to enable ProcessBlock to be threaded, as it may be due to this causing nodes to take a long time to respond to messages that’s causing them to time-out and ask elsewhere.

11. 
    rebroad commented at 5:15 am on May 12, 2013: contributor

    I’d like to understand exactly why pull request #1382 was closed. The issue was that nodes were being disconnected when they were simply doing as they were told - but what is the alternative to this? How do we avoid receiving duplicate blocks? The only way I see this is to NEVER request the same block from another node unless the nodes previously requested from have disconnected. Surely there can be circumstances in which this could take a very long time for these nodes to disconnect, during which time only orphan blocks can be requested, and the blockchain would appear stuck during this time. Or am I missing something?
12. 
    sipa commented at 8:46 am on May 12, 2013: member

    @rebroad If a node is slow in responding, you need to switch to another peer for downloading. If a node is really slow, you can choose to disconnect him. If a node doesn’t respond at all, he’ll timeout after one minute anyway. But without logic to at least prevent unintentional double requests, disconnecting a peer simply because they do what you asked seems pretty much cutting in your own flesh.
13. 
    laanwj commented at 7:53 am on October 26, 2013: member

    Closing, as it “works as intended”. Pull requests to speed up block downloading are on the way and could use testing, see #2964.
14. laanwj closed this on Oct 26, 2013

    ---

15. suprnurd referenced this in commit ee60f6e76a on Dec 5, 2017
16. lateminer referenced this in commit 964f326bb1 on Jan 22, 2019
17. lateminer referenced this in commit 0d04267c89 on Dec 25, 2019
18. Bushstar referenced this in commit 92bc19f610 on May 2, 2020
19. DrahtBot locked this on Sep 8, 2021

Contributors
rebroad sipa laanwj