Full blocks are sent despite a filter being set #18324

1. 
   schildbach commented at 10:36 pm on March 11, 2020: contributor

   When downloading the block chain, some 0.19.0.1 nodes are sending full blocks rather than filtered blocks despite the peer having used filterload to set a filter. I’d expect to only send filtered blocks or drop the connection.

   By sending full blocks, bitcoind is currently causing high phone bills to mobile users with metered data plans.

2. schildbach added the label Bug on Mar 11, 2020
3. MarcoFalke added this to the milestone 0.19.2 on Mar 12, 2020
4. MarcoFalke added the label P2P on Mar 12, 2020
5. 
   MarcoFalke commented at 5:29 pm on March 12, 2020: member

   Can you explain the steps to reproduce or show a log excerpt? As I understand BIP 37, merely setting filterload does not make the node to send only filtered blocks. Blocks still need to be queried with a getdata that sets MSG_FILTERED_BLOCK == 3.
6. 
   MarcoFalke commented at 6:13 pm on March 12, 2020: member

   Also, if 0.19.0.1 is broken, is 0.18.* the last version that was working?
7. 
   schildbach commented at 3:01 pm on March 13, 2020: contributor

   At the current time I can’t provide logs because I didn’t see this on my own bitcoind. What I see is a massive number of txns per block (2000-3000) right from the beginning of the download from a checkpoint.

   It’s probably too early to talk about affected versions, but so far I only remember seeing 0.19.0.1.

8. 
   MarcoFalke commented at 3:22 pm on March 16, 2020: member

   With checkpoint do you mean wallet birthday?

   And to double check, you only send getdata of type 3?

   This is the function that sends out blocks: https://github.com/bitcoin/bitcoin/blob/v0.19.0.1/src/net_processing.cpp#L1357-L1510 . You can highlight the lines that have a PushMessage in it and can see that each of them is conditioned on the inv.type.

9. MarcoFalke removed this from the milestone 0.19.2 on Mar 31, 2020
10. 
    MarcoFalke commented at 2:49 am on April 1, 2020: member

    Is the type of the block message block or merkleblock?
11. 
    theStack commented at 12:35 pm on April 1, 2020: member

    @schildbach: Is there a possibility that the filterload command has not arrived at the node? Since TCP is used I think that’s unlikely to not be discovered by the client, but maybe for whatever internal reasons the packet was not even transmitted by the client. Due to the bug described in #18483, you would receive full blocks merkle blocks if you request merkleblocks, but no filterload was received by the node before. (//EDIT: Updated after the hint that there is no possibility to reply with full blocks to inv message with MSG_FILTERED_BLOCK set).
12. 
    schildbach commented at 1:00 pm on April 1, 2020: contributor

    @theStack I can’t say, because I don’t have any bitcoind logs of the issue. Our code certainly sets a filter, but of course there could be subtle bugs. I must admit that I’m not super happy about the quality and maintainability of bitcoinj’s network code. That said, I think bitcoind should just drop the connection if it detects an unexpected state. (Ideally send a REJECT first, but that’s another topic.)
13. 
    MarcoFalke commented at 3:46 pm on April 1, 2020: member

    @theStack I don’t think it is possible for Bitcoin Core to respond with a block to an inv that sets MSG_FILTERED_BLOCK. Even with the code mess from #18483.
14. 
    theStack commented at 4:26 pm on April 1, 2020: member

    Sorry for the confusion and my wrong conclusion, I was not reading thoroughly – @MarcoFalke is right, I also definitely not see a way that regular blocks are sent as reply to received invs with MSG_FILTERED_BLOCK inside.
15. 
    MarcoFalke commented at 11:14 am on April 20, 2020: member

    I’d suggest to close this issue. Absent any information on how to reproduce, or even what message types were sent in what order, this is hard to debug.

    If you have the code available of the exact version of your client software you were using, I can take a look, but other than that I don’t think we can solve this issue.

16. 
    schildbach commented at 2:37 pm on April 20, 2020: contributor

    Ok, fair enough. I’ll keep an eye on it and try to produce better logs.
17. schildbach closed this on Apr 20, 2020

    ---

18. MarcoFalke locked this on Feb 15, 2022

Contributors
schildbach MarcoFalke theStack

Labels
Bug P2P