WIP: net processing: Don’t reach into CBlockIndex to check for block mutation #17485

1. 
   jnewbery commented at 10:24 pm on November 14, 2019: member

   BUILDS ON #17479. PLEASE REVIEW THAT PR FIRST.

   If a CMPCTBLOCK is in flight from peer A and we then succesfully reconstruct it during CMPCTBLOCK processing from peer B, we need to clear the in-flight state for the block from peer A.

   We can only do that once we’ve ensured that the block hasn’t been mutated (otherwise peer B could interfere with our block relay from peer A by providing a mutated block).

   Mutation-checking used to be done indirectly by checking that the block had been writted to disk by checking the CBlockIndex. Now that ProcessNewBlock returns a BlockValidationState, we can check that state directly to determine whether to mark the block as no longer in-flight.

   This PR also renames MarkBlockAsReceived() to MarkBlockAsNotInFlight() since that function can be called when the block has not been received. It also improves the comments for MarkBlockAsNotFlight() and MarkBlockAsNotInFlight()

2. [tests] Remove unnecessary cs_mains in denialofservice_tests

   9fdf05d70cac4a62d1aeeb4299e2c3a9a866f8af resolved some lock
   inversion warnings in denialofservice_tests, but left in a number
   of cs_main locks that are unnecessary (introducing lock inversion
   warnings in future changes).

   8a50e9a316
3. fanquake added the label P2P on Nov 14, 2019
4. 
   DrahtBot commented at 0:51 am on November 15, 2019: member

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #17693 (rpc: Add generateblock to mine a custom set of transactions by andrewtoth)
   • #17601 (Validation: Move CheckBlock() mutation guard to AcceptBlock() by jnewbery)
   • #17479 (Return BlockValidationState from ProcessNewBlock if CheckBlock/AcceptBlock fails by jnewbery)
   • #17399 (validation: Templatize ValidationState instead of subclassing by jkczyz)
   • #16442 (Serve BIP 157 compact filters by jimpo)
   • #16411 (BIP-325: Signet support by kallewoof)
   • #15606 ([experimental] UTXO snapshots by jamesob)
   • #15545 ([doc] explain why CheckBlock() is called before AcceptBlock by Sjors)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

5. jnewbery force-pushed on Nov 15, 2019
6. [net processing] Deduplicate post-block-processing code

   Co-authored-by: Carl Dong <contact@carldong.me>

   8cd7501997
7. [validation] Add BlockValidationState inout param to ProcessNewBlock

   This is a pure refactor commit.
   
   This commit enables the caller of ProcessNewBlock to access the final
   BlockValidationState passed around between CheckBlock(), AcceptBlock(),
   and BlockChecked() inside ProcessNewBlock(). This is useful because in a
   future commit, we will move the BlockChecked() call out of
   ProcessNewBlock(), and BlockChecked() still needs to be able to access
   the BlockValidationState.
   
   Co-authored-by: John Newbery <john@johnnewbery.com>
   Co-authored-by: Carl Dong <contact@carldong.me>

   57782d5f70
8. [net processing] Make BlockChecked a standalone, static function

   This is a pure refactor commit.
   
   Since BlockChecked() doesn't actually depend on all of
   PeerLogicValidation but just PeerLogicValidation's CConnman, we can make
   a standalone, static function that simply has an extra CConnman
   parameter and have the non-static version call the static one.
   
   This also means that, in a future commit, when we move the
   BlockChecked() call out of ProcessNewBlock(), the caller of
   ProcessNewBlock() can call BlockChecked() directly even if they only
   have a CConnman.
   
   Co-authored-by: John Newbery <john@johnnewbery.com>
   Co-authored-by: Carl Dong <contact@carldong.me>

   bbc1bca53d
9. [validation] Return the AcceptBlock BlockValidationState directly in ProcessNewBlock

   Net processing now passes a BlockValidationState object into
   ProcessNewBlock(). If CheckBlock() or AcceptBlock() fails, then PNB
   returns to net processing without calling the (asynchronous)
   BlockChecked Validation Interface method. net processing can use the
   invalid BlockValidationState returned to punish peers.
   
   CheckBlock() and AcceptBlock() represent the DoS checks on a block (ie
   PoW and malleability). Net processing wants to know about those failed
   checks immediately and shouldn't have to wait on a callback. Other
   validation interface clients don't care about net processing submitting
   bogus malleated blocks to validation, so they don't need to be notified
   of BlockChecked.
   
   Furthermore, if PNB returns a valid BlockValidationState, we never need
   to try to process (non-malleated) copies of the block from other peers.
   That makes it much easier to move the best chain activation logic to a
   background thread in future work.
   
   Co-authored-by: John Newbery <john@johnnewbery.com>
   Co-authored-by: Carl Dong <contact@carldong.me>

   a49a153f7c
10. [validation] trivial: Rename state to dummy_state for clarity

    This is a pure refactor commit.
    
    Co-authored-by: John Newbery <john@johnnewbery.com>
    Co-authored-by: Carl Dong <contact@carldong.me>

    91d26915cd
11. [test/rpc] Additional checks for dos_state validity

    Co-authored-by: John Newbery <john@johnnewbery.com>
    Co-authored-by: Carl Dong <contact@carldong.me>

    0bf87d2034
12. [net processing] Improve comments for MarkBlockAsInFlight and MarkBlockAsReceived e6a70e5b8f
13. [net processing] rename MarkBlockAsReceived -> MarkBlockAsNotInFlight

    The previous name was misleading, since we can call the function even
    when the block has not been received.

    367ce754aa
14. [net processing] Don't reach into CBlockIndex to check for block mutation

    If a CMPCTBLOCK is in flight from peer A and we then succesfully
    reconstruct it during CMPCTBLOCK processing from peer B, we need to
    clear the in-flight state for the block from peer A.
    
    We can only do that once we've ensured that the block hasn't been
    mutated (otherwise peer B could interfere with our block relay from peer
    A by providing a mutated block).
    
    Mutation-checking used to be done indirectly by checking that the block
    had been writted to disk by checking the CBlockIndex. Now that
    ProcessNewBlock returns a BlockValidationState, we can check that state
    directly to determine whether to mark the block as no longer in-flight.

    11b7abdb42
15. jnewbery force-pushed on Nov 18, 2019
16. DrahtBot added the label Needs rebase on Dec 16, 2019
17. 
    DrahtBot commented at 9:53 pm on December 16, 2019: member
18. jnewbery renamed this:
    net processing: Don't reach into CBlockIndex to check for block mutation
    WIP: net processing: Don't reach into CBlockIndex to check for block mutation
    on Jan 2, 2020
19. 
    jnewbery commented at 8:56 pm on March 13, 2020: member

    Closing for now. No need for this to be open while #17479 is not yet merged. Will reopen when #17479 is merged.
20. jnewbery closed this on Mar 13, 2020

    ---

21. DrahtBot locked this on Feb 15, 2022

Contributors
jnewbery DrahtBot

Labels
P2P Needs rebase