validation: Add eligible ancestors of reconsidered block to setBlockIndexCandidates #30479

pull mzumsande wants to merge 3 commits into bitcoin:master from mzumsande:202407_fix_resetfailure changing 5 files +30 −15
  1. mzumsande commented at 5:55 pm on July 18, 2024: contributor

    When we call reconsiderblock for some block,  Chainstate::ResetBlockFailureFlags puts the descendants of that block into setBlockIndexCandidates (if they meet the criteria, i.e. have more work than the tip etc.), but never put any ancestors into the set even though we do clear their failure flags.

    I think that this is wrong, because setBlockIndexCandidates should always contain all eligible indexes that have at least as much work as the current tip, which can include ancestors of the reconsidered block. This is being checked by CheckBlockIndex(), which could fail if it was invoked after ActivateBestChain connects a block and releases cs_main:

     0diff --git a/src/validation.cpp b/src/validation.cpp
     1index 7b04bd9a5b..ff0c3c9f58 100644
     2--- a/src/validation.cpp
     3+++ b/src/validation.cpp
     4@@ -3551,6 +3551,7 @@ bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr<
     5             }
     6         }
     7         // When we reach this point, we switched to a new tip (stored in pindexNewTip).
     8+        m_chainman.CheckBlockIndex();
     9 
    10         if (exited_ibd) {
    11             // If a background chainstate is in use, we may need to rebalance our
    

    makes rpc_invalidateblock.py fail on master.

    Even though we don’t currently have a CheckBlockIndex() in that place, after cs_main is released other threads could invoke it, which is happening in the rare failures of #16444 where an invalid header received from another peer could trigger a CheckBlockIndex() call that would fail.

    Fix this by adding eligible ancestors to setBlockIndexCandidates in Chainstate::ResetBlockFailureFlags (also simplifying that function a bit).

    Fixes #16444

  2. DrahtBot commented at 5:55 pm on July 18, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30479.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK TheCharlatan
    Stale ACK stratospher

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31405 (validation: stricter internal handling of invalid blocks by mzumsande)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Validation on Jul 18, 2024
  4. TheCharlatan approved
  5. TheCharlatan commented at 3:33 pm on August 23, 2024: contributor

    ACK 4a76a8300f295e8eab0ea8ab1ea1580450f131c8

    This has been the behaviour since reconsiderblock was introduced ~10 years ago https://github.com/bitcoin/bitcoin/commit/9b0a8d3152b43b63c99878d0223a1681993ad608#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R2176 .

  6. DrahtBot added the label CI failed on Sep 10, 2024
  7. DrahtBot removed the label CI failed on Sep 14, 2024
  8. achow101 requested review from sr-gi on Oct 15, 2024
  9. in src/validation.cpp:3808 in 4a76a8300f outdated
    3776@@ -3777,9 +3777,9 @@ void Chainstate::ResetBlockFailureFlags(CBlockIndex *pindex) {
    3777 
    3778     int nHeight = pindex->nHeight;
    3779 
    3780-    // Remove the invalidity flag from this block and all its descendants.
    3781+    // Remove the invalidity flag from this block and all its descendants and ancestors.
    


    stratospher commented at 4:42 am on February 25, 2025:
    4a76a83: also update comment in src/validation.h

    mzumsande commented at 7:46 pm on March 24, 2025:
    updated, thanks!
  10. stratospher commented at 6:23 am on March 1, 2025: contributor

    tested ACK 4a76a8300f295e8eab0ea8ab1ea1580450f131c8.

    makes sense that setBlockIndexCandidates should ideally contain all block indices with as much/more work than current tip regardless of whether it is an ancestor/descendant.

    I’ve tested it by demonstrating a situation where the chain tip doesn’t get correctly updated on master - https://github.com/stratospher/bitcoin/commit/57959f9b659f9df13aca3d08b2a14adc9588ac07, feel free to cherry-pick if useful.

  11. DrahtBot added the label CI failed on Mar 17, 2025
  12. mzumsande force-pushed on Mar 24, 2025
  13. mzumsande commented at 7:54 pm on March 24, 2025: contributor

    4a76a83 to adcbb2a:

    • Rebased onto master because the CI failed, as far as I can see unrelated to the PR.
    • Added test by @stratospher, thanks! (with minor changes, no need to re-format block.sha256, can just use block.hash).
    • Added a commit that replaces IsValid() by a comparison with BLOCK_FAILED_MASK. IsValid() is kind of weird in that it returns false not only if the block is invalid, but also when it hasn’t been validated yet to the given level (BLOCK_VALID_TRANSACTIONS by default). This could have led to unnecessary processing of blocks - but since that processing wouldn’t have had an effect, it’s not a bad bug.
  14. DrahtBot removed the label CI failed on Mar 24, 2025
  15. stratospher commented at 7:41 am on March 28, 2025: contributor
    reACK adcbb2a. nice catch regarding isValid() usage in ResetBlockFailureFlags!
  16. DrahtBot requested review from TheCharlatan on Mar 28, 2025
  17. in src/validation.cpp:3810 in adcbb2a32e outdated
    3838@@ -3839,7 +3839,7 @@ void Chainstate::ResetBlockFailureFlags(CBlockIndex *pindex) {
    3839 
    3840     // Remove the invalidity flag from this block and all its descendants and ancestors.
    3841     for (auto& [_, block_index] : m_blockman.m_block_index) {
    3842-        if (!block_index.IsValid() && (block_index.GetAncestor(nHeight) == pindex || pindex->GetAncestor(block_index.nHeight) == &block_index)) {
    3843+        if ((block_index.nStatus & BLOCK_FAILED_MASK) && (block_index.GetAncestor(nHeight) == pindex || pindex->GetAncestor(block_index.nHeight) == &block_index)) {
    


    TheCharlatan commented at 11:53 am on May 17, 2025:

    Nice catch! How about removing the default argument altogether:

     0diff --git a/src/chain.h b/src/chain.h
     1index c6d1640768..f5bfdb2fb4 100644
     2--- a/src/chain.h
     3+++ b/src/chain.h
     4@@ -295 +295 @@ public:
     5-    bool IsValid(enum BlockStatus nUpTo = BLOCK_VALID_TRANSACTIONS) const
     6+    bool IsValid(enum BlockStatus nUpTo) const
     7diff --git a/src/test/fuzz/chain.cpp b/src/test/fuzz/chain.cpp
     8index 0363f317b6..09053e4815 100644
     9--- a/src/test/fuzz/chain.cpp
    10+++ b/src/test/fuzz/chain.cpp
    11@@ -33 +33 @@ FUZZ_TARGET(chain)
    12-        (void)disk_block_index->IsValid();
    13+        (void)disk_block_index->IsValid(BLOCK_VALID_TRANSACTIONS);
    

    mzumsande commented at 8:54 pm on May 19, 2025:
    Good idea - done!
  18. DrahtBot requested review from TheCharlatan on May 17, 2025
  19. mzumsande force-pushed on May 19, 2025
  20. mzumsande commented at 1:35 am on May 20, 2025: contributor
    Looks like there is a intermittent failure in the added test - I’ll look into it soon. [Edit]: moved the added subtest up a bit, there was a race with the chain created / invalidated at node 1.
  21. validation: Add ancestors of reconsiderblock to setBlockIndexCandidates
    When we call reconsiderblock for some block, ResetBlockFailureFlags puts the descendants of that block
    into setBlockIndexCandidates (if they meet the criteria, i.e. have more work than the tip etc.)
    We also clear the failure flags of the ancestors, but we never put any of those into setBlockIndexCandidates
    this is wrong and could lead to failures in CheckBlockIndex().
    3c974ed7f8
  22. test: verify that ancestors of a reconsidered block can become the chain tip
    when we reconsiderblock, previously only block and it's
    descendants were considered as chain tip candidates/inserted into
    setBlockIndexCandidates
    
    ex: on this chain, with block 4 invalidated
    1 -> 2 -> 3 -> 4 -> 5 -> 6 -> header 7
    blocks 4, 5, 6, header 7 have BLOCK_FAILED_* flags set
    
    previously:
    - if we reconsiderblock header 7, the chain would have all the
    BLOCK_FAILED_* flags cleared but would report chain tip as block 3.
    - after restart, it reports correct chain tip block 6.
    
    now:
    - if we reconsiderblock header 7, the correct chain tip block 6 is
    reported since ancestors are also considered as chain tip
    candidates/inserted into setBlockIndexCandidates.
    
    Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
    2d0f583be7
  23. validation: Don't use IsValid() to filter for invalid blocks
    IsValid() also returns false for blocks that have not been
    validated yet up to the default validity level of BLOCK_VALID_TRANSACTIONS but
    are not marked as invalid - e.g. if we only know the header.
    Here, we specifically want to filter for invalid blocks.
    
    Also removes the default arg from IsValid() which is now unused outside
    of tests, to prevent this kind of misuse for the future.
    
    Co-authored-by: TheCharlatan <seb.kung@gmail.com>
    843f786029
  24. mzumsande force-pushed on May 20, 2025
  25. TheCharlatan approved
  26. TheCharlatan commented at 8:22 am on May 22, 2025: contributor
    Re-ACK 843f7860298081d0a8d12294d23b962f5586a862
  27. DrahtBot requested review from stratospher on May 22, 2025

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-05-28 18:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me