validation: stricter internal handling of invalid blocks #31405

pull mzumsande wants to merge 7 commits into bitcoin:master from mzumsande:202411_stricter_invalidblock_handling changing 2 files +38 −76
  1. mzumsande commented at 7:29 pm on December 2, 2024: contributor

    Some fields in validation are set opportunistically by “best effort”:

    • The BLOCK_FAILED_CHILD status (which means that the block index has an invalid predecessor)
    • m_best_header (the most-work header not known to be invalid).

    This means that there are known situations in which these fields are not set when they should be, or set to wrong values. This is tolerated because the fields are not used for anything consensus-critical and triggering these situations involved creating invalid blocks with valid PoW header, so would have a cost attached. Also, having stricter guarantees for these fields requires iterating over the entire block index, which has some DoS potential, especially with any header above the checkpoint being accepted int he past (see e.g. #11531).

    However, there are reasons to change this now:

    • RPCs use these fields and can report wrong results
    • There is the constant possibility that someone could add code that expects these fields to be correct, especially because it is not well documented that these fields cannot always be relied upon.
    • DoS concerns have become less of an issue after #25717 - now an attacker would need to invest much more work because they can’t fork off the last checkpoint anymore

    This PR continues the work from #30666 to ensure that BLOCK_FAILED_CHILD status and m_best_header are always correct:

    • it adds a call to InvalidChainFound() in AcceptBlock().
    • it adds checks for BLOCK_FAILED_CHILD and m_best_header to CheckBlockIndex(). In order to be able to do this, two more caches were added to the RPC-only InvalidateBlock() similar to the existing candidate_blocks_by_work for setBlockIndexCandidates. These are performance optimizations with the goal of avoiding having a call of InvalidChainFound() / looping over the block index after each disconnected block. I also wrote a fuzz test to find possible edge cases violating CheckBlockIndex, which I will PR separately soon.
    • it removes the m_failed_blocks set, which was a heuristic necessary when we couldn’t be sure if a given block index had an invalid predecessor or not. Now that we have that guarantee, the set is no longer needed.
  2. DrahtBot commented at 7:29 pm on December 2, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31405.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK sipa, TheCharlatan
    Concept ACK stickies-v, stringintech
    Stale ACK stratospher

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31835 (validation: set BLOCK_FAILED_CHILD correctly by stratospher)
    • #30479 (validation: Add eligible ancestors of reconsidered block to setBlockIndexCandidates by mzumsande)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Validation on Dec 2, 2024
  4. mzumsande force-pushed on Dec 2, 2024
  5. DrahtBot added the label CI failed on Dec 2, 2024
  6. DrahtBot commented at 8:47 pm on December 2, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33808912013

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  7. DrahtBot removed the label CI failed on Dec 2, 2024
  8. mzumsande marked this as ready for review on Dec 3, 2024
  9. stratospher commented at 2:14 pm on January 13, 2025: contributor

    ACK b2136da9

    • Went through the logs in rpc_invalidateblock.py and checked how the children were marked invalid/m_best_header was set on master vs in this PR
    • Went through logs of some functional tests like feature_block.py and verified that BLOCK_FAILED_CHILD wasn’t being set in m_failed_blocks loop
    • Ran invalidateblock RPC on a full node.
  10. in src/validation.cpp:5381 in b2136da98d outdated
    5358@@ -5367,6 +5359,8 @@ void ChainstateManager::CheckBlockIndex()
    5359         if (pindexFirstInvalid == nullptr) {
    5360             // Checks for not-invalid blocks.
    5361             assert((pindex->nStatus & BLOCK_FAILED_MASK) == 0); // The failed mask cannot be set for blocks without invalid parents.
    5362+        } else {
    


    stratospher commented at 6:41 am on January 14, 2025:

    528b8cc: this would be more accurate for descendants.

    0        } else if (pindexFirstInvalid != pindex) {
    

    mzumsande commented at 6:07 pm on January 28, 2025:
    I changed the comment to “Invalid blocks and their descendants must be marked as invalid” - doesn’t really make sense to me to make an exception for the block itself.
  11. in src/validation.cpp:3745 in b2136da98d outdated
    3738@@ -3729,6 +3739,28 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
    3739             m_blockman.m_dirty_blockindex.insert(to_mark_failed);
    3740         }
    3741 
    3742+        // Mark descendants of the invalidated block as invalid
    3743+        auto range = cand_invalid_descendants.equal_range(invalid_walk_tip);
    3744+        for (auto it = range.first; it != range.second; ++it) {
    3745+            it->second->nStatus |= BLOCK_FAILED_CHILD;
    


    stratospher commented at 7:33 am on January 14, 2025:

    7f88af8b: when invalidateblock is called multiple times, we would need to clear the BLOCK_FAILED_VALID from previous iteration so that both BLOCK_FAILED_VALID and BLOCK_FAILED_CHILD don’t happen together on the same block. ex: this situation

    0if (it->second->nStatus & BLOCK_FAILED_VALID) {
    1    it->second->nStatus = (it->second->nStatus ^ BLOCK_FAILED_VALID) | BLOCK_FAILED_CHILD;
    2} else {
    3    it->second->nStatus |= BLOCK_FAILED_CHILD;
    4}
    

    mzumsande commented at 6:08 pm on January 28, 2025:
    Done, made it a bit shorter: it->second->nStatus = (it->second->nStatus & ~BLOCK_FAILED_VALID) | BLOCK_FAILED_CHILD;
  12. validation: call InvalidChainfound also from AcceptBlock
    In this scenario we have stored a valid header with an invalid (but not
    mutated!) block, so it can only be triggered by doing the
    necessary PoW.
    The added call ensures that descendant blocks failure flags and m_best_header
    will be set correctly - at the cost of looping over the entire block
    index, which, because of the mentioned necessary PoW, is not a DoS
    vector.
    163b01eae8
  13. mzumsande force-pushed on Jan 28, 2025
  14. mzumsande force-pushed on Jan 28, 2025
  15. mzumsande commented at 6:10 pm on January 28, 2025: contributor
    b2136da to 4ba2e48: Addressed review comments by @stratospher (Thanks!) and rebased.
  16. in src/validation.cpp:5382 in eca8d9182e outdated
    5434@@ -5434,6 +5435,8 @@ void ChainstateManager::CheckBlockIndex()
    5435         if (pindexFirstInvalid == nullptr) {
    5436             // Checks for not-invalid blocks.
    5437             assert((pindex->nStatus & BLOCK_FAILED_MASK) == 0); // The failed mask cannot be set for blocks without invalid parents.
    5438+        } else {
    5439+            assert((pindex->nStatus & BLOCK_FAILED_MASK)); // Invalid blocks and their descendants must be marked as invalid
    


    stratospher commented at 8:40 am on February 6, 2025:
    eca8d91: micro-style-nit only if you have to retouch - could make double (()) to ().
  17. stratospher commented at 3:17 pm on February 10, 2025: contributor
    reACK 4ba2e48.
  18. stickies-v commented at 4:48 pm on March 3, 2025: contributor
    PSA: Bitcoin Core PR Review Club will cover this PR in its next meeting at 2025-03-05 at 17:00 UTC. See https://bitcoincore.reviews/31405 for notes, questions, and instructions on how to join.
  19. in src/validation.cpp:3773 in 4ba2e480ff outdated
    3763@@ -3754,6 +3764,29 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
    3764             m_blockman.m_dirty_blockindex.insert(to_mark_failed);
    3765         }
    3766 
    3767+        // Mark descendants of the invalidated block as invalid
    3768+        // (possibly replacing a pre-existing BLOCK_FAILED_VALID with BLOCK_FAILED_CHILD)
    3769+        auto range = cand_invalid_descendants.equal_range(invalid_walk_tip);
    3770+        for (auto it = range.first; it != range.second; ++it) {
    3771+            it->second->nStatus = (it->second->nStatus & ~BLOCK_FAILED_VALID) | BLOCK_FAILED_CHILD;
    3772+        }
    


    stickies-v commented at 9:32 pm on March 3, 2025:
    Is there a reason why we forcefully unset BLOCK_FAILED_VALID? Since both flags store orthogonal information in separate bits, intuitively I’d expect we could have both co-exist? If this breaks other assumptions, perhaps good to add to the documentation here?

    mzumsande commented at 6:34 pm on March 4, 2025:
    There is some more discussion about this in #31835, where I wrote my opinion. Since BLOCK_FAILED_CHILD has never been used for anything, it’s hard to say what the intentions really were, but my impression is that the first invalid block should be marked as BLOCK_FAILED_VALID and all of its descendants as BLOCK_FAILED_CHILD - so having both BLOCK_FAILED_VALID and BLOCK_FAILED_CHILD at the same time doesn’t make sense to me.
  20. in src/validation.cpp:3717 in 4ba2e480ff outdated
    3710@@ -3707,6 +3711,12 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
    3711                     candidate->HaveNumChainTxs()) {
    3712                 candidate_blocks_by_work.insert(std::make_pair(candidate->nChainWork, candidate));
    3713             }
    3714+            // Similarly, populate cache for blocks not in main chain to invalidate
    3715+            if (!m_chain.Contains(candidate) &&
    3716+                !CBlockIndexWorkComparator()(candidate, pindex->pprev)) {
    3717+                cand_invalid_descendants.insert(std::make_pair(m_chain.FindFork(candidate), candidate));
    


    stickies-v commented at 9:48 pm on March 3, 2025:
    It’s not clear to me why we would only want to mark blocks with large-enough PoW as invalid descendants. If this is not a bug, I think behaviour is inconsistent with the commit message and should probably be document in the code too?

    mzumsande commented at 6:52 pm on March 4, 2025:
    With pindex->pprev being the predecessor of the block passed to the invalidateblock rpc call, the logic in InvalidateBlock() is to go from the tip back to pindex->pprev - and we may need to update other forked descendants of the blocks encountered along that way (which must have more accumulated work than their parents), which is why we put them in the cache. But all blocks with lower work can never be encountered by that process and therefore don’t need to be in the caches. I’ll add more documentation with my next push.

    stickies-v commented at 12:07 pm on March 5, 2025:
    Oh sorry, I get it now, thanks. For some reason (I think I had just been reviewing the next section dealing with disconnecting the tip) I was confusing pindex for the chain tip here. I don’t really have any doc suggestions, so can be marked as resolved.
  21. in src/validation.cpp:3697 in 4ba2e480ff outdated
    3689@@ -3691,6 +3690,11 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
    3690     // build a map once so that we can look up candidate blocks by chain
    3691     // work as we go.
    3692     std::multimap<const arith_uint256, CBlockIndex *> candidate_blocks_by_work;
    3693+    // Map to cache candidates not in the main chain that might need invalidating.
    3694+    // Maps fork block in chain to the candidates for invalidation.
    3695+    std::multimap<const CBlockIndex*, CBlockIndex*> cand_invalid_descendants;
    3696+    // Map to cache candidates for m_best_header
    3697+    std::multimap<const arith_uint256, CBlockIndex*> best_header_blocks_by_work;
    


    stickies-v commented at 0:49 am on March 4, 2025:

    Since this map is a superset of candidate_blocks_by_work , I think we could just have one, and instead check for IsValid(BLOCK_VALID_TRANSACTIONS) && HaveNumChainTxs() further down when we update setBlockIndexCandidates? I suspect the memory impllications should be negligible either way, but I think it would clean up the code a bit?

    E.g. something like:

     0diff --git a/src/validation.cpp b/src/validation.cpp
     1index f9c900ef27..ead0454f7b 100644
     2--- a/src/validation.cpp
     3+++ b/src/validation.cpp
     4@@ -3689,12 +3689,10 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
     5     // To avoid walking the block index repeatedly in search of candidates,
     6     // build a map once so that we can look up candidate blocks by chain
     7     // work as we go.
     8-    std::multimap<const arith_uint256, CBlockIndex *> candidate_blocks_by_work;
     9+    std::multimap<const arith_uint256, CBlockIndex *> highpow_outofchain_headers;
    10     // Map to cache candidates not in the main chain that might need invalidating.
    11     // Maps fork block in chain to the candidates for invalidation.
    12     std::multimap<const CBlockIndex*, CBlockIndex*> cand_invalid_descendants;
    13-    // Map to cache candidates for m_best_header
    14-    std::multimap<const arith_uint256, CBlockIndex*> best_header_blocks_by_work;
    15 
    16     {
    17         LOCK(cs_main);
    18@@ -3706,16 +3704,9 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
    19             // Instead, consider only non-active-chain blocks that have at
    20             // least as much work as where we expect the new tip to end up.
    21             if (!m_chain.Contains(candidate) &&
    22-                    !CBlockIndexWorkComparator()(candidate, pindex->pprev) &&
    23-                    candidate->IsValid(BLOCK_VALID_TRANSACTIONS) &&
    24-                    candidate->HaveNumChainTxs()) {
    25-                candidate_blocks_by_work.insert(std::make_pair(candidate->nChainWork, candidate));
    26-            }
    27-            // Similarly, populate cache for blocks not in main chain to invalidate
    28-            if (!m_chain.Contains(candidate) &&
    29-                !CBlockIndexWorkComparator()(candidate, pindex->pprev)) {
    30+                    !CBlockIndexWorkComparator()(candidate, pindex->pprev)) {
    31+                highpow_outofchain_headers.insert(std::make_pair(candidate->nChainWork, candidate));
    32                 cand_invalid_descendants.insert(std::make_pair(m_chain.FindFork(candidate), candidate));
    33-                best_header_blocks_by_work.insert(std::make_pair(candidate->nChainWork, candidate));
    34             }
    35         }
    36     }
    37@@ -3774,11 +3765,9 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
    38         // Determine new best header
    39         if (m_chainman.m_best_header->nStatus & BLOCK_FAILED_MASK) {
    40             m_chainman.m_best_header = invalid_walk_tip->pprev;
    41-            auto best_header_it = best_header_blocks_by_work.lower_bound(m_chainman.m_best_header->nChainWork);
    42-            while (best_header_it != best_header_blocks_by_work.end()) {
    43-                if (best_header_it->second->nStatus & BLOCK_FAILED_MASK) {
    44-                    best_header_it = best_header_blocks_by_work.erase(best_header_it);
    45-                } else {
    46+            auto best_header_it = highpow_outofchain_headers.lower_bound(m_chainman.m_best_header->nChainWork);
    47+            while (best_header_it != highpow_outofchain_headers.end()) {
    48+                if (!(best_header_it->second->nStatus & BLOCK_FAILED_MASK)) {
    49                     if (!CBlockIndexWorkComparator()(best_header_it->second, m_chainman.m_best_header)) {
    50                         m_chainman.m_best_header = best_header_it->second;
    51                     }
    52@@ -3788,11 +3777,12 @@ bool Chainstate::InvalidateBlock(BlockValidationState& state, CBlockIndex* pinde
    53         }
    54 
    55         // Add any equal or more work headers to setBlockIndexCandidates
    56-        auto candidate_it = candidate_blocks_by_work.lower_bound(invalid_walk_tip->pprev->nChainWork);
    57-        while (candidate_it != candidate_blocks_by_work.end()) {
    58-            if (!CBlockIndexWorkComparator()(candidate_it->second, invalid_walk_tip->pprev)) {
    59-                setBlockIndexCandidates.insert(candidate_it->second);
    60-                candidate_it = candidate_blocks_by_work.erase(candidate_it);
    61+        auto candidate_it = highpow_outofchain_headers.lower_bound(invalid_walk_tip->pprev->nChainWork);
    62+        while (candidate_it != highpow_outofchain_headers.end()) {
    63+            if (!CBlockIndexWorkComparator()(candidate_it->second, invalid_walk_tip->pprev) &&
    64+                candidate_it->second->IsValid(BLOCK_VALID_TRANSACTIONS) &&
    65+                candidate_it->second->HaveNumChainTxs()) {
    66+                    setBlockIndexCandidates.insert(candidate_it->second);
    67             } else {
    68                 ++candidate_it;
    69             }
    

    mzumsande commented at 10:26 pm on March 18, 2025:
    That’s a good idea! I think we can even go one step further and use highpow_outofchain_headers for everything, also for cand_invalid_descendants (at the cost of doing a few GetAncestor() calls, which should be cheap enough). I pushed an updated version that uses this approach.
  22. stickies-v commented at 0:58 am on March 4, 2025: contributor
    Concept ACK for making m_best_header and nStatus more reliably correct, and the trade-offs seem reasonable. Even though the diff is small, I find reasoning about these changes fairly complex and require a lot of context, so I’ll need to think through the implications more.
  23. stringintech commented at 6:37 pm on March 8, 2025: none

    Concept ACK.

    I wrote two test cases to understand the general behavior of InvalidateBlock, not specifically to test this PR’s changes:

    0A->B->C->D->E->F (main chain)
    1A->B->C->G (fork 1)
    2A->B->H->I->J (fork 2)
    3A->B->H->K (fork 3)
    

    In the first test, I invalidate block C (on active chain) which should invalidate C, D, E, F, and G. The second test invalidates H (on a fork) which should invalidate H, I, J, and K.

    Both tests behave the same on the master and PR branches. The first test fails on both because D, E, and F are not marked as BLOCK_FAILED_CHILD, which I did not expect.

    I have two questions:

    1 ) I found what appears to be a bug in InvalidateBlock: https://github.com/bitcoin/bitcoin/blob/4637cb1eec48d1af8d23eeae1bb4c6f8de55eed9/src/validation.cpp#L3750

    The first part of the condition seems incorrect and should probably be:

    0if (to_mark_failed->pprev == invalid_walk_tip ...) {
    

    The assertions that fail in the first test are for blocks D, E, and F which were all part of the best chain before invalidation, which led me to this part of the code. I’m not sure if fixing this is in scope for this PR.

    Update: Just saw the PR #31835 which fixes this issue. :)

    2 ) To validate my understanding: Since both branches behave similarly (even on master the m_best_header and BLOCK_FAILED_CHILD statuses are set correctly because of the final call to InvalidChainFound - except for the ones not set as BLOCK_FAILED_CHILD because of the potential bug), is it correct that the main goal of changes applied in InvalidateBlock is to maintain proper block statuses and best header between tip disconnections by marking children as invalid right away and recalculating the best header immediately rather than doing it later?

  24. mzumsande commented at 7:23 pm on March 17, 2025: contributor

    Update: Just saw the PR #31835 which fixes this issue. :)

    Heh, looks like you are at least the third person who found that bug independently, first one was in #16856!

    Since both branches behave similarly (even on master the m_best_header and BLOCK_FAILED_CHILD statuses are set correctly because of the final call to InvalidChainFound - except for the ones not set as BLOCK_FAILED_CHILD because of the potential bug), is it correct that the main goal of changes applied in InvalidateBlock is to maintain proper block statuses and best header between tip disconnections by marking children as invalid right away and recalculating the best header immediately rather than doing it later?

    Yes, that’s correct. The main goal of this PR to add these assumptions to CheckBlockIndex() - this means they should always be true when cs_main is not held. Since InvalidateBlock will release cs_main and call CheckBlockIndex() after each step, some tests would fail without the changes to InvalidateBlock() - even if the final InvalidChainFound() in that function will eventually correct it!

  25. mzumsande marked this as a draft on Mar 17, 2025
  26. validation: add call to CheckBlockIndex in InvalidateBlock
    since we release cs_main after each step, that means we have to
    make sure that the block index is left in a consistent state.
    Since CheckBlockIndex, by default, is only enabled on regtest, this
    does not affect performance.
    e2b69d48a0
  27. mzumsande force-pushed on Mar 18, 2025
  28. validation: cache all headers with enough PoW in invalidateblock
    We now include blocks without HaveNumChainTxs() / lower validation status
    than BLOCK_VALID_TRANSACTIONS. These checks are still performed at the
    spot where we use the cache to insert into setBlockIndexCandidates.
    
    This is in preparation for using the cache for more things than
    just setBlockIndexCandidates candidates in the following commits.
    
    Since we don't remove blocks after inserting into
    setBlockIndexCandidates anymore, this means that blocks could be inserted
    multiple times now into the set - this won't have any effect, so
    behavior isn't changed.
    
    Co-authored-by: stickies-v <stickies-v@protonmail.com>
    fbe9e2a391
  29. mzumsande force-pushed on Mar 18, 2025
  30. mzumsande commented at 10:37 pm on March 18, 2025: contributor

    4ba2e48 to fe83133: I changed the approach to InvalidateBlock() after the suggestion by @stickies-v: Instead of having 3 slightly different caches, now only one cache is used for everything (highpow_outofchain_headers). This should make the changes more simple and have a negligible overhead of a few additional GetAncestor() calls.

    I also added a call to CheckBlockIndex after each iteration of InvalidateBlock() (which releases cs_main, so any other thread could call CBI, which means that the block index cannot be in an inconsistent state, even temporarily). This should make it clear why the changes to InvalidateBlock() are necessary, even if (as @stringintech noted) the additional checks to CheckBlockIndex might not have led to an failure otherwise.

  31. mzumsande marked this as ready for review on Mar 19, 2025
  32. in src/validation.cpp:3785 in fe83133998 outdated
    3787+                if (!CBlockIndexWorkComparator()(best_header_it->second, m_chainman.m_best_header)) {
    3788+                    m_chainman.m_best_header = best_header_it->second;
    3789+                }
    3790+                ++best_header_it;
    3791             }
    3792         }
    


    stringintech commented at 10:03 pm on March 19, 2025:

    Can the m_best_header update logic be merged into the while loop that handles the nStatus updates? Not sure if it is necessarily better; but I was thinking of something like:

    Adding this just before the while loop:

    0const bool best_header_needs_update = m_chainman.m_best_header->GetAncestor(invalid_walk_tip->nHeight) == invalid_walk_tip;
    1if (best_header_needs_update) {
    2  m_chainman.m_best_header = invalid_walk_tip->pprev;
    3}
    

    And then here:

    0            if (best_header_needs_update &&
    1                !CBlockIndexWorkComparator()(candidate_it->second, m_chainman.m_best_header)) {
    2                m_chainman.m_best_header = candidate_it->second;
    3            }
    4            ++candidate_it;
    5        }
    

    mzumsande commented at 7:54 pm on March 21, 2025:
    Yes, I like that solution, it’s less code and since we would continue when encountering an invalid header before we’d never set an invalid block as best header, so it’s correct. I added you as co-author of that commit with the email from your github profile - let me know if you want me to change anything about that!

    stringintech commented at 10:30 am on March 22, 2025:
    Thanks a lot! Yes the email is okay.
  33. in src/validation.cpp:3769 in fe83133998 outdated
    3761+        // (possibly replacing a pre-existing BLOCK_FAILED_VALID with BLOCK_FAILED_CHILD)
    3762+        // Add any equal or more work headers that are not invalidated to setBlockIndexCandidates
    3763+        auto candidate_it = highpow_outofchain_headers.lower_bound(invalid_walk_tip->pprev->nChainWork);
    3764+        while (candidate_it != highpow_outofchain_headers.end()) {
    3765+            if (candidate_it->second->GetAncestor(invalid_walk_tip->nHeight) == invalid_walk_tip) {
    3766+                candidate_it->second->nStatus = (candidate_it->second->nStatus & ~BLOCK_FAILED_VALID) | BLOCK_FAILED_CHILD;
    


    stringintech commented at 10:04 pm on March 19, 2025:
    Shouldn’t we mark the candidate dirty after this?

    mzumsande commented at 7:49 pm on March 21, 2025:
    yes added - nothing bad would happen if we didn’t (because BLOCK_FAILED_CHILD would be recalculated at next startup) but I think it’s definitely better to do it.
  34. in src/validation.cpp:3763 in fe83133998 outdated
    3762+        // Add any equal or more work headers that are not invalidated to setBlockIndexCandidates
    3763+        auto candidate_it = highpow_outofchain_headers.lower_bound(invalid_walk_tip->pprev->nChainWork);
    3764+        while (candidate_it != highpow_outofchain_headers.end()) {
    3765+            if (candidate_it->second->GetAncestor(invalid_walk_tip->nHeight) == invalid_walk_tip) {
    3766+                candidate_it->second->nStatus = (candidate_it->second->nStatus & ~BLOCK_FAILED_VALID) | BLOCK_FAILED_CHILD;
    3767+                // If invalidated, the block is irrelevant for setBlockIndexCandidates and m_best_header and can be removed fro mthe cache
    


    stringintech commented at 10:06 pm on March 19, 2025:
    Small typo at the end of the comment

    mzumsande commented at 7:48 pm on March 21, 2025:
    fixed
  35. validation: in invalidateblock, mark children as invalid right away
    Before, they would be marked as invalid only after disconnecting
    multiple blocks, letting go of cs_main in the meantime.
    This is in preparation for adding a check to
    CheckBlockIndex() requiring that descendants of invalid block indexes
    are always marked as invalid.
    b8a68fed83
  36. mzumsande force-pushed on Mar 21, 2025
  37. validation: in invalidateblock, calculate m_best_header right away
    Before, m_best_header would be calculated only after disconnecting
    multiple blocks, letting go of cs_main in the meantime.
    This is in preparation for adding checks to CheckBlockIndex()
    requiring that m_best_header is the most-work header not known to be invalid.
    
    Co-authored-by: stringintech <stringintech@gmail.com>
    ad54231314
  38. validation: Add more checks to CheckBlockIndex()
    This adds checks that
    1) Descendants of invalid block indexes are also marked invalid
    2) m_best_header cannot be invalid, and there can be no valid
    block with more work than it.
    f2362fa3a0
  39. validation: remove m_failed_blocks
    We now mark all blocks that descend from an invalid block
    immediately as the invalid block is encountered
    (by iterating over the entire block index).
    As a result, m_failed_blocks, which was a heuristic to only
    mark descendants of failed blocks as failed when necessary,
    (i.e., when we have do decide whether to add another descendant or not)
    is no longer required.
    7b6fe5a219
  40. mzumsande force-pushed on Mar 21, 2025
  41. mzumsande commented at 7:55 pm on March 21, 2025: contributor
    fe83133 to 7b6fe5a - addressed feedback by @stringintech, thanks!
  42. sipa commented at 3:48 pm on March 28, 2025: member

    utACK 7b6fe5a2196d53a3024dfe7f11b316a50acc3cb7

    The code changes make sense, I think, given the fact that the block index failed flags are generally kept consistent already, apart from some edge cases around invalidateblock. I think the existing design stemmed from a desire not to have full block index iterations wherever possible, but if we’ve already bit the bullet on that, there is no need to for the complexity laziness brings here.

  43. DrahtBot requested review from stratospher on Mar 28, 2025
  44. DrahtBot requested review from stringintech on Mar 28, 2025
  45. DrahtBot requested review from stickies-v on Mar 28, 2025
  46. in src/validation.cpp:3763 in fbe9e2a391 outdated
    3762+        while (candidate_it != highpow_outofchain_headers.end()) {
    3763+            if (!CBlockIndexWorkComparator()(candidate_it->second, invalid_walk_tip->pprev) &&
    3764+                candidate_it->second->IsValid(BLOCK_VALID_TRANSACTIONS) &&
    3765+                candidate_it->second->HaveNumChainTxs()) {
    3766                 setBlockIndexCandidates.insert(candidate_it->second);
    3767-                candidate_it = candidate_blocks_by_work.erase(candidate_it);
    


    TheCharlatan commented at 2:47 pm on March 29, 2025:

    In commit fbe9e2a391d5f6e14998776b03d4ffbbf43f33c8:

    Nit: Should this change really be in this commit? It does make sense in the next commit. I don’t think it would be a bug per se to never erase, but it might be less efficient.

  47. TheCharlatan approved
  48. TheCharlatan commented at 9:11 pm on March 29, 2025: contributor

    crACK 7b6fe5a2196d53a3024dfe7f11b316a50acc3cb7

    This reduces complexity overall and I am happy that the slightly verbose block in AcceptBlockHeader and the m_dirty_blockindex are gone. It is not really straight forward to review though and I hope I did not miss any edge cases or real performance drawbacks. Might be good to get some more eyes on this. Having a better abstraction on the BlockMap / block tree data structure that just handles all of this internally could make things easier in the future.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-03-31 09:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me