fuzz: Add fuzz target for block index tree and related validation events #31533

pull mzumsande wants to merge 3 commits into bitcoin:master from mzumsande:202412_fuzz_checkblockindex_pr changing 6 files +302 −9
  1. mzumsande commented at 5:53 pm on December 18, 2024: contributor

    This adds a fuzz target for the block index and various events in validation that interact with it.

    It can create arbitrary tree-like structure of block indexes, simulating (so far) the following events:

    • Adding a header
    • Receiving the full block (may be valid or not)
    • ActivateBestChain() - Reorging the chain to a new chain tip (possibly encountering invalid blocks on the way)
    • Pruning

    It might be interesting / possible to extend this to more events, such as dealing with more than one chainstate (assumeutxo).

    The test skips all actual validation of header/ block / transaction data by just simulating the outcome, and also doesn’t interact with the data directory. The main goal is to ensure the integrity of the block index tree in all fuzzed constellations, by calling CheckBlockIndex() at the end of each iteration.

    Compared to #29158 this approach has a more limited scope (by skipping all actual validation), but it is fast - it doesn’t do a full init sequence on each iteration, but “cleans up” after itself by resetting the global validation state after each iteration.

  2. DrahtBot commented at 5:53 pm on December 18, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31533.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK sedited, Crypt-iQ
    Concept ACK ismaelsadeeq, dergoegge

    If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

    Conflicts

    No conflicts as of last run.

  3. DrahtBot added the label Tests on Dec 18, 2024
  4. mzumsande marked this as a draft on Dec 18, 2024
  5. DrahtBot added the label Needs rebase on Jan 22, 2025
  6. mzumsande force-pushed on Jan 23, 2025
  7. DrahtBot removed the label Needs rebase on Jan 23, 2025
  8. in src/test/fuzz/block_index_tree.cpp:63 in 98d68e00c2 outdated
    51+        CallOneOf(
    52+            fuzzed_data_provider,
    53+            [&] {
    54+                // Receive a header building on an existing one. This assumes headers are valid, so PoW is not relevant here.
    55+                LOCK(cs_main);
    56+                CBlockIndex* prev_block = PickValue(fuzzed_data_provider, blocks);
    


    stratospher commented at 4:46 am on February 5, 2025:

    98d68e0: shouldn’t we have a blocks.push_back(index) in this block of code?

    right now, only genesis block gets inserted into std::vector<CBlockIndex*> blocks and we don’t enter into the interesting test cases.


    mzumsande commented at 8:47 pm on February 5, 2025:
    Good point - fixed, and also cleared blocks in the end! While working /testing this branch I directly did picked a value from blockman.m_block_index and introduced blocks right before pushing because picking from a std::unordered_map mad runs non-deterministic.
  9. mzumsande force-pushed on Feb 5, 2025
  10. DrahtBot added the label Needs rebase on Feb 5, 2025
  11. mzumsande force-pushed on Feb 6, 2025
  12. DrahtBot removed the label Needs rebase on Feb 6, 2025
  13. in src/test/fuzz/block_index_tree.cpp:102 in 7dd613b268 outdated
    90+                CBlockIndex* old_tip = chain.Tip();
    91+                assert(old_tip);
    92+                do {
    93+                    CBlockIndex* best_tip = chainman.ActiveChainstate().FindMostWorkChain();
    94+                    assert(best_tip);                   // Should at least return current tip
    95+                    if (best_tip == chain.Tip()) break; // Nothing to do
    


    stratospher commented at 7:30 pm on March 3, 2025:
    looks like we always break here since old_tip, best_tip and chain.Tip() is the genesis block initially. EDIT: oops no problem here, please mark as resolved! best_tip can be block at any height n. I got confused when I saw logs with lots of prune height = 0.
  14. mzumsande force-pushed on Apr 4, 2025
  15. mzumsande force-pushed on Apr 7, 2025
  16. DrahtBot added the label CI failed on May 16, 2025
  17. DrahtBot removed the label CI failed on May 17, 2025
  18. DrahtBot added the label CI failed on Jun 15, 2025
  19. DrahtBot commented at 3:55 pm on June 15, 2025: contributor

    🚧 At least one of the CI tasks failed. Task tidy: https://github.com/bitcoin/bitcoin/runs/40120572487 LLM reason (✨ experimental): Missing or undefined member ’m_failed_blocks’ in ‘ChainstateManager’ caused compilation error, leading to CI failure.

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  20. in src/test/fuzz/block_index_tree.cpp:28 in 9622aab389 outdated
    23+    CBlockHeader header;
    24+    header.nVersion = provider.ConsumeIntegral<decltype(header.nVersion)>();
    25+    header.hashPrevBlock = prev_hash;
    26+    header.hashMerkleRoot = uint256{}; // never used
    27+    header.nTime = provider.ConsumeIntegral<decltype(header.nTime)>();
    28+    header.nBits = Params().GenesisBlock().nBits;
    


    ismaelsadeeq commented at 12:11 pm on June 16, 2025:
    Should also indicate why the nBits is static?

    mzumsande commented at 9:20 pm on June 16, 2025:
    done
  21. in src/test/fuzz/block_index_tree.cpp:57 in 9622aab389 outdated
    52+    {
    53+        if (abort_run) break;
    54+        CallOneOf(
    55+            fuzzed_data_provider,
    56+            [&] {
    57+                // Receive a header building on an existing one. This assumes headers are valid, so PoW is not relevant here.
    


    ismaelsadeeq commented at 12:12 pm on June 16, 2025:
    0                // Receive a header building on an existing valid one. This assumes headers are valid, so PoW is not relevant here.
    

    mzumsande commented at 9:20 pm on June 16, 2025:
    added
  22. in src/test/fuzz/block_index_tree.cpp:67 in 9622aab389 outdated
    58+                LOCK(cs_main);
    59+                CBlockIndex* prev_block = PickValue(fuzzed_data_provider, blocks);
    60+                if (!(prev_block->nStatus & BLOCK_FAILED_MASK)) {
    61+                    CBlockHeader header = ConsumeBlockHeader(fuzzed_data_provider, prev_block->GetBlockHash(), nonce_counter);
    62+                    CBlockIndex* index = blockman.AddToBlockIndex(header, chainman.m_best_header);
    63+                    assert(index->nStatus & BLOCK_VALID_TREE);
    


    ismaelsadeeq commented at 12:14 pm on June 16, 2025:
    Assert that we connect to the picked index?

    mzumsande commented at 9:20 pm on June 16, 2025:
    good idea, added an assert
  23. in src/test/fuzz/block_index_tree.cpp:148 in 9622aab389 outdated
    143+                assert(chain.Tip()->nChainWork >= old_tip->nChainWork);
    144+            },
    145+            [&] {
    146+                // Prune chain - dealing with block files is beyond the scope of this test, so just prune random blocks, making no assumptions what must
    147+                // be together in a block file.
    148+                // Also don't prune blocks outside of the chain for now - this would make the fuzzer crash because of the problem describted in
    


    ismaelsadeeq commented at 12:25 pm on June 16, 2025:

    Maybe lil too early, but a nit when updating.

    0                // Also don't prune blocks outside of the chain for now - this would make the fuzzer crash because of the problem described in
    

    mzumsande commented at 9:21 pm on June 16, 2025:
    fixed
  24. ismaelsadeeq commented at 12:30 pm on June 16, 2025: member

    Concept ACK

    Simple implementation, comments below are just a suggestions for minor improvements

  25. mzumsande force-pushed on Jun 16, 2025
  26. mzumsande commented at 9:22 pm on June 16, 2025: contributor
    Rebased due to silent conflict with #31405 and to address feedback by @ismaelsadeeq (thanks!). I think I’ll fuzz this for a few more days and take it out of draft then.
  27. in src/test/fuzz/block_index_tree.cpp:138 in 3d53859074 outdated
    133+                                block->nStatus |= BLOCK_HAVE_UNDO;
    134+                            }
    135+                        }
    136+                        chain.SetTip(*block);
    137+                        chainman.ActiveChainstate().PruneBlockIndexCandidates();
    138+                        // ABC may release cs_main / not connect all blocks in one go - but only if we have at least much chain work as we had at the start.
    


    DrahtBot commented at 7:29 am on June 17, 2025:
    “at least much chain work as we had at the start” -> “at least as much chain work as we had at the start” [missing “as” after “at least”]
    
  28. Crypt-iQ commented at 3:54 pm on June 17, 2025: contributor

    Concept ACK, will test out.

    I think both approaches (this PR and the i/o-based #29158) have their merits and could both be merged. I like that this PR focuses on the internal state we want to check. I also like that the other PR does round-trip reads/writes to disk since I think there should be asserts on i/o like this.

  29. DrahtBot removed the label CI failed on Jun 17, 2025
  30. Crypt-iQ commented at 6:00 pm on June 17, 2025: contributor

    Have you seen the below crash before / is it the same root cause as reported in #31512?

     0Running: crash-3b11c210b246fa09b395166b5007c451399115b8
     1Assertion failed: (c->setBlockIndexCandidates.contains(const_cast<CBlockIndex*>(pindex))), function CheckBlockIndex, file validation.cpp, line 5416.
     2==90212== ERROR: libFuzzer: deadly signal
     3    [#0](/bitcoin-bitcoin/0/) 0x10876cd70 in __sanitizer_print_stack_trace+0x28 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x5cd70)
     4    [#1](/bitcoin-bitcoin/1/) 0x105276794 in fuzzer::PrintStackTrace() FuzzerUtil.cpp:210
     5    [#2](/bitcoin-bitcoin/2/) 0x10525c57c in fuzzer::Fuzzer::CrashCallback() FuzzerLoop.cpp:231
     6    [#3](/bitcoin-bitcoin/3/) 0x19b787620 in _sigtramp+0x34 (libsystem_platform.dylib:arm64+0x3620)
     7    [#4](/bitcoin-bitcoin/4/) 0x19b74d888 in pthread_kill+0x124 (libsystem_pthread.dylib:arm64+0x6888)
     8    [#5](/bitcoin-bitcoin/5/) 0x19b656c5c in abort+0x78 (libsystem_c.dylib:arm64+0x78c5c)
     9    [#6](/bitcoin-bitcoin/6/) 0x19b655ee8 in __assert_rtn+0x118 (libsystem_c.dylib:arm64+0x77ee8)
    10    [#7](/bitcoin-bitcoin/7/) 0x10506d1e4 in ChainstateManager::CheckBlockIndex() const validation.cpp:5416
    11    [#8](/bitcoin-bitcoin/8/) 0x102c40e84 in block_index_tree_fuzz_target(std::__1::span<unsigned char const, 18446744073709551615ul>) block_index_tree.cpp:173
    12    [#9](/bitcoin-bitcoin/9/) 0x1035242fc in LLVMFuzzerTestOneInput fuzz.cpp:215
    13    [#10](/bitcoin-bitcoin/10/) 0x10525d9a4 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) FuzzerLoop.cpp:614
    14    [#11](/bitcoin-bitcoin/11/) 0x10524a6e8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) FuzzerDriver.cpp:327
    15    [#12](/bitcoin-bitcoin/12/) 0x10524f980 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) FuzzerDriver.cpp:862
    16    [#13](/bitcoin-bitcoin/13/) 0x10527707c in main FuzzerMain.cpp:20
    17    [#14](/bitcoin-bitcoin/14/) 0x19b3aeb48  (<unknown module>)
    
  31. mzumsande commented at 6:18 pm on June 17, 2025: contributor

    Have you seen the below crash before / is it the same root cause as reported in #31512?

    no, I haven’t seen it, and it shouldn’t be the same root cause - could you provide me the seed?

  32. mzumsande force-pushed on Jun 17, 2025
  33. mzumsande commented at 8:09 pm on June 17, 2025: contributor

    Thanks for providing me with the seed! It was a bug in the fuzz test: when we set abort_run (because reorging over pruned blocks also makes real nodes crash), we shouldn’t call CheckBlockIndex in the fuzz test.

    There are no known crashes with the current version. The problem of #31512 is being avoided by only allowing to prune blocks inside the chain for now, instead of pruning arbitrary blocks.

  34. in src/test/fuzz/block_index_tree.cpp:7 in ab8bc7a1f8
    0@@ -0,0 +1,205 @@
    1+// Copyright (c) 2020-2022 The Bitcoin Core developers
    2+// Distributed under the MIT software license, see the accompanying
    3+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
    4+
    5+#include <chain.h>
    6+#include <chainparams.h>
    7+#include <cstdint>
    


    maflcko commented at 8:56 am on June 20, 2025:
    nit: wrong section?

    mzumsande commented at 4:59 pm on October 30, 2025:
    fixed
  35. in src/test/fuzz/block_index_tree.cpp:16 in ab8bc7a1f8
    11+#include <test/fuzz/util.h>
    12+#include <test/util/setup_common.h>
    13+
    14+#include <optional>
    15+#include <ranges>
    16+#include <validation.h>
    


    maflcko commented at 8:56 am on June 20, 2025:
    same nit
  36. in src/test/fuzz/block_index_tree.cpp:155 in ab8bc7a1f8 outdated
    144+                assert(chain.Tip()->nChainWork >= old_tip->nChainWork);
    145+            },
    146+            [&] {
    147+                // Prune chain - dealing with block files is beyond the scope of this test, so just prune random blocks, making no assumptions what must
    148+                // be together in a block file.
    149+                // Also don't prune blocks outside of the chain for now - this would make the fuzzer crash because of the problem described in
    


    Crypt-iQ commented at 2:37 pm on June 23, 2025:
    Could there also be a lambda that downloads a previously valid, pruned block that is in the main chain?

    mzumsande commented at 5:41 pm on June 23, 2025:
    you mean to simulate getblockfrompeer? I think it would be good to try! I thought about it and was a bit concerned it would open a can of worms of issues in CheckBlockIndex(), and I’m doubtful if we want to accomodate CBI too much just for getblockfrompeer - it’s a rather recent and artifical rpc after all which I view as kind of a hack - but maybe these concerns are unfounded because I didn’t try it!

    Crypt-iQ commented at 6:39 pm on June 23, 2025:

    you mean to simulate getblockfrompeer?

    Yup just to see that it’s tolerated properly. I completely agree that getblockfrompeer is kind of a hack.


    Crypt-iQ commented at 4:01 pm on December 4, 2025:

    I hacked together a patch that does this, it hasn’t found anything. Feel free to use if you want:

     0
     1diff --git a/src/test/fuzz/block_index_tree.cpp b/src/test/fuzz/block_index_tree.cpp
     2index 037d168e0e..a8c0dc98cf 100644
     3--- a/src/test/fuzz/block_index_tree.cpp
     4+++ b/src/test/fuzz/block_index_tree.cpp
     5@@ -49,6 +49,8 @@ FUZZ_TARGET(block_index_tree, .init = initialize_block_index_tree)
     6     blocks.push_back(genesis);
     7     bool abort_run{false};
     8 
     9+    std::vector<CBlockIndex*> pruned_blocks;
    10+
    11     LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 1000)
    12     {
    13         if (abort_run) break;
    14@@ -168,7 +170,23 @@ FUZZ_TARGET(block_index_tree, .init = initialize_block_index_tree)
    15                             blockman.m_blocks_unlinked.erase(_it);
    16                         }
    17                     }
    18+                    pruned_blocks.push_back(prune_block);
    19                 }
    20+            },
    21+            [&] {
    22+                // Download a previously pruned block
    23+                LOCK(cs_main);
    24+                size_t num_pruned = pruned_blocks.size();
    25+                if (num_pruned == 0) return;
    26+                size_t i = fuzzed_data_provider.ConsumeIntegralInRange<size_t>(0, num_pruned - 1);
    27+                CBlockIndex* index = pruned_blocks[i];
    28+                CBlock block;
    29+                block.vtx = std::vector<CTransactionRef>(index->nTx); // Set the number of tx to the prior value.
    30+                FlatFilePos pos(0, fuzzed_data_provider.ConsumeIntegralInRange<int>(1, 1000));
    31+                chainman.ReceivedBlockTransactions(block, index, pos);
    32+                assert(index->nStatus & BLOCK_VALID_TRANSACTIONS);
    33+                assert(index->nStatus & BLOCK_HAVE_DATA);
    34+                pruned_blocks.erase(pruned_blocks.begin() + i);
    35             });
    36     }
    37     if (!abort_run) {
    

    EDIT: This introduces non-determinism, not sure why.


    mzumsande commented at 11:24 pm on December 8, 2025:
    looks good, I’ll do some fuzzing with it / try to understand the non-determinism before I include it.

    mzumsande commented at 8:46 pm on December 9, 2025:
    I added it now with an additional assert that !(index->nStatus & BLOCK_HAVE_DATA) before we redownload - which initially could crash, because we could have duplicates in pruned_blocks by pruning twice. So I also added a check to the previous commit that we don’t attempt to prune an already pruned block - maybe that was also the reason for the non-determinism?

    mzumsande commented at 8:51 pm on December 9, 2025:
    Also, the issues I was worried about were for situations where we prune/redownload blocks outside of the chain - only if we fixed #31512 and relaxed the pruning behaviour in the fuzz test, I would expect other issues to pop up.

    Crypt-iQ commented at 6:28 pm on December 10, 2025:

    because we could have duplicates in pruned_blocks by pruning twice

    Ah, nice catch.

    So I also added a check to the previous commit that we don’t attempt to prune an already pruned block - maybe that was also the reason for the non-determinism?

    The non-determinism is gone now, it was either because I had a small local patch to change m_block_index to std::map that I forgot about or because of the lack of this new check.

  37. Crypt-iQ commented at 2:41 pm on June 23, 2025: contributor

    It would be nice if we could also test out LoadBlockIndex since it inserts into m_blocks_unlinked for pruned blocks, but I think that will require a bit more mocking and not necessary for this PR.

    There is some non-determinism in the fuzz test that I’ll investigate:

     0diff --git a/Users/eugenesiegel/btc/bitcoin-review/build/fuzz_det_cov.show.t0.a.txt b/Users/eugenesiegel/btc/bitcoin-review/build/fuzz_det_cov.show.t0.b.txt
     1index 45c6f3edb4..592f5964eb 100644
     2--- a/Users/eugenesiegel/btc/bitcoin-review/build/fuzz_det_cov.show.t0.a.txt
     3+++ b/Users/eugenesiegel/btc/bitcoin-review/build/fuzz_det_cov.show.t0.b.txt
     4@@ -298059,13 +298059,13 @@
     5  5513|  19.6k|            } else if (pindexPar == best_hdr_chain[nHeight - 1]) {
     6                                  ^15.6k^15.6k
     7   ------------------
     8-  |  Branch (5513:24): [True: 6.04k, False: 9.57k]
     9+  |  Branch (5513:24): [True: 6.03k, False: 9.57k]
    10   ------------------
    11  5514|       |                // Move to pindex's sibling on the best-chain, if it has one.
    12- 5515|  6.04k|                pindex = best_hdr_chain[nHeight];
    13+ 5515|  6.03k|                pindex = best_hdr_chain[nHeight];
    14  5516|       |                // There will not be a next block if (and only if) parent block is the best header.
    15- 5517|  6.04k|                assert((pindex == nullptr) == (pindexPar == best_hdr_chain.Tip()));
    16- 5518|  6.04k|                break;
    17+ 5517|  6.03k|                assert((pindex == nullptr) == (pindexPar == best_hdr_chain.Tip()));
    18+ 5518|  6.03k|                break;
    19  5519|  9.57k|            } else {
    20  5520|       |                // Move up further.
    21  5521|  9.57k|                pindex = pindexPar;
    22@@ -299753,10 +299753,10 @@
    23                                                                          ^6.06k
    24   ------------------
    25   |  Branch (6425:13): [True: 6.06k, False: 5.70k]
    26-  |  Branch (6425:60): [True: 647, False: 5.41k]
    27+  |  Branch (6425:60): [True: 645, False: 5.42k]
    28   ------------------
    29- 6426|    647|            m_best_header = &entry.second;
    30- 6427|    647|        }
    31+ 6426|    645|            m_best_header = &entry.second;
    32+ 6427|    645|        }
    33  6428|  11.7k|    }
    34  6429|  1.98k|}
    35  6430|       |
    36⚠️
    37
    38The coverage was not deterministic between runs.
    
  38. Crypt-iQ commented at 4:04 pm on July 10, 2025: contributor

    Still looking into the above source of non-determinism, it points to RecalculateBestHeader.

    A separate source of non-determinism is because m_cached_finished_ibd is not reset (if set) at the end of the run.

  39. maflcko commented at 4:14 pm on July 10, 2025: member

    A separate source of non-determinism is because m_cached_finished_ibd is not reset (if set) at the end of the run.

    This could be fixed by calling ResetIbd(), if there is need to.

  40. in src/test/fuzz/block_index_tree.cpp:180 in ab8bc7a1f8
    175+    }
    176+
    177+    // clean up global state changed by last iteration and prepare for next iteration
    178+    {
    179+        LOCK(cs_main);
    180+        blocks.clear();
    


    Crypt-iQ commented at 6:05 pm on July 14, 2025:
    blocks is reset each iteration, so can this be removed?

    mzumsande commented at 5:00 pm on October 30, 2025:
    done, thanks
  41. Crypt-iQ commented at 6:08 pm on July 14, 2025: contributor

    I tracked down the m_best_header non-determinism I posted above to RecalculateBestHeader. It seems that since m_block_index is a std::unordered_map, the iteration order in RecalculateBestHeader is not guaranteed.

    I narrowed down my corpus to two files that, when run in separate processes (the first step of the deterministic-fuzz-coverage script), are deterministic. When they are run in the same process (the second step of the script), it is non-deterministic. I found this pretty confusing because m_block_index is cleared between runs (besides genesis). It seems that depending on the execution order, m_block_index may have changed internal map state (bucketing?) that is not reset by the calls to erase in this fuzz test. When I change the map to be a std::map, the non-determinism goes away. I am not sure if this can be fixed in this PR?

  42. mzumsande commented at 9:52 pm on July 14, 2025: contributor

    It seems that since m_block_index is a std::unordered_map, the iteration order in RecalculateBestHeader is not guaranteed.

    Oh yes, this bit me in an earlier iteration of this PR where I made the mistake to do PickValue(fuzzed_data_provider, blockman.m_block_index) directly to pick a random block index (so that no blocks vector was needed).

    It’s also related to the fact that we don’t use CBlockIndexWorkComparator tiebreak rules for m_best_header (documented in #32843), so I think it’s currently intrinsically non-deterministic (not just in the context of the fuzz test) which block is the best header in case of multiple candidates with the same work?!

  43. dergoegge commented at 3:34 pm on October 22, 2025: member
    Concept ACK
  44. fanquake added the label Fuzzing on Oct 30, 2025
  45. mzumsande force-pushed on Oct 30, 2025
  46. mzumsande commented at 5:04 pm on October 30, 2025: contributor
    ab8bc7a to 6abbb6e: Rebased, addressed remaining feedback, including resetting m_cached_finished_ibd reset to cleanup. Moved out of draft.
  47. mzumsande marked this as ready for review on Oct 30, 2025
  48. in src/test/fuzz/block_index_tree.cpp:81 in 48d11516da
    76+                        BlockValidationState state;
    77+                        state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "consensus-invalid");
    78+                        chainman.ActiveChainstate().InvalidBlockFound(index, state);
    79+                    } else {
    80+                        size_t nTx = fuzzed_data_provider.ConsumeIntegralInRange<size_t>(1, 1000);
    81+                        CBlock block; // Dummy block, so that ReceivedBlockTransaction can infer a nTx value.
    


    maflcko commented at 5:06 pm on October 30, 2025:

    from the llm:

    ReceivedBlockTransaction -> ReceivedBlockTransactions [the comment refers to the ReceivedBlockTransactions function; singular "ReceivedBlockTransaction" is incorrect and may confuse readers]
    making no assumptions what must -> making no assumptions about what must [missing preposition "about" makes the phrase ungrammatical and harder to parse]
    

    mzumsande commented at 5:12 pm on October 30, 2025:
    fixed those.
  49. mzumsande force-pushed on Oct 30, 2025
  50. mzumsande force-pushed on Oct 30, 2025
  51. in src/validation.h:811 in 6abbb6ed6c
    808@@ -807,8 +809,6 @@ class Chainstate
    809         ConnectTrace& connectTrace,
    810         DisconnectedBlockTransactions& disconnectpool) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool->cs);
    811 
    812-    void InvalidBlockFound(CBlockIndex* pindex, const BlockValidationState& state) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    813-    CBlockIndex* FindMostWorkChain() EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    


    sedited commented at 1:45 pm on November 28, 2025:

    It would be nice if these could stay where they are. How about the following:

     0diff --git a/src/test/fuzz/block_index_tree.cpp b/src/test/fuzz/block_index_tree.cpp
     1index 037d168e0e..04fbbc5473 100644
     2--- a/src/test/fuzz/block_index_tree.cpp
     3+++ b/src/test/fuzz/block_index_tree.cpp
     4@@ -14,0 +15 @@
     5+#include <test/util/validation.h>
     6@@ -44 +45 @@ FUZZ_TARGET(block_index_tree, .init = initialize_block_index_tree)
     7-    ChainstateManager& chainman = *g_setup->m_node.chainman;
     8+    auto& chainman = static_cast<TestChainstateManager&>(*g_setup->m_node.chainman);
     9@@ -78 +79 @@ FUZZ_TARGET(block_index_tree, .init = initialize_block_index_tree)
    10-                        chainman.ActiveChainstate().InvalidBlockFound(index, state);
    11+                        chainman.InvalidBlockFound(index, state);
    12@@ -97 +98 @@ FUZZ_TARGET(block_index_tree, .init = initialize_block_index_tree)
    13-                    CBlockIndex* best_tip = chainman.ActiveChainstate().FindMostWorkChain();
    14+                    CBlockIndex* best_tip = chainman.FindMostWorkChain();
    15@@ -130 +131 @@ FUZZ_TARGET(block_index_tree, .init = initialize_block_index_tree)
    16-                                chainman.ActiveChainstate().InvalidBlockFound(block, state);
    17+                                chainman.InvalidBlockFound(block, state);
    18diff --git a/src/test/util/validation.cpp b/src/test/util/validation.cpp
    19index ce558078a6..b419069b19 100644
    20--- a/src/test/util/validation.cpp
    21+++ b/src/test/util/validation.cpp
    22@@ -33,0 +34,22 @@ void TestChainstateManager::JumpOutOfIbd()
    23+void TestChainstateManager::InvalidBlockFound(CBlockIndex* pindex, const BlockValidationState& state)
    24+{
    25+    struct TestChainstate : public Chainstate {
    26+        void CallInvalidBlockFound(CBlockIndex* pindex, const BlockValidationState& state) EXCLUSIVE_LOCKS_REQUIRED(cs_main) {
    27+            InvalidBlockFound(pindex, state);
    28+        }
    29+    };
    30+
    31+    static_cast<TestChainstate*>(&ActiveChainstate())->CallInvalidBlockFound(pindex, state);
    32+}
    33+
    34+CBlockIndex* TestChainstateManager::FindMostWorkChain()
    35+{
    36+    struct TestChainstate : public Chainstate {
    37+        CBlockIndex* CallFindMostWorkChain() EXCLUSIVE_LOCKS_REQUIRED(cs_main) {
    38+            return FindMostWorkChain();
    39+        }
    40+    };
    41+
    42+    return static_cast<TestChainstate*>(&ActiveChainstate())->CallFindMostWorkChain();
    43+}
    44+
    45diff --git a/src/test/util/validation.h b/src/test/util/validation.h
    46index f9c6a8ac02..ae2e13b826 100644
    47--- a/src/test/util/validation.h
    48+++ b/src/test/util/validation.h
    49@@ -18,0 +19,4 @@ struct TestChainstateManager : public ChainstateManager {
    50+
    51+    void InvalidBlockFound(CBlockIndex* pindex, const BlockValidationState& state) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    52+
    53+    CBlockIndex* FindMostWorkChain() EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    54diff --git a/src/validation.h b/src/validation.h
    55index 7f284da272..9e58bc9aa7 100644
    56--- a/src/validation.h
    57+++ b/src/validation.h
    58@@ -800,2 +799,0 @@ public:
    59-    void InvalidBlockFound(CBlockIndex* pindex, const BlockValidationState& state) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    60-    CBlockIndex* FindMostWorkChain() EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    61@@ -811,0 +810,2 @@ protected:
    62+    void InvalidBlockFound(CBlockIndex* pindex, const BlockValidationState& state) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    63+    CBlockIndex* FindMostWorkChain() EXCLUSIVE_LOCKS_REQUIRED(cs_main);
    

    We could also add similar methods for wrapping the block manager and the best invalid entry.


    mzumsande commented at 9:27 pm on December 5, 2025:

    I like this. The only downside is that moving the functions from Chainstate to TestChainstateManager is less flexible if ever we wanted to have multiple chainstates.

    I chose a similar approach for blockmanager etc., so that now we only need to move a few things from private to protected in the production part of the code.

  52. sedited approved
  53. sedited commented at 1:50 pm on November 28, 2025: contributor
    ACK 6abbb6ed6cb29b20645a7ef78bcf2e51fc7bde49
  54. DrahtBot requested review from Crypt-iQ on Nov 28, 2025
  55. DrahtBot requested review from dergoegge on Nov 28, 2025
  56. DrahtBot requested review from ismaelsadeeq on Nov 28, 2025
  57. in src/test/fuzz/block_index_tree.cpp:107 in 6abbb6ed6c outdated
    101+                    const CBlockIndex* fork = chain.FindFork(best_tip);
    102+                    // If we can't go back to the fork point due to pruned data, abort this run. In reality, a pruned node would also currently just crash in this scenario.
    103+                    // This is very unlikely to happen due to the minimum pruning threshold of 550MiB.
    104+                    CBlockIndex* it = chain.Tip();
    105+                    while (it && it->nHeight != fork->nHeight) {
    106+                        if (!(it->nStatus & BLOCK_HAVE_UNDO) && it->nHeight > 0) {
    


    Crypt-iQ commented at 6:00 pm on December 3, 2025:
    nit: I think the it->nHeight > 0 branch is unnecessary? If nHeight is 0 and we are here, then that means the fork point was before the genesis block which should be impossible?

    mzumsande commented at 11:19 pm on December 8, 2025:
    makes sense, removed the condition
  58. in src/test/fuzz/block_index_tree.cpp:130 in 6abbb6ed6c
    125+                        assert(block->nStatus & BLOCK_HAVE_DATA);
    126+                        if (!block->IsValid(BLOCK_VALID_SCRIPTS)) {
    127+                            if (fuzzed_data_provider.ConsumeBool()) { // Invalid
    128+                                BlockValidationState state;
    129+                                state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "consensus-invalid");
    130+                                chainman.ActiveChainstate().InvalidBlockFound(block, state);
    


    Crypt-iQ commented at 6:58 pm on December 3, 2025:
    Would it be a good idea to also call InvalidChainFound(to_connect.front()) similar to here?

    mzumsande commented at 10:06 pm on December 8, 2025:

    I have been thinking about removing this call from ABCStep for a while, because it doesn’t seem necessary: During ConnectTip(), we call InvalidBlockFound(), which calls InvalidChainFound() for the block that fails validation. The additional call to InvalidChainFound() with the highest connectable block from the chain (which could be a successor of the invalid block, or the same block again) can lead to additional unnecessary work and duplicate logging. The only part from InvalidChainFound() that would make sense to do again is to update m_best_invalid if the invalid block is not the highest connectable block, but then again m_best_invalid is only used for logging.

    Let me know if this makes sense, this change always seemed a bit scary so I never PRed this / included this in other PRs from that area.

    That being said, for now it seems reasonable to imitate the behavior, so I will add the call with the next push.


    Crypt-iQ commented at 3:44 pm on December 10, 2025:

    I was pretty confused about the second call to InvalidChainFound when reading the code, so I think it makes sense to PR and remove. It seems to have been introduced in #4148 and only affects the warnings in CheckForkWarningConditions.

    Since vpindexToConnect.front() must descend (or be equal to) from the CBlockIndex* passed to InvalidBlockFound and all descendants of this CBlockIndex* will have been marked as invalid in SetBlockFailureFlags, the extra call to InvalidChainFound(vpindexToConnect.front()) can’t affect m_best_header so it’s safe to delete imo.

  59. in src/test/fuzz/block_index_tree.cpp:140 in 6abbb6ed6c outdated
    134+                                block->nStatus |= BLOCK_HAVE_UNDO;
    135+                            }
    136+                        }
    137+                        chain.SetTip(*block);
    138+                        chainman.ActiveChainstate().PruneBlockIndexCandidates();
    139+                        // ABC may release cs_main / not connect all blocks in one go - but only if we have at least as much chain work as we had at the start.
    


    Crypt-iQ commented at 7:01 pm on December 3, 2025:

    (Just a clarifying question)

    Is this true of ABC? From my reading, this is true of ABCStep but not ABC since ABC will stop (besides failure or interrupt) only when our tip is the block with most work (see: here).


    mzumsande commented at 11:19 pm on December 8, 2025:
    True, changed to ABCStep. The point is that we want to sometimes stop here (and possibly call CheckBlockIndex etc.), because whenever we would release cs_main, the same would be possible from another thread.
  60. in src/test/fuzz/block_index_tree.cpp:186 in 6abbb6ed6c outdated
    180+        LOCK(cs_main);
    181+        genesis->nStatus |= BLOCK_HAVE_DATA;
    182+        genesis->nStatus |= BLOCK_HAVE_UNDO;
    183+        chainman.m_best_header = genesis;
    184+        chainman.m_best_invalid = nullptr;
    185+        chainman.nBlockSequenceId = 1;
    


    Crypt-iQ commented at 10:23 pm on December 3, 2025:
    Should this be 2 instead? See here.

    mzumsande commented at 11:20 pm on December 8, 2025:
    right, that changed recently with #29640
  61. Crypt-iQ commented at 5:11 pm on December 4, 2025: contributor
    Looks good, left some comments.
  62. DrahtBot requested review from Crypt-iQ on Dec 4, 2025
  63. mzumsande force-pushed on Dec 5, 2025
  64. mzumsande commented at 9:27 pm on December 5, 2025: contributor
    Thanks for the reviews - with the most recent push I addressed #31533 (review) - will address the comments by @Crypt-iQ a bit later.
  65. mzumsande force-pushed on Dec 5, 2025
  66. DrahtBot added the label CI failed on Dec 5, 2025
  67. DrahtBot removed the label CI failed on Dec 5, 2025
  68. test: Wrap validation functions with TestChainstateManager
    This allows to access them in the fuzz test in the next commit
    without making them public.
    
    Co-authored-by: TheCharlatan <seb.kung@gmail.com>
    9f9d0e059f
  69. mzumsande force-pushed on Dec 8, 2025
  70. mzumsande commented at 11:25 pm on December 8, 2025: contributor
    7927473 to 62edcea: Addressed feedback (didn’t include getblockfrompeer behavior of re-downloading prune blocks yet, see above).
  71. fuzz: Add fuzzer for block index
    This fuzz target creates arbitrary tree-like structure of indices,
    simulating the following events:
    - Adding a header to the block tree db
    - Receiving the full block (may be valid or not)
    - Reorging to a new chain tip (possibly encountering invalid blocks on
      the way)
    - pruning
    The test skips all actual validation of header/ block / transaction data
    by just simulating the outcome, and also doesn't interact with the data directory.
    
    The main goal is to test the integrity of the block index tree in
    all fuzzed constellations, by calling CheckBlockIndex()
    at the end of each iteration.
    78f04a82ef
  72. fuzz: add subtest for re-downloading a previously pruned block
    This imitates the use of the getblockfrompeer rpc.
    Note that currently pruning is limited to blocks in the active chain.
    
    Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
    ac67ab7647
  73. mzumsande force-pushed on Dec 9, 2025
  74. mzumsande commented at 8:47 pm on December 9, 2025: contributor
    62edcea to ac67ab7: Added commit by @Crypt-iQ to redownload already pruned blocks (getblockfrompeer).
  75. sedited approved
  76. sedited commented at 10:08 pm on December 9, 2025: contributor

    re-ACK ac67ab76470441ed7539eb780efa3912f1281faa

    The addition of the re-download case is nice.

  77. in src/test/util/validation.cpp:62 in 9f9d0e059f outdated
    57+    };
    58+
    59+    static_cast<TestChainstate*>(&ActiveChainstate())->CallInvalidBlockFound(pindex, state);
    60+}
    61+
    62+void TestChainstateManager::InvalidChainFound(CBlockIndex* pindex)
    


    Crypt-iQ commented at 6:36 pm on December 10, 2025:
    nit: (if you push up again) this could be named pindexNew to match the header?
  78. Crypt-iQ commented at 6:42 pm on December 10, 2025: contributor

    crACK ac67ab76470441ed7539eb780efa3912f1281faa

    Will continue to fuzz and see if anything pops up.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-12-11 15:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me