bench: unrealistic ConnectBlock benchmarks #33375

issue Raimo33 openend this issue on September 12, 2025
  1. Raimo33 commented at 12:53 pm on September 12, 2025: none

    The current ConnectBlock benchmarks in bench/connectblock.cpp do not reflect realistic mainnet workloads due to three key issues:

    1. Unrealistic block composition

    Every benchmarked block is constructed with a highly artificial transaction pattern:

    0/*
    1 * - Each transaction has the same number of inputs and outputs
    2 * - All Taproot inputs use simple key path spends (no script path spends)
    3 * - All signatures use SIGHASH_ALL (default sighash)
    4 * - Each transaction spends all outputs from the previous transaction
    5 */
    

    This setup avoids realistic UTXO set fragmentation and script diversity. The benchmark effectively measures validation of a synthetic “ladder” of transactions rather than a block resembling mainnet traffic.

    2. Unrealistic UTXO cache state

    Before benchmarking, the code creates a block that produces the outputs, then immediately spends them all in the benchmark block. This keeps the entire UTXO set hot in memory (CoinsTip()).

    In reality:

    • Many UTXO lookups hit LevelDB and require disk access.
    • Cache misses and eviction policies significantly impact block validation cost.

    3. Unrealistic repetition

    Each benchmark repeatedly validates the same synthetic block:

    0    const auto& test_block{CreateTestBlock(test_setup, keys, outputs)};
    1    bench.unit("block").run([&] {
    2        /* ... */
    3    });
    

    There is no variability in transaction graph, script mix, or UTXO evolution across iterations. As a result, the benchmark never exercises cache churn, block-to-block dependency patterns, or realistic workload diversity.


    Why this matters

    These issues mean the benchmark results do not reflect real-world ConnectBlock performance. Instead, they measure a best-case, memory-only workload on a synthetic block structure.

  2. davidgumberg commented at 0:06 am on September 13, 2025: contributor
    I am skeptical of trying to measure end-to-end performance of Bitcoin Core systems like block connection including LevelDB performance using a microbenchmark suite. In my opinion, these benchmarks work best at measuring the performance of specific components and functions a-la unit tests, and I would not object to adding more scenarios to ConnectBlock() that are meant to exercise the performance of certain segments of block connection, but I think to get measurements of realistic scenarios, there is no substitute for just measuring Bitcoin Core nodes doing IBD, of interest might be the benchkit project for measuring IBD’s: https://github.com/bitcoin-dev-tools/benchkit.
  3. Raimo33 commented at 2:44 pm on September 13, 2025: none
    it’s not “a little bit of faithfulness” that you sacrifice. you’re ignoring the cache completely.
  4. l0rinc commented at 2:11 am on September 15, 2025: contributor

    @Raimo33 if you can provide an alternative that’s reasonably simple for a micro-benchmark, I will happily review it.

    But note that I already have a few micro-benchmark improvements in that area where review is needed:

  5. HowHsu commented at 1:34 pm on September 15, 2025: none
    I can confirm that this issue is true, during my testing for my PR: https://github.com/bitcoin/bitcoin/pull/32791 . To test ConnectBlock() realistically, I think we can leverage CVerifyDB::VerifyDB(), it is called on node init. It mainly replays the latest -check_blocks blocks in the best chain when you set -check_level to 4. Considering cache state, you have to manually set it at the begining (flush it for no-cache case, pre-load entries for cache-hit case, .etc).Surely you have to find a good height range of blocks first.

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-09-18 21:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me