bench: add index benchmarks #35827

arejula27 commented at 7:53 PM on July 27, 2026: none

There is a fair amount of work going into the indices at the moment, and no cheap way to tell whether a change is promising, or whether it actually improves what it set out to improve. Checking it end to end means a full IBD. The only index benchmark in the tree, BlockFilterIndexSync, syncs coinbase-only blocks, so it cannot show anything that scales with the number of inputs or outputs per block, and there is none at all for TxIndex or TxoSpenderIndex.

This adds shared helpers in src/bench/index_sync_util.{h,cpp} that build a chain of blocks with chained, validly-signed transactions (~2 inputs and ~2 outputs each, every output paying to a distinct key), and three benchmarks on top of them, applied to txindex, txospenderindex and blockfilterindex:

<Index>SyncDisk : full sync against a disk-backed LevelDB, the real write path.
<Index>SyncMem: the same in memory; the difference against SyncDisk separates I/O from CPU.
<Index>Lookup: one query per key known to be in the index (FindTx, FindSpender).

The existing coinbase-only BlockFilterIndexSync is left untouched. The whole suite runs in well under a minute.

A last commit documents in doc/benchmarking.md how to measure syscalls, on-disk size and peak memory around bench_bitcoin, which nanobench does not report. Which of those matters depends on the change: on-disk size is the whole point of #35531, while peak memory is the open question in #34489. Being able to measure them alongside the benchmark seems generally useful, so that section is written for any benchmark, not just these.

I have used this on those two pull requests, and in both the interesting number was one the existing benchmark could not produce: #35531 (5-byte siphash keys) trades a 49% smaller index on disk for a lookup regression that grows with index size, and #34489 (batch db writes) cuts write syscalls by 91% with CPU unchanged. The numbers are in those threads.

DrahtBot added the label Tests on Jul 27, 2026

DrahtBot commented at 7:53 PM on July 27, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35827.

Reviews

See the guideline and AI policy for information on the review process.

Type	Reviewers
Concept ACK	l0rinc

If your review is incorrectly listed, please copy-paste <code></code> into the comment that the bot should ignore.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

du -sb '<testdatadir>/test_common bitcoin/<benchmark>/datadir/regtest/<dir>' -> du -sb '<testdatadir>/test_common/bitcoin/<benchmark>/datadir/regtest/<dir>' [the current path has a stray space, making the directory path incorrect/confusing]

arejula27 commented at 8:04 PM on July 27, 2026: none

I think this PR can be splited into two, one for the doc and another for the index benchmarks

bench: add reusable index sync benchmark helpers

The existing BlockFilterIndexSync syncs coinbase-only blocks, which
cannot exercise anything that scales with the number of inputs or
outputs per block.

Add src/bench/index_sync_util.{h,cpp} with the chain setup and timing
loops shared by index benchmarks. ExtendChainWithSpends() builds blocks
of validly-signed transactions with ~2 inputs each (the average ratio
observed on mainnet), paying every output to a distinct key: block
filter elements are deduplicated, so reusing a couple of scripts would
keep every filter tiny no matter how many transactions the block holds.

34d3d28d4b

bench: add TxoSpenderIndex, TxIndex and BlockFilterIndex sync benchmarks

Add, on top of the shared helpers:
- TxoSpenderIndexSync{Disk,Mem} and TxoSpenderIndexLookup (FindSpender)
- TxIndexSync{Disk,Mem} and TxIndexLookup (FindTx)
- BlockFilterIndexSync{Disk,Mem}

The Disk variant exercises the real LevelDB write path; the Mem variant
isolates CPU cost from I/O. The pre-existing coinbase-only
BlockFilterIndexSync is left untouched as a cheap baseline.

2d30d9de98

in src/bench/index_blockfilter.cpp:70 in 6bf3b13e53 outdated

  65 | +// Returns a fresh, not-yet-initialized BASIC BlockFilterIndex. `f_memory=false`
  66 | +// exercises the real disk write path, true isolates CPU cost from I/O.
  67 | +static std::unique_ptr<BlockFilterIndex> MakeBlockFilterIndex(TestChain100Setup& test_setup, bool f_memory)
  68 | +{
  69 | +    return std::make_unique<BlockFilterIndex>(interfaces::MakeChain(test_setup.m_node), BlockFilterType::BASIC,
  70 | +                                              /*n_cache_size=*/1 << 20, f_memory, /*f_wipe=*/true);

l0rinc commented at 8:10 PM on July 27, 2026:

nit: you can use 1_MiB

in src/bench/index_sync_util.cpp:49 in 6bf3b13e53 outdated

  44 | +    // Coinbase from block 1 matures exactly when spent at height 101 (chain
  45 | +    // tip is at height 100 right after TestChain100Setup).
  46 | +    auto& coinbase_to_spend{test_setup.m_coinbase_txns[0]};
  47 | +    size_t k0{0};
  48 | +    size_t k1{1};
  49 | +    const CMutableTransaction first_tx{test_setup.CreateValidTransaction({coinbase_to_spend}, {COutPoint(coinbase_to_spend->GetHash(), 0)},

l0rinc commented at 8:11 PM on July 27, 2026:

I don't usually mind long lines but formatting is a bit off on GitHub

in src/bench/index_sync_util.h:25 in 6bf3b13e53 outdated

  20 | +/**
  21 | + * Size of the chain built by the index benchmarks. Building it is not timed but
  22 | + * dominates the wall time of a run, so keep these just large enough for the
  23 | + * timed part to stand out of the noise.
  24 | + */
  25 | +static constexpr size_t BENCH_INDEX_NUM_BLOCKS{50};

l0rinc commented at 8:12 PM on July 27, 2026:

we should avoid adding new arch dependent types if possible

arejula27 commented at 5:11 PM on July 28, 2026:

mb

in src/bench/index_sync_util.h:59 in 6bf3b13e53 outdated

  54 | +        assert(!index->BlockUntilSyncedToCurrentChain());
  55 | +        index->Sync();
  56 | +
  57 | +        IndexSummary summary{index->GetSummary()};
  58 | +        assert(summary.synced);
  59 | +        assert(summary.best_block_hash == WITH_LOCK(::cs_main, return test_setup.m_node.chainman->ActiveTip()->GetBlockHash()));

l0rinc commented at 8:13 PM on July 27, 2026:

locking inside the measured block will likely distort the results. Consider precalculating and maybe using setup block

in src/bench/index_sync_util.h:97 in 6bf3b13e53 outdated

  92 | +    assert(!keys.empty());
  93 | +    bench.batch(keys.size()).unit("lookup").run([&] {
  94 | +        for (const Key& key : keys) {
  95 | +            const bool ok{lookup_one(key)};
  96 | +            assert(ok);
  97 | +            ankerl::nanobench::doNotOptimizeAway(ok);

l0rinc commented at 8:14 PM on July 27, 2026:

either assert or doNotOptimizeAway, no need for both. Alternatively have a bool at the begining, and each value and assert at the end

in src/bench/index_txindex.cpp:38 in 6bf3b13e53 outdated

  33 | +}
  34 | +
  35 | +static void TxIndexSyncDisk(benchmark::Bench& bench) { TxIndexSync(bench, /*f_memory=*/false); }
  36 | +static void TxIndexSyncMem(benchmark::Bench& bench) { TxIndexSync(bench, /*f_memory=*/true); }
  37 | +BENCHMARK(TxIndexSyncDisk);
  38 | +BENCHMARK(TxIndexSyncMem);

l0rinc commented at 8:15 PM on July 27, 2026:

I'd prefer having all of these at the very end instead

in src/bench/index_txindex.cpp:49 in 6bf3b13e53 outdated

  44 | +{
  45 | +    const auto test_setup = MakeNoLogFileContext<TestChain100Setup>();
  46 | +    ExtendChainWithSpends(*test_setup, BENCH_INDEX_NUM_BLOCKS, BENCH_INDEX_TXS_PER_BLOCK);
  47 | +
  48 | +    // Build and fully sync a persistent txindex, kept alive for the lookups.
  49 | +    auto index{MakeTxIndex(*test_setup, /*f_memory=*/false)};

l0rinc commented at 8:15 PM on July 27, 2026:

these are likely small, so even if they're not in memory, the OS will likely make sure they're still actually in memory...

arejula27 commented at 5:04 PM on July 28, 2026:

Thats a problem i am having with this PR, based on my results looks like Im not using the disk, not sure if it is only the size or am i doing something wrong elsewhere

l0rinc commented at 5:19 PM on July 28, 2026:

You can try invalidating the OS file caches

arejula27 commented at 5:57 PM on July 28, 2026:

I didn't find a way to do it from inside the benchmark itself, only things the user can do outside the process, but I don't know this area well enough. Do you know a simpler way?

Also I noticed the writes are not fsynced (fSync=false in BaseIndex::Commit) and the index is smaller than the write buffer, so it never leaves the memtable. I'll try with a bigger chain and see what happens

in src/bench/index_txospender.cpp:56 in 6bf3b13e53 outdated

  51 | +
  52 | +    const std::vector<COutPoint> outpoints{CollectChainSpentOutpoints(*test_setup, BENCH_INDEX_FIRST_SPEND_HEIGHT)};
  53 | +
  54 | +    BenchIndexLookup(bench, outpoints, [&](const COutPoint& txo) {
  55 | +        const auto result{index->FindSpender(txo)};
  56 | +        return result && result->has_value();

l0rinc commented at 8:16 PM on July 27, 2026:

aren't these two conditions exactly the same?

arejula27 commented at 5:34 PM on July 28, 2026:

 /**
     * Search the index for a transaction that spends the given outpoint.
     *
     * [@param](/bitcoin-bitcoin/contributor/param/)[in] txo  The outpoint to search for.
     *
     * [@return](/bitcoin-bitcoin/contributor/return/)  std::nullopt               if the outpoint has not been spent on-chain.
     *          std::optional{TxoSpender}  if the output has been spent on-chain. Contains the spending transaction
     *                                     and the block it was confirmed in.
     *          util::Unexpected{error}    if something unexpected happened (i.e. disk or deserialization error).
     */
    util::Expected<std::optional<TxoSpender>, std::string> FindSpender(const COutPoint& txo) const;

FindSpender returns util::Expected<std::optional<TxoSpender>, std::string>, which might be complex read. My understanding:

The first check is "no read error".
The second check is "a spender was found".

How do you understand it?

l0rinc commented at 8:17 PM on July 27, 2026: contributor

Concept ACK

Thanks for taking care of these, I will review them later in more detail, just quickly glanced at the structure.

doc: describe measuring I/O and memory cost of benchmarks

The framework reports wall-clock time and, where available, cycles and
instructions, but not syscalls, bytes written, on-disk size or peak
memory. Document how to measure those around bench_bitcoin with strace,
/usr/bin/time and du, and the -testdatadir knob needed to avoid tmpfs.

4f79b7275a

arejula27 force-pushed on Jul 27, 2026

bench: address review feedback on index benchmarks 0584290638

DrahtBot added the label CI failed on Jul 29, 2026