fuzz: target concurrent leveldb reads #34866

pull andrewtoth wants to merge 4 commits into bitcoin:master from andrewtoth:fuzz_db_reads changing 5 files +385 −2
  1. andrewtoth commented at 1:41 pm on March 19, 2026: contributor

    Inspired by #31132 (comment).

    We currently do concurrent leveldb reads when accessing our indexes.

    1. txindex - we call FindTx() from multiple RPC threads.
    2. blockfilterindex - we call LookupFilter/Header() concurrently from msghand thread for p2p requests as well as RPC threads.
    3. coinstatsindex - we call LookUpStats() from multiple RPC threads.
    4. txospenderindex - we call FindSpender() from multiple RPC threads.

    We also read from our chainstate and blocks index while background compactions are writing.

    While OSS-Fuzz does cover leveldb (https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc), it doesn’t cover multi threaded access. Without a deterministic hypervisor this fuzz harness won’t be deterministic, but we can at least run it with TSan to get a higher confidence that the synchronization code in leveldb is correct. Hopefully other reviewers find this useful.

    This harness creates a global threadpool with 16 threads, and then creates an in-memory levelDB which it seeds with deterministically random values. It selects up to 16 threads and chooses a random set of keys to query. It first performs all queries on the db on a single thread to get a baseline, then synchronizes all threads on a latch so they hit the db at the same time. Each thread performs the same queries, and afterwards are all checked against the baseline.

    It uses a DeterministicEnv to capture background compaction work when seeding the db, which is also run immediately after the latch is released. This causes a race between compaction and reading, ensuring we exercise many thread synchronization code paths in leveldb.

    I ran both TSan and ASan/UBSan overnight with no issues.

  2. DrahtBot added the label Fuzzing on Mar 19, 2026
  3. DrahtBot commented at 1:41 pm on March 19, 2026: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK furszy

    If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

    Conflicts

    No conflicts as of last run.

  4. maflcko commented at 1:52 pm on March 19, 2026: member

    While OSS-Fuzz does cover leveldb (https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc), it …

    I wouldn’t put too much value in this. I think there is only a single fuzz target, which fails, according to https://issues.oss-fuzz.com/issues/447252244, so it may be unmaintained.

  5. sedited requested review from marcofleon on Mar 23, 2026
  6. dbwrapper: make max_file_size a configurable DBParams field
    Useful for fuzzing different values.
    9e444624bb
  7. dbwrapper: accept optional testing leveldb::Env in DBParams
    Allow callers to inject a custom leveldb::Env via DBParams::testing_env,
    which takes priority over the memory_only in-memory environment. This
    enables fuzz harnesses to supply a deterministic environment.
    e903203536
  8. test: add fuzz harness for CDBWrapper
    Introduces a libFuzzer harness that exercises CDBWrapper operations
    against a std::map oracle, with a DeterministicEnv that captures LevelDB
    background compaction for single-threaded determinism.
    430f357f7e
  9. fuzz: target concurrent leveldb reads 15b7a8fa10
  10. andrewtoth force-pushed on Mar 29, 2026
  11. andrewtoth commented at 6:51 pm on March 29, 2026: contributor
    Updated this to be a commit on top of #34887, so that we can reuse the DeterministicEnv to run compaction concurrently with reading.
  12. furszy commented at 11:59 pm on March 29, 2026: member
    Concept ACK. Useful to verify this behavior on our end.

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-12 09:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me