AssumeUTXO: crowdsource trust through independent snapshot verification #32377

issue l0rinc opened this issue on April 29, 2025
  1. l0rinc commented at 1:34 PM on April 29, 2025: contributor

    Problem

    With AssumeUTXO, a node starts from a UTXO snapshot in an effectively pruned state and validates new blocks immediately, while downloading and validating earlier blocks in the background to become an archival node. This leaves us in a confusing limbo: we trust the expected UTXO set hash enough to use the snapshot, but still treat the blocks preceding it as unvalidated until background validation finishes. If we trust the hash, why does every snapshot user have to validate those blocks? If we do not, what assurance are users relying on in the meantime? If background validation finds that the reconstructed UTXO set does not match the snapshot, it is unclear what users should do.

    Managing multiple chainstates adds substantial maintenance work across RPC and wallet reporting, notifications, pruning, cache allocation, indexes, and reorgs, and introduces race conditions and unintuitive corner cases. Script-check logs need to distinguish snapshot and background validation, restarts need extra completion checks (even for a fully validated snapshot), and RPCs report different progress for the same background chainstate. Reviewing even an ordinary validation fix requires reasoning about block candidates in both chainstates, making the most critical part of the codebase harder to understand and review.

    AssumeUTXO shortens the wait for a usable node, but completing background validation still requires downloading the snapshot in addition to all the blocks and temporarily keeping two UTXO databases. Each chainstate also starts its own prevout-fetching worker pool when enabled, doubling the number of workers. The cache budget follows a fixed 95/5 split that reverses once the snapshot chainstate catches up, then goes entirely to that chainstate when background validation finishes. These changing budgets complicate presizing the UTXO cache, and resizing can require flushing and discarding cached entries or reopening the database. AssumeUTXO adoption remains below expectations.

    @ryanofsky raised similar concerns about code complexity in the original implementation PR, arguing that validation outside ChainstateManager should not need to know about the different AssumeUTXO phases and roles. David Harding's original feedback also questioned keeping old snapshots and requiring every snapshot user to validate earlier blocks, while calling for easy hash checks by ordinary nodes.

    We already bake the expected UTXO set hash at the snapshot block into the source code and binary, but normal IBD ignores it completely.

    Proposed approach

    Nodes doing a full IBD could use the baked-in hash to check their own UTXO set, giving others confidence to bootstrap more quickly from the snapshot. These routine checks could replace background validation on snapshot nodes, letting them skip the blocks before the snapshot and run validation on a single chainstate.

    1. Check the UTXO set hash during IBD

    When a node doing IBD reaches the snapshot height, it could hash the flushed UTXO set and compare the result with the hash in Core. With this check enabled by default, every full IBD would strengthen trust in the hash, much like reproducible builds do for release binaries (we don't have a background-recompilation either): a single honest participant is enough to expose a mismatch for everyone to investigate.

    2. Keep only the latest UTXO set hash

    Keeping only the latest UTXO snapshot hash would let nodes running the same release share one reference and scan the UTXO set only once during IBD. Older snapshots (including those whose background validation is unfinished) would remain usable with a compatible older release.

    3. Make UTXO snapshot trust an explicit choice

    Users already choose how much to verify during IBD: -assumevalid=0 checks every script, while the default can skip scripts in older blocks. Relying on the independent checks above would extend that choice to snapshot startup, removing the chainstate role changes, cache splitting, and special handling of progress reporting and block downloads.

    Without background validation, a snapshot node would lack the earlier blocks, so wallet rescans, indexes, and serving old blocks could follow the existing pruning behavior. We could also improve wallet and index support for pruning, so they handle missing blocks more reliably. Users who later want an unpruned node could disable pruning and run with -reindex to download and validate all blocks, rebuilding the UTXO set from scratch. Since the snapshot would be trusted from the outset, restoring the missing blocks would be optional rather than a required step before trusting the node.

    Why the IBD check is not a new checkpoint

    The IBD check would compare against a hash that already exists, without changing the rules for accepting blocks or choosing the valid chain with the most work. The node would warn if its UTXO set did not match the baked-in hash, without stopping IBD. An extreme reorg could replace the snapshot block itself, in which case the node would report that change and skip the comparison, since the baked-in hash describes a different chain.

    Cost and related work

    Checking the UTXO set takes a few minutes, and the check could be disabled or moved to a separate thread if needed.

    The check could also build confidence in faster approaches such as pruneassumevalid and SwiftSync by confirming that they produce the expected UTXO set.

    P2P UTXO snapshot distribution could make AssumeUTXO easier to use and to integrate into node packages, and the single-chainstate approach proposed here could address concerns about extending multiple-chainstate validation. The discussion already considered removing background validation to save bandwidth and reduce complexity, and users who want to validate every block locally could do so afterward with -reindex.

  2. mzumsande commented at 2:46 PM on April 29, 2025: contributor

    Depending on what we do if the hash doesn't match, wouldn't this be effectively be reintroducing something similar to checkpoints (which we just got rid of in #31649)?

  3. l0rinc commented at 3:03 PM on April 29, 2025: contributor

    Yes, we already have the "checkpoints", we're just ignoring them. The difference is that we're also validating the checkpoints themselves - giving the assumeutxo scenario more credence ("every IBD verifies these hashes").

  4. mzumsande commented at 3:29 PM on May 5, 2025: contributor

    Removing the existing checkpoints is something that has been done to a large part for ideological reasons (devs shouldn't prescribe / hardcode what the "correct" chain is), so introducing something similar back in (and not just historical but new ones) just for the sake of AssumeUxto would be quite contentious I think - I view AssumeUtxo as a nice feature, but a second-class citizen to normal IBD.

    I guess if we'd just log a warning in case of a mismatch there is no harm (except for the added time spent calculating the hash), but asserting / refusing to continue normal IBD in that case would not be something I'd want.

  5. l0rinc commented at 3:37 PM on May 5, 2025: contributor

    second-class citizen to normal IBD

    Agree, IBD would validate that the AssumeUTXO hashes are correct - lending its credibility to the quicker alternative.

  6. willcl-ark added the label Feature on Jan 15, 2026
  7. willcl-ark added the label Brainstorming on Jan 15, 2026
  8. willcl-ark added the label Validation on Jan 15, 2026
  9. l0rinc renamed this:
    Verify AssumeUTXO snapshot hashes during full validation as well
    AssumeUTXO: crowdsource trust through independent snapshot verification
    on Oct 9, 2026
  10. l0rinc commented at 4:05 AM on October 10, 2026: contributor

    I've expanded the description into a proposal to replace background validation with crowdsourced UTXO snapshot verification through default IBD hash checks, keeping only the latest hash. This extends the existing choice between -assumevalid=0 and the default to the UTXO starting state, with confidence in its hash coming from routine checks across the network. The hash check would warn on a mismatch without stopping IBD, replacing my original proposal to assert. Snapshot imports would still require the expected UTXO set hash. The update adds local hash timings, the pruning model and optional reindexing, resource and maintenance costs, and the differing views on P2P snapshot distribution.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-10-12 02:52 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me