[Demo] validation: remove AssumeUTXO #36598

pull l0rinc wants to merge 35 commits into bitcoin:master from l0rinc:l0rinc/remove-assumeutxo changing 82 files +672 −5795
  1. l0rinc commented at 6:38 AM on October 22, 2026: contributor

    Draft demo: this is meant to showcase what complete AssumeUTXO removal could look like and the simplifications that AssumeUTXO currently prevents. The decision about complete removal is of course still open. This draft is meant to serve as a concrete reference for the smaller proposals below - it's not meant to be merged.

    Smaller PRs: I will carve out individual changes to gauge interest in each part, and link them here as they are opened. Some of the findings are useful even if snapshot loading or background validation stays. Feedback on which parts seem worth pursuing, and in which order, is welcome.

    Motivation: this follows our in-person discussion at the October 2026 Core Dev meeting. My impression was that there was rough agreement about the ambiguity background validation introduces: we trust a snapshot enough to use it immediately, but describe that trust as temporary until the same node finishes checking history.

    Multiple chainstates affect much more than snapshot loading: ordinary validation changes have to account for different chainstate roles, candidate sets, notifications, downloads, pruning, cache allocation, database lifetimes and locks. I am not sure this amount of complexity in the most critical part of our codebase is still worth it, given that adoption isn't where we hoped it would be. Ordinary IBD is also getting faster: #35295 is already merged, and the upcoming pruneassumevalid will make pruned IBD even faster.

    Snapshot bootstrapping without background validation can still offer an advantage: a new node becomes usable before ordinary IBD would finish because snapshot loading time scales with the UTXO set size, while ordinary IBD time scales with the blockchain size. Whether that is worth its maintenance cost is a fair question, and we can discuss it here or in the dedicated PR where I will propose the removal. Even after removal, we could decide to add it back in a cleaner way, for example isolated through IPC. This demo makes that cost, and the code we could remove, visible for that discussion.

    Background validation can be considered separately: if we keep snapshot loading, I would prefer making its trust assumption explicit and distributing verification across ordinary IBD nodes, as proposed in #32377 and #36543. That could give us wider independent checking of the commitment without making every snapshot user maintain a second chainstate.

    Users who want to validate history themselves and transition to an archival node could rebuild with -reindex and pruning disabled. Alternatively, they could run IBD in a separate datadir while their snapshot node remains usable, then switch once it is ready. @Sjors's #36557 discusses dropping mandatory background sync and these alternatives.

    What the series demonstrates:

    • After the fixture cleanup in #36601, the next eight commits remove background validation while snapshot loading keeps working, a stopping point that can be reviewed on its own.
    • The middle commits remove snapshot imports and startup resumption, then the validation, block storage, notification and release-process special cases they needed.
    • The final seven commits add the simplifications this removal enables: startup in one pass, stable database ownership, one download path, a single prune height, and removal of the locks and diagnostics used to select and identify chainstates.

    Ordinary validation, -assumevalid, pruning, ChainstateManager and UTXO export (dumptxoutset) remain.

    Related work:

  2. DrahtBot commented at 6:38 AM on October 22, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    Concept ACK ismaelsadeeq

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

    LLM Linter (✨ experimental)

    Possible places where comparison-specific test macros should replace generic comparisons:

    • src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW((ChainMan{context, chainman_opts}), std::runtime_error) → Prefer BOOST_CHECK_EXCEPTION with a predicate that checks the expected error message.
    • test/functional/feature_assumeutxo_compat.py: assert len(list(snapshot.blocks_path.glob("blk*.dat"))) >= 2 → assert_greater_than_or_equal(len(list(snapshot.blocks_path.glob("blk*.dat"))), 2)

    <sup>2026-10-22 09:50:32</sup>

  3. test: avoid snapshot insertion in coin fixtures
    Use `AddCoin` and `SpendCoin` to set up the flush, reset and overlay tests, leaving direct coverage of `EmplaceCoinInternalDANGER` in the simulation, fuzz target and duplicate-insertion test.
    Set `possible_overwrite=true` to keep the entries DIRTY but not FRESH, as before.
    1c62336903
  4. DrahtBot added the label CI failed on Oct 22, 2026
  5. test: prepare coverage for AssumeUTXO removal
    Preserve tip notifications, candidate rebuilding and explicit rollback exports before removing mixed snapshot tests.
    6d3f3c2852
  6. validation: stop background cache reprioritization
    Keep the existing snapshot-first cache split after the snapshot reaches tip.
    Remove the IBD-exit resize trigger and the allocation reversal that prioritizes historical validation.
    Activation and startup still resize caches while snapshot loading is supported.
    The existing allocation test no longer needs to force IBD to select this split.
    d72e3a7333
  7. rpc: remove background validation progress
    Remove the backgroundvalidation object from getblockchaininfo and its dedicated assertions.
    Snapshot import, background validation and the getchainstates diagnostic remain available.
    05f043e63e
  8. wallet: remove background progress hints
    Use the ordinary missing-block rescan errors instead of making wallets depend on background progress.
    Keep the snapshot wallet-load restriction and describe explicit reindex recovery.
    Update only the existing error assertions while background validation and wallet scenarios remain supported.
    cc4ba39fe9
  9. index: stop following the historical chainstate
    Use the active chain and existing block-availability checks for index startup.
    Reject snapshot imports while indexes are running and remove automatic index restarts at snapshot completion.
    Keep background validation and its network-service restoration.
    The existing snapshot test starts indexes on its ordinary restart after historical validation completes.
    d084a285cc
  10. validation: stop historical synchronization
    Download blocks only for the active chain and stop historical activation at its common entry point.
    Keep the guard with download removal so startup and direct callers cannot advance incomplete historical candidates.
    Adapt the existing import, wallet and native scenarios to stopped history and explicit reindex recovery.
    46f5f90c04
  11. validation: delete historical activation callers
    Historical activation is already disabled. Remove its callers in ProcessNewBlock and ActivateBestChains.
    Remove completion handling from ConnectTip and the remaining network-service callback.
    Keep periodic flushing and the genesis interrupt ordering.
    Snapshot imports and startup resumption remain supported.
    ba09b96c9a
  12. validation: delete background completion
    Remove historical UTXO hashing, its failure handling and the restart-time directory normalization it drove.
    Remove the hash-mismatch test and reuse the completion scenario to check that restart keeps the snapshot assumed.
    Snapshot loading still checks the compiled commitment before activation.
    Snapshot databases now remain assumed across restart until explicitly discarded by reindexing.
    Previously completed imports can be normalized with a compatible older release before upgrading.
    Remove background progress and prefixes from the existing tip-log helper.
    436d2ec197
  13. rpc: stop new snapshot imports
    Retire loadtxoutset after background validation has been removed independently.
    Keep a hidden RPC tombstone and string argument conversion to explain the removal to existing callers.
    Existing snapshot datadirs still resume, so keep getchainstates and startup coverage until resumption is retired.
    Remove import instructions and end-to-end import scenarios, preserving ordinary kernel-tool coverage.
    Reuse the export test to check the removed RPC's error code with both RPC and CLI transports.
    e5ac899739
  14. net: drop mutable AssumeUTXO service flags
    AssumeUTXO import and completion were the only runtime writers of local services.
    Their removal lets CConnman store the startup services without an atomic.
    Init sets them before network threads start and before RPC warmup ends.
    Keep the constructor's initial NODE_NONE value through its existing Init call.
    060ea8908b
  15. validation: reject legacy snapshot startup
    After snapshot import removal, refuse snapshot datadirs with an active marker before destructive node or kernel initialization.
    Keep normalized ordinary datadirs usable and preserve rejected data for recovery with a compatible older release.
    Remove getchainstates and activation, restart and import fuzz coverage now that supported entry points are closed.
    Keep cache resizing, commitment and metadata parsing tests until their corresponding APIs are removed.
    Document this transition alongside the compatibility tests.
    Delete candidate clearing with its last snapshot-test caller.
    Remove unused database and validation-cache fixture options with their last snapshot test callers.
    ad4cefb50f
  16. rpc: require an explicit UTXO rollback target
    Keep UTXO exports available without trusted snapshot commitments.
    Require a rollback height or hash instead of choosing a supported AssumeUTXO snapshot.
    a9ce3d5d6c
  17. chainparams: remove AssumeUTXO commitments
    Remove the trusted snapshot entries now that import and startup resumption are unavailable.
    Keep the lookup interfaces until their remaining callers are removed.
    52ee1d9a16
  18. validation: drop snapshot chain restrictions
    Snapshot startup and imports are disabled. Remove the remaining snapshot-base peer filter and historical fork-warning exception.
    d8b0ddc02f
  19. validation: load only the active chainstate
    Snapshot imports and startup resumption are unavailable.
    Initialize and verify the ordinary chainstate directly, retaining interruption and database errors.
    Completion hashing and directory normalization were removed with background validation.
    Delete the unused restart loader and chainstate deletion helper with their last startup callers.
    Remove the deletion test while retaining ordinary kernel operation without a mempool.
    95fd82307d
  20. validation: delete AssumeUTXO coin loading
    Delete snapshot activation, coin import and their unused helpers.
    Keep AddChainstate until its remaining cache-rebalancing test is removed.
    d5c7d02f56
  21. coins: delete AssumeUTXO insertion bypass
    Snapshot import bypassed the normal AddCoin spentness checks through EmplaceCoinInternalDANGER.
    With the importer removed and fixtures using AddCoin and SpendCoin, delete the bypass and its entry constructor together with the remaining simulation, fuzz and duplicate-insertion coverage.
    69a90c6644
  22. validation: delete obsolete cache resizing
    AssumeUTXO activation and cache sharing no longer need runtime resizing.
    Remove the final startup rebalance call and the unused rebalancing and resizing APIs.
    Keep startup cache-budget storage and use the existing test budget field directly.
    The database is no longer replaced, so compaction needs no extra mutex.
    Remove AddChainstate with its final cache-rebalancing test caller.
    Remove the resizing and rebalancing tests with the APIs they exercise.
    9880f12c59
  23. node: delete AssumeUTXO directory metadata
    Use the ordinary chainstate directory without snapshot path selection.
    Remove the base-block metadata readers, writers and StoragePath helper.
    Retain the UTXO export file format and its metadata parser.
    e749dc5bdd
  24. validation: remove AssumeUTXO index exceptions
    Snapshot loading has already been removed, so candidates require transaction history back to genesis.
    Remove snapshot-base exceptions from transaction counts and consistency checks.
    Ordinary script, sigop, BIP30 and proof-of-work checks are unchanged.
    c824984cd3
  25. node: load block counts without AssumeUTXO
    Reconstruct cumulative transaction counts from downloaded block history.
    Remove the trusted snapshot count supplied during block-index loading.
    Remove the commitment type, storage and lookup interfaces with their last caller.
    Delete FindFirst with the lookup methods that used it.
    Include util/vector.h directly for the external signer now that chainparams no longer supplies it.
    9cf0eaa0b7
  26. node: use one block-file cursor and prune budget
    Without AssumeUTXO, historical and tip downloads no longer need separate block files or divided prune budgets.
    Write blocks and undo data through the ordinary block-file cursor.
    Keep cs_main locking, undo flush failures and pruning retention unchanged.
    With one cursor, flushing always targets the current block file. Remove the separate file selector and forwarding wrapper.
    The undivided prune target already satisfies the configured minimum, so it needs no per-chainstate clamp.
    Delete the obsolete cursor selector and printer with their last callers.
    Remove the snapshot lower prune bound while preserving the existing pruning helper interfaces.
    f3d6b9f3cb
  27. wallet: use ordinary block availability checks
    Retain pruning and rescan errors without the removed AssumeUTXO state.
    Do not change wallet transaction accounting.
    Remove hasAssumedValidChain with its last wallet callers.
    d08f939dc3
  28. refactor: drop AssumeUTXO notification roles
    All remaining notifications come from the ordinary chainstate.
    Remove the role argument from internal callbacks and their callers.
    The public kernel callback ABI is unchanged.
    Delete the unused role accessor, formatter and type with their consumers.
    48c5977694
  29. refactor: use the ordinary chainstate accessors
    Use the active chainstate for indexing and progress.
    No snapshot chainstate can be constructed or loaded.
    Remove the unused snapshot target, base and chainstate-selection methods with their last callers.
    ce6993c4f3
  30. validation: retain one ordinary chainstate
    Replace the chainstate vector with one owned chainstate.
    Remove snapshot targets, validity state and obsolete aliases.
    Snapshot activation no longer transfers the optional mempool between chainstates.
    Keep its pointer mutable for fuzz fixtures that replace the pool.
    Keep shutdown flushing and block activation on the ordinary path.
    Remove ResetChainstates with its last caller instead of rewriting its unused body.
    A successfully created kernel manager always owns that state at destruction.
    Tip notifications no longer need to check which chainstate is active.
    8372144e28
  31. rpc: remove snapshot metadata decoding
    Deleting the AssumeUTXO importer leaves only the export side of the snapshot metadata format.
    Delete the unused metadata parser, its deserialization fuzzer and the parser-only network-magic lookup.
    Keep the existing export serializer and its callers unchanged.
    19a40435cc
  32. node: initialize one chainstate in one pass
    AssumeUTXO completion no longer tears down and reloads chainstates during startup.
    Fold the remaining initialization pass into LoadChainstate and pass the cache budgets directly to the sole chainstate instead of retaining totals for redistribution.
    bbdaa554a7
  33. txdb: own the coins database directly
    Removing AssumeUTXO cache resizing leaves the coins database open for the lifetime of its view.
    Store the wrapper by value and retain only the path used in compaction logs instead of parameters for reopening it.
    Pass database parameters by reference now that neither view retains them.
    Keep the compaction future, overlap check and destructor wait so a worker cannot outlive the database.
    646997a903
  34. net: use one block download path
    Historical AssumeUTXO downloads were the only caller that omitted the active chain and staller tracking.
    Fold the shared traversal into the ordinary download path and evaluate its in-flight budget once now that there is no second download pass.
    f5f80498b0
  35. node: prune using a single upper height
    The snapshot base was the only nonzero lower bound on pruning.
    With AssumeUTXO removed, keep the tip-distance and requested-height limits in block storage and drop the always-zero lower bound and chainstate range accessor.
    7c084b9070
  36. validation: drop chainstate selection locking
    Snapshot activation no longer replaces the active chainstate while the node is running.
    Read the fixed owner without cs_main and remove locks that existed only to select it, including the outer fee-estimation lock around a block assembler that locks internally.
    Chain contents and validation still retain their existing synchronization.
    701ada3a74
  37. validation: drop chainstate identity diagnostics
    With AssumeUTXO gone, activation visits one chainstate and no longer needs to identify which state failed.
    Use the singular activation name and remove the chainstate formatter, its startup log and the lock used only to format the activation error.
    824d0fb577
  38. rpc: keep UTXO export serialization local
    Removing snapshot imports leaves dumptxoutset as the only consumer of the snapshot header and export helpers.
    Write the unchanged version 2 header at its sole call site and keep the helpers inside the RPC translation unit.
    Drop unused default arguments from these now-local export helpers.
    bc62b59b8d
  39. l0rinc force-pushed on Oct 22, 2026
  40. ismaelsadeeq commented at 10:46 AM on October 22, 2026: member

    Concept ACK.

    Following on from what we discussed in person: a feature-ready, usable assumeUTXO is with P2P utxo snapshot sharing, no background validation, and potentially on by default. I don't think it's a state we want to be in. That variant of assumeUTXO trusts the whole money supply and UTXO set without ever reconstructing it. Making that the default new user experience, for the dominant full node implementation, is a shift in what running a node means, I'd rather we lean on fewer assumptions, e.g. even removing assumevalid as default.

    If the feature-ready version lands and most new nodes come up from a snapshot and never pull the history, then over time the archival nodes thin out and validating/having the chain's history becomes something only a handful of nodes still do. Worst case, you can recover your funds but not the history. Pruning already nudges us that way (though it's not on by default, and most users only prune when they actually need to), but a snapshot default would speed it up. It's a bit of an extreme scenario, but I don't think it's an implausible state to be in, if the direction is future ready assumeUTXO.

    And as discussed, as-is it's not even that usable, loading the snapshot takes too long. I remember giving up when trying it for the first time. (This can be fixed though)

    I've also hit a maintenance cost directly. e.g. in #34075 we do block building, and query the tip which needs the chainstate, and multiple chainstates make that harder than it should be. Any node component that needs chainstate has no guarantee the chainstate it grabbed is still live, and the active one can get swapped under it, so you end up holding onto the chainstate manager and re-fetching the active chainstate every time instead of just keeping a stable handle. The last seven commits here show what you get back once this is gone.

    I do think there's probably still a group of users who'd want assumeUTXO like snapshot bootstrapping. But I'd rather that live outside the dominant full node, libbitcoinkernel c header api should gives downstream projects a way to embed a ready UTXO set without risking a consensus split? and with LLMs now, that node-like projects can easily be vibe coded.

    Snapshot loading scales with the UTXO set size while IBD scales with chain length indeed, so there is some argument on the linear growth concern and that #35295 and your pruneassumevalid idea don't fully close that gap for a new node that validates/semi validates the chain history, and that gap only grows as the chain gets longer. That's a genuine advantage of assumeUTXO. But I don't think it's worth the complexity and assumptions it adds to the the reference implementation. @l0rinc should we do it through the usual deprecate then remove style? Also thanks for putting the demo together, having the removable code and the simplifications it shows is nice to see.

  41. DrahtBot removed the label CI failed on Oct 22, 2026

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-10-23 11:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me