AssumeUTXO: replace `loadtxoutset` RPC with a startup arg #36778

pull l0rinc wants to merge 12 commits into bitcoin:master from l0rinc:l0rinc/assumeutxo-startup changing 22 files +264 −358
  1. l0rinc commented at 2:31 PM on November 5, 2026: contributor

    Problem: Loading a UTXO snapshot through RPC requires coordinating with the active chain, mempool, caches and advertised services while the node is running.

    Fix: Replace the loadtxoutset RPC with -loadtxoutset=<file> and finish importing during chainstate initialization, before networking and mempool loading begin. Require a receiver that has not advanced beyond genesis, already knows the snapshot base header and has no stored blocks beyond genesis or unfinished reindexing. This removes runtime activation checks, temporary import cache resizing and service demotion while preserving snapshot validation, resumption and background validation.

  2. DrahtBot commented at 2:31 PM on November 5, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    Concept ACK fjahr

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #36629 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36629.svg"></sub> (Add output script index and corresponding script history RPC by andrewtoth)
    • #36543 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36543.svg"></sub> (validation: check the most recent AssumeUTXO commitment during IBD by l0rinc)
    • #36542 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36542.svg"></sub> (AssumeUTXO: allow retrying loadtxoutset after an interrupted load by l0rinc)
    • #36503 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36503.svg"></sub> (rpc: reject AssumeUTXO rollback without tip data by l0rinc)
    • #36500 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36500.svg"></sub> (AssumeUTXO: reject duplicate coins and empty snapshot groups by l0rinc)
    • #36346 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36346.svg"></sub> (validation: don't accept txs to the mempool while loading a snapshot by 0xShadowX)
    • #36268 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36268.svg"></sub> (refactor: prune unused semi-colons by fanquake)
    • #34534 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/34534.svg"></sub> (rpc: Manual prune lock management (Take 2) by fjahr)
    • #33324 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/33324.svg"></sub> (blocks: add resumable reobfuscation for existing block files by l0rinc)
    • #31260 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/31260.svg"></sub> (scripted-diff: Type-safe settings retrieval by ryanofsky)
    • #26022 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/26022.svg"></sub> (Add util::ResultPtr class by ryanofsky)
    • #25665 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/25665.svg"></sub> (refactor: Add util::Result failure types and ability to merge result values by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  3. DrahtBot added the label Needs rebase on Nov 5, 2026
  4. DrahtBot added the label CI failed on Nov 5, 2026
  5. DrahtBot commented at 5:31 PM on November 5, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/42045836029/job/127308391176</sub> <sub>LLM reason (✨ experimental): CI failed because IWYU/clang-format detected an uncommitted include change in src/init.cpp (<unordered_map>); the diff check then exited nonzero.</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  6. l0rinc force-pushed on Nov 5, 2026
  7. DrahtBot removed the label CI failed on Nov 5, 2026
  8. DrahtBot removed the label Needs rebase on Nov 6, 2026
  9. l0rinc renamed this:
    assumeutxo: replace `loadtxoutset` RPC with a startup arg
    AssumeUTXO: replace `loadtxoutset` RPC with a startup arg
    on Nov 9, 2026
  10. index: ignore repeated genesis notifications
    When startup connects genesis to an empty chainstate, an index can catch up to genesis before its queued BlockConnected notification arrives.
    Ignore that notification when a best block is already indexed, so Rewind never receives a null parent.
    bd2e531c3e
  11. test: remove snapshot import test with a mempool
    Startup snapshot loading will happen before the mempool is loaded, making this RPC scenario obsolete.
    
    Drop the receiver setting that disabled mempool persistence for this scenario.
    9735c1e372
  12. test: remove cases incompatible with loading a snapshot at startup
    Startup snapshot loading will require a chainstate at genesis.
    Remove the RPC scenarios that import with an active chain with at least as much work, or a different active fork.
    Preserve the difficulty-field checks in the surviving import scenario.
    This avoids adapting those scenarios only to delete them when the RPC is retired.
    35c897a05c
  13. test: start assumeutxo snapshot receivers at genesis
    Start the receivers in the main, wallet and chainstate-tool tests at genesis while their producers keep the cached chain.
    Use the producer's tip time for every node's mocktime in the main and wallet tests.
    Supply the headers that the fresh receivers need in the wallet and chainstate-tool tests.
    
    Run the file-path and invalidated-block checks on a receiver, using the producer for block-hash lookups.
    Adjust the height and transaction-count checks and use the producer's snapshot metadata for the expected hash and coin count.
    
    In the wallet pruning case, check the imported chainstates first, then supply the historical blocks the cached chain used to provide and keep the manual prune sanity check.
    
    Remove the height check after invalidation on the headers-only receiver, whose chainstate remains at genesis.
    
    Exercise a database-open failure before switching the test calls to startup loading.
    249c7a0c1f
  14. node: add startup loading for UTXO snapshots
    Add -loadtxoutset=<file> and call the existing snapshot loader during chainstate initialization, before networking starts.
    Both this option and the existing RPC use the same snapshot activation checks, including the requirement to know the base header.
    The main import test retains the header index but removes the ordinary chainstate before importing.
    Permit a missing active tip in the work comparisons while the RPC still needs those checks.
    Publish the imported tip so startup can complete without an ordinary chainstate loaded from disk.
    Treat interrupted imports as cancellation and report exceptions through the initialization error path.
    
    Keep snapshot loading separate from reindexing and manual block-import modes.
    Skip the GUI reindex retry when an import is requested, since these startup modes cannot be combined.
    Check conflicting startup modes, missing snapshot files and option negation.
    01b7a9562d
  15. node: require genesis for startup snapshot imports
    Require a receiving chainstate at genesis so a new snapshot import does not replace an existing chain.
    Also reject unfinished reindexing and stored blocks beyond genesis, which startup could otherwise connect to that chainstate.
    
    Check both a populated chainstate and an empty chainstate with stored blocks.
    The latter must fail before connecting any of those blocks.
    21337135dc
  16. test: use startup loading in snapshot checks
    Use the startup option for the existing snapshot, wallet and chainstate-tool checks.
    Update the failure checks in place.
    A missing base header now fails startup.
    Mock only the producer's clock, since receivers restart to import and lose theirs.
    
    Restore the wallet after importing its snapshot so it can remain active during background validation.
    This prepares the tests for retiring the RPC without mixing their setup changes into that commit.
    55e1e3c194
  17. rpc: replace loadtxoutset with startup loading
    Retire snapshot loading through RPC now that the startup option is available.
    Keep a hidden method for existing callers and let the RPC fuzzer exercise it because it no longer reads files.
    Leave the help text and migration guidance for the final documentation commit.
    Existing imported snapshots still resume normally without the option.
    dc2cfae8b4
  18. validation: remove redundant snapshot checks
    Now that imports happen only at startup, the active chain cannot advance during an import and the mempool has not been loaded yet.
    Startup also rejects a new import if a snapshot chainstate already exists.
    
    Remove these checks from snapshot activation.
    A supported snapshot has more work than the genesis chainstate, which cannot advance during import, so both work comparisons are redundant.
    Remove the test helper's temporary tip rewind as well.
    Also remove the unit and fuzz checks that attempt another import.
    
    Only the old RPC needed the details of the snapshot's base block in the result.
    Return no value on success and keep the error message on failure.
    Keep the checks on the base header, the supported snapshot commitment and the snapshot contents.
    99b7c177cf
  19. validation: remove snapshot import cache resizing
    Startup imports begin with an empty ordinary chainstate and finish before it can process blocks.
    Give the snapshot loader the full cache budget directly, without shrinking and flushing the ordinary cache first or restoring it after failure.
    Remove the old percentage split and its conversions.
    
    Keep cache redistribution after the import and during background validation, including the forced flush of the imported block-index state.
    60bdb5dc31
  20. fuzz: stop removing local service flags
    The node now announces its limited ability to serve blocks before it accepts any peers.
    Loading a snapshot therefore no longer needs to remove service flags from a running node.
    
    Delete `RemoveLocalServices`, and update the fuzz test to keep track of flags as they are added.
    The node can still announce full block service once background validation completes.
    8de7988a19
  21. doc: describe snapshot loading at startup
    Describe the -loadtxoutset startup option in the user and design docs, the option and RPC help, and a release note.
    Update comments, log messages and errors for startup loading, including the base-header error that asked to call loadtxoutset again.
    
    Correct the swapped file and directory paths in the missing-base-blockhash warning.
    393d05b33c
  22. l0rinc force-pushed on Nov 11, 2026
  23. fjahr commented at 2:58 PM on November 17, 2026: contributor

    Concept ACK

    This would also resolve https://github.com/bitcoin/bitcoin/pull/36346


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-20 15:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me