p2p: UTXO set sharing #35054

pull fjahr wants to merge 13 commits into bitcoin:master from fjahr:2026-02-utxo-set-share-safe-take-2 changing 27 files +1714 −1
  1. fjahr commented at 9:58 PM on April 10, 2026: contributor

    This implements a draft BIP for a protocol to share a UTXO set over P2P. Ideally, please review the BIP draft along side this PR if you already want to look at the code.

    Motivation

    The motivation is to make it possible to use the assumeutxo feature without having to acquire a snapshot from a third party.

    UX

    The UX for the user is very similar to how loadtxoutset works today:

    • The user starts their new node and has to wait until the headers have synced
    • The user then invokes the RPC downloadutxoset with their selected height
    • The utxo set download finishes in the background
    • Upon completion the snapshot is compared to assumeutxo params and if those match, it is activated

    Design choices

    • Capability to serve UTXO sets is signaled with a service bit
    • The UTXO set is shared in chunks of 3.9 MB
    • A merkle root of these chunks serves as integrity check to prevent sending useless data as a trivial DoS vector
    • UTXO set snapshots for sharing are placed in a share folder in the datadir, they are atomically loaded on startup and then shared
    • A node that downloaded a snapshot will share it themselves until the snapshot has been deleted from their share folder
    • The merkle root is introduced to the assumeutxo data in the chainparams to ensure peers can not lie to us about it

    Status

    This is certainly still rough around the edges and the merkle roots in the chain params have not all been filled in yet. I am primarily looking for concept and approach feedback as well as potentially overlooked DoS vectors to start.

    If you find a DoS vector on the serving side feel free to crash the live demo node below for bragging rights. Just please give me a heads up afterwards :)

    Live demo

    The feature can be tried out on mainnet:

    1. Build this branch (duh)
    2. Start with a fresh node (datadir) and -connect=178.104.141.103:8333 -debug=utxosetshare. You can also use addnode but this will make it harder to watch the logs since historical blocks are still synced at the same time. Note that since the demo node is pruned, your node will need to be restarted to sync blocks after snapshot validation if you are using -connect.
    3. Wait until the headers have synced, then use bitcoin-cli downloadutxoset 935000
    4. Watch the logs to see the the chunks come in and then see the completed snapshot getting activated
  2. net: Add NODE_UTXO_SET service bit and message types
    Add NODE_UTXO_SET (bit 12) service flag indicating the node can
    serve a UTXO set over P2P.
    
    Also add the P2P message types for UTXO set sharing.
    350985da5e
  3. log: Add UTXOSETSHARE category 34fd9f6c3e
  4. consensus: Expose ComputeMerklePath and add merkle proof verification
    Allows reuse of ComputeMerklePath() for UTXO set chunk Merkle proofs.
    78969203f5
  5. net: Define P2P message serialization for UTXO set sharing 9a6fd56aea
  6. node: Implement UTXO set share provider
    Scans a share directory for valid dumptxoutset snapshot files, validates
    them against known assumeutxo parameters and then serves chunks with Merkle
    proofs. A sidecar file is generated to cache the necessary data and
    signal the file has already been verified in future restarts.
    d16e35ac62
  7. init: Scan share folder in datadir for sharable snapshots 3f8e5c0dc9
  8. net: Handle getutxostinf and getutxoset P2P messages ef15e22d58
  9. node: Implement UTXO set download manager
    The fetched UTXO set is written to the share dir so the node can
    serve the downloaded snapshot afterwards.
    a00eebc6b5
  10. rpc: Add downloadutxoset RPC 0943ed1f17
  11. contrib: Add snapshot merkle root tool
    Standalone script that computes the merkle root of a
    dumptxoutset snapshot file. The output can be used to set
    the chunk_merkle_root field in the assumeutxo chain params.
    0cda5f60a1
  12. net: Validate chunk Merkle root against assumeutxo chain params
    Add chunk_merkle_root to AssumeutxoData so the download manager can
    reject peers that advertise a different Merkle root than expected.
    This prevents a peer serving a UTXO set where the merkle root does
    not correspond to the UTXO set a the expected height or has the
    correct serialized hash.
    
    Check is skipped for now if no value is set for easier testing and
    backwards compatibility.
    b131adeb25
  13. test: Add functional test for P2P UTXO set sharing
    Makes the necessary changes to the functional test framework
    like adding constants.
    
    Also tests common failure cases:
    - Disconnect on wrong block hash
    - Disconnect on wrong height
    - Non-serving node doesn't advertise NODE_UTXO_SET
    - Non-serving node ignores getutxostinf
    - RPC error on downloadutxoset with invalid height
    - RPC error on duplicate downloadutxoset calls
    3fffb9f085
  14. chainparams: Add missing snapshot merkle roots cf5d5b9533
  15. DrahtBot added the label P2P on Apr 10, 2026
  16. DrahtBot commented at 9:59 PM on April 10, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK andrewtoth, svanstaa, danielabrozzoni

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #35148 (refactor: Remove confusing DataStream::in_avail() alias by maflcko)
    • #34860 (mining: always pad scriptSig at low heights, drop include_dummy_extranonce by Sjors)
    • #34628 (p2p: Replace per-peer transaction rate-limiting with global rate limits by ajtowns)
    • #34075 (fees: Introduce Mempool Based Fee Estimation to reduce overestimation by ismaelsadeeq)
    • #33421 (node: add BlockTemplateCache by ismaelsadeeq)
    • #31974 (Drop testnet3 by Sjors)
    • #28690 (build: Introduce internal kernel library by sedited)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • Compute leaf hashes by from snapshot file. -> Compute leaf hashes from snapshot file. [“by from” is a grammatical error that obscures the intended meaning]
    • the peer is be accepted. -> the peer is accepted. [“is be” is a typo that breaks the grammar]

    <sup>2026-04-10 21:59:29</sup>

  17. DrahtBot added the label CI failed on Apr 10, 2026
  18. ?
    added_to_project_v2 fanquake
  19. ?
    project_v2_item_status_changed fanquake
  20. andrewtoth commented at 7:50 PM on April 12, 2026: contributor

    Concept ACK

    Why invoke an RPC, instead of a configuration option (which can be set by default at a later point)?

    If ActivateSnapshot fails, download manager is forever in COMPLETE state. Not sure what we should do in that case. It would mean something is very wrong if our committed merkle root doesn't result in a correct snapshot, or there's disk error. Probably want to abort?

    DoS on downloading side: Server can set utxosetinfo's data_length field to 10 PB, which causes the receiving node to allocate >60 GB immediately. Recommend committing the data_length field alongside the merkle root.

  21. svanstaa commented at 5:30 PM on April 20, 2026: none

    Concept ACK

    Built the branch, set up fresh data dir, connected exclusively to the demo node (-connect=178.104.141.103:8333 -debug=utxosetshare). Waited for headers to sync, then started the snapshot download: bitcoin-cli downloadutxoset 935000.

    Observations:

    • snapshot was activated and node came up at height 935000
    • restarted without -connect to reconnect to the broader network
    • Node synced from the snapshot to current tip normally
    • Background validation from genesis also started

    Overall feature works as advertised.

    Also tried to shoot down the test node via several methods, but it survived all attempts. Will do another post later with the details.

  22. DrahtBot commented at 10:19 AM on April 25, 2026: contributor

    <!--cf906140f33d8803c4a75a2196329ecb-->

    🐙 This pull request conflicts with the target branch and needs rebase.

  23. DrahtBot added the label Needs rebase on Apr 25, 2026
  24. in src/rpc/blockchain.cpp:3462 in cf5d5b9533
    3457 | +        "downloadutxoset",
    3458 | +        "Download a UTXO set snapshot from peers through P2P network.\n"
    3459 | +        "The download happens asynchronously in the background. Once complete, the snapshot is automatically activated. "
    3460 | +        "The snapshot is saved to the share directory inside the data dir, so the node can re-serve it to other peers.",
    3461 | +        {
    3462 | +            {"height", RPCArg::Type::NUM, RPCArg::Optional::NO, "The block height of the UTXO set to download."},
    


    danielabrozzoni commented at 4:21 PM on April 29, 2026:

    I think this is fine for the first implementation, but do you think that in the future we could be smarter and decide the height by ourselves, without the user specifying it? Based on what we have in chainparams, or what we see in utxosetinfo

  25. in src/node/utxo_set_share.cpp:263 in cf5d5b9533
     258 | +                     peer_id, entry.merkle_root.ToString(),
     259 | +                     m_expected_chunk_merkle_root.ToString());
     260 | +            continue;
     261 | +        }
     262 | +
     263 | +        m_data_length = entry.data_length;
    


    danielabrozzoni commented at 5:02 PM on April 29, 2026:

    @andrewtoth said:

    Recommend committing the data_length field alongside the merkle root.

    Yes, I think right now if a peer sends us a utxosetinfo with the wrong data length, we would record it and never notice the mistake, since we are currently only processing the first utxosetinfo message that we receive that contains the merkle_root we're interested in (we switch to DOWNLOADING state in L268)


    andrewtoth commented at 4:50 PM on April 30, 2026:

    It's also a little worse than that currently, maybe I wasn't clear. If they specify a size of say 10 petabytes (or just uint64 max) then the receiving node will allocate 60 GB or more just to track the chunks. So immediate OOM DoS by a malicious server.

  26. danielabrozzoni commented at 4:45 PM on April 30, 2026: member

    Concept ACK!

    I'm slowly going through the BIP and the code, haven't tested yet :) I left a couple of questions.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-02 12:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me