From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Tue, 05 May 2026 09:18:01 -0700 Received: from mail-oa1-f57.google.com ([209.85.160.57]) by mail.fairlystable.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1wKIT9-0005Hu-Q2 for bitcoindev@gnusha.org; Tue, 05 May 2026 09:18:01 -0700 Received: by mail-oa1-f57.google.com with SMTP id 586e51a60fabf-40f25e55f20sf8214272fac.0 for ; Tue, 05 May 2026 09:17:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1777997874; x=1778602674; darn=gnusha.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:sender:from :to:cc:subject:date:message-id:reply-to; bh=E2eyoMigcOauVr5kpXn45k9iPm0eSmO00Zr8KehJmzs=; b=c1DjdMLWJn599EhebCYZMUo5dUgx5sV5jpPl4al6Z3RSdTvwuhnnrQ7pMuoLcEdlcb GgzyHODZo0xPyn3JetnpodCHDx1BZIMEKAptKJwdhbt8/CitNODNc/ok1ivxSwIWXn5e +KlzaLpL72aIBkSn0Y4VXNvbG1Xt9AQt5YUfndoTSLhQYWNeODIhZAXVA/MITAOLvgnG BIDGCugO7IZMZXphCYdIG5/uCVufXy3ruZ74subSOwFWRCxXr8T/TYoE1qg8FzSCAnv8 7NSclVU4kvpzStF62nk3f8G8vGbn8uVr4/7leT1RFOs0eFVA7Z3JUyXhRgffn3NBwzk3 x65w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=voskuil.org; s=google; t=1777997874; x=1778602674; darn=gnusha.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=E2eyoMigcOauVr5kpXn45k9iPm0eSmO00Zr8KehJmzs=; b=Fh+vj0vWbNCaNiI8ftavIiczWT/K90F3/x215zyP3S434FJdXHoEQuh/mEQ94ia3T1 TG8leJbdWjys5EgpO7wl5g/Ebjq0R2Wo+ZAYg1Z55iExatRv+mO1mmGvxfE/hsXq939l Nf0gkqN7XSgi4qkeOqp6IVeZpsGtpvwItjBcsN79IUIFc/KRbjq/HiwE3TOaoWDnRFYJ mKeyIZIlbiySs7CZyPSth3guKQOH7FvdtidGxHh5upLAzQDkit6sjUukRwnpwN4ULp3w 1p909tmn2fXD9Y51qYGdxc6TTLWChczb6gIs/lDQajCYxalFQLnzFO0LIub/muoR8wiP kjvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777997874; x=1778602674; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:x-beenthere :x-gm-message-state:sender:from:to:cc:subject:date:message-id :reply-to; bh=E2eyoMigcOauVr5kpXn45k9iPm0eSmO00Zr8KehJmzs=; b=iraew6DzEGPCUVvnp0uqXTWkEKpV10z5N6T6pMx8S2ukdSLxNrzg8caJJQ9VULEoaP B09EuznsHu+8pAtBogWAL4+wSw/QNmlL1iEC4jZoLprRfEZa5dYlea6Yp8fo8V71T0K3 SmjuLO6ImyrbkdRaVOqjYQOKRhl+v+pDSVSdBdG0YIuoP07HKk+AkQlx1I8Dlycinr4B iOavc5ucJSh8g4sdHZel/S6aoDqt/vrw2ByZbJIvAAvAfE2nlvFjntCpN1kKfxotqQ4h HfvG1IpOia3fRkj6jrlQvvGmxeEWYFpQAoVBZ1Lm0yCzMLvZg0EAFYsqgWTxkPau1FNF YI+Q== Sender: bitcoindev@googlegroups.com X-Forwarded-Encrypted: i=1; AFNElJ8qJevLweFPALY0ptUwZ8mYvIFyrKKv9Q+fP0Gg8CsMumA+YmUc5dPqKlK4P7lKaCqzsttSVYfV8+yI@gnusha.org X-Gm-Message-State: AOJu0Yxy4l4JAjM5cAKg2fDtl48GoEjOJAmo/Q96z8MeVTBgmKT5eWgv 0N9/8Zl4N/w4uLisC8UPSrX2raCTn0chxbL0rY3SLxldDIssqtDXFrj3 X-Received: by 2002:a05:6870:f629:b0:414:93cb:b992 with SMTP id 586e51a60fabf-434762ccd72mr7597652fac.33.1777997873584; Tue, 05 May 2026 09:17:53 -0700 (PDT) X-BeenThere: bitcoindev@googlegroups.com; h="AUV6zMPXtwnqtFD2wYjRTeKqCBxrQLraLeleYbGU0wTwUa7xyg==" Received: by 2002:a05:6870:5108:b0:42b:cff6:f28e with SMTP id 586e51a60fabf-4343033bd7cls3874670fac.2.-pod-prod-03-us; Tue, 05 May 2026 09:17:48 -0700 (PDT) X-Received: by 2002:a05:6808:e40c:b0:479:eb19:6e66 with SMTP id 5614622812f47-47c892c7568mr5805839b6e.38.1777997867995; Tue, 05 May 2026 09:17:47 -0700 (PDT) Received: by 2002:a05:690c:578f:b0:79a:c9dc:1f8d with SMTP id 00721157ae682-7bd7670b083ms7b3; Tue, 5 May 2026 09:01:45 -0700 (PDT) X-Received: by 2002:a05:690c:e3c7:b0:7b2:9347:7be1 with SMTP id 00721157ae682-7bd770df5aemr156689237b3.29.1777996904695; Tue, 05 May 2026 09:01:44 -0700 (PDT) Date: Tue, 5 May 2026 09:01:44 -0700 (PDT) From: Eric Voskuil To: Bitcoin Development Mailing List Message-Id: <19616822-8a03-4de1-99be-72d50479208fn@googlegroups.com> In-Reply-To: References: Subject: [bitcoindev] Re: [BIP Draft] P2P UTXO Set Sharing MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1005932_918551034.1777996904044" X-Original-Sender: eric@voskuil.org Precedence: list Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com List-ID: X-Google-Group-Id: 786775582512 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Score: -0.7 (/) ------=_Part_1005932_918551034.1777996904044 Content-Type: multipart/alternative; boundary="----=_Part_1005933_992242423.1777996904044" ------=_Part_1005933_992242423.1777996904044 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Concept NACK. It's bad enough that nodes are formalizing this off network,= =20 but incorporating it into p2p is another level of awful. On Tuesday, May 5, 2026 at 11:39:56=E2=80=AFAM UTC-4 Fabian wrote: > Dear list, > > I am sharing a BIP draft for sharing the UTXO set over the P2P network, a= n=20 > old idea that makes it possible to utilize AssumeUTXO without sourcing a= =20 > UTXO set dump from a third party source. You can find the full text below= =20 > or comment on the BIPs repository pull directly:=20 > https://github.com/bitcoin/bips/pull/2137 > > Fabian > > > ``` > BIP: ? > Layer: Peer Services > Title: P2P UTXO Set Sharing > Authors: Fabian Jahr > Status: Draft > Type: Specification > Assigned: ? > Discussion: ? > Version: 0.2.0 > License: BSD-2-Clause > ``` > > ## Abstract > > This BIP defines a P2P protocol extension for sharing full UTXO sets=20 > between peers. It introduces > a new service bit `NODE_UTXO_SET`, four new P2P messages (`getutxotree`,= =20 > `utxotree`, `getutxoset`, > `utxoset`), and a chunk-hash list anchored to a Merkle root known to the= =20 > requesting node, enabling > per-chunk verification. This allows nodes to bootstrap from a recent=20 > height by obtaining the > required UTXO set directly from the P2P network via mechanisms such as=20 > assumeutxo. > > ## Motivation > > The assumeutxo feature (implemented in Bitcoin Core) allows nodes to begi= n=20 > operating from a serialized > UTXO set while validating > historical blocks in the background. However, there is currently no=20 > canonical source for obtaining this > data. Users must either generate one themselves from a fully synced node= =20 > (using `dumptxoutset` in=20 > Bitcoin Core), or download one from a third party. > > By enabling UTXO set sharing over the P2P network, new nodes can obtain= =20 > the data directly from > peers, removing the dependency on external infrastructure. > > ## Specification > > The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in=20 > this document are to be > interpreted as described in RFC 2119. > > ### Service Bit > > | Name | Bit | Description | > |------|-----|-------------| > | `NODE_UTXO_SET` | 12 (0x1000) | The node can serve complete UTXO set=20 > data for at least one height. | > > A node MUST NOT set this bit unless it has at least one full UTXO set=20 > available to serve. > A node signaling `NODE_UTXO_SET` MUST be capable of responding to=20 > `getutxotree` and `getutxoset` > requests for every UTXO set it is willing to serve, including the full=20 > chunk-hash list and every > chunk of those sets. > > ### Data Structures > > #### Serialized UTXO Set > > The serialized UTXO set uses the format established by the Bitcoin Core= =20 > RPC `dumptxoutset` (as of Bitcoin Core v31). > > **Header (55 bytes):** > > | Field | Type | Size | Description | > |-------|------|------|-------------| > | `magic` | `bytes` | 5 | `0x7574786fff` (ASCII `utxo` + `0xff`). | > | `version` | `uint16_t` | 2 | Format version. | > | `network_magic` | `bytes` | 4 | Network message start bytes. | > | `base_height` | `uint32_t` | 4 | Block height of the UTXO set. | > | `base_blockhash` | `uint256` | 32 | Block hash of the UTXO set. | > | `coins_count` | `uint64_t` | 8 | Total number of coins (UTXOs) in the= =20 > set. | > > **Body (coin data):** > > Coins are grouped by transaction hash. For each group: > > | Field | Type | Size | Description | > |-------|------|------|-------------| > | `txid` | `uint256` | 32 | Transaction hash. | > | `num_coins` | `compact_size` | 1=E2=80=939 | Number of outputs for this= txid. | > > For each coin in the group: > > | Field | Type | Size | Description | > |-------|------|------|-------------| > | `vout_index` | `compact_size` | 1=E2=80=939 | Output index. | > | `coin` | `Coin` | variable | Serialized coin (varint-encoded code for= =20 > height/coinbase, then compressed txout). | > > Coins are ordered lexicographically by outpoint (txid, then vout index),= =20 > matching the LevelDB iteration > order of the coins database. > > #### Chunk Merkle Tree > > The serialized UTXO set (header + body) is split into chunks of exactly= =20 > 3,900,000 bytes (3.9 MB). The > last chunk contains the remaining bytes and may be smaller. > > The leaf hash for each chunk is `SHA256d(chunk_data)`. The tree is built= =20 > as a balanced binary tree. When > the number of nodes at a level is odd, the last node is duplicated before= =20 > hashing the next level. > Interior nodes are computed as `SHA256d(left_child || right_child)`. > > The leaves are delivered to the node in a single `utxotree` response. A= =20 > node that knows > the Merkle root for a given UTXO set checks a received list of leaves by= =20 > recomputing the root and > comparing. The Merkle root is the sole trust input required to verify the= =20 > integrity of the received UTXO set. > > `SHA256d` denotes double-SHA256: `SHA256d(x) =3D SHA256(SHA256(x))`. > > ### Messages > > #### `getutxotree` > > Sent to request the chunk-hash list for a specific UTXO set. > > | Field | Type | Size | Description | > |-------|------|------|-------------| > | `block_hash` | `uint256` | 32 | Block hash identifying the requested=20 > UTXO set. | > > A node that has advertised `NODE_UTXO_SET` and can serve the requested=20 > UTXO set MUST respond with > `utxotree`. If the serving node cannot fulfill the request, it MUST NOT= =20 > respond. The requesting > node SHOULD apply a reasonable timeout and try another peer. > > #### `utxotree` > > Sent in response to `getutxotree`, delivering the full chunk-hash list=20 > along with per-snapshot > metadata. > > | Field | Type | Size | Description | > |-------|------|------|-------------| > | `block_hash` | `uint256` | 32 | Block hash this data corresponds to. | > | `version` | `uint16_t` | 2 | Format version of the serialized UTXO set.= | > | `data_length` | `uint64_t` | 8 | Total size of the serialized UTXO set= =20 > in bytes (header + body). | > | `num_chunks` | `compact_size` | 1=E2=80=939 | Number of chunks the seri= alized=20 > UTXO set is split into. | > | `chunk_hashes` | `uint256[]` | 32 =C3=97 `num_chunks` | The ordered lis= t of=20 > chunk hashes. | > > Upon receiving a `utxotree` message, the node MUST recompute the Merkle= =20 > root from > `chunk_hashes` and compare it against the Merkle root it knows for the=20 > corresponding UTXO set. If > the roots do not match, the node MUST discard the response and MUST=20 > disconnect the peer. > > #### `getutxoset` > > Sent to request a single chunk of UTXO set data. The requesting node MUST= =20 > have received a `utxotree` > for the corresponding UTXO set before sending this message. > > | Field | Type | Size | Description | > |-------|------|------|-------------| > | `block_hash` | `uint256` | 32 | Block hash identifying the requested=20 > UTXO set. | > | `chunk_index` | `uint32_t` | 4 | Zero-based index of the requested=20 > chunk. | > > If the serving node cannot fulfill the request, it MUST NOT respond. The= =20 > requesting node SHOULD apply > a reasonable timeout and try another peer. > > #### `utxoset` > > Sent in response to `getutxoset`, delivering one chunk. > > | Field | Type | Size | Description | > |-------|------|------|-------------| > | `block_hash` | `uint256` | 32 | Block hash this data corresponds to. | > | `chunk_index` | `uint32_t` | 4 | Zero-based index of this chunk. | > | `data` | `bytes` | variable | Chunk payload, exactly 3.9 MB except for= =20 > the last chunk. | > > The transfer is receiver-driven: the requesting node sends one=20 > `getutxoset` per chunk. Chunks MAY be > requested in any order and from different peers. > > Upon receiving a `utxoset` message, the node MUST compute `SHA256d(data)`= =20 > and compare it against > `chunk_hashes[chunk_index]` from the `utxotree` it accepted for this UTXO= =20 > set. If the hashes do not > match, the node MUST discard the chunk and MUST disconnect the peer. A=20 > node SHOULD also disconnect > a peer that sends a `utxoset` message with fields (`chunk_index`,=20 > `block_hash`) that do not match > the outstanding request. > > After all chunks have been received, the node SHOULD parse the reassemble= d=20 > UTXO set against the > serialized UTXO set format to confirm it is well-formed. > > ### Protocol Flow > > 1. The requesting node identifies peers advertising `NODE_UTXO_SET`. > 2. The requesting node sends `getutxotree` for the desired block hash to= =20 > one or more of these peers. > 3. Each peer responds with `utxotree`. The requesting node verifies the= =20 > response by recomputing > the Merkle root against a value it knows for the given UTXO set, eithe= r=20 > from a trusted source > or by selecting a root with agreement among multiple peers. > 4. The requesting node downloads chunks via `getutxoset`/`utxoset`=20 > exchanges, verifying each chunk > against its entry in the accepted `utxotree` on receipt. On=20 > verification failure the peer is > disconnected and download continues from another peer without losing= =20 > already-verified chunks. > 5. After all chunks are received, the node parses the reassembled UTXO se= t=20 > against the serialized > UTXO set format to confirm that it is well-formed. > > Serving nodes are free to limit the number of concurrent and repeated=20 > transfers per peer at their own > discretion to manage resource consumption. > > ## Rationale > > **Usage of service bit 12:** Service bits allow selective peer discovery= =20 > through > DNS seeds and addr relay. Bit 12 is chosen as the next unassigned bit=20 > after `NODE_P2P_V2` (bit 11, BIP 324). > > **Direct request model:** Peers signal availability of UTXO sets via the= =20 > `NODE_UTXO_SET` > service bit; the requesting node identifies the desired UTXO set by block= =20 > hash when sending > `getutxotree`. The serving node responds only if it can serve that=20 > specific UTXO set. > > **Per-chunk verification:** The chunk-hash list returned in `utxotree`=20 > enables each chunk to be verified > by direct lookup against the accepted list as it arrives, allowing=20 > immediate detection of corrupt data, > peer switching without data loss, and parallel download from multiple=20 > peers. The list itself is small > (~80 KB for a ~10 GB set). The specified serialization is deterministic,= =20 > so all honest nodes produce > byte-identical output, guaranteeing Merkle root agreement. > > **3.9 MB chunk size:** The number balances round trips (~2,560 for a ~10= =20 > GB set) against memory usage > for buffering and verifying a single chunk. Smaller chunks would increase= =20 > protocol overhead; larger > chunks would increase memory pressure on constrained devices commonly use= d=20 > to run Bitcoin nodes. > Together with the additional message overhead, the `utxoset` message=20 > including the chunk data also > sits just below the theoretical maximum block size which means any=20 > implementation should be able to > handle messages of this size. > > **Reusing the `dumptxoutset` format:** Avoids introducing a new=20 > serialization format and ensures > compatibility with UTXO sets already being generated and shared. > > **Relationship to BIP 64:** BIP 64 defined a protocol for querying=20 > individual UTXOs by outpoint and is > now closed. This BIP addresses a different use case: bulk transfer of the= =20 > entire UTXO set for node > bootstrapping. > > ## Reference Implementation > > [Bitcoin Core implementation pull request]( > https://github.com/bitcoin/bitcoin/pull/35054) > > ## Copyright > > This BIP is made available under the terms of the 2-clause BSD license. S= ee > https://opensource.org/license/BSD-2-Clause for more information. > > ## Changelog > > * __0.2.0__ (2026-05-04): > * Dropped discovery before download approach, instead request the=20 > chunk-hash list via `getutxotree`/`utxotree` > * Dropped per-chunk Merkle proofs; chunks verified directly against= =20 > the chunk-hash list > * Dropped `height` from requests (`block_hash` is the sole=20 > identifier); added format `version` to `utxotree` > * Dropped references to the serialized hash; the Merkle root is the= =20 > sole integrity check > * __0.1.0__ (2026-04-10): > * Initial draft > --=20 You received this message because you are subscribed to the Google Groups "= Bitcoin Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoindev+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/= 19616822-8a03-4de1-99be-72d50479208fn%40googlegroups.com. ------=_Part_1005933_992242423.1777996904044 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Concept NACK. It's bad enough that nodes are formalizing this off network, = but incorporating it into p2p is another level of awful.

On Tuesday, May = 5, 2026 at 11:39:56=E2=80=AFAM UTC-4 Fabian wrote:
De= ar list,

I am sharing a BIP draft for sharing the UTXO set over t= he P2P network, an old idea that makes it possible to utilize AssumeUTXO wi= thout sourcing a UTXO set dump from a third party source. You can find the = full text below or comment on the BIPs repository pull directly:=C2=A0https://github.com/bitcoin/bips/pull/2137

Fabia= n


```=C2=A0 BIP: ?
=C2=A0 Layer: Peer Services
=C2=A0 Title: P2P UTXO Set Sharing
<= span>=C2=A0 Authors: Fabian Jahr <https://github.com/bitcoin/bi= tcoin/pull/35054)

## Copyright

This BIP is made available under the te= rms of the 2-clause BSD license. See
https:= //opensource.org/license/BSD-2-Clause for more information.

## Changelog

* __0.2.0__ (2026-05-04):
=C2=A0 =C2=A0 * Dropped= discovery before download approach, instead request the chunk-hash list vi= a `getutxotree`/`utxotree`
=C2=A0 =C2=A0 * Dropped p= er-chunk Merkle proofs; chunks verified directly against the chunk-hash lis= t
=C2=A0 =C2=A0 * Dropped `height` from requests (`b= lock_hash` is the sole identifier); added format `version` to `utxotree`
=C2=A0 =C2=A0 * Dropped references to the serialized h= ash; the Merkle root is the sole integrity check
* _= _0.1.0__ (2026-04-10):
=C2=A0 =C2=A0 * Initial draft

--
You received this message because you are subscribed to the Google Groups &= quot;Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to bitcoind= ev+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoind= ev/19616822-8a03-4de1-99be-72d50479208fn%40googlegroups.com.
------=_Part_1005933_992242423.1777996904044-- ------=_Part_1005932_918551034.1777996904044--