BIP 152: Compact Block Relay #389

 184 | +====Protocol design====
 185 | +There have been many proposals to save wire bytes when relaying blocks. Many of them have a two-fold goal of reducing block relay time and thus rely on the use of significant processing power in order to avoid introducing additional worst-case RTTs. Because this work is not focused primarily on reducing block relay time, its design is much simpler (ie does not rely on set reconciliation protocols). Still, in testing at the time of writing, nodes are able to relay blocks without the extra getblocktxn/blocktxn RTT around 90% of the time. With a smart compact-block-announcement policy, it is thus expected that this work might allow blocks to be relayed between nodes in 0.5*RTT instead of 1.5*RTT at least 75% of the time.
 186 | +
 187 | +====Use of New VarInts====
 188 | +Bitcoin has long had a variable-length integer implementation (referred to as CompactSize in this document), making a second a strange protocol quirk. However, in this protocol most of our variable-length integers are between 0 and 2000. For both encodings, small numbers (<100) are encoded as 1-byte. For numbers over 250, the CompactSize encoding begins to use 3 bytes instead of 1, whereas the New VarInt encoding uses 2. Because the primary motivation for this work is to save bytes during block relay, the extra byte of saving per transaction-difference is considered worth the extra design complexity.
 189 | +

---

   0 | @@ -0,0 +1,212 @@
   1 | +<pre>
   2 | +  BIP: 152
   3 | +  Title: Compact block relay

---

  24 | +==Specification==
  25 | +
  26 | +===Intended Protocol Flow===
  27 | +<img src=bip-0152/protocol-flow.png></img>
  28 | +
  29 | +The protocol is intended to be used in two ways, depending on the peers and bandwidth available, as discussed [[#Implementation_Details|later]]. The "high-bandwidth" mode, which nodes may only enable for a few of their peers, is enabled by setting the first boolean to 1 in a "sendcmpct" message. In this mode, peers send new block announcements with the short transaction IDs already (via a cmpctblock message), possibly even before fully validating the block (as indicated by the grey box in the image above). In some cases no further round-trip is needed, and the receiver can reconstruct the block and process it as usual immediately. When some transactions were not available from local sources (ie mempool), a getblocktxn/blocktxn roundtrip is necessary, bringing the best-case latency to the same 1.5*RTT minimum time that nodes take today, though with significantly less bandwidth usage.

---

  34 | +Several new data structures are added to the P2P network to relay compact blocks: PrefilledTransaction, HeaderAndShortIDs, BlockTransactionsRequest, and BlockTransactions. Additionally, we introduce a new variable-length integer encoding for use in these data structures.
  35 | +
  36 | +For the purposes of this section, CompactSize refers to the variable-length integer encoding used across the existing P2P protocol to encode array lengths, among other things, in 1, 3, 5 or 9 bytes.
  37 | +
  38 | +====New VarInt====
  39 | +TODO: I just copied this out of the src...Something that is wiki-formatted and more descriptive should be used here instead.

---

  94 | +|}
  95 | +
  96 | +====Short transaction IDs====
  97 | +Short transaction IDs are used to represent a transaction without sending a full 256-bit hash. They are calculated by:
  98 | +# single-SHA256 hashing the block header with the nonce appended (in little-endian)
  99 | +# Running SipHash2-4 with the input being the transaction ID and the keys (k0/k1) set to the first two little-endian 64-bit integers from the above hash, respectively.

---