net: serve `MSG_BLOCK` by stripping witnesses in place #36548

pull l0rinc wants to merge 8 commits into bitcoin:master from l0rinc:l0rinc/msg-block-serving changing 8 files +482 −48

l0rinc commented at 1:08 AM on October 16, 2026: contributor

Problem: Serving blocks to legacy nodes currently decodes and reserializes them just to remove witnesses. MSG_WITNESS_BLOCK already avoids this work by serving the stored bytes directly. Faster witnessless IBD, as proposed in #36476, also needs peers to serve these blocks efficiently.

Fix: Read disk-backed, non-Signet MSG_BLOCK replies into an owned byte buffer, check the header's proof of work and expected hash, and strip witness ranges from that buffer in place.

Extract a shared helper so decoding transactions and skipping their bytes follow the same encoding rules. Signet keeps ReadBlock for its solution check. Header checks now precede transaction parsing, so records corrupted in both places can log a different error.

Performance:

Time to read a block from disk and build its non-witness message:

Block Before After Speedup

block413567 1.31 ms 0.147 ms 8.9x

Same block with synthetic witnesses 3.47 ms 0.374 ms 9.3x

The synthetic fixture adds a 512-byte witness item and an empty item to every input.

Related work: This extends the raw witness-block serving added in #13151. The #13011 discussion and the closed #13098 identify unnecessary transaction hashing when serving historical blocks. The implementation also builds on the bulk block reads introduced in #31551.

Block	Before	After	Speedup
`block413567`	1.31 ms	0.147 ms	8.9x
Same block with synthetic witnesses	3.47 ms	0.374 ms	9.3x

test: cover block serving serialization

Check exact `MSG_BLOCK` and `MSG_WITNESS_BLOCK` payloads before and after recent-block cache eviction.
Cover successful and failing `ReadBlock` cases before introducing the serialized reader.

98ce82547b

refactor: share block read hash checks

Extract the expected-hash check for reuse by the serialized reader.
Keep Signet solution validation before the shared check.

7c6e5549f5

bench: measure non-witness block serving

Measure disk reads followed by `TX_NO_WITNESS` message serialization before replacing the full block reader.

Witness-free: 1.31 ms/block
Witness-bearing: 3.47 ms/block

e2b4f330cb

node: read non-witness block payloads

Add `ReadBlockWithoutWitness` as a decode-and-reserialize reference implementation.
It checks proof of work and the expected header hash before transaction decoding, drops trailing bytes, and requires non-Signet consensus because it omits the Signet solution check.

The benchmark takes 1.42 ms/block for witness-free data and 3.65 ms/block for witness-bearing data.

9f15a0da42

net: serve `MSG_BLOCK` from serialized data

Use `ReadBlockWithoutWitness` for disk-backed `MSG_BLOCK` requests.
Keep Signet and cached, witness, compact, and filtered block requests on their existing paths.

f0d2fb57be

refactor: extract transaction decoding rules

Extract marker, optional-data, input, output, and witness handling into `UnserializeTransactionBody` so block witness stripping can reuse the decoder's validation.
Keep the ordinary decoder's reads, clears, and error order unchanged.

8e2e518d2a

node: strip witnesses without decoding

Walk serialized transactions with `UnserializeTransactionBody` and copy only retained ranges.
This shares the ordinary decoder's marker, optional-data, CompactSize, and witness checks without allocating transactions or computing their cached hashes.
Witness-free encodings are still walked completely, and trailing bytes remain excluded.

The benchmark falls from 1.41 to 0.161 ms/block for witness-free data and from 3.65 to 0.384 ms/block for witness-bearing data.

3557ab74e6

DrahtBot added the label P2P on Oct 16, 2026
DrahtBot commented at 1:08 AM on October 16, 2026: contributor


The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.



External sites
- Code coverage & benchmarks: https://corecheck.dev/bitcoin/bitcoin/pulls/36548
- Ralph review <img src=https://review.fish.foo/assets/bitcoin/bitcoin/36548.png alt="Ralph review status"> https://git.fish.foo/bitcoin/bitcoin/pulls/36548#author-ralph


Reviews

See the guideline and AI policy for information on the review process. A summary of reviews will appear here.



Conflicts

Reviewers, this pull request conflicts with the following ones:
- #36521 <img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36521.svg"> (Keep witness stacks in serialized form in CScriptWitness by sipa)
- #36074 <img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36074.svg"> (scripted-diff: [test] Add util/check.h includes for assertions by maflcko)
- #36000 <img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36000.svg"> (validation: prefetch blocks while connecting by l0rinc)
- #35731 <img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/35731.svg"> (Indexes: Harden the flush-error notification invariant by arejula27)
- #32554 <img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/32554.svg"> (bench: replace embedded raw block with configurable block generator by l0rinc)
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.



LLM Linter (✨ experimental)

Possible places where comparison-specific test macros should replace generic comparisons:
- test/functional/p2p_getdata.py: assert len(from_binary(CBlock, full).serialize(with_witness=False)) < len(full) → Prefer assert_greater_than(len(full), len(from_binary(CBlock, full).serialize(with_witness=False))).
2026-10-16 13:37:48
DrahtBot added the label CI failed on Oct 16, 2026
DrahtBot commented at 2:21 AM on October 16, 2026: contributor


🚧 At least one of the CI tasks failed. Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/38833479928/job/116757258223 LLM reason (✨ experimental): CI failed because IWYU modified includes in src/bench/readwriteblock.cpp, and the formatting check detected uncommitted changes.

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:
- Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
- A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
- An intermittent issue.
Leave a comment here, if you need help tracking down a confusing failure.

</details>

node: strip block witnesses in place

Reuse the owned input buffer and move retained ranges only after a removed witness gap.
Each destination is at or before its source, so `memmove` leaves unread bytes intact.
Erase the unused tail after the walk.

The benchmark falls from 0.161 to 0.147 ms/block for witness-free data and from 0.384 to 0.374 ms/block for witness-bearing data.

0b01ce05e1

l0rinc force-pushed on Oct 16, 2026
fametrano referenced this in commit 906e8f75dd on Oct 17, 2026
fametrano referenced this in commit 2108530cd4 on Oct 17, 2026
fametrano referenced this in commit 39348d4034 on Oct 17, 2026
l0rinc closed this on Oct 18, 2026
l0rinc reopened this on Oct 18, 2026
DrahtBot removed the label CI failed on Oct 18, 2026
pmazzocchi referenced this in commit ca94fe44d6 on Oct 18, 2026

Contributors

l0rinc

DrahtBot

Labels

P2P

Linked (view graph)

#13011 Cache witness hash in CTransaction #13098 Skip tx-rehashing on historic blocks #13151 net: Serve blocks directly from disk when possible #31551 [IBD] batch block reads/writes during `AutoFile` serialization #36476 pruneassumevalid: avoid downloading witnesses and writing blocks/undos during pruned assumevalid IBD

net: serve `MSG_BLOCK` by stripping witnesses in place #36548

External sites

Reviews

Conflicts

LLM Linter (✨ experimental)