[IBD] specialize block serialization #31868

This change is part of [IBD] - Tracking PR for speeding up Initial Block Download

This PR is drafted until I remeasure everything after the recent merges and I need to find a way to simplify the 1 byte writes more nicely, I don't like all the specializations.

Summary: This PR contains a few different optimizations found by IBD profiling and via the newly added block serialization benchmarks. It also takes advantage of the recently merged std::span changes enabling propagating static extents.

The commits merge similar (de)serialization methods and separate them internally with if constexpr, similarly to how it has been done here before. This enables further SizeComputer optimizations as well.

Context: Since single byte writes are used very often (for every (u)int8_t, std::byte, or bool, and for every VarInt's first byte, which is also needed for every (pre)Vector), it makes sense to avoid the generalized serialization infrastructure that isn't needed:

• AutoFile write doesn't need to allocate a 4k buffer for a single byte now;
• VectorWriter and DataStream avoid memcpy/insert calls;
• CSHA256::Write can avoid memcpy.

DeserializeBlock is dominated by hash calculations, so the optimizations barely affect it.

Measurements:

<details> <summary>C compiler ............................ AppleClang 16.0.0.16000026</summary>

Before: | ns/block | block/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 195,610.62 | 5,112.20 | 0.3% | 11.00 | SerializeBlock | 12,061.83 | 82,906.19 | 0.1% | 11.01 | SizeComputerBlock After: | ns/block | block/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 174,569.19 | 5,728.39 | 0.6% | 10.89 | SerializeBlock | 10,241.16 | 97,645.21 | 0.0% | 11.00 | SizeComputerBlock

</details>

SerializeBlock - ~12.% faster SizeComputerBlock - ~17.7% faster

<details> <summary>C++ compiler .......................... GNU 13.3.0</summary>

Before: | ns/block | block/s | err% | ins/block | cyc/block | IPC | bra/block | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 867,857.55 | 1,152.26 | 0.0% | 8,015,883.90 | 3,116,099.08 | 2.572 | 1,517,035.87 | 0.5% | 10.81 | SerializeBlock | 30,928.27 | 32,332.88 | 0.0% | 221,683.03 | 111,055.84 | 1.996 | 53,037.03 | 0.8% | 11.03 | SizeComputerBlock After: | ns/block | block/s | err% | ins/block | cyc/block | IPC | bra/block | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 615,000.56 | 1,626.01 | 0.0% | 8,015,883.64 | 2,208,340.88 | 3.630 | 1,517,035.62 | 0.5% | 10.56 | SerializeBlock | 25,676.76 | 38,945.72 | 0.0% | 159,390.03 | 92,202.10 | 1.729 | 42,131.03 | 0.9% | 11.00 | SizeComputerBlock

</details>

SerializeBlock - ~41.1% faster SizeComputerBlock - ~20.4% faster

While this wasn't the main motivation for the change, IBD on Ubuntu/GCC on SSD with i9 indicates a 2% speedup as well:

<details> <summary>Details</summary> ```bash COMMITS="05314bde0b06b820225f10c6529b5afae128ff81 1cd94ec2511874ec68b92db34ad7ec7d9534fed1"; \ STOP_HEIGHT=880000; DBCACHE=10000; \ C_COMPILER=gcc; CXX_COMPILER=g++; \ hyperfine \ --export-json "/mnt/my_storage/ibd-${COMMITS// /-}-${STOP_HEIGHT}-${DBCACHE}-${C_COMPILER}.json" \ --runs 3 \ --parameter-list COMMIT ${COMMITS// /,} \ --prepare "killall bitcoind || true; rm -rf /mnt/my_storage/BitcoinData/*; git checkout {COMMIT}; git clean -fxd; git reset --hard; cmake -B build -DCMAKE_BUILD_TYPE=Release -DENABLE_WALLET=OFF -DCMAKE_C_COMPILER=$C_COMPILER -DCMAKE_CXX_COMPILER=$CXX_COMPILER && cmake --build build -j$(nproc) --target bitcoind && ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=1 -printtoconsole=0 || true" \ --cleanup "cp /mnt/my_storage/BitcoinData/debug.log /mnt/my_storage/logs/debug-{COMMIT}-$(date +%s).log || true" \ "COMPILER=$C_COMPILER COMMIT={COMMIT} ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=$STOP_HEIGHT -dbcache=$DBCACHE -prune=550 -printtoconsole=0" Benchmark 1: COMPILER=gcc COMMIT=05314bde0b06b820225f10c6529b5afae128ff81 ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=10000 -prune=550 -printtoconsole=0 Time (mean ± σ): 33647.918 s ± 508.655 s [User: 71503.409 s, System: 4404.899 s] Range (min … max): 33283.439 s … 34229.026 s 3 runs

Benchmark 2: COMPILER=gcc COMMIT=1cd94ec2511874ec68b92db34ad7ec7d9534fed1 ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=10000 -prune=550 -printtoconsole=0 Time (mean ± σ): 33062.491 s ± 183.335 s [User: 71246.532 s, System: 4318.490 s] Range (min … max): 32888.211 s … 33253.706 s 3 runs

Summary COMPILER=gcc COMMIT=1cd94ec2511874ec68b92db34ad7ec7d9534fed1 ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=10000 -prune=550 -printtoconsole=0 ran 1.02 ± 0.02 times faster than COMPILER=gcc COMMIT=05314bde0b06b820225f10c6529b5afae128ff81 ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=10000 -prune=550 -printtoconsole=0

</details>

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31868.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #34436 (refactor: add overflow-safe CeilDiv helper and use it in unsigned callsites by l0rinc)
• #34208 (bench: add fluent API for untimed setup steps in nanobench by l0rinc)
• #32729 (test,refactor: extract script template helpers & widen sigop count coverage by l0rinc)
• #31682 ([IBD] specialize CheckBlock's input & coinbase checks by l0rinc)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

LLM Linter (✨ experimental)

Possible places where named args for integral literals may be used (e.g. func(x, /*named_arg=*/0) in C++, and func(x, named_arg=0) in Python):

• Transform(s, buf, 1) in src/crypto/sha256.cpp

^2026-01-15

Drafting until #31519 is merged, as recommended in #31868 (review)

<!--85328a0da195eb286784d51f73fa0af9-->

🚧 At least one of the CI tasks failed. _{Debug: https://github.com/bitcoin/bitcoin/runs/38458137649}

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

This looks good, but I am not really reproducing any of the performance changes on my machine yet. Maybe my laptop is just too unreliable though.

but I am not really reproducing any of the performance changes on my machine yet

I just removed the single-byte specializations, they were quite ugly, will have to add it back somehow later - will draft it until I do that, thanks for checking!

@TheCharlatan, I've added back the single-byte optimizations (simplified slightly after rebase), your feedback is appreciated.

This looks ok, albeit that I'm seeing more modest performance improvements on my end.

While this wasn't the main motivation for the change, IBD on Ubuntu/GCC on SSD with i9 indicates a 2% speedup as well:

If IBD speedup is not the main motivation, what else is the motivation, given that the title mentions [IBD]?

Also, what code part is hitting this during IBD, given that reads and writes are now buffered?

what code part is hitting this during IBD

During serialization we have very many 1 byte writes, this change avoids the heavy allocations. But drafting until I have time to properly remeasure everything after the recent merge.

I understand that there are many 1-byte writes, but they go into the BufferedReader/Writer, which is not modified here. The Buffered* then passes them to AutoFile, which is modified here, but never receives 1-byte writes anymore

If IBD speedup is not the main motivation, what else is the motivation, given that the title mentions [IBD]?

I think block serialization being fast could be useful for future kernel users. Reading and de-serializing blocks is one of the common operations I'd expect outside of just pure block validation.

If IBD speedup is not the main motivation, what else is the motivation, given that the title mentions [IBD]?

I think block serialization being fast could be useful for future kernel users. Reading and de-serializing blocks is one of the common operations I'd expect outside of just pure block validation.

Ok, seems fine. It would be good to list the main motivation(s) in the OP. Currently, it only says IBD and that IBD isn't the main motivation. However, I don't understand how this could have any effect at all, from looking at the code, see #31868 (comment).

I will work on this more in the upcoming weeks, drafted it for now since I don't like the current single-byte specializations, have to find a better way. Suggestions are welcome.

My guess is that the concept specialization here at most could have an effect on xor'd IO? This is because anything not xor'd won't need a buffer to (un)do the xor before reading or writing.

Thus, it just seems easier to use the existing BufferedReader/Writer in places that could benefit from it?

However, for block serialization they are already used.

(Edit: I misunderstood that this has benefits outside of BufferedReader. Please ignore this comment)

Again, no objection here, but it would be good to explain:

• What is the goal of the changes here, and what real-world usage do they affect
• How is it achieved and how do the benchmarks represent the real-world usage

Right now:

• It only states IBD as motivation and also that IBD isn't the motivation. Also the IBD speedup measurement seems outdated.
• The added benchmark is about test-only code paths (DataStream) or p2p code paths (VectorWriter). It is unclear what the AutoFile changes are about. Also, it is unclear how this could possibly affect the kernel, given that the kernel doesn't call into test-only or p2p-only code paths and instead uses BufferedReader/Writer.

Rebased to solve conflict, the PR is still in draft mode until I remeasure its effect on IBD - and I need to find a nicer way to do the single-byte specializations, since they have a measurable effect but make the code super-ugly... Suggestions are welcome. In the meantime I have split out the first refactor into #33093.

Are you still working on this?

Of course, paused it since I'm not exactly sure how do handle single-byte entries: they measurably speed up the scenarios, but the resulting code is quite ugly. Do you have a suggestion? Should I attempt that in a separate PR and ignore it here?

Edit: draft because of the requests in #31868 (comment) which I will need to remeasure

What was the speedup? I can only see your last comment, which says:

the PR is still in draft mode until I remeasure its effect on IBD

<!--cf906140f33d8803c4a75a2196329ecb-->

🐙 This pull request conflicts with the target branch and needs rebase.