node: batch block file pruning in a single pass over block index #36621

mutatrum commented at 11:39 AM on October 23, 2026: contributor

When starting a node with pruning enabled (e.g. -prune=550) on a node with full chain history or pruning many files at once, FindFilesToPrune and FindFilesToPruneManual previously called PruneOneBlockFile once per file.

Because PruneOneBlockFile iterated over the entire m_block_index for each file, this was an O(M * N) operation (M = files to prune, N = total blocks). When pruning a full chain (~5,770 files and ~972k blocks), this triggered ~5.6 billion iterations and froze the node running on a Pi 5 for ~10 minutes on startup while holding cs_main.

This PR replaces PruneOneBlockFile with PruneBlockFiles(const std::set<int>&), which collects all candidate files and prunes them in a single O(M + N) pass over m_block_index using a compact dynamic bitfield (std::vector<bool>). Also skips entries without block or undo data to avoid unneeded checks and prevent file 0 pruning from dirtying header-only or already-pruned index entries. The new signature is analogous to UnlinkPrunedFiles({file_number});.

Benchmarked with ~972k blocks and 5,770 files to prune:

x86_64 (i7-8665U @ 1.90 GHz): 75.1s -> 14.7ms (~5,100x speedup)
ARM64 (Raspberry Pi 5): 606.0s -> 26.3ms (>3,500x speedup)

Add unit test blockmanager_prune_block_files_batch to verify batch pruning, empty set no-ops, and clean dirty-set behavior. Update caller tests in blockmanager_tests and rescan_tests.

DrahtBot commented at 11:40 AM on October 23, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

External sites

Code coverage & benchmarks: https://corecheck.dev/bitcoin/bitcoin/pulls/36621
Ralph review <img src=https://review.fish.foo/assets/bitcoin/bitcoin/36621.png alt="Ralph review status"> https://git.fish.foo/bitcoin/bitcoin/pulls/36621#author-ralph

Reviews

See the guideline and AI policy for information on the review process. A summary of reviews will appear here.

Conflicts

No conflicts as of last run.

mutatrum force-pushed on Oct 23, 2026

DrahtBot added the label CI failed on Oct 23, 2026

DrahtBot commented at 12:41 PM on October 23, 2026: contributor

🚧 At least one of the CI tasks failed. <sub>Task macOS-cross to x86_64: https://github.com/bitcoin/bitcoin/actions/runs/39947671739/job/120394208316</sub> <sub>LLM reason (✨ experimental): CI failed compiling blockmanager_tests.cpp: Clang’s thread-safety checks report accesses to nFile and nStatus without holding cs_main, and -Werror treats them as errors.</sub>

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

node: batch block file pruning in a single pass over block index

When starting the node with pruning enabled (e.g. -prune=550) on a
node with full chain history or pruning many files at once,
FindFilesToPrune and FindFilesToPruneManual previously called
PruneOneBlockFile once per file.

Because PruneOneBlockFile iterated over the entire m_block_index for
each file, this was an O(M * N) operation (M = files to prune,
N = total blocks). When pruning a full chain (~5,770 files and ~972k
blocks), this triggered ~5.6 billion iterations and froze the node
for ~10 minutes on startup while holding cs_main.

Replace PruneOneBlockFile with PruneBlockFiles(const std::set<int>&),
which collects all candidate files and prunes them in a single O(M + N)
pass over m_block_index using a compact dynamic bitfield
(std::vector<bool>). Also skip entries without block or undo data to
avoid unneeded checks and prevent file 0 pruning from dirtying
header-only or already-pruned index entries.

Benchmarked with ~972k blocks and 5,770 files to prune:
- x86_64 (i7-8665U @ 1.90 GHz): 75.1s -> 14.7ms (~5,100x speedup)
- ARM64 (Raspberry Pi 5): 606.0s -> 26.3ms (>3,500x speedup)

Add unit test blockmanager_prune_block_files_batch to verify batch
pruning, empty set no-ops, and clean dirty-set behavior. Update
caller tests in blockmanager_tests and rescan_tests.

b96e669c60

mutatrum force-pushed on Oct 23, 2026

DrahtBot removed the label CI failed on Oct 23, 2026