kernel: add manual pruning #36856

pull pzafonte wants to merge 13 commits into bitcoin:master from pzafonte:kernel-prune-up-to-height changing 18 files +1215 −355

pzafonte commented at 3:25 PM on November 13, 2026: contributor

Part of #35627 and kernel-node/kernel-node#111. The kernel can't open a data directory that a pruned node created, and it has no way to prune.

This adds a prune option on the block manager, btck_block_manager_prune_up_to_height and btck_chainstate_manager_flush. With the option set, the kernel opens pruned data directories. The chainstate manager created with the block manager decides what may be deleted, so pruning fails without one.

Pruning writes only block storage, never the chainstate. After an unclean shutdown the chainstate on disk is at the last flush, and recovering from there needs the blocks above it. Pruning therefore keeps every block above the fork point between the last flushed block and the active chain, and reports btck_PruneStatus_NEEDS_FLUSH when that bound limited what it could delete. The caller can flush and prune again.

The fork point matters when the flushed block was reorged out. Recovery then disconnects the stale blocks, which needs their block and undo data, and connects the new ones from their block data. btck_block_manager_prune_reorg_tests covers that case and fails if the bound uses the flushed block's height instead.

A fast prune option lets tests prune, since test chains never fill a 128 MiB block file.

Based on #36738 and #36854. Draft until both are merged.

test: apply BlockManager args in test fixtures

Without this, -prune and -fastprune in extra_args have no effect.

24ad36d929

test: check flush retry after failure

Move the blk file aside instead of deleting it, so it can be restored.

2334d27a24

test: characterize a failed prune flush

The pruned blk/rev files are not deleted, even after a later flush
succeeds.

108d7ff03c

validation: delete pruned files on a later flush

Files selected for pruning were kept in a local, so a failed flush left
them on disk until the next startup.

Keep them in BlockManager and delete them after the next block index
write. Take them with the dirty entries, which WriteBlockIndexDB() drops
even if the write fails. Drop a file from the set once a block is
written to it.

Pending files wait for the next periodic, forced or shutdown flush
rather than forcing one, which would retry a failing write on every
block.

d224fb8721

refactor: let BlockManager write block storage

Move the block and undo file flush, block index write and pruned file
deletion from FlushStateToDisk into BlockManager. Their bench log lines
change from FlushStateToDisk to WriteBlockStorage.

fb911aecde

refactor: apply prune locks in BlockManager

Validation read BlockManager's prune locks directly. Move that code and
PRUNE_LOCK_BUFFER into BlockManager.

14de6dfdf5

refactor: pass prune limits to BlockManager

BlockManager read the chainstate to decide what to prune. Decide in
validation and pass the height range and sizes instead. An Assume checks
the heights are non-negative.

ae7d92d02e

refactor: move prune bookkeeping into BlockManager

Move FlushStateToDisk's handling of the prune request, prune locks,
pruned flag, files to delete and blocks directory space check into
BlockManager. The prune request is cleared before the prune locks are
applied rather than after selection, which differs only if selection
throws.

An Assume checks that the pruned flag is set before files are queued
for deletion, so a restart knows blocks were deleted.

777b2f90de

kernel: destroy chainstate manager first

ChainMan declares m_context last, which destroys it before the
ChainstateManager that refers into it. Reset the chainstate manager
explicitly, making the order independent of member declaration order.

4f9f742b3e

refactor: pass BlockManager to ChainstateManager

Create the BlockManager first and pass it in by reference. NodeContext
and the kernel own both and destroy ChainstateManager first.

The block tree database opens before the script verification threads
start, which swaps their startup log lines.

462dcf9966

refactor: remove ChainstateLoadOptions::prune

The flag duplicated the BlockManager's prune mode. Init and the test
setup copied it, and the kernel left it false. Check the BlockManager
directly, keeping one source of truth before the kernel can enable
pruning.

df1007a5c2

kernel: let callers create the block manager

Block storage was configured and read through the chainstate manager.
Add btck_BlockManagerOptions and btck_BlockManager, and pass the block
manager to btck_chainstate_manager_create, which keeps a shared
reference, so either can be destroyed first. A block manager serves
one chainstate manager.

btck_block_read and btck_block_spent_outputs_read take the block
manager. Each options object takes the total database cache and uses
its part. Wiping the block tree db also wipes the chainstate db.

7f70fed152

kernel: add manual pruning

Add a block manager prune option and
btck_block_manager_prune_up_to_height. The option also lets the kernel
open data directories Bitcoin Core has pruned. The chainstate manager
created with the block manager decides what may be deleted, so pruning
fails without one.

Pruning writes only block storage. It keeps blocks above the last
chainstate flush, which the chainstate on disk may still need after an
unclean shutdown, and reports when it did. Add
btck_chainstate_manager_flush to flush before pruning again.

Add a fast prune option for tests, since test chains never fill a
128 MiB block file.

2e21a991cb

DrahtBot commented at 3:25 PM on November 13, 2026: contributor


The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.



External sites
- Code coverage & benchmarks: https://corecheck.dev/bitcoin/bitcoin/pulls/36856
- Ralph review <img src=https://review.fish.foo/assets/bitcoin/bitcoin/36856.png alt="Ralph review status"> https://git.fish.foo/bitcoin/bitcoin/pulls/36856#author-ralph


Reviews

See the guideline and AI policy for information on the review process. A summary of reviews will appear here.



LLM Linter (✨ experimental)

Possible places where comparison-specific test macros should replace generic comparisons:
- src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(BlockManagerOptions(context, data_dir, blocks_dir), std::runtime_error) → Prefer BOOST_CHECK_EXCEPTION(..., std::runtime_error, HasReason{"..."}) to verify the failure reason.
- src/test/kernel/test_kernel.cpp: Both BOOST_CHECK_THROW checks for empty ChainstateManagerOptions paths → Prefer BOOST_CHECK_EXCEPTION(..., std::runtime_error, HasReason{"..."}).
- src/test/kernel/test_kernel.cpp: The BOOST_CHECK_THROW checks for a different context and for reusing a block manager → Prefer BOOST_CHECK_EXCEPTION(..., std::runtime_error, HasReason{"..."}) to check the specific failure message.
- src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(create_chainman(...), std::runtime_error) when opening a pruned directory without pruning enabled → Prefer BOOST_CHECK_EXCEPTION(..., std::runtime_error, HasReason{"..."}).
- src/test/kernel/test_kernel.cpp: The two BOOST_CHECK_THROW(blockman.ReadBlockSpentOutputs(...), std::runtime_error) checks after spent-output data is unavailable → Prefer BOOST_CHECK_EXCEPTION(..., std::runtime_error, HasReason{"..."}).
2026-11-13 15:25:50
pzafonte renamed this:
~~Kernel prune up to height~~
kernel: add manual pruning
on Nov 13, 2026
DrahtBot added the label Validation on Nov 13, 2026
DrahtBot added the label CI failed on Nov 13, 2026
DrahtBot commented at 4:33 PM on November 13, 2026: contributor


🚧 At least one of the CI tasks failed. Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/43452418783/job/131977732637 LLM reason (✨ experimental): CI failed because IWYU added missing includes in src/kernel/bitcoinkernel.cpp, leaving an uncommitted diff that the lint check rejects.

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:
- Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
- A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
- An intermittent issue.
Leave a comment here, if you need help tracking down a confusing failure.

</details>

Contributors

pzafonte

DrahtBot

Labels

Validation CI failed

Linked (view graph)

#35627 kernel: can't work with pruned blocksdir #36738 refactor: move block writes and pruning into BlockManager #36739 refactor: pass BlockManager to ChainstateManager #36854 kernel: let callers create the block manager #36857 kernel: add prune locks and target pruning

kernel: add manual pruning #36856

External sites

Reviews

LLM Linter (✨ experimental)