kernel: add prune locks and target pruning #36857

pull pzafonte wants to merge 16 commits into bitcoin:master from pzafonte:kernel-prune-locks-and-target changing 18 files +1388 −356
  1. pzafonte commented at 3:37 PM on November 13, 2026: contributor

    This adds more of the pruning API discussed in #35627:

    • btck_block_manager_update_prune_lock and btck_block_manager_delete_prune_lock keep blocks at and above a height until the application deletes the lock. Like the node's prune locks, they are held in memory, so an application sets them again after a restart before it prunes.
    • btck_block_manager_prune_to_target deletes the oldest block files until usage, with room for the next allocation, is below a target. It keeps the same blocks as btck_block_manager_prune_up_to_height. Unlike -prune, the target has no minimum, and pruning doesn't wait for the chain to reach a minimum height.

    Target pruning uses BeginManualPrune. BeginTargetPrune only proceeds after a new block file is allocated, and the periodic flush during block processing already consumes that, so it would never prune.

    Names the buffer FindFilesToPrune keeps below a target. The kernel uses it to tell whether a flush would help.

    Based on #36856. Draft until it is merged.

  2. test: apply BlockManager args in test fixtures
    Without this, -prune and -fastprune in extra_args have no effect.
    24ad36d929
  3. test: check flush retry after failure
    Move the blk file aside instead of deleting it, so it can be restored.
    2334d27a24
  4. test: characterize a failed prune flush
    The pruned blk/rev files are not deleted, even after a later flush
    succeeds.
    108d7ff03c
  5. validation: delete pruned files on a later flush
    Files selected for pruning were kept in a local, so a failed flush left
    them on disk until the next startup.
    
    Keep them in BlockManager and delete them after the next block index
    write. Take them with the dirty entries, which WriteBlockIndexDB() drops
    even if the write fails. Drop a file from the set once a block is
    written to it.
    
    Pending files wait for the next periodic, forced or shutdown flush
    rather than forcing one, which would retry a failing write on every
    block.
    d224fb8721
  6. refactor: let BlockManager write block storage
    Move the block and undo file flush, block index write and pruned file
    deletion from FlushStateToDisk into BlockManager. Their bench log lines
    change from FlushStateToDisk to WriteBlockStorage.
    fb911aecde
  7. refactor: apply prune locks in BlockManager
    Validation read BlockManager's prune locks directly. Move that code and
    PRUNE_LOCK_BUFFER into BlockManager.
    14de6dfdf5
  8. refactor: pass prune limits to BlockManager
    BlockManager read the chainstate to decide what to prune. Decide in
    validation and pass the height range and sizes instead. An Assume checks
    the heights are non-negative.
    ae7d92d02e
  9. refactor: move prune bookkeeping into BlockManager
    Move FlushStateToDisk's handling of the prune request, prune locks,
    pruned flag, files to delete and blocks directory space check into
    BlockManager. The prune request is cleared before the prune locks are
    applied rather than after selection, which differs only if selection
    throws.
    
    An Assume checks that the pruned flag is set before files are queued
    for deletion, so a restart knows blocks were deleted.
    777b2f90de
  10. kernel: destroy chainstate manager first
    ChainMan declares m_context last, which destroys it before the
    ChainstateManager that refers into it. Reset the chainstate manager
    explicitly, making the order independent of member declaration order.
    4f9f742b3e
  11. refactor: pass BlockManager to ChainstateManager
    Create the BlockManager first and pass it in by reference. NodeContext
    and the kernel own both and destroy ChainstateManager first.
    
    The block tree database opens before the script verification threads
    start, which swaps their startup log lines.
    462dcf9966
  12. refactor: remove ChainstateLoadOptions::prune
    The flag duplicated the BlockManager's prune mode. Init and the test
    setup copied it, and the kernel left it false. Check the BlockManager
    directly, keeping one source of truth before the kernel can enable
    pruning.
    df1007a5c2
  13. kernel: let callers create the block manager
    Block storage was configured and read through the chainstate manager.
    Add btck_BlockManagerOptions and btck_BlockManager, and pass the block
    manager to btck_chainstate_manager_create, which keeps a shared
    reference, so either can be destroyed first. A block manager serves
    one chainstate manager.
    
    btck_block_read and btck_block_spent_outputs_read take the block
    manager. Each options object takes the total database cache and uses
    its part. Wiping the block tree db also wipes the chainstate db.
    7f70fed152
  14. kernel: add manual pruning
    Add a block manager prune option and
    btck_block_manager_prune_up_to_height. The option also lets the kernel
    open data directories Bitcoin Core has pruned. The chainstate manager
    created with the block manager decides what may be deleted, so pruning
    fails without one.
    
    Pruning writes only block storage. It keeps blocks above the last
    chainstate flush, which the chainstate on disk may still need after an
    unclean shutdown, and reports when it did. Add
    btck_chainstate_manager_flush to flush before pruning again.
    
    Add a fast prune option for tests, since test chains never fill a
    128 MiB block file.
    2e21a991cb
  15. kernel: add prune locks
    An application that reads blocks after they are connected needs them
    kept until it has read them. Add block manager functions to set and
    delete prune locks.
    37e40a00e9
  16. refactor: name the prune target buffer
    The kernel needs the room FindFilesToPrune keeps below a prune target.
    907e974e73
  17. kernel: add target pruning
    An application that wants block storage kept below a size would have
    to pick prune heights itself. Add btck_block_manager_prune_to_target,
    which deletes the oldest blocks until usage is below a target and keeps
    the same blocks as btck_block_manager_prune_up_to_height. The kernel
    never prunes on its own, so the caller decides when.
    
    Target pruning uses BeginManualPrune. BeginTargetPrune only proceeds
    after a new block file is allocated, and the periodic flush in block
    processing already consumes that.
    3a95f31c1e
  18. DrahtBot commented at 3:38 PM on November 13, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process. A summary of reviews will appear here.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

    LLM Linter (✨ experimental)

    Possible places where comparison-specific test macros should replace generic comparisons:

    • src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(BlockManagerOptions(context, data_dir, blocks_dir), std::runtime_error) → Prefer BOOST_CHECK_EXCEPTION with a HasReason predicate checking the expected message.
    • src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(ChainstateManagerOptions(context, ""), std::runtime_error) and BOOST_CHECK_THROW(ChainstateManagerOptions(context, {nullptr, 0}), std::runtime_error) → Prefer BOOST_CHECK_EXCEPTION with predicates checking the expected messages.
    • src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(ChainMan(context, chainman_opts, blockman), std::runtime_error) (context mismatch and reused block manager) → Prefer BOOST_CHECK_EXCEPTION with predicates checking the respective expected messages.
    • src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(blockman.ReadBlockSpentOutputs(chain.GetByHeight(1)), std::runtime_error) (both occurrences) → Prefer BOOST_CHECK_EXCEPTION with a predicate checking the expected message.
    • src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(create_chainman(...), std::runtime_error) → Prefer BOOST_CHECK_EXCEPTION with a predicate checking the expected message.
    • src/test/kernel/test_kernel.cpp: BOOST_CHECK_THROW(blockman.ReadBlockSpentOutputs(tip), std::runtime_error) → Prefer BOOST_CHECK_EXCEPTION with a predicate checking the expected message.

    <sup>2026-11-13 15:38:38</sup>

  19. pzafonte renamed this:
    Kernel prune locks and target
    kernel: add prune locks and target pruning
    on Nov 13, 2026
  20. DrahtBot added the label Validation on Nov 13, 2026
  21. DrahtBot added the label Needs rebase on Nov 13, 2026
  22. DrahtBot commented at 5:19 PM on November 13, 2026: contributor

    <!--cf906140f33d8803c4a75a2196329ecb-->

    🐙 This pull request conflicts with the target branch and needs rebase.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-14 11:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me