[RFC] BlockMap and CChain Concurrency Improvement #34654

pull alexanderwiederin wants to merge 12 commits into bitcoin:master from alexanderwiederin:blockmap-chain-concurrency changing 38 files +1123 −317
  1. alexanderwiederin commented at 9:58 am on February 23, 2026: none

    RFC: BlockMap and CChain Concurrency Improvement

    Follow-up to #34424.

    This RFC applies a stable and recent + copy-on-write design to BlockMap, adds an internal Mutex to CChain and demonstrates how cs_main locks can be removed.

    Motivation

    The current BlockMap and CChain are protected by cs_main, creating bottlenecks for multi-threaded access.

    BlockMap Design

    TL;DR: Split the block index into stable (bulk of history) + recent (~1,000 blocks). Use nested stlab::copy_on_write so updates copy only recent, keeping stable shared.

    Changes

    1. Stores block hash directly in CBlockIndex (required due to loss of address stability)
    2. Implements copy-on-write BlockMap with stable + recent architecture
    3. Adds internal mutex to CChain
    4. Removes cs_main locks across multiple call sites, using getchaintips as an illustrative example

    Note: the lock removals in this RFC are non-exhaustive

    Request for Comments

    If you have opinions on the following design decisions, please weigh in:

    • Storing block hashes directly in CBlockIndex is required due to loss of address stability from the copy-on-write design, is the memory cost an acceptable tradeoff?
    • Are there simpler alternatives to copy-on-write here? A plain mutex might be sufficient, but full BlockMap iteration would hold the lock for extended periods.
  2. DrahtBot commented at 9:58 am on February 23, 2026: contributor

    ♻️ Automatically closing for now based on heuristics. Please leave a comment, if this was erroneous. Generally, please focus on creating high-quality, original content that demonstrates a clear understanding of the project’s requirements and goals.

    📝 Moderators: If this is spam, please replace the title with ., so that the thread does not appear in search results.

  3. DrahtBot closed this on Feb 23, 2026

  4. DrahtBot commented at 9:59 am on February 23, 2026: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #34489 (index: batch db writes during initial sync by furszy)
    • #34451 (rpc: fix race condition in gettxoutsetinfo by w0xlt)
    • #34440 (Refactor CChain methods to use references, tests by optout21)
    • #34416 (Add nullptr-check to CChain::Contains(), tests by optout21)
    • #33752 (rest: Query predecessor headers using negative count param by A-Manning)
    • #33477 (Rollback for dumptxoutset without invalidating blocks by fjahr)
    • #32554 (bench: replace embedded raw block with configurable block generator by l0rinc)
    • #32317 (kernel: Separate UTXO set access from validation functions by sedited)
    • #28690 (build: Introduce internal kernel library by sedited)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • exisitng -> existing [misspelling in comment “If block already exists, returns (pointer to exisitng, false).”]
    • Chec -> Check [misspelling in comment “Chec if block exists in the map.”]

    2026-03-07 16:51:13

  5. alexanderwiederin commented at 10:05 am on February 23, 2026: none
    This was closed in error.
  6. maflcko reopened this on Feb 23, 2026

  7. DrahtBot added the label CI failed on Feb 23, 2026
  8. alexanderwiederin force-pushed on Feb 23, 2026
  9. alexanderwiederin renamed this:
    [RFC] BlockMap and CChain Concurrency Improvement
    [RFC] `BlockMap` and `CChain` Concurrency Improvement
    on Feb 23, 2026
  10. DrahtBot removed the label CI failed on Feb 23, 2026
  11. alexanderwiederin force-pushed on Mar 4, 2026
  12. alexanderwiederin force-pushed on Mar 4, 2026
  13. DrahtBot added the label CI failed on Mar 4, 2026
  14. ?
    added_to_project_v2 sedited
  15. ?
    project_v2_item_status_changed github-project-automation[bot]
  16. util: add copy-on-write smart pointer
    Introduce copy-on-write wrapper from stlab library to enable efficient
    value semantics with shared ownership. Uses atomic reference counting to
    share instances and only copies on modification when non-unique.
    
    Based on Adobe's stlab library, distributed under Boost Software License
    1.0
    
    Original source: https://github.com/stlab/stlab-copy-on-write/tree/abb4445
    
    Co-authored-by: janb84 <608446+janb84@users.noreply.github.com>
    da39e4e89b
  17. chain: store block hash directly in CBlockIndex
    replace the external pointer `phashBlock` with an owned `m_block_hash`
    member in CBlockIndex. Previously the hash was stored in the BlockMap
    key and CBlockIndex held a pointer to it, creating a lifetime dependency
    on the map's stable addressing. Storing the hash directly makes
    CBlockIndex self-contained, which is a prerequisite for moving to a
    copy-on-write BlockMap with shared_ptr<CBlockIndex>.
    
    Add a new constructor CBlockIndex(const uint256& hash) and delegate the
    existing CBlockHeader constructor to it. Remove all phashBlock
    assignemnts and replace with m_block_hash.
    ebfa877640
  18. blockindex: introduce copy-on-write BlockMap
    Replace the previous BlockMap (unordered_map<uint256, CBlockIndex>) with
    a new copy-on-write BlockMap that stores shared_pointer<CBlockIndex>.
    This is a prefactor for lock-free block index reads.
    
    The new BlockMap uses a two-tier structure (stable + recent) backed by
    stlab::copy-on-write. New entries are inserted into a small recent map
    and promoted to the stable COW map when the recent map exceeds a
    threshold. Copies of the BlockMap are cheap (refcount bump) and produce
    consistent point-in-time snapshots.
    
    Call sites are updated from value semantics (&senty.second) to pointer
    semantica (entry.second).
    
    No locking changes in this commit; cs_main guards are preserved as-is.
    d7a63e54b5
  19. blockmanager: remove cs_main from block index access
    LookupBlockIndex and GetAllBlockIndices now take a snapshot copy instead
    of requiring cs_main.
    
    Add m_block_index_mutex to serialize writes in AddToBlockIndex and
    InsertBlockIndex. Split AddToBlockIndex into separate lock scopes so
    m_block_index_mutex and cs_main are not held simultaneously.
    8f42c7196e
  20. refactor: remove cs_main locks from some LookupBlockIndex call sites
    Remove cs_main locks that are no longer needed when calling
    BlockManager::LookupBlockIndex.
    
    Changes:
    - Remove cs_main locks from index/base.cpp, kernel/bitcoinkernel.cpp,
      net_processing.cpp, node/interfaces.cpp, node/miner.cpp,
      rpc/gettxoutproof.cpp, rpc/blockchain.cpp and rest.cpp
    - Remove unused validation.h and sync.h includes from
      kernel/coinstats.cpp
    - Update expected circular dependency path for kernel/coinstats due to
      removal of validation.h include
    aef61cf565
  21. alexanderwiederin force-pushed on Mar 5, 2026
  22. refactor: add thread safety to CChain with internal mutex
    Introduce a mutex to protect CChain's internal vector from concurrent
    access, eliminating the need for external locking with cs_main.
    
    Changes:
    - Add m_mutex member to CChain class and mark vChain as
      GUARDED_BY(m_mutex)
    - Add LOCK(m_mutex) to all public methods accessing vChain
    - Refactor Contains() and Next() to avoid calling operator[] (which
      would acquire the lock twice) by directly acessing vChain under a
      single lock
    - Update FindFork() to use local height variable and inline the
      Contains() check to avoid nested locking
    02d9a7609c
  23. refactor: remove cs_main lock requirements from CChain accessor methods
    Remove EXCLUSIVE_LOCKS_REQURIED annotations from ActiveChain(),
    ActiveHeight() and ActiveTip() methods, as CChain now handles its own
    thread safety internally with m_mutex.
    
    This change allows callers to access the active chain without holding
    cs_main.
    7f02912254
  24. refactor: remove cs_main lock requirements from CChain call sites
    Now that CChain handles its own thread safety internally via m_mutex,
    callers no longer need to hold cs_main when accessing chain data.
    Removes the redundant LOCK(cs_main) and WITH_LCOK(cs_main) annotations
    from call sites across RPC, indexes, node interfaces, REST and tests.
    f6cb486983
  25. validation: add BlockIndexSnapshot method ae3ba076f6
  26. refactor: remove cs_main lock from getchaintips RPC
    Replace cs_main lock with targeted locking and use BlockIndexSnapshot()
    to iterate over the block index.
    b6aa626d53
  27. refactor: remove cs_main lock requirement from NextSyncBlock 544d0cad8e
  28. refactor: remove cs_main locks from CChain and BlockIndex accessor call sites 5849975570
  29. alexanderwiederin force-pushed on Mar 7, 2026
  30. DrahtBot removed the label CI failed on Mar 7, 2026
  31. DrahtBot added the label Needs rebase on Mar 19, 2026
  32. DrahtBot commented at 9:19 am on March 19, 2026: contributor
    🐙 This pull request conflicts with the target branch and needs rebase.
  33. alexanderwiederin closed this on Mar 19, 2026

  34. ?
    project_v2_item_status_changed github-project-automation[bot]
  35. ismaelsadeeq commented at 3:55 pm on March 19, 2026: member

    Are there simpler alternatives to copy-on-write here? A plain mutex might be sufficient, but full BlockMap iteration would hold the lock for extended periods.

    I would be interested in seeing a plain mutex approach to this first that does not need a full copy-on-write structure.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-03-21 12:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me