txorphanage: replace boost::multi_index with dual std::map #34586

pull w0xlt wants to merge 1 commits into bitcoin:master from w0xlt:replace-multi-index-orphanage changing 1 files +216 −114
  1. w0xlt commented at 8:16 pm on February 13, 2026: contributor

    Motivation

    TxOrphanage needs to look up orphan announcements in two different orders — by (wtxid, peer) for transaction lookups, and by (peer, reconsider, sequence) for per-peer iteration and eviction.

    Currently this is done with boost::multi_index_container, one of the last remaining Boost uses in the codebase. This PR removes it in favor of standard library containers, contributing to the ongoing effort to eliminate the Boost dependency from bitcoind.

    Approach

    Replace the boost::multi_index_container with a hand-rolled AnnouncementIndex class wrapping two std::map containers cross-linked by iterators:

    • m_by_wtxid std::map<(Wtxid, NodeId), Announcement> — owns the data
    • m_by_peer std::map<(NodeId, bool, Seq), WtxidMap::iterator> — stores cross-references, no data duplication

    std::map iterator stability guarantees that cross-references remain valid across insertions and erasures of other elements. All mutations go through AnnouncementIndex member functions (Insert/Erase/SetReconsider) to keep both indices in sync, with Assume() assertions verifying every mutation.

    No behavior change.

    Operation mapping

    Operation Before (boost) After (AnnouncementIndex)
    Insert m_orphans.get<ByWtxid>().emplace(tx, peer, seq) m_orphans.Insert(tx, peer, seq)
    Erase m_orphans.get<Tag>().erase(it) m_orphans.Erase(it) (overloaded)
    Modify reconsider index.modify(it, [](auto& a){ a.m_reconsider = val; }) m_orphans.SetReconsider(it, val)
    Project peer→wtxid m_orphans.project<ByWtxid>(peer_it) m_orphans.ToWtxid(peer_it)
    Access field it->m_tx Ann(it).m_tx
  2. txorphanage: replace boost::multi_index with dual std::map
    Replace the boost::multi_index_container in TxOrphanage with a
    hand-rolled AnnouncementIndex class wrapping two std::map containers:
    
    - WtxidMap: keyed by (wtxid, peer), stores Announcement objects
    - PeerMap: keyed by (peer, reconsider, sequence), stores WtxidMap
      iterators as cross-references
    
    All mutations go through AnnouncementIndex member functions
    (Insert/Erase/SetReconsider) to keep both indices in sync. std::map
    iterator stability guarantees that cross-references remain valid across
    insertions and erasures of other elements.
    
    Harden index consistency checks by asserting mirror-index emplace/erase
    success in mutators, and strengthen SanityCheck() to validate full
    two-way bijection and key coherence between ByWtxid and ByPeer.
    
    No behavior change is intended.
    
    This is a step toward removing the boost::multi_index dependency from
    bitcoind.
    345ae35f07
  3. DrahtBot commented at 8:17 pm on February 13, 2026: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

  4. sipa commented at 8:18 pm on February 13, 2026: member
    boost::multi_index is headers-only and not a runtime dependency, why would we replace it with something less efficient?
  5. w0xlt commented at 8:42 pm on February 13, 2026: contributor

    Minimizing external dependencies is a project-wide goal, as Boost has been systematically removed over the years (boost::filesystem → std::filesystem, boost::thread → std::thread, boost::optional → std::optional, etc.).

    Even though boost::multi_index is headers-only, it is still required as a build dependency and and it transitively pulls in ~24 Boost sub-libraries (mpl, fusion, preprocessor, bind, etc.)

    The orphanage is small (bounded by DEFAULT_MAX_ORPHANAGE_LATENCY_SCORE (3000) and per-peer memory limits) - typically hundreds to low thousands of entries. The trade-off is dependency reduction vs. marginal overhead in a small data structure.

  6. sipa commented at 9:02 pm on February 13, 2026: member

    Minimizing external dependencies is a project-wide goal

    I don’t think that view is shared by everyone; I personally disagree with it.

    Dependencies certainly have costs: build-time logistics, reviewer competence, and security surface. All else being equal, we’d certainly prefer fewer. But I don’t think that’s the case here: emulating multi_index with STL containers means additional allocations, more indirections at runtime, some amount of duplication, and complexity of keeping all synchronized with one another. At the same time, the downsides seem minimal to me here; a headers-only build dependency seems like not much of a burden, the review burden seems less than that required to keep data structures in sync, and security surface of a pure internal data structure seems as low as it can get.

    (boost::filesystem → std::filesystem, boost::thread → std::thread, boost::optional → std::optional, etc.).

    I don’t think those are comparable. Obviously we’d prefer standard library versions over additional dependencies if they are functionally equivalent.

    The orphanage is small (bounded by DEFAULT_MAX_ORPHANAGE_LATENCY_SCORE (3000) and per-peer memory limits) - typically hundreds to low thousands of entries. The trade-off is dependency reduction vs. marginal overhead in a small data structure.

    The orphanage isn’t the only usage of boost::multi_index. I don’t think replacing all of them should be a goal, and I think there are other data structures that could be improved by replacing them with boost::multi_index ones, like addrman or the block header index.

  7. w0xlt marked this as a draft on Feb 13, 2026
  8. ajtowns commented at 11:31 pm on February 14, 2026: contributor
    Yeah, concept nack – boost::multi_index is useful and better than the alternatives. It might make some sense to try to remove it from the kernel (which would involve splitting the mempool out of the kernel, and just using CCoinsView as the interface to link them or similar), but removing it from p2p transaction handling makes no sense.
  9. w0xlt commented at 11:44 pm on February 14, 2026: contributor
    Thanks for the feedback. Since there’s no current interest in minimizing boost::multi_index usage, I’ll close the PR for now.
  10. w0xlt closed this on Feb 14, 2026

  11. fanquake commented at 10:31 am on February 16, 2026: member
    cc @theuni given you were also working on a replacement for this at some point (https://github.com/theuni/tmi iirc).

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-02-17 06:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me