mempool: asynchronous mempool fee rate diagram updates via validation interface

ismaelsadeeq commented at 1:39 pm on March 11, 2026: member

Motivation

Now that we have the full linearization of mempool clusters, whenever a cluster in the mempool gets updated either due to replacement, eviction for various reasons, or block connection — we now have the previous linearization of the affected cluster and the new linearization of the new clusters available.

This brings two advantages — the ability for outside observers to make decisions based on fee rate diagram updates without direct access to the mempool, and reduced lock contention.

It can used primarily for:

Making CBlockPolicyEstimator package aware
Preventing redundant block template builds in the proposed block template manager BlockTemplateManager Proposal.

During the development of cluster mempool those usage was discussed. Note: This is not the motivation behind cluster mempool just a part of it.

See discussion in Package aware fee estimator and Determining BlockTemplate Fee Increase Using Fee Rate Diagram.

Currently the fee estimator is asynchronous but not package-aware. After this PR it is possible to make it package-aware — a follow-up branch on top of this does exactly that see commit.

For the mining interface, the node every 1 seconds rebuilds a block template regardless of whether any significant mempool change occurred to the top block.

Even after cluster mempool, building a block template is not free since we have to pause the node (holding cs_main).

With this notification a proposed block template manager that observes the mempool update will only rebuild after an inflow in the mempool top block template warrants it. Redundant builds where no significant fee rate change occurred are eliminated.

And doing this becomes even more useful because of proposed improvements like mempool-based fee rate estimation which also build a block template right now it uses a constant 7-second interval and rebuilds regardless of whether an inflow affects the top block template or not. This will fix that.

A proposed sendtemplate in the p2p layer also build block template, I propose it also use the proposed block template manager which should be built on top of this so that redundant block template builts are totally eliminated.

There is a different approach to determining updates in the top block template, but that still requires some direct interaction with the mempool. Since the fee estimator needs this notification anyway and we already have it, the block template manager can use it too and not interact with the mempool at all — it becomes a purely passive observer.

Changes

TxGraph::Chunk a new struct that has with transaction TxGraph::Ref’s, and fee rate FeeFrac so the before/after diagram carries enough information to compute chunk hashes.
MemPoolChunk (kernel/mempool_entry.h) — new struct representing a mempool chunk: a FeeFrac fee rate and a uint256 chunk hash identifying the set of witness transactions in the chunk.
MemPoolChunksUpdate (txmempool.h) — new struct carrying:
- old_chunks — chunks before the update.
- new_chunks — chunks after the update
- reason — MemPoolRemovalReason identifying the update type
- block_height — set only when reason == BLOCK, carries the connecting block’s height; defaults to std::nullopt
MemPoolUpdate — new ValidationInterface callback receiving a MemPoolChunksUpdate.
GetHashFromWitnesses (src/util/witnesses_hash.{h,cpp}) — sorts a vector of Wtxids lexicographically and hashes them deterministically to produce a stable chunk hash. GetPackageHash in src/policy/packages.cpp is refactored to use this.
Caches the fee rate diagram computed in GetAndSaveMainStagingDiagram / GetFeeRateDiagramChunks so it is immediately available when the notification is emitted without recomputing.
Extract the dependency addition subroutine in UpdateTransactionsFromBlock to a new method private method addDependenciesFromBlock

MempoolUpdated is fired after every mempool mutation. Specifically:

Transaction addition and RBF replacement: fired inside CTxMemPool::ChangeSet::Apply with reason REPLACED.
Block connection: fired inside removeForBlock with reason BLOCK. Confirmed transactions are fully removed before conflicts are evicted. The connecting block’s height is carried in block_height.
Reorg: fired inside removeForReorg with reason REORG.
Recursive removal: fired inside removeRecursive with a caller-supplied reason e.g. CONFLICT, EXPIRY, REORG.
Expiry: fired inside Expire with reason EXPIRY.
Size limit eviction: fired inside TrimToSize with reason SIZELIMIT.
Post-reorg dependency update: fired inside UpdateTransactionsFromBlock with reason SIZELIMIT.

In all cases the before/after chunks are obtained from changeSet->GetFeeRateDiagramChunks() after committing the staging graph.

Note on UpdateTransactionsFromBlock, a naive single-pass approach calling Trim() directly on the main graph fails because oversized clusters cannot be linearized, so GetMainStagingDiagram cannot produce a valid before/after diff. The fix is a two-stage process:
Phase 1 (optimistic): Create a txgraph staging , call addDependenciesFromBlock to register all new parent-child relationship, then call Trim().
If Trim() returns nothing — no cluster exceeded the limit — commit staging, compute the diagram, emit MempoolUpdated(SIZELIMIT), done.
If Trim() returns evicted transactions, the optimistic path failed. Abort the staging we roll back to main, open a fresh staging, call RemoveTransaction for each evicted tx to remove them from staging, call addDependenciesFromBlock again (evicted txs are silently skipped since they are no longer in the graph), commit, compute the before/after diagram, emit MempoolUpdated(SIZELIMIT), then call RemoveStaged to fully remove evicted transactions from the mempool.
Reorgs are rare, and reorgs that burst the cluster size limit are also rarer still, so in the optimistic common case we apply dependencies only once.
Added a new mempool_update_tests: unit test suite covering all MempoolUpdated emission paths:

DrahtBot added the label Mempool on Mar 11, 2026

DrahtBot commented at 1:40 pm on March 11, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	sedited

If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#34669 (feefrac: drop comparison and operator{«,»} for sorted wrappers by sipa)
#34448 (ci, iwyu: Fix warnings in src/util and treat them as errors by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

test/validationinterface.h comment: “Returns MemPoolChunkUpdate struct.” -> “Returns MemPoolChunksUpdate struct.” [The comment’s type name is singular (“MemPoolChunkUpdate”) while the code/actual struct is plural (“MemPoolChunksUpdate”), which can mislead readers.]
src/txgraph.h comment: “Datatype that represent a chunk along with it’s feerate …” -> “Datatype that represents a chunk along with its feerate …” [Incorrect possessive “it’s” instead of “its”, and verb agreement (“represent”) reduces readability/clarity.]

2026-03-19 15:06:46

ismaelsadeeq commented at 1:43 pm on March 11, 2026: member

This PR is targeted for v32.0.

The main changes are not significant; the test in the last commit constitutes the bulk of the diff, 493 lines of code addition.

sedited added this to the milestone 32.0 on Mar 14, 2026

sedited commented at 8:26 am on March 14, 2026: contributor

Concept ACK

Put this on the milestone. I am a bit surprised at the amount of changes this requires, but I guess this is what it takes to wire up the new cluster mempool data structures.

DrahtBot added the label Needs rebase on Mar 19, 2026

txgraph: populate chunk refs in `GetMainStagingDiagrams` result

GetMainStagingDiagrams currently returns vector<FeeFrac> pair
containing only chunk feerates.
Some callers might need to know which transactions belong to
each e.g chunk in order to compute chunk hash.

This commit introduces TxGraph::Chunk which pairs a chunk feerate with
the refs of all transactions in that chunk. Update GetMainStagingDiagrams
to return pair of vector<TxGraph::Chunk> and rename AppendChunkFeerates
to AppendChunks to reflect the richer return type.

This commit then added a SanityCheck coverage for the new behaviour.

bcc62e7773

refactor: move-only: split GetPackageHash 4e1827b595

mempool: cache fee rate diagrams as chunks in ChangeSet

CalculateChunksForRBF extracts only the FeeFrac from each TxGraph::Chunk
and discards the rest.

Cache the fee rate diagrams chunks in ChangeSet.

The cached chunks are going to be used in subsequent commit.

3b300c6b6b

mempool: add `MemPoolChunksUpdate` to `kernel/mempool_entry.h` d68252544c

mempool: add GetAndSaveMainStagingDiagram to ChangeSet

GetAndSaveMainStagingDiagram is extracted from CalculateChunksForRBF so
the diagram snapshot can be taken on removal paths that do not go through
the RBF codepath.

The const fix to RemoveStaged is required because GetRemovals()
returns a const reference.

1e9ef1d448

validation interface: add `MempoolUpdated` signal

This signal allows subscribers to observe mempool fee rate diagram
changes in an asynchronous way without holding the mempool lock.

8e791b5adf

refactor: move-only: extract dependency addition into a seperate method

The newly created method addDependenciesFromBlock is going to be
used multiple times in subsequent commit.

9adafcfaea

mempool: fire MempoolUpdated signal on each mempool update path

Each removal path now captures the before/after fee rate diagram chunks
and fires MempoolUpdated so subscribers can observe every mempool change.

This changes the order of operations in removeForBlock: all block
transactions in the mempool are now removed before conflict
removals.

2c5d15383b

test: add unit test to mempool update validation events db0a2b1a4b

ismaelsadeeq force-pushed on Mar 19, 2026

DrahtBot removed the label Needs rebase on Mar 19, 2026

mempool: asynchronous mempool fee rate diagram updates via validation interface #34803

Motivation

Changes

Reviews

Conflicts

LLM Linter (✨ experimental)