Mempool Expiry eviction might remove txs that could be mined in the next block #33510

LimitMempoolSize always try to remove transactions from the mempool whose age has passed the mempool expiry time limit, which is 336 hours (i.e., two weeks) by default.

There might be a case where you evict something that might be mined next. For example, the mempool was cleared and some low feerate transaction that has been in the mempool and has not confirmed in a long time due to having a low feerate is now at the top of the mempool. However, when we want to limit the mempool size, we will evict such transactions.

We can limit the harm of this by only expiring when the mempool is full now then post Cluster mempool, in addition to the transaction entry time, it would be possible to keep track of how many times you expect the transaction to confirm and it has not. If it has reached some threshold and the expiry time of the tx has been reached, then drop the transaction from the mempool because you know the transaction is likely deliberately ignored by miners (On the assumption that the node policy rules is sane and miners use tx fee as incentive for selection of transaction into their block template) hence no need to save it in the nodes mempool.

How about we make the expiration time dynamic based on the position (computer by fee rate) in the mempool?

Txs more likely to be mined have a bigger expiry time. If the mempool clears a bit, txs with low feerate will be on top of the mempool and their expiry time can be re-calculated to increase and avoid evicting txs that will be mined.

This way transactions more likely to be mined will not be evicted while the txs less likely to be mined will be evicted and only kept if the mempool clears a bit and they change their "likely grade".

I don't know if this is really a concern in practice, because it's already compensated for by the fact that after a long time, transactions near the top of the mempool have had many chances of being mined already.

We could just get rid of expiration entirely if a time-based limit wasn't valuable, and just rely on eviction due to fullness. But if time-based expiration is valuable, then it's inevitable that a chance exists that a soon-to-be mined transaction expires.

We could just get rid of expiration entirely if a time-based limit wasn't valuable

I might agree with this. @sipa do you know the reason why this was implemented in the first place?

Archeology

Time-based mempool expiry was added here: https://github.com/bitcoin/bitcoin/commit/49b6fd5663dfe081d127cd1eb11407c4d3eaf93d as part of #6722, from what I found, the most relevant discussion for time expiry was in #6455, e.g. this comment (https://github.com/bitcoin/bitcoin/pull/6455#issuecomment-122580009)):

I do understand the worry about mempools becoming filled with high-fee but unconfirming transactions, leading to (ever) increasing fees until the mempool clears. I've thought about using fees actually in the mempool, or time-based experation.

There are also two relevant irc discussions: here and here

<details><summary>Mempool Expiry PR History Diagram</summary>

flowchart TD
    A[#6331] --> D(#6421)
    B[#6410] --> D
    C[#6281] --> D
    D --> |time expiry added| G[#6455]
    E[#6452] --> G
    F[#6453] --> G
    G --> H[#6470]
    H --> I[#6557]
    I --> J[#6722]
    J --> |merged| K[master]
    
    click A "https://github.com/bitcoin/bitcoin/pull/6331" _blank
    click B "https://github.com/bitcoin/bitcoin/pull/6410" _blank
    click C "https://github.com/bitcoin/bitcoin/pull/6281" _blank
    click D "https://github.com/bitcoin/bitcoin/pull/6421" _blank
    click E "https://github.com/bitcoin/bitcoin/pull/6452" _blank
    click F "https://github.com/bitcoin/bitcoin/pull/6453" _blank
    click G "https://github.com/bitcoin/bitcoin/pull/6455" _blank
    click H "https://github.com/bitcoin/bitcoin/pull/6470" _blank
    click I "https://github.com/bitcoin/bitcoin/pull/6557" _blank
    click J "https://github.com/bitcoin/bitcoin/pull/6722" _blank
    click K "https://github.com/bitcoin/bitcoin/commit/49b6fd5663dfe081d127cd1eb11407c4d3eaf93d" _blank

</details>

From this and my own assumptions, I gather that the time-based expiry is generally intended to evict transactions that a node believes are likely to be mined but empirical evidence has shown miners are not including for a long enough period of time that we think those transactions will never be included, maybe because miners:

• Are censoring these transactions
• Have a different estimation of their feerate
• Have a soft-fork activated that your node doesn't that makes the high feerate transactions consensus invalid

How about we make the expiration time dynamic based on the position (computer by fee rate) in the mempool?

Txs more likely to be mined have a bigger expiry time

I think this should be the opposite, the transactions that are the best candidates for time-based expiry, are the ones that are at the top of our mempool, but block after block remain unconfirmed.

We could just get rid of expiration entirely if a time-based limit wasn't valuable, and just rely on eviction due to fullness.

My gut reaction is that time-based expiry isn't helpful, but without it, it seems like it would be easy to break the mempools of nodes that are missing a soft fork, since you could broadcast a mempool's worth of post-soft-fork-invalid high-fee transactions for "free" to pre-soft-fork nodes, and they will never let go of them, but I suppose the current behavior where an attacker has to periodically rebroadcast is not that big of a barrier.

Thanks for the context @davidgumberg

How about we make the expiration time dynamic based on the position (computer by fee rate) in the mempool? Txs more likely to be mined have a bigger expiry time

I think this should be the opposite, the transactions that are the best candidates for time-based expiry, are the ones that are at the top of our mempool, but block after block remain unconfirmed.

Yep, you are right, makes more sense in the opposite 😅

but I suppose the current behavior where an attacker has to periodically rebroadcast is not that big of a barrier.

It's not a big barrier but still more uncomfortable; you have to be connected to the victim node and re-rebroadcast every default expiry_time, and eventually, if the attacker gets tired, the node will go run normal again. Without the expiry time, you can dos a mempool with 1 single attack.

@davidgumberg I like the idea of expiring based on how many times a transaction was expected to be mined, but wasn't. E.g. every block, or every 10 minutes, or on a Poisson timer, run the block building code and increase a counter in the mempool for every transaction in the template. Whenever the counter reaches a certain configurable value, expire. Note that that may well mean that nothing ever expires, if everything gets mined once it's spent enough time near the top of the mempool - but that may be ok, if it's a sign there is no persistent policy difference between your mempool and miners'.

One potential issue I see that whenever we expire, we also need to expire all descendants. What if you have a very old transaction that isn't confirming (for whatever reason) but it has a very recent child? Should the child be permitted to keep its parent alive longer? This might matter in CPFP settings, if the child somehow helps the parent getting confirmed.

Also thanks for digging up the history for expiration; I was wondering if it predated RBF, eviction, and CPFP, but it looks like all those things were added roughly around the same time. I can imagine one other reason for expiration, which probably didn't exist at the time it was added: allowing transactions to be replaced for free (e.g. without the incremental relay fee, as a counter to the pinning that's caused by that). But I'm guessing that the 2-week timeout currently doesn't really help with that anyway.

Also thanks for digging up the history for expiration; I was wondering if it predated RBF, eviction, and CPFP, but it looks like all those things were added roughly around the same time. I can imagine one other reason for expiration, which probably didn't exist at the time it was added: allowing transactions to be replaced for free (e.g. without the incremental relay fee, as a counter to the pinning that's caused by that).

I'll add that, back in the day, expiry would have been the only way to "replace" (i.e. get rid of) a non-signaling transaction that wasn't confirming.

One potential issue I see that whenever we expire, we also need to expire all descendants. What if you have a very old transaction that isn't confirming (for whatever reason) but it has a very recent child? Should the child be permitted to keep its parent alive longer? This might matter in CPFP settings, if the child somehow helps the parent getting confirmed.

Seems the package has to be removed at all, otherwise attackers can use this to keep the bigger and bigger package.

This appears detrimental to protocols that expect a high-fee child to bump their package. IMO this is a clear incentive compatibility issue. I wanted to investigate how this mechanism works and if we can construct a high fee rate package that is evicted simply from time. Claude was able to produce a clear example here. However, after reading the rest of the file, I was shocked to find this behavior is already tested and expected - the current test asserts a child is also evicted when the parent times out. After reading the history, thanks David, I am not convinced time is very useful, if not harmful.

This appears detrimental to protocols that expect a high-fee child to bump their package.

I agree, I think we should get rid of expiry.

The other idea of evicting something you think should be mined consecutively and it's now and not can be pursued later, or not?

1. Some people might be on the fence on that, because it can be seen as aiding censorship.
2. If your node did not upgrade due to a tightening of the rules, then someone can cheaply evict your mempool txs with high fee paying ones continuously with these transactions that won't get mined because of the rules tightening, using e.g by just recycling them after you evict (So in this case you are better off to just update or turn off the mempool). etc

For fee estimation, it can be done in that layer and should be discussed there, e.g. #34075 uses some heuristic to prevent that issue and detect that your node is not incentive compatible.

Friendly ping to @0xB10C can you grep one of your nodes for how often we hit this expiry on mainnet?

• Some people might be on the fence on that, because it can be seen as aiding censorship.

• If your node did not upgrade due to a tightening of the rules, then someone can cheaply evict your mempool txs with high fee paying ones continuously with these transactions that won't get mined because of the rules tightening, using e.g by just recycling them after you evict (So in this case you are better off to just update or turn off the mempool). etc

These two arguments also affect expiry, so I don't see a downgrade in that sense.

Using the "how many times this tx had a chance to be mined but it wasn't" metric sounds like a win. We would not evict CPFP cases as the parent is the one paying low fees, so it didn't have a realistic chance to get mined. But we would evict things that are not getting mined after a certain time. And we would not evict things that did not have a chance to be mined, but after a certain time they do have a chance (the initial issue presented here).

Just thinking out loud and without giving it much thought, but it sounds "easy" to keep track of that thanks to #33389. Probably thanks to cluster mempool we could do it in a package way and not per-tx.

Friendly ping to @0xB10C can you grep one of your nodes for how often we hit this expiry on mainnet?

Already working on this for https://bnoc.xyz/t/data-request-how-many-mempool-transactions-are-evicted-by-time/132 by grepping for reason=expiry.

Will post something with more details soon: We frequently remove transaction for expiry. There might be days without an expiry, and then days with hundreds or thousands transactions expiring. They were likely all broadcast at the same time with a low fee. On 2025-09-30 multiple of my nodes saw more than 114k transaction expire, which is something I want to take a closer look at..