mempool transaction expiration

jgarzik commented at 5:01 pm on February 21, 2014: contributor

Sparked by another discussion (#3721), it seemed useful to open an issue on transaction expiration.

It has been my longstanding contention that transaction lifetimes should be more deterministic. If a transaction does not confirm within a set period, the mempool should forget it, rather than accumulating garbage and leaving the transaction in limbo.

One proposal bandied about on IRC is to expire any transaction from the memory pool, that does not confirm within X blocks. Proposed value for X is 2 days, 144*2.

It is always the responsibility of the client to retransmit a transaction until it is confirmed. If a client ceases retransmission, and waits X blocks, they have a reasonable chance of presuming the network has forgotten the transaction [assuming the network has mostly upgraded]. That enables the client to recover the dead transaction, including double-spending with a higher fee.

This seems to accomplish two useful goals: preventing the mempool from accumulating garbage, and making transaction lifetime more deterministic [within a certain probability].

Related: mempool superblock redesign (#3723).

jgarzik added the label Improvement on Feb 21, 2014

gmaxwell commented at 5:06 pm on February 21, 2014: contributor

One thing I’ve wondered about this is if its possible to avoid an expiration replacement race.

Say 2 days minus one second after your tx is originally broadcast, you rebroadcast: your peers still have it so it goes nowhere and your broadcast has no effect. Say it’s 2 days plus one second: some nodes still have it and so the broadcast is not very effective… and your transaction still falls out of memory pools.

This all creates a window where an aggressively broadcasting double-spender could replace the transaction in memory pools.

Obviously double-spend exclusion can’t be counted on, and perhaps will someday be mooted by a more greedy miner policy, but I think it would be unfortunate to weaken the exclusion as a side effect of this if it can be avoided.

jgarzik added the label Brainstorming on Feb 21, 2014

christophebiocca commented at 6:04 pm on February 21, 2014: none

This all creates a window where an aggressively broadcasting double-spender could replace the transaction in memory pools.

So the idea here is:

Attacker creates a deliberately low fee transaction (t1) paying himself, broadcasts it to merchant node (they in turn relay to the entire network).
Original transaction (t1) falls out of merchant’s mempool.
Attacker double-spends that original transaction (t1), but this time with a transaction (t2) paying the merchant, and sends it to the merchant node.
Broadcast of t2 mostly fails, because the rest of the network still has t1 in their mempools.
Attacker waits a few more seconds (or more, time isn’t critical for this step if the previous steps failed), t1 is dropped from most nodes.
Attacker broadcasts t3, which pays him (double-spend of both t1 and t2).

The simplest countermeasure I can think of is to have a longer expiration time for transactions if you’re running a merchant node (so you never accept a tx before your peers would relay it), but I think the attacker can still juggle broadcasts to merchant/rest of network to get their mempools out of sync (just requires more steps), unless the merchant mempool’s expiration time is effectively infinite.

I don’t think there’s a reject-doublespend peer message in #3185, but if there were (or if the first doublespend relaying code was deployed), wouldn’t that let the merchant node know there’s a problem?

That way it can try broadcasting t2 and very quickly spot the double spend conflict between t1 and t2.

jgarzik commented at 1:47 am on February 27, 2014: contributor

Mempool janitor (#3753) expires transactions.

jgarzik commented at 2:15 am on February 27, 2014: contributor

Agree that TX mempool expiration weakens not-in-blockchain, zero-conf double-spend exclusion ever so slightly. That seems lost in the noise. Merchants who accept zero-conf transactions are already aware of the risks of operating outside of consensus, and expiration should not measurably change their business costs or calculus. Nobody is going to bother with an expensive, complicated attack on a 3-day-old transaction when a simple network race can more easily execute a partitioning double-spend attack.

It seems worth it, to give bitcoin users for the first time a mostly-predictable way to handle stuck transactions.

christophebiocca commented at 3:28 pm on February 27, 2014: none

I think you can get the best of both worlds by making the expiry logic have some randomness.

For example, the mempool janitor running exactly every 24 hrs means that an attacker can know when you’ll expire a transaction (by doing some probing) and use it for new attacks, but even making that interval have a ±1 hour variation completely removes that risk (without losing the roughly predictable expiration of transactions on the network overall).

gmaxwell commented at 5:12 pm on February 27, 2014: contributor

I don’t think the attacker needs to know. I think he can just try constantly to relay the replacement and nodes will take it when they take it, it just gives him an excuse to also flood the network.

Unreliability in expiration will mean that you really ought to wait even longer in re-announcing.

Consider:

send -> [has] -> [has] -> [has] [has] - [has] - [!has] [!has] - [has] - [!has] resend-> [has] -> [has] - [!has] [has] - [!has] - [!has]

Because the resend was before everyone dropped the transaction, the middle node in that example rejected it on resend and didn’t forward to the further node that had already dropped it.

dgenr8 commented at 0:13 am on August 2, 2014: contributor

The alternative to mempool expiration, tx permanently locking up its inputs until some day far in the future when it is confirmed, can’t be right.

So the question is: until when should double-spends be guarded against with whatever tools are available? A reasonable answer would be somewhere between “until next block” through “until sender being able to un-stick it is more important than protecting receiver from a malicious double-spend”.

The latter time is much sooner than the time when slow-confirming transactions should be expired. So by the time expiration (72 hours, 432 blocks, or whatever) rolls around, sender is way past the time when he should have realized he no longer had any double-spend protection.

MarcoFalke commented at 7:52 pm on September 27, 2016: member

This seems to be implemented by the current mempool limiting logic.

MarcoFalke closed this on Sep 27, 2016

sidhujag referenced this in commit 68283411df on Oct 19, 2020

DrahtBot locked this on Sep 8, 2021

mempool transaction expiration #3722