Rethinking Transaction Rebroadcasting #36403

Current Behavior

Rebroadcasting is handled entirely by loaded wallets, there is no rebroadcast behavior within the node. With the default of -walletbroadcast=1, a wallet will resubmit transactions into the node's mempool upon loading, and forcibly rebroadcast transactions at a uniformly random time between 12 and 36 hours since the last rebroadcast.

Transactions eligible to be rebroadcast are those known to the wallet that the wallet believes are unconfirmed (transactions confirmed in a pending blockConnected will still be considered eligible). On loading, all such unconfirmed transactions will be rebroadcast. On the timer, the transaction must also have a received time that is older than 5 minutes since the last updatedBlockTip notification.

On wallet loading, transactions that are successfully added to the node's mempool are not added to the unbroadcast set and no broadcast is initiated. On the timer, all transactions that are either already in the mempool or were just added to the mempool are rebroadcast unconditionally; they are added to the unbroadcast set and InitiateTxBroadcastToAll is called to begin broadcast to all tx relay peers.

Why Rebroadcast

Rebroadcasting serves 2 purposes for a wallet:

1. Reminding the network of transactions that may have fallen out of node mempools so that these transactions can be mined
2. Placing transactions into the node's mempool allows the wallet to utilize unconfirmed transactions so that users are not surprised by things that suggest they have less money than they have.
   • Unconfirmed change can be spent, but the transaction must be in the node's mempool
   • Change outputs of inactive (unconfirmed, not abandoned, and not in mempool) transactions are not included in the usual balances, while the spends of the inputs are, resulting in balances that are lower than expected. #33671 helps to address this, but the nonmempool balance does not indicate the amount returned by change as there are double counting issues, so it ultimately understates the balance. Once in the node's mempool, change is counted again.

Issues

Privacy Leak

Since the wallet unconditionally broadcasts transactions, a connected peer will learn about the initial broadcast, and then 12 to 24 hours later, will receive the transaction again from that node. Every 12 to 24 hours, it receives the rebroadcast. Since rebroadcasting is not part of typical node behavior. This indicates that the node doing these rebroadcasts is likely to be a party involved in that transaction, either sending or receiving (#3828).

Private Broadcast Circumvention

The wallet does not use the private broadcast feature at all. All transactions (re)broadcast by the wallet itself are to all tx relay peers and -privatebroadcast is ignored. As such, transactions initially sent via private broadcast are rebroadcast without private broadcast, which reveals the originating IP, as well as receiving IPs.

Doesn't Submit Packages

The wallet does not currently attempt to do any form of package submission to make use of opportunistic package relay that the node may do. While rebroadcasting does submit transactions in order from oldest first to ensure that parents are accepted before children, if a parent is unable to make it into the node's mempool on its own, that parent will not be broadcast, and subsequently, neither will the child. This applies to both new transactions and to rebroadcasts.

Potential Improvements

• Wallets should submit packages to node's mempool, probably limited to 1p1c so that they can be relayed
• Rebroadcast could be performed only on transactions that are newly added to the mempool, rather than unconditionally rebroadcasting every unconfirmed transaction that meets the rebroadcast criteria.
• The node could have a rebroadcast pool which can rebroadcast transactions that are not relevant to loaded wallets, thereby increasing the anonymity set of rebroadcasts, and enabling other nodes to potentially rebroadcast transactions for us. See #21061 for the last attempt at this.

Current PRs

These are currently open PRs that are touching something in rebroadcasting. This is not a list of things to review, and this is not a tracking issue for PRs to be reviewed. Rather, this list is to serve as a collection point of all PRs attempting to do something to rebroadcasting because I am having a hard time keeping track of everything.

PRs Related to Private Broadcast

• #34533
• #36330

Settings For Controlling Rebroadcast

• #36380

Changing What Gets Rebroadcast

• #36398

Thanks for the summary!

This indicates that the node doing these rebroadcasts is likely to be a party involved in that transaction, either sending or receiving

While much of the summary focuses on the case where the wallet is the sender, I think there is an asymmetry: In both the "Privacy Leak" and "Private Broadcast Circumvention" situations, the case where the wallet is the receiver is much worse because

1. The sender (attacker) chooses the fee and can make confirmation within 24h unlikely / rebroadcast likely
2. The sender chooses the timing: a wallet is affected indefinitely, not just in the timeframe until a tx is confirmed.

As such, transactions initially sent via private broadcast are rebroadcast without private broadcast, which reveals the originating IP, as well as receiving IPs.

Same here: The issue is not so much the private broadcast tx itself (which will most of the time confirm before rebroadcast), but potential new transactions sending to the involved addresses, maybe at a later time when the user previously running on tor is now running that same wallet on clearnet.

Thanks for this writeup.

One potential improvement that I would like to advocate for is something similar to #30471.

The rebroadcasting logic should be separated from the wallet into its own module, and the wallet would be a consumer of the rebroadcast module alongside other consumers like sendrawtransaction callers.

Rebroadcasting until your transaction gets mined is an important feature, and makes sending from the wallet more reliable. Other wallets/users should also be able to benefit from this, instead of having to create a daemon or other retry mechanism on their own.

This entire module could be optional, so that users who don't want any potential privacy leaks could just turn it off.

Improving private broadcast dovetails nicely with this. Others have suggested it should be separated into its own module as well. If we tied private broadcast to rebroadcasting it substantially mitigates the privacy leak concerns.

I had an offline discussion about this with @davidgumberg and others at the localhost office. The conclusion was that the mempool expiry timer should be significantly extended or dropped (see current discussion in #33510), the wallet should still resubmit to the mempool but not force relay, nodes should periodically choose random transactions from their mempool to rebroadcast, and sendtemplate and erlay would be immensely helpful. Lastly, we should have an RPC and GUI option for users to do a single rebroadcast of a specified transaction.

The idea is that this addresses the 2 ways that transactions get dropped from mempools (they reach the mempool expiry time limit, or the mempool minimum feerate is increased to the point that the transaction is evicted) while still preserving sender and receiver privacy.

On the expiry timer, the discussion in #33510 is going in the direction of dropping the expiry timer in favor of eviction of things that should have been but weren't mined after some time. With the expiry time extended, low feerate transactions that exceed the timer no longer need to be rebroadcast. High feerate transactions that are not mined could still get evicted, but if they were not mined, then probably something else is going on that would render rebroadcast useless. Additionally, sendtemplate and erlay would help to ensure that that transaction had propagated well.

For low feerate transactions, the goal is to get such transactions into mempools as soon as they are ready to accept them. If mempools are heterogeneous, there should be some node somewhere that has the the low feerate transactions it its mempools. With a random rebroadcast, those transactions should be rebroadcast eventually. Additionally, if wallets still resubmit to their own mempools periodically, our node will also rebroadcast it eventually. No preference is given to transactions that come from loaded wallets or RPC.

And of course, if a user wants to rebroadcast something manually, then they should be able to do so with a RPC or GUI button.

the case where the wallet is the receiver is much worse because

But also when the wallet is the receiver, it is more specifically interested in rebroadcasting. As an intentional receiver, you want to make sure that incoming transactions get mined so that you can get paid. Hence rebroadcasting such transactions is also in the user's interest. Keeping these transactions in our mempool, and in other mempools, allows us to CPFP such transactions. With CPFP, the receiver can increase the package feerate such that a transaction does get confirmed in a timely manner.

Fundamentally, both senders and receivers should want to rebroadcast - senders so that their payment can go through and they can get whatever service, and receivers so that they can get paid. At the same time, there are times where the sender may not be interested in rebroadcast (e.g. if the transaction is not for delivery of a good or service), and that the receiver may not be interested in rebroadcast (e.g. getting dust spam). But I don't think that the wallet should cater to one or the other specifically, as in the average case, both senders and receivers are interested in rebroadcast.

One potential improvement that I would like to advocate for is something similar to #30471.

One issue that came up in the offline discussion was how to decide what to evict from the (re)broadcast pool. It seems like it would be DoS-able if an attacker makes a bunch of low feerate dust to addresses in your wallet, then they could fill up the rebroadcast pool and prevent rebroadcast of desired transactions.

There is an argument for dropping automated rebroadcast entirely, other than mempool submission on load. With the current expiry timer of 2 weeks, it could be that rebroadcast isn't meaningfully helping transactions get confirmed since that is a fairly long timeframe, certainly much longer than when rebroadcast was first introduced. In a dynamic fee environment, rebroadcast is also possibly less helpful as the things that would get rebroadcast might not even be accepted into mempools. So maybe automated rebroadcast is not all that useful? But I would be interested in some data on this before jumping to this conclusion.

In a world without automated rebroadcast, we definitely should make manual rebroadcast easier by adding a RPC and GUI button for it.

choose random transactions from their mempool to rebroadcast

re:privacy, I think this is doing the heavy lifting. I'd expect some sort of heuristic like a tx that's been in local mempool fox "too long" given its chunk feerate. One primary cost to this strategy is that if a non-standard-to-other-nodes is sitting in you mempool too long, it magnifies how much bandwidth will be wasted serving it, vs giving up and letting an operator figure out why.

rebroadcast pool historical attempts: #16698 and #21061 in case its helpful. At least one blocker seems to be better off: we have cluster mempool now for very efficient estimation of "should have been mined". Worth more investigation in light of the latest few releases? I might try to categorize the list of objections historically.

<details> <summary>Click here to expand prior attempt higher level comments</summary>

Prior rebroadcast attempts: objections/concerns

#16698 (https://github.com/bitcoin/bitcoin/pull/16698, 2019-08 → 2020-03; closed by author pending #18038)

• "Should have been mined" is ill-defined without looking at actual blocks (mzumsande). The local mempool's top may differ from what miners had; a template doesn't tell you what was skipped. #16698#pullrequestreview-279845040 #16698 (comment)
• Dependency on mining code to decide what to rebroadcast; testing of edge conditions (JeremyRubin) #16698#pullrequestreview-321309451
• Undocumented parameters, flaky functional test (jonatack) #16698#pullrequestreview-323826007
• Wallet resend timer interaction, unbroadcast/GETDATA delivery logic should be split out (ariard) #16698#pullrequestreview-323793323
• Empty blocks; full blocks at lower feerate than mempool (out-of-band miner deals, miner self-sweeps) trigger spurious rebroadcast (fjahr) #16698 (comment)
• gmaxwell, 2020-01-07: #16698 (comment)
  • Parent/child handling: a high-feerate child with a low-feerate parent has no chance of inclusion.
  • Rebroadcast far less frequently. After one successful broadcast, extra rebroadcasts mostly just leak data to reconnecting spies.
  • Wait until a tx is missed by N≈3 blocks, not 1, to handle out-of-band miner deals.
  • Applying this to the whole mempool has "considerable implications". Average bandwidth isn't enough; what happens when all nodes are at mempool limits under load? Could the network DoS itself?
  • This is effectively mempool sync, which erlay's design rejected: policy differences (standardness, minfee, size limits, ancestor limits, softforks) cause persistent network-wide bandwidth leaks. Erlay-style reconciliation would be the right primitive, not INVs.
  • Attack: give miner-adjacent nodes a low-feerate tx and everyone else a high-feerate conflict, so the network re-advertises forever.
• Per-peer known-inventory bloom filters are unreliable for this: they both block intended re-sends and forget (gmaxwell) #16698 (comment)
• Closed pending #18038 (unbroadcast set): #16698 (comment)

#21061 (https://github.com/bitcoin/bitcoin/pull/21061, 2021-02 → 2022-03; drafted for rework, closed by fanquake)

• Privacy objection: new fee/time-in-mempool pattern; suggests randomization (prayank23). sipa rebutted: the leak is negligible compared to the ownership signal removed. #21061 (comment) #21061 (comment) #21061 (comment)
• Use incoming block size instead of max weight (empty blocks); IBD handling; tracker eviction (glozow) #21061#pullrequestreview-626660219
• Whole network rebroadcasting the same well-propagated txs right after each block; require missed in N blocks? (vasild) #21061 (comment) #21061 (comment)
• Measured volume: median ~15 missed txs/block, max 3226 (0xB10C); 75% of blocks ≤51 queued, top 1% 586–1075 (ajtowns) #21061 (comment) #21061 (comment)
• Spam via txs valid only under new softfork rules / outdated nodes (fjahr; ajtowns reply: bounded by attempt tracker/expiry) #21061 (comment) #21061 (comment)
• L2/app nodes with custom rebroadcast logic get no anonymity-set benefit; default-on would raise everyone's bandwidth for a minority; large mempools acting as tx archives (ariard) #21061#pullrequestreview-638098056 #21061 (comment) #21061 (comment)
• sdaftuar, 2021-05-31 (simulation on historical data); this effectively ended the design: #21061 (comment)
  • Well-propagated txs shouldn't be re-INVed by the whole network at once; for poorly propagated ones, a single node suffices.
  • Small benefit: a tx that doesn't propagate to nodes probably doesn't reach miners either.
  • The block-connection trigger synchronizes bandwidth spikes exactly when block relay needs the bandwidth.
  • Mass re-INVs back up the announcement rate limiter, so relay latency goes up.
  • Little value in rebroadcasting txs only marginally better than the rest of the mempool.
  • A node coming back online fires rapid rebroadcasts with a stale mempool/tip.
  • The CNB-plus-cached-feerate heuristic can select thousands of old low-feerate txs.
  • Packages: a parent below the peer's mempool min fee, or descendant limits, block relay. Skip txs with unconfirmed parents until package relay exists.
  • Only rebroadcast to wtxidrelay peers (older peers don't cache segwit rejects).
  • Possible ping-pong resurrection of near-expiry txs. Maybe restrict to RBF-signaling txs?
  • Counter-proposal: random timer about once every 24–48h, at most ~25 parentless txs older than 30 min from the top 1MB by ancestor feerate, with no mining-code dependency.
• Outdated nodes (unaware of new standard types) would rebroadcast below-market txs; infer the inclusion feerate from actual blocks instead of CreateNewBlock (sdaftuar) #21061 (comment)
• Author drafts it for rework ("only INV after missed from 3 blocks") after offline discussion with ajtowns/sdaftuar/jnewbery. The rework was never pushed: #21061 (comment)
• Closed: #21061 (comment)

</details>

re: #30471 I'm very wary of assuming how each wallet would want to behave and moderate their privacy leaks. There is no one size fits all, and I don't think we should be trying to invent some omin-wallet-protocol. We should be staying as simple and understandable as we can.

From the OP:

Rebroadcasting serves 2 purposes for a wallet:

1. Reminding the network of transactions that may have fallen out of node mempools so that these transactions can be mined

This seems one area where we can do better and improve the privacy as a side effect. Currently we rebroadcast all wallet transactions and only wallet transactions because they may have fallen out of mempools. But they may not have fallen out.

Instead of that, broadcast all transactions (wallet or not) that should have been mined in the last 3 blocks but were not. That seems to be more useful for the network and as useful for the wallet, plus has the obvious privacy advantage that we do not treat the wallet transactions differently than non wallet ones.

Thanks for the writeup, my opinion is that wallet rebroadcasting should be removed entirely.

The current rebroadcasting behavior causes irreversible harm to users by revealing[^1] their node to be the origin or attached to a wallet that is the recipient (@mzumsande above; see #36398) of a transaction, and the problem that it only-sometimes (if feerates ever get low enough) solves is that a user's transaction never gets confirmed because the feerate dropped below what any miner with meaningful hashrate was willing to hold onto. If there were no automatic rebroadcasting, the worst case for users would be confusion, troubleshooting, and manual intervention.[^2] That's much better than leaking transaction origin.

Problems currently mitigated by wallet rebroadcasting

1. Transactions with a confirmation horizon close to or beyond the mempool expiry (2 weeks) may need to be rebroadcast in order to be confirmed.
   • This problem can and probably should be mitigated by replacing or supplementing the time-based mempool expiry with another heuristic that does not evict transactions until after they were expected to be included in blocks but weren't. (#33510)
2. During certain mempool conditions which have historically been rare but will be more common in the future, transactions that are evicted from miner's mempools but later become worth of inclusion in blocks might benefit from rebroadcast.
   • While not a guarantee, if a single miner with a reasonable amount of hashrate has a large enough mempool, the transaction will eventually be included. And I would argue that is likely since miners are financially incentivized to run nodes with large mempools, since the marginal cost of the increased memory usage is very far below the marginal utility of the transaction.[^3]

Harms of wallet rebroadcasting

• With current behavior, when a transaction is wallet rebroadcast, there is a risk of revealing that the node is the origin of a transaction, or has a wallet that is interested in one of the inputs or outputs of the transaction.
• The current behavior is also a footgun/confusing for developers. In the PR that introduced private broadcast, a reviewer raised the question of wallet rebroadcast, but both the author and reviewers missed the way that existing behavior undermines private broadcast. This is not the fault of the reviewers or author, the behavior itself is a side effect that is confusing and hard to reason about / understand.
• While a lot of this harm could be partially mitigated with various fixes / improvements to the existing rebroadcasting behavior, I am skeptical that there is a variation of the current design that solves all of the problems.

Alternatives to wallet rebroadcasting

• I think that problem $2$ which wallet rebroadcasting mitigates, where a transaction that gets evicted from many miner's mempools but is now a good candidate for block inclusion is worth solving.
• In my opinion, I don't think there is likely to be a good design where a node treats a subset of transactions with different broadcast rules than all other transactions without leaking this. Even if there is, I think it would be hard to implement in a way that didn't leave footguns for other developers to accidentally leak that the node is treating the subset differently.
  • See prior discussions linked by @instagibbs above.
• Either of sendtemplate (proposed BIP-0153 and erlay (#35591) with no special treatment of one's own transactions are better solutions, since that makes the "Transactions eventually get confirmed" assumption equivalent to the assumption that a single node on the network has a large enough mempool to hold that transaction, instead of relying on there being sufficient hashrate that knows about the transaction.
• Failing all of this, there should be a good UX for discovering stuck transactions and doing an RBF or rebroadcast manually, both for RPC and GUI.

v32 and v31 backports

I don't think the bigger discussion about rebroadcasting should get in the way of delivering a simple fix for users, since the current rebroadcasting behavior makes private broadcast too dangerous to offer as-is IMO, but it is functionality that is worth fixing. I think #36330 which disables wallet rebroadcast if the node was started with private broadcast enabled, is a really simple and good-enough solution that is easy to backport.

[^1]: Rebroadcasting always leaks information but it doesn't always happen that a transaction is rebroadcast, and it's not necessarily the case that it a rebroadcast leaks enough information to unambiguously identify transaction participants.
[^2]: Unless either: - The user is under the mistaken impression that unconfirmed transactions are final. In this case there are many other ways to separate them from their bitcoin. - The user isn't depending on timely confirmation. In this case, rebroadcasting is a pretty hopeless strategy, the user should be RBF'ing their transaction. [^3]: Even at current memory prices of 300,000 sats for 16GB of DDR5, 1 byte of memory costs 0.00001875 sats to buy, far below current minimum relay settings. And the marginal cost will be a fraction of that since most valid transactions will get confirmed before the useful life of a stick of DDR5 ends.

Problems currently mitigated by wallet rebroadcasting

The one corner case I think is most likely is misconfiguration or poor connection on original submission. Maybe just give a button that does the resubmit for all unconfirmed, and let the user decide when to leak info?

I would not be troubled to remove it, provided there was a manual way to fix issues

I've never dived into this part of the code and maybe this is a stupid idea but, couldn't we try to "protect" our transactions in the mempool from being evicted? I guess if we don't want to mix and create dependencies between wallet and mempool code we could just use some technique similar to what prioritisetransaction does, thus making it more valuable than other transactions. If it cannot be evicted and we increase the expiry time enough or we simply remove expiry, wouldn't we get rid of the rebroadcast problem? Eventually we will INV the tx to someone and we would still be able to CPFP it.

The one corner case I think is most likely is misconfiguration or poor connection on original submission. Maybe just give a button that does the resubmit for all unconfirmed, and let the user decide when to leak info?

Yeah, I agree that this is a legitimate case for rebroadcast, this is similar to the reason why private broadcast does automatic rebroadcasting but for non-private broadcasting I agree that the user should decide whether or not to risk losing privacy.

couldn't we try to "protect" our transactions in the mempool from being evicted?

There are two problems with this: the first is that a node's mempool is easy to discover, and this I suspect is an inescapable property of Bitcoin Core's mempool or any bitcoin implementation's transaction gossip pool. Keeping a transaction in your mempool that an external observer can see you should have evicted would be a leak.

Eventually we will INV the tx to someone and we would still be able to CPFP it.

The second is that as far as I understand, in Bitcoin Core's current behavior, transactions are only INV'ed when they are first learned about:

https://github.com/bitcoin/bitcoin/blob/acdb32693417c1b6596edfbdd19ab793c010d34f/src/net_processing.cpp#L2489-L2497 https://github.com/bitcoin/bitcoin/blob/acdb32693417c1b6596edfbdd19ab793c010d34f/src/net_processing.cpp#L3406-L3407 https://github.com/bitcoin/bitcoin/blob/acdb32693417c1b6596edfbdd19ab793c010d34f/src/net_processing.cpp#L3420

Or when they are added to the mempool's unbroadcasted tx set, which only happens for transactions that are broadcast either via RPC or via the wallet.

I think most if not all nodes on the network also behave like this, so there is no mechanism for old transactions to get propagated around the network, I think general tx gossip designs that include or imply rebroadcasting like sendtemplate and erlay are better solutions to the """mempool backlog propagation problem""". Solving this generally is useful for helping get juicy fees in the tx backlog to miners and makes broadcast a little more reliable, and it improves mempool fee estimation, and it would help with compact block reconstruction.

From the perspective of a random node rebroadcasting other people's transactions, I think there's two approaches that make sense:

• share the top of your mempool, because that's what's expected to be mined really soon. this is the template sharing approach #36369
• share the "bottom" of your mempool, triggered when the peer's minfeefilter reduces (if the feefilter reduces from 0.14 sat/vb to 0.12 sat/vb, send all the txs in your mempool with a feerate between those two values)

If the wallet is always trying to keep a tx in your mempool, then that can result in having it fall out of the mempool when the mempool minfee is 0.15 sat/vb and re-enter when your minfee drops to 0.13 sat/vb. To resist probing attacks, you'd want random nodes on the network to experience the same behaviour, so having that occur on feefilter changes may be plausible. However getting the right set of transactions on a feefilter change is probably a little expensive (it's not what cluster mempool is optimised for), and sending all the txs between 0.14 sat/vb and 0.12 sat/vb has a good chance of overflowing the extra space available in your mempool, so I think it'd end up being something of a waste of bandwidth and not terribly stable if you took that approach.

If template sharing is deployed and gets adoption, then matching the wallet rebroadcast behaviour to that could work -- ie, only re-add to the mempool (and hence your own generated templates) when the (cluster/chunk) feerate of the tx would be high enough to include in a template.

#36482 states about the wallet rebroadcast “while this is broadcast is limited to peers that have not previously been inv'd the transaction[…]”, but the summary here in the OP mentions “Since the wallet unconditionally broadcasts transactions, a connected peer will learn about the initial broadcast, and then 12 to 24 hours later, will receive the transaction again from that node. Every 12 to 24 hours, it receives the rebroadcast.” Perhaps one of the two should be updated to clarify which of the two is the case.