Ephemeral Dust #30239

A replacement for #29001

Now that we have 1P1C relay, TRUC transactions and sibling eviction, it makes sense to retarget this feature more narrowly by not introducing a new output type, and simple focusing on the feature of allowing temporary dust in the mempool.

Users of this can immediately use dust outputs as:

1. Single keyed anchor (can be shared by multiple parties)
2. Single unkeyed anchor, ala P2A

Which is useful when the parent transaction cannot have fees for technical or accounting reasons.

What I’m calling “keyed” anchors would be used anytime you don’t want a third party to be able to run off with the utxo. As a motivating example, in Ark there is the concept of a “forfeit transaction” which spends a “connector output”. The connector output would ideally be 0-value, but you would not want that utxo spend by anyone, because this would cause financial loss for the coordinator of the service: https://arkdev.info/docs/learn/concepts#forfeit-transaction

Note that this specific use-case likely doesn’t work as it involves a tree of dust, but the connector idea in general demonstrates how it could be used.

Another related example is connector outputs in BitVM2: https://bitvm.org/bitvm2.html .

Note that non-TRUC usage will be impractical unless the minrelay requirement on individual transactions are dropped in general, which should happen post-cluster mempool.

Lightning Network intends to use this feature post-29.0 if available: https://github.com/lightning/bolts/issues/1171#issuecomment-2373748582

It’s also useful for Ark, ln-symmetry, spacechains, Timeout Trees, and other constructs with large presigned trees or other large-N party smart contracts.

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30239.

Reviews

See the guideline for information on the review process.

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #31153 (bench: Remove various extraneous benchmarks by dergoegge)
• #29247 (CAT in Tapscript (BIP-347) by 0xBEEFCAF3)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/25906194302

@theStack the primary motivation is to cover cases where non-0 value is attached to handle cases where a smart contract may want to “throw away” a few sats to fees, but otherwise cannot because of the 0-fee requirement of this PR for transactions with ephemeral anchors. If the ephemeral anchor-having transaction had non-0-fee, that would allow endogenous incentives to get it mined on its own, leaving the dust in the utxo set. As an example from the LN spec, [trimmed outputs(https://github.com/lightning/bolts/blob/master/03-transactions.md#trimmed-outputs) are directly added to the commitment transaction fee. Instead, spec writers could have the value flow to the ephemeral anchor, which is then spent to fees by the child transaction. Example spec here: https://github.com/instagibbs/bolts/commits/zero_fee_commitment

It’s a fairly narrow motivation, and honestly I don’t love the timmed output to fees scheme, but also doesn’t make the code anymore complicated, so I think it’s worth consideration.

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/25953908933

Concept NACK

There’s no reason to limit this behavior to TRUC/V3 transactions. The implementation should not ship with this restriction.

@petertodd I’ll give a non-TRUC version another try and report back. I may have been over-thinking things. If it works out it will be very similar in code structure…

edit: oh of course I already noted in OP: We don’t allow individual transactions with 0-fee outside of TRUC, for now due to trimming computation concerns. Cluster mempool does away with this restriction entirely. I’ll rework the code to upgrade to non-TRUC usage once cluster mempool hits.

@petertodd I’ll give a non-TRUC version another try and report back. I may have been over-thinking things. If it works out it will be very similar in code structure…

edit: oh of course I already noted in OP: We don’t allow individual transactions with 0-fee outside of TRUC, for now due to trimming computation concerns. Cluster mempool does away with this restriction entirely. I’ll rework the code to upgrade to non-TRUC usage once cluster mempool hits.

I think the worthwhile question here is why are 0-fee CPFP’s required to signal TRUC/V3? I don’t see any reason why transaction creators should be forced to adhere to the restrictive TRUC/V3 rules if they do not wish to.

I’m also going to point out that restricting ephemeral dust transactions to zero fee seems to create protocol design problems around HTLCs: https://delvingbitcoin.org/t/ephemeral-anchors-and-mevil/383

The simplest solution with dust HTLCs is to just put the value towards mining fees. Forcing them to be put towards the keyless ephemeral output potentially creates incentive problems around stealing that output. If ephemeral dust transactions have fewer rules around them, that kind of design problem goes away and protocols can do what makes sense for them without trying to meet TRUC/V3 requirements.

Forcing them to be put towards the keyless ephemeral output potentially creates incentive problems around stealing that output.

Not really, because it will always be the miner that will steal it (since they can replace any spend by a spend to themselves). As was detailed in the delving post you’re referencing, putting HTLC values in the keyless ephemeral output ensures that it’s spent and doesn’t pollute the utxo set, while putting the fees directly to the commitment transaction doesn’t incentivize spending that output?

Stealing an ephemeral anchor output can only be done in certain conditions, at specific ranges of values. It is not always profitable due to overheads, and you can reduce those overheads by doing complex things.

The game theory is much simpler if the dust HTLC funds go directly to fees.

On July 19, 2024 5:50:49 PM GMT+02:00, Bastien Teinturier @.***> wrote:

Forcing them to be put towards the keyless ephemeral output potentially creates incentive problems around stealing that output.

Not really, because it will always be the miner that will steal it (since they can replace any spend by a spend to themselves). As was detailed in the delving post you’re referencing, putting HTLC values in the keyless ephemeral output ensures that it’s spent and doesn’t pollute the utxo set, while putting the fees directly to the commitment transaction doesn’t incentivize spending that output?

Ok, the rework ended up being quite easy.

Pushed an update that generalizes to non-TRUC transactions, but as noted earlier, individual transactions have to meet minrelay for now unless they are TRUC, see/divert discussion here for that specific point: #26933 . If this restriction goes away, it will be usable(see tests I added with v2 transactions at the bottom where I checked it). Note that we are still constrained to 1P1C relay until we get general package relay.

re:0-fee, yes it’s obviously sub-optimal for transaction creation for things like trimmed values. There’s no simple way that I can think of that ensures that dust isn’t mined. E.g., TxA with dust, followed by TxB spending the dust. Both enter mempool. Then TxB' replaces TxB, but doesn’t spend the dust output from TxA. If TxA has fees at or above minrelay, it may well be mined on its own.

I don’t have any clean ways of doing that at present. If we figure it out, it’s another clean relaxation that should have easy logic to follow. @glozow had suggested long ago that we could allow any fee under minrelay for ephemeral tx, but this is still pretty limited in what amounts are allowed for fees. I’m not convinced it’s worth it vs thinking harder about figuring out how to allow unbounded feerates.

Stealing an ephemeral anchor output can only be done in certain conditions, at specific ranges of values. It is not always profitable due to overheads, and you can reduce those overheads by doing complex things.

I don’t understand this hand-wavy comment, can you detail? I don’t see any complexity here. Whenever a miner would include a transaction spending an ephemeral output in a block:

• if that ephemeral output is economical, the miner replaces the spending transaction by one that sends funds to themselves
• otherwise, they keep the existing transaction (which in most cases will contain an unrelated input to bring fees)

Sounds trivial enough?

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/27674254741

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/27880415572

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

It looks like we’ve created a new, rare, pinning vector here: in the event that a dust-generating transaction is reorged back into the mempool, transactions that otherwise could have spent an ordinary output of that transaction alone now can’t be accepted into the mempool because they fail the “spends all parent dust” check.

For example, imagine a coinjoin is funded by a transaction A containing a dust output. After a reorg, the transaction A gets forced back into the mempool. Now the coinjoin (or double-spends of it) fails the spends all parents dust check, because it doesn’t spend the dust of A, and A may never get mined due to fee-rates increasing.

@petertodd I believe that’s incorrect, and looks like I wrote a test case for this: https://github.com/bitcoin/bitcoin/pull/30239/commits/63e097092253b25fa9d72af167d83d911c805c9c#diff-f3bb3623abbd14a7140e405175278c9187eb8c2cf6ff0279700ac507c5f4180cR334

I show that a child spend of parent tx gets into mempool even if it has dust of its own, and fails to spend parent’s dust.

Reorgs should hopefully be expensive, so we simply allow things back in mempool even if the spentness is violated.

You can glance at the new usages of bypass_limits in the code changes to see. I should probably add a test showing that a tx with two dust or fee-having can’t use bypass_limits.

@petertodd I believe that’s incorrect, and looks like I wrote a test case for this: 63e0970#diff-f3bb3623abbd14a7140e405175278c9187eb8c2cf6ff0279700ac507c5f4180cR334

I show that a child spend of parent tx gets into mempool even if it has dust of its own, and fails to spend parent’s dust.

Reorgs should hopefully be expensive, so we simply allow things back in mempool even if the spentness is violated.

You can glance at the new usages of bypass_limits in the code changes to see. I should probably add a test showing that a tx with two dust or fee-having can’t use bypass_limits.

I think you’re misunderstanding the scenario I’m talking about. I’m not saying that transactions in a block won’t get back into your mempool when your node reorgs the block - they of course do get it due to the bypass_limits flag. What I’m saying is that subsequent transactions won’t be accepted, resulting in a pin.

Concretely, imagine that the top block contains transaction A, which contains a dust output. Transaction B, which spends an output of A, can be accepted into your mempool as A is mined, and we can double-spend B as usual with a higher fee-rate.

Now let’s reorg out the top block. Transaction A is forced into the mempool using bypass_limits; transaction B remains in the mempool. However, we can no longer double-spend B with a higher fee-rate, as any B2 is rejected on the basis that it does not spend the dust output of A. Thus, B is pinned by the existence of A in your mempool.

While this is a relatively rare scenario, it’s not that rare. And there’s probably circumstances where attackers could take advantage of this opportunistically by waiting until they happened to have a transaction in the right state to use in an attack.

Got it. Normally a reorg like this would result in an effective feerate reduction, as the entire package is now having to be paid for. With the dust spending restriction, it makes it impossible to replace later, exactly like a package limit pin.

I should point out that the TRUC code may suffer from this same category of problem around reorgs too…

Depends on the topo but yep, post-reorg package limits may be hit unexpectedly. Parent->child->grandchild, where parent was formerly confirmed, as an example. Regular chain limits are also possible to hit, f.e. if the re-entered parent was 100kvB, or full chain limits hit.

Circling back to dust, I suppose bypass-ness of a transaction entry(the re-entered parent in this context) could be tracked and spenders could bypass the dust spentness check as well? Will mull it over.

Got it. Normally a reorg like this would result in an effective feerate reduction, as the entire package is now having to be paid for. With the dust spending restriction, it makes it impossible to replace later, exactly like a package limit pin.

Exactly. And unlike the inherent pin possible by the fact that the reorged transaction could simply be big, this variant works even if the transaction is small, making pulling it off potentially a lot easier.

Personally I’d consider removing the restrictions on spending outputs of ephemeral dust transactions. For what Core is trying to accomplish here, I think it’s enough that the transaction was accepted into your mempool with the dust being spent. That’s enough of a nudge to get the vast majority of use-cases to do what we want them to do. It’s more important that this feature can be used securely by L2 implementations, without having to worry about security problems from complex edge cases.

People who want to actually create dust intentionally can do so pretty easily anyway by just sending their transactions to one of the pools that doesn’t enforce the dust limit; most of the recent growth is probably due to intentional token protocols doing things for which rules like this are irrelevant.

Suppose we do remove that restriction. You would be able to create an unspent dust UTXO by broadcasting a transaction package creating the UTXO in A, a transaction with a dust output and a non-dust output, and then spending it with a two input transaction B spending both those outputs. Then you double-spend B to remove the dust input while still providing miners an incentive to mine A.

Thing is, to do this you have to spend about as much money in fees as it would have cost you in lost sats to create an unspent non-dust output instead. Probably more, because the dust limit is based on an unrealistically low 1sat/vb fee-rate. This isn’t actually any worse than the status quo for an attacker trying to create dust.

Similarly, I don’t think we actually need the restriction only having one dust output per transaction. If they’ve all been spent to get the transaction into your mempool, playing games to get them unspent costs as much or more money as just creating non-dust UTXOs.

Regular chain limits are also possible to hit, f.e. if the re-entered parent was 100kvB, or full chain limits hit.

While I’m not sure off the top of my head if this is actually true, it should be the case that RBF’s of existing transactions should always be allowed even if package limits have been exceeded somehow. I proposed this principle awhile back as the “always replaceable invariant”: the mempool should always allow you to replace a transaction already in the mempool with a similar one at higher fee so that you can make forward progress.

This specific issue contradicts that principal because it’s a hard restriction on the type of transaction allowed in a specific circumstance.

Reworked the PR to relax IsStandardTx to allow a single dust output, and hopefully simplified the remaining checks that enforces spend of dusty parent’s dust. This should assure the same mempool invariants as before, but with more understandable and minimal changes to logic flow. @petertodd

It’s more important that this feature can be used securely by L2 implementations, without having to worry about security problems from complex edge cases.

I can’t speak for all wallets, but at least for CLN the fresh spend of funding outputs will be detected and the anchor spending will start anew. I think for first cut the tradeoff of having invariants that are easy to reason about is best. Given that there are actionable things wallets can do and the minimal complexity, I’m sticking with the overall current approach.

While I’m not sure off the top of my head if this is actually true, it should be the case that RBF’s of existing transactions should always be allowed even if package limits have been exceeded somehow.

Today’s mempool does a suboptimal job of dealing with chain limits and RBFs. Assuming cluster mempool, we still have the issue of what to do with cluster limits that are exceeded even after removing RBF conflicts, if any. This has been called “sibling eviction” and in general seems doable in a DoS-resistant way but is left to future work. It’s a problem I’m very interested in solving post-cluster mempool.

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/29626041615

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

Today’s mempool does a suboptimal job of dealing with chain limits and RBFs. Assuming cluster mempool, we still have the issue of what to do with cluster limits that are exceeded even after removing RBF conflicts, if any. This has been called “sibling eviction” and in general seems doable in a DoS-resistant way but is left to future work. It’s a problem I’m very interested in solving post-cluster mempool.

What I mean is, if you are doing an RBF replacement of one transaction with another transaction paying a higher fee-rate, with the same or smaller size, this should always be allowed even if somehow your mempool got into a state where a cluster limit was exceeded.

Point is, the replacement isn’t making anything worse, so it should be allowed. Sibling eviction isn’t needed to support this case.

Looks good. New rules seem easier to reason about (up to 1 as long as 0-fee, must spend all of my parent’s dust no matter what).

Can you remind me of the use cases of a keyed dust output again? It seems slightly safer to require ephemeral dust be P2A, as it would allow anybody to clean them up in case something goes wrong, but perhaps too narrow.

Additionally, could add a release note fragment.

With the new changes, I’m considering disallowing entry into the mempool if either the base fee or the modified fee is non-0. If miners really want to mine things generally with 0 fee that’s one thing, but offering an interface to avoid the dust cleanup seems suboptimal. @glozow

Can you remind me of the use cases of a keyed dust output again?

What I’m calling “keyed” anchors would be used anytime you don’t want a third party to be able to run off with the utxo. As a motivating example, in Ark there is the concept of a “forfeit transaction” which spends a “connector output”. The connector output would ideally be 0-value, but you would not want that utxo spend by anyone, because this would cause financial loss for the coordinator of the service: https://arkdev.info/docs/learn/concepts#forfeit-transaction

Note that this specific use-case likely doesn’t work as it involves a tree of dust, but the connector idea in general demonstrates how it could be used.

Another related example is connector outputs in BitVM2: https://bitvm.org/bitvm2.html . You could imag

It seems slightly safer to require ephemeral dust be P2A, as it would allow anybody to clean them up in case something goes wrong, but perhaps too narrow.

In general. if we have 0-value dust, I find it unlikely to hope that miners will clean it up for no reason?

I’m also pretty hesitant to guess what people will find useful in general, e.g., if they’re worried about third party replacement cycling, they can be more cautious. The can also include a tapscript branch for anyonecanspend cleanup, just like LN does today!

I feel moderately strongly about this, but of course relaxing policy further later is also another valid choice.

Additionally, could add a release note fragment.

Added

I’m also pretty hesitant to guess what people will find useful in general, e.g., if they’re worried about third party replacement cycling, they can be more cautious. The can also include a tapscript branch for anyonecanspend cleanup, just like LN does today!

Seems reasonable to want keyed. Mostly wanted the rationale to be written up here.

In general. if we have 0-value dust, I find it unlikely to hope that miners will clean it up for no reason?

Anyone can pay fees to clean up the dust. Doesn’t have to be miners specifically.

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/31849856149

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

Concept ACK

Mostly looked and reasoned about the core commit 92d7ad2b135668cb7f6b0fd32be15fd6dd2f915c so far. From what I understand the “only one dust output” limit would not be strictly necessary and is far less important than the other two rules (must be 0-fee, child must spend all dust from the parent) in order to disisincentivize creating dust UTXOs, or am I missing something?

What I left below are largely nits and follow-up ideas, feel free to ignore. Still want to look deeper into unit tests and the fuzzer.