Remove mempoolfullrbf option #26438

sdaftuar commented at 1:57 PM on November 1, 2022: member

I've laid out my reasoning for removing this option on the mailing list (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021135.html).

If any reviewers are unconvinced by my rationale, I'd appreciate answers to the questions I laid out at the end of that email:

Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?
Is it reasonable to enforce BIP 125's rbf rules on all transactions, if those rules themselves are not always incentive compatible?
If someone were to propose a command line option that breaks v3 transaction relay in the future, is there a logical basis for opposing that which is consistent with moving towards fullrbf now?

Remove mempoolfullrbf option

Reverts 25353

4a4086603e

glozow added the label TX fees and policy on Nov 1, 2022

glozow added the label Mempool on Nov 1, 2022

glozow added this to the milestone 24.0 on Nov 1, 2022

ryanofsky commented at 2:43 PM on November 1, 2022: member

Concept ACK. Admittedly, I haven't been following this issue very closely, but wow, that mailing list post was very persuasive. The analogy between the -mempoolfullrbf flag and a hypothetical -disable_v3_transaction_enforcement flag to me was the most compelling part, as well as the framing:

Relay has only ever been a best-efforts concept, where we carve out a small subset of the entire transaction universe for which we try to optimize propagation.

Adding options for different nodes to see different subsets of the transaction universe seems something we would really want to avoid unless the options provide very compelling benefits. And given limitations of -mempoolfullrbf described in footnote 3, the benefits it provides seem scant compared to the cost and complexity having to design for multiple mempool policies on the same network going forward.

EDIT 2022-11-07: After discussion below I still support main idea behind the PR: that adding mempool options which cause different nodes to have different views of transactions could be bad in the long term for transaction propagation and network reliability and complexity of the mempool implementation. So it would be bad to add options that don't have practical individual use cases and let them accumulate over time.

But I'd also support postponing this PR, and letting people experiment with the option until we can see whether it gains usage or has any practical effect at all. I just think in the long term, we should streamline options that make the network behave less homogeneously and don't have useful local behavior.

ghost commented at 3:32 PM on November 1, 2022: none

NACK

Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?

Zeroconf businesses and projects are vulnerable by design. This can be fixed without changing anything in Bitcoin Core.

Full RBF offers 2 benefits:

Better security for coinjoin and multi party contracts which is hard to fix without users trying full RBF using this option
Provide another policy option for users to replace transactions

Is it reasonable to enforce BIP 125's rbf rules on all transactions, if those rules themselves are not always incentive compatible?

Its reasonable to try and collect some insights based on usage. This could be used to improve things in future.

If someone were to propose a command line option that breaks v3 transaction relay in the future, is there a logical basis for opposing that which is consistent with moving towards fullrbf now?

I do not understand this question, v3 tx relay is still being discussed afaik and not sure how it's related to an option to use full RBF as Gloria mentioned in this email:

"If you don't want your transactions to be subject to these rules, just continue whatever you're doing and don't use nVersion=3."

And some projects would continue to use nVersion=1 and nVersion=2.

Full RBF is already available as an option in Knots and custom bitcoind if someone running core with patches. This option could only make it easier for average users to try it and better insights.

sdaftuar commented at 3:54 PM on November 1, 2022: member

Full RBF offers 2 benefits:

Better security for coinjoin and multi party contracts which is hard to fix without users trying full RBF using this option

As discussed in the mailing list thread, the security problem with a coinjoin protocol/multiparty funded transaction protocol that Antoine described can still occur even if fullrbf were the norm. So I believe this to be false (in the sense that I don't think @ariard or @instagibbs, who seem most familiar with the protocol being described, think the problem would be solved with fullrbf -- if I'm misunderstanding that viewpoint though I would welcome being corrected).

Provide another policy option for users to replace transactions

"Providing another policy option" is a circular reason. The question I'm trying to get at is: what harm is there to the network from some users choosing to opt their transactions out of RBF? The example Antoine gave would be in that category if fullrbf actually solved the problem he described, but since it doesn't seem to solve the problem I'm looking for other examples.

Is it reasonable to enforce BIP 125's rbf rules on all transactions, if those rules themselves are not always incentive compatible?

Its reasonable to try and collect some insights based on usage. This could be used to improve things in future.

We have had 7 years to gain insights based on usage. Something like 20-30% of all transactions opt-in already, I think? I don't think we're going to learn anything more that we can't already learn from looking at current usage patterns.

Perhaps another way of phrasing this argument is, if you're going to advocate for fullrbf because non-replacement is not incentive compatible, then you should also be able to argue that our fullrbf policy is incentive compatible. And I'm pointing out that fullrbf as it exists in master (when this mempoolfullrbf flag is enabled) is not incentive compatible in some obvious situations. So it seems to me that an incentive compatibility argument cuts both ways, and those arguing that non-replacement is bad should implement an incentive compatible set of RBF rules first, before we enforce that on all transactions.

If someone were to propose a command line option that breaks v3 transaction relay in the future, is there a logical basis for opposing that which is consistent with moving towards fullrbf now?

I do not understand this question, v3 tx relay is still being discussed afaik and not sure how it's related to an option to use full RBF as Gloria as mentioned in this email:

If you read my full email and still do not understand my question then I'm not sure I can explain this any better here, but I tried to argue that the restriction on not allowing replacements of non-rbf-signaling transactions is similar in concept to a restriction to not allow arbitrary children of a v3 transaction, in the v3 transaction relay proposal that is being advanced. I like the v3 proposal, and I think we should pursue it, and I worry that setting a precedent of breaking a policy for no network benefit could be used in the future to interfere with v3 down the road, because many of the same arguments would seem to apply.

So I'm looking for people who think that (a) the v3 transaction policy proposal is good and (b) that fullrbf is good to explain how those can be mutually compatible viewpoints, because from how I think about both of these issues, I'm unable to justify supporting fullrbf right now (given my understanding of current usage patterns on the network, where there seems to be little demand for transactions that opt-out of replacement under BIP 125 being doublespent) without also justifying offering knobs to break v3 in the future, which I would be opposed to doing.

instagibbs commented at 4:06 PM on November 1, 2022: member

NACK with longer form here: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021136.html

And I'm pointing out that fullrbf as it exists in master (when this mempoolfullrbf flag is enabled) is not incentive compatible in some obvious situations.

"I want to steal from 0-conf merchants" is the "use-case" you're ignoring, and where the argument falls apart imo. It's the whole reason we're having this debate, it's that merchants are scared about it. If people didn't believe it was incentive compatible, we wouldn't be having this conversation at all.

Ostensibly, we are trying to allow people to bid fees for inclusion to blocks. Excluding people from bidding against themselves is clearly different than V3 type patterns which are for making bidding a robust process.

edit: and for anyone reading I'm completely against stealing from merchants, and personally will not be working towards fullrbf-default-on efforts because of lack of ethical use-cases.

ryanofsky commented at 4:36 PM on November 1, 2022: member

NACK with longer form here: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021136.html

One question I have from that post is about:

Removing a quite-likely-incentive-compatible option from the software just encourages miners to adopt an additional patch if they ever deem it necessary to increase their revenue

To me this is sounds like an argument for removing the option, not an argument against removing the option, because if we see miners actually taking the effort to patch their software in this way it gives us a real empirical confirmation that our mempool policy is not incentive-compatible, and that it needs to change.

In general, if bitcoin core software is designed around following a single mempool policy, and if we can observe whether or not miners are following that policy, it puts us in a solid position to be able to improve the policy over time, and have a network that is reliable and performant without painful edge cases and usability problems. If different mempools have different policies, it seems like it would be a lot harder to evaluate any one policy or for the network to be reliable and improve over time.

instagibbs commented at 4:42 PM on November 1, 2022: member

To me this is sounds like an argument for removing the option, not an argument against removing the option, because if we see miners actually taking the effort to patch their software in this way it gives us a real empirical confirmation that our mempool policy is not incentive-compatible, and that it needs to change.

I don't think we should be encouraging miners to run even more patches to be profitable. That said, I think it's an ok argument for leaving it in as false unless other data comes in. (this is my current stance fwiw)

sdaftuar commented at 8:30 PM on November 1, 2022: member

And I'm pointing out that fullrbf as it exists in master (when this mempoolfullrbf flag is enabled) is not incentive compatible in some obvious situations.

"I want to steal from 0-conf merchants" is the "use-case" you're ignoring, and where the argument falls apart imo. It's the whole reason we're having this debate, it's that merchants are scared about it. If people didn't believe it was incentive compatible, we wouldn't be having this conversation at all.

I completely agree that the fullrbf policy currently in master is incentive compatible enough to help people steal from 0-conf merchants.

The issue with incentive compatibility that I was trying to bring up here is that if you look at our rbf policy as a whole, you'd find problems with its design (namely, the lack of comparison between the ancestor feerate of an incoming transaction, and the actual feerates of all the transactions that would be evicted, can mean that we'd evict transactions that would be mined in the next block in favor of a transaction that may not be mined for a long time). So in that sense, adopting fullrbf is trading one set of incentive compatibility problems for another. It's not like we have a policy A that is so perfectly designed and incentive compatible on the one hand, and this irrational non-replacement policy B on the other hand, and we're just trying to move everyone who is suffering in the backwards B world into the enlightened A world. Instead, A itself has some good things about it, and it has some issues, and B has some good things about it, and some issues of its own.

So if you put aside the question of 0-conf and just tried to look at the policies on their face, it's hard to argue that this is a strict improvement over giving people a choice of which policy regime to opt into.

edit: and for anyone reading I'm completely against stealing from merchants, and personally will not be working towards fullrbf-default-on efforts because of lack of ethical use-cases.

I think this gets at the fundamental issue, which I think we both largely agree on, which is that the main use case of fullrbf is helping users steal from 0-conf merchants. It's not that an opt-in non-replacement policy is fundamentally bad; it's just that businesses offering 0-conf services seem to be offering a reason to subvert the very policy on which they (predominantly) rely.

Ostensibly, we are trying to allow people to bid fees for inclusion to blocks. Excluding people from bidding against themselves is clearly different than V3 type patterns which are for making bidding a robust process.

I just want to make sure that no subtlety in my argument here was overlooked -- I do think that the v3 transaction proposal, as an opt-in set of restrictions on certain transactions to support a particular use case, makes sense. I also think that when applied to the use cases we have in mind, the v3 proposal should make it easier for the best transactions to propagate to miners.

I would also argue that a non-replacement policy, when used correctly, can also help miners get the best transactions, if the transaction participants are better able to coordinate when to use CPFP and when not to. The pinning issues with RBF highlight this issue exactly (wouldn't it make sense for wallets to be more willing to spend unconfirmed outputs from non-signaling incoming transactions and just fee bump them appropriately? with rbf transactions, pinning and the greater chance of having your child transaction becoming invalid make this messier).

My comparison of v3 to non-replacement transactions is in the use cases that we do not foresee, which might in turn create incentives that could be argued would theoretically make the v3 ruleset not incentive compatible. This is essentially how I see 0-conf merchants' use of non-replacement transactions; they have turned a neutral policy into something that theoretically may not be incentive compatible, by creating an incentive to break it.

What I'm arguing though is that we should wait to see evidence that there are actually transactions being broadcast and mined that subvert the policy, and not just postulate that there are reasons for the policy to be subverted and therefore we should move to break it prematurely. This is because there may be countering reasons why the policy isn't being subverted (perhaps because people are mostly honest), and I don't think it's great to break a theoretically useful policy (non-replacement) without a really good reason.

In the analogy to v3 transactions, a similar scenario could occur if lots of non-lightning services start using v3 transactions for reasons that (likely) wouldn't make any sense, creating theoretical demand for more child transactions for fee bumping and access to unconfirmed coins. I would hope that we would try to support the v3 use case by just communicating that this is a bad idea and not the intent of the policy, and wait until we were to actually see evidence of transactions being relayed and mined that violate the policy, rather than offer tools to subvert the policy prematurely just because of a theoretical incentive compatibility problem.

instagibbs commented at 8:58 PM on November 1, 2022: member

I think we're primarily disagreeing on if a knob should exist; I think it's fine and respectful, while picking a "good" default.

DrahtBot commented at 10:50 PM on November 1, 2022: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#26403 ([POLICY] Ephemeral anchors by instagibbs)
#25038 (policy: nVersion=3 and Package RBF by glozow)
#21422 (Add feerate histogram to getmempoolinfo by kiminuo)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

ariard commented at 3:39 AM on November 2, 2022: member

Reading the reasoning on the mailing post, I think there are more long-term implications in the suggestion of adopting the transaction-relay policy of non-interfering use cases to what the network supposes. Especially the concern we might create mining income asymmetries due to unequal access to transactions flows bypassing policies, in a future where fees should substantially contribute to block reward. Beyond, policy design miner-incentive misalignement could create "hidden" security risks for protocols like Lightning, leading to sudden wreckage (e.g miner-bribing contract to ignore version=3 policy). I think a more complete layout of those implications should be pursued on the mailing list. To be clear, I'm not saying a transaction-relay philosophy "to each use-case, a set of policy rules" AFAIU isn't a reasonable path forward, just we should rather understand better the implications.

Answering on the direct questions, as argued multiple times, full-rbf removes one DoS concern for multi-party transactions flows. The solution is satisfying in the sense, if full-rbf is deployed on the majority of the network, a participant to a coinjoin transaction can expect the target transaction, if it offers a better fee in the definition of BIP125 to propagate to the miners. The coinjoin protocol execution doesn't have to halt, and it can keep progressing forward as long as you have fee-bumping reserves available to compete with blockspace demand >

As discussed in the mailing list thread, the security problem with a coinjoin protocol/multiparty funded transaction protocol that Antoine described can still occur even if fullrbf were the norm.

However, reading this statement, I'm not sure if we all think about the same success/failure terms for the security problem raised ? The arguments why the solution is not satisfying are not clear to me.

Another more distant concern (yet to be backed up by more research), full-rbf removes a mass mempool-partitioning vector, where an attacker could partition all the network mempools states and from then alter fee-estimation or leverage this as a building block for advanced pinning (cf. the last attack scenario [0]). Of course, partitioning mempools states due to policy difference is always a potential outcome (e.g Taproot transactions between upgraded and non-upgrade nodes), however I wonder what level of difficulty and cost we should burden on an attacker, or consider this as inevitable.

On the second question, if we had a stronger model of what we mean by incentive-compatible, rather to evaluate any rule on a binary approach (i.e either compatible or not), we could adopt a quantitative approach. Each policy rule impact could be measured in deviation from an "ideal" mining income (as we're doing in cryptanalysis where we deviate from the perfect security of the one-time pad) and to be pondered with other dimensions like privacy or full-node DoS surface increased. If we had a consistent modeling of miner incentive-compatibility that we'll agree on, we could make legible statement if enforcing BIP125 rules on all transactions or not is a improvement. One could argue today that the non-replacement policy regime as expected by zeroconf operators is "incentive-compatible" in the sense it favors Bitcoin adoption as a payment system, as such increase fees volume (all depends how you construct your model, and what variables you select or not).

On the last question, if someone proposes a command line option that breaks v3 transaction relay in the future, I think I would be in favor of such change. I could understand a full-node operator which is not partaking in Lightning neither interested to have compelling fee-estimation, and willing to reduce the CPU DoS surface of a low-performance node. Any transaction-relay rules or mempool acceptance rules, even wisely designed and reviewed, increases the bug surface of a full-node, a risk a node operator could choose to not burden with.

[0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-June/018011.html

petertodd commented at 9:07 AM on November 2, 2022: contributor

Strong NACK

By doing this we're setting an extremely bad precedent that zeroconf "breaking" is our responsibility. Zeroconf is antithetical to the trustless, economic incentive driven, design of Bitcoin. It's a dangerous practice that has repeatedly bitten naive users leading to large losses.

Any option that changes mempool policy breaks zeroconf if used by miners because any difference in mempool policy can be exploited. mempoolfullrbf isn't special and claiming otherwise is dishonest for users.

sdaftuar commented at 1:04 PM on November 2, 2022: member

By doing this we're setting an extremely bad precedent that zeroconf "breaking" is our responsibility. Zeroconf is antithetical to the trustless, economic incentive driven, design of Bitcoin. It's a dangerous practice that has repeatedly bitten naive users leading to large losses. @petertodd I'm not entirely sure I follow your argument, so I'd appreciate a clarification. I understand that you think two things: (a) zeroconf is antithetical to Bitcoin and should be eliminated, and (b) this PR (or the one that introduced -mempoolfullrbf) does not affect the safety of zeroconf in a material way, because it's inherently dangerous.

I think I can accept both of those statements as your position, but if they are both true, why would you care strongly whether this PR is merged or not? Why does anyone need the ability to enable fullrbf in the way that -mempoolfullrbf does, if not to improve the ability to steal from zeroconf merchants? I'd like to understand if there's another use case for the fullrbf flag that I'm missing.

petertodd commented at 2:15 PM on November 2, 2022: contributor

On Wed, Nov 02, 2022 at 06:04:51AM -0700, Suhas Daftuar wrote:

By doing this we're setting an extremely bad precedent that zeroconf "breaking" is our responsibility. Zeroconf is antithetical to the trustless, economic incentive driven, design of Bitcoin. It's a dangerous practice that has repeatedly bitten naive users leading to large losses.

@petertodd I'm not entirely sure I follow your argument, so I'd appreciate a clarification. I understand that you think two things: (a) zeroconf is antithetical to Bitcoin and should be eliminated, and (b) this PR (or the one that introduced -mempoolfullrbf) does not affect the safety of zeroconf in a material way, because it's inherently dangerous.

I think I can accept both of those statements as your position, but if they are both true, why would you care strongly whether this PR is merged or not? Why does anyone need the ability to enable fullrbf in the way that -mempoolfullrbf does, if not to improve the ability to steal from zeroconf merchants? I'd like to understand if there's another case for the fullrbf flag that I'm missing.

First of all, people do on occasion send non-opt-in-signalling transactions that need to be fee-bumped to get mined in a reasonable amount of time. Similarly, it's perfectly valid to try to cancel a transaction sent in error, regardless of opt-in status. That alone is more than enough reason to support full-rbf. The need to signal opt-in status is also of course a privacy harm. Again, that's enough reason to support full-rbf.

More generally, zeroconf is dangerous. Both to users to mistakenly believe it to be secure and get defrauded, and all Bitcoin users who are harmed by attempts to make it secure. What you are doing here with this pull-req is politics: you're trying to take away an option in an attempt to make it inconvenient enough for miners and other users that this dangerous feature continues to live on. And in the process, you're setting precedents that will harm Bitcoin development by constraining what we can do in the future without running into yet more nonsense politics from the people trying to continue to keep zeroconf on life support.

We're lucky that we haven't yet invited legal action from businesses trying to keep zeroconf a thing. We're much better off killing it while we can, at a time when it's hardly used.

ryanofsky commented at 4:02 PM on November 2, 2022: member

I have a question that is making me reconsider my concept ACK for this PR: Do people who support keeping the -mempoolfullrbf option believe keeping it will have a practical effect on network transaction propagation, and that removing it would be a real change? Or would removing the option basically be symbolic, and not likely to have a significant and observable effect on the behavior of network?

I might have gotten the misconception from not reading deeply enough that having this option or not having it would basically make no practical difference for the network, because few people would be interested enough to set the option, and unless a large fraction of nodes did set the option, it wouldn't practically affect what transactions ultimately get propagated and confirmed. But maybe this is not the case, and only a few nodes and miners need to set the option for it to have a real, noticeable effect. If this is the case, I'd rescind my ACK, because I do think miners should have freedom to decide what transactions they want to confirm (whatever their reasoning), or even if they shouldn't have that freedom, trying to remove it by deleting a few lines of mempool code would just be a dumb/heavy-handed/ineffective tactic.

Following this logic, I'd also support having -disable_v3_transaction_enforcement option if we believed enough people might enable that option for it to have a practical effect on the network (good or bad). I don't think bitcoin core should support options that are just symbolic ineffective signaling mechanisms without good use cases. I think if policy options like -mempoolfullrbf or -disable_v3_transaction_enforcement are added that are mostly intended to affect global network behavior rather than local node behavior, they should only be added on a temporary basis, as ways of providing backwards compatibility, or as ways of experimenting and gathering information so default P2P behavior can be more performant and reliable in the future. I think it would be bad thing to let mempool policy options accumulate over time, because it would make network behavior harder to reason about and create corner cases that make it less reliable.

(I am only thinking here about what policy options it is good for bitcoin core software to support, and don't have an informed opinion about whether full RBF is good policy or not. Suhas's arguments that full RBF behavior is actually complicated and hard to reason about, and that RBF signalling can make usage simpler and mempool optimizations possible make sense to me. But ariard's concerns that RBF signalling could create attack vectors and be unsustainable also ring true. Peter Todd's concerns about zero conf may also be reasonable.)

I just think regardless of whether full RBF policy is good or bad, that the -mempoolfullrbf option should have a shelf-life. It does not seem like a good thing in the long term for individual nodes to consciously choose different views of the transaction universe.

petertodd commented at 6:01 PM on November 2, 2022: contributor

mempoolfullrbf=1 by itself won't do much for transaction propagation unless a non-trivial fraction of nodes enable it. In conjunction with addnode however it does make it much easier for people to opt-in to effective full-rbf relay policy for their own nodes, regardless of what % of nodes choose to run it. For example, a friend of mine told me that he was waiting for v24.0 to be released to run full-rbf, because his node was on Umbrel, and installing a custom patch would be annoying; he was going to peer with another full-rbf node to ensure propagation worked.

As for miners, again, if they only use the mempoolfullrbf=1 option it's symbolic until a non-trivial percentage of nodes enable it. But again, by using addnode to connect to other full-rbf nodes, it will get full-rbf replacements mined even with very little hash power running it.

I and others are currently running a dozen or so nodes that have both full-rbf enabled, and preferential peering via a full-rbf service bit. So finding a node to peer with to get tx propagation to work reliably isn't too hard.

tl;dr: if people choose too, they can use mempoolfullrbf to make a genuine difference. If people don't use it, it'll make no difference at all.

On November 2, 2022 5:02:27 PM GMT+01:00, Ryan Ofsky @.***> wrote:

I have a question that is making me reconsider my concept ACK for this PR: Do people who support keeping the -mempoolfullrbf option believe keeping it will have a practical effect on network transaction propagation, and that removing it would be a real change? Or would removing the option basically be symbolic, and not likely to have a significant and observable effect on the behavior of network?

I might have gotten the misconception from not reading deeply enough that having this option or not having it would basically make no practical difference for the network, because few people would be interested enough to set the option, and unless a large fraction of nodes did set the option, it wouldn't practically affect what transactions ultimately get propagated and confirmed. But maybe this is not the case, and only a few nodes and miners need to set the option for it to have a real, noticeable effect. If this is the case, I'd rescind my ACK, because I do think miners should have freedom to decide what transactions they want to confirm (whatever their reasoning), or even if they shouldn't have that freedom, trying to remove it by deleting a few lines of mempool code would just be a dumb/heavy-handed/ineffective tactic.

Following this logic, I'd also support having -disable_v3_transaction_enforcement option if we believed enough people might enable that option for it to have a practical effect on the network (good or bad). I don't think bitcoin core should support options that are just symbolic ineffective signaling mechanisms without good use cases. I think if policy options like -mempoolfullrbf or -disable_v3_transaction_enforcement are added that are mostly intended to affect global network behavior rather than local node behavior, they should only be added on a temporary basis, as ways of providing backwards compatibility, or as ways of experimenting and gathering information so default P2P behavior can be more performant and reliable in the future. I think it would be bad thing to let mempool policy options accumulate over time, because it would make network behavior harder to reason about and create corner cases that make it less reliable.

(I am only thinking here about what policy options it is good for bitcoin core software to support, and don't have an informed opinion about whether full RBF is good policy or not. Suhas's arguments that full RBF behavior is actually complicated and hard to reason about, and that RBF signalling can make usage simpler and mempool optimizations possible make sense to me. But ariard's concerns that RBF signalling could create attack vectors and be unsustainable also ring true. Peter Todd's concerns about zero conf may also be reasonable.)

I just think regardless of whether full RBF policy is good or bad, that the -mempoolfullrbf option should have a shelf-life. It does not seem like a good thing in the long term for individual nodes to consciously choose different views of the transaction universe.

-- Reply to this email directly or view it on GitHub: https://github.com/bitcoin/bitcoin/pull/26438#issuecomment-1300767663 You are receiving this because you were mentioned.

Message ID: @.***>

Sjors commented at 8:03 AM on November 3, 2022: member

If we merge this, anyone who wants to use the feature can do git cherry-pick 3e27e317270fdc2dd02794fea9da016387699636 and compile. So the cat crawled slightly further out of the bag.

I haven't had time to read all the mailinglist discussion. It could make sense to punt this feature, reopen the original PR and merge it later - depending on the outcome of that discussion. Unless there is an urgent reason to have this feature in the current release.

Nit: commit message could point to the previous commit instead of the PR (Github will find the PR for you once you click on the commit).

petertodd commented at 9:42 AM on November 3, 2022: contributor

As I said above:

“For example, a friend of mine told me that he was waiting for v24.0 to be released to run full-rbf, because his node was on Umbrel, and installing a custom patch would be annoying; he was going to peer with another full-rbf node to ensure propagation worked.”

The fact is, we're punting on this in response to a tiny handful of people complaining that they wanted zero conf to continue to work. And the effect of taking away options is to just put up barriers to people who quite rationally, think economically rational mempool policy is preferable over trust.

On November 3, 2022 9:03:38 AM GMT+01:00, Sjors Provoost @.***> wrote:

If we merge this, anyone who wants to use the feature can do git cherry-pick 3e27e317270fdc2dd02794fea9da016387699636 and compile. So the cat crawled slightly further out of the bag.

I haven't had time to read all the mailinglist discussion. It could make sense to punt this feature, reopen the original PR and merge it later - depending on the outcome of that discussion. Unless there is an urgent reason to have this feature in the current release.

Nit: commit message could point to the previous commit instead of the PR (Github will find the PR for you once you click on the commit).

-- Reply to this email directly or view it on GitHub: https://github.com/bitcoin/bitcoin/pull/26438#issuecomment-1301755629 You are receiving this because you were mentioned.

Message ID: @.***>

hugohn commented at 1:35 PM on November 3, 2022: contributor

My two cents:

IMHO, “full RBF” will be inevitable - without BIP125, an explicit flag in Core, or anyone’s blessings.

Once the block subsidy is gone, miners will fight for scraps for survival and do everything they can to get better margins (fees). If Core doesn't explicitly allow full RBF transactions in its mempool, there will be other non-Core mempools or some sort of black market forming.

So either we formalize full RBF, which is preferable, or it will arise out-of-band.

If you want to give zero-confs businesses some grace period, shipping mempoolfullrbf defaulting to false seems reasonable. But removing the option doesn't make sense to me.

Sjors commented at 1:41 PM on November 3, 2022: member

@petertodd but is that an urgent need?

As you point out in your mailinglist post, your friend could run the v24.0rc3 release candidate binary, in case we drop it from the final release. That said, this wouldn't be safe if there are additional bug fixes in future release candidates, and running release candidates in production is not a good idea in general.

The debate around RBF has been going on for almost a decade. I think it can wait a few more months for a point release. We can determine if your bounty program worked out (i.e. created a fait accompli) through something like miningpool.observer.

Sjors commented at 3:35 PM on November 3, 2022: member

ACK 4a4086603e6b548a21c85eb4cad9e4d4e02573fd

(update 2022-11-07: merging this still has my preference, but I don't consider this PR release blocking)

v24.0 release notes will have to be edited (when this is backported)

The term Full RBF is confusing. As @sdaftuar points out in his mailinglist post, -mempoolfullrbf really just means "BIP 125 without the signal". Without also getting rid of other BIP 125 constraints, we leave at least some pinning attacks unaddressed. So it's not a huge improvement for Lightning and other multi-party protocols, not an improvement for those who already signal BIP 125 by default. So imo the change is not useful enough to harm even "a tiny handful of people".

It would be even more confusing to have "full rbf" available now and "maximal rbf" later. Maybe it turns out we can't come up with a "maximal RBF" that is meaningfully more incentive compatible than "full rbf", but I think this should be explored a bit more.

I also agree with @ajtowns that the general issue of divergent mempool policies is worth thinking through more. Since -mempoolfullrbf doesn't seem to solve any urgent problem, it seems reasonable to take more time to discuss things.

ghost commented at 5:32 PM on November 3, 2022: none

We have had 7 years to gain insights based on usage. Something like 20-30% of all transactions opt-in already, I think? I don't think we're going to learn anything more that we can't already learn from looking at current usage patterns. @sdaftuar we have NO insights about usage of full rbf. I remember you suggesting full-rbf a few week backs in IRC. I think @petertodd is right and its just egos and politics else technically its very clear this option will help bitcoin.

sdaftuar commented at 5:57 PM on November 3, 2022: member

First of all, people do on occasion send non-opt-in-signalling transactions that need to be fee-bumped to get mined in a reasonable amount of time. Similarly, it's perfectly valid to try to cancel a transaction sent in error, regardless of opt-in status. [...] The need to signal opt-in status is also of course a privacy harm. @petertodd Thanks for providing these examples; I think they’re worth consideration and reviewers should decide how compelling they are.

What you are doing here with this pull-req is politics: you're trying to take away an option in an attempt to make it inconvenient enough for miners and other users that this dangerous feature continues to live on.

This is either poor phrasing or you have misunderstood my motivation: I have no intent with this PR to keep zeroconf going, and I am not arguing that zeroconf is a good or safe practice for the network. It appears from Antoine’s recent mailing list post that he may also misunderstand me as somehow trying to protect zeroconf merchants or their business practices, so I will try to clarify my views here because that is not my interest at all.

Let me start by saying that I think I mostly share your views that zeroconf – ie selling a product in an irrevocable way prior to a transaction having any confirmations – should not be relied upon for anything of material value. It seems to me that engaging in a practice like that is asking for trouble, and as I have thought about this issue in recent weeks it is unfortunate that zeroconf merchants exist, as their existence appears to be a lightning rod that attracts interest to pushing for fullrbf (which I now view as a protocol development mistake, at least at this time).

Because I share your view, I’m surprised that zeroconf services appear to still be in operation, and it further makes me wonder if there’s anything that we can do at the protocol level that would affect the existence of those operations. Perhaps the services that exist today only engage in these practices in situations where theft could be detected and deterred via off-chain mechanisms (and hence, it may not be “zeroconf” in the sense I defined it above, if there is recourse in the event the transaction doesn’t confirm). If so, then even if such practices are bad for Bitcoin (which I think is hard to say without looking at concrete examples of merchant behavior), they may not go away in a fullrbf world either.

If non-replacement were something that I believed only existed to support zeroconf merchants, I would have stayed out of this debate. However, I realized that what we are actually doing here is taking our opt-in non-replacement policy regime and deciding to get rid of it. And it occurred to me that this is a policy that is independent of zeroconf practices – which are an outside-of-the-protocol merchant behavior, which we may be able to influence but not definitively eliminate – and moreover that non-replacement is actually a useful idea on its face. In fact it’s such a useful idea that I think I’d propose it or something like it in the future, if we were in a fullrbf world with the rbf policies we have today – but not for anything that has to do with zeroconf merchant practices, which seem insecure regardless of whether we have a non-replacement policy.

Since I perhaps didn’t explain this thoroughly in my mailing list post, I will expand on it here. The RBF rules we have today are clumsy, and I have personally had countless discussions over the years with other developers about how to improve on them, primarily due to the pinning issues that exist and the incentive compatibility considerations that come with it. On top of that, transaction chains themselves are hard to reason about in the context of coming up with optimal transaction selection for a miner and preventing mempool DoS in various ways (these are also topics I have substantial experience working on for this project). In thinking about all this, one conclusion I’ve reached is that Bitcoin’s transaction relay protocol and user behavior would likely have been far better from the start if the expectation was that the network wouldn’t generally relay transactions with unconfirmed parents – a position that I think is easy to justify if you look at the various anti-DoS rules we’ve put in place over the last 10 years which have generally served to limit the transactions which get relayed on the network, in comparison to what early versions of the software supported.

From a blank slate/initial policy of never allowing transactions with unconfirmed parents, relaxing that policy in specific situations to support narrow use cases which can be easily reasoned about would have been much better than the free-for-all we have now (where arbitrary transaction chains topologies are permitted, with the only limits being on certain kinds of package size and count).

In such a world, there would be a meaningful difference between transactions that can be replaced via feerate considerations, and transactions that the sender does not intend to replace. In the latter case, CPFP is a far more reasonable strategy than in the former, because the recipient who chains a transaction has greater reason to believe that they are doing something helpful rather than harmful by creating the chained transaction which feebumps the parent (and not just pinning the transaction and irritating the sender, or creating a transaction that will disappear when the parent is replaced and need to be reissued, or creating a DoS vector for the network by relaying something that will need to be evicted later – which is the central issue that creates the pinning consideration in the first place).

Now I get that we aren’t starting from a blank slate; I wanted to take everyone through that thought experiment to try to understand that non-replacement is not intrinsically evil. Note that the CPFP use case I describe is completely benign, provided that the wallet engaging in it can reasonably handle the (likely rare, but certainly possible) situations where the first transaction doesn’t confirm for some reason (eg by being able to reissue a payment if needed).

Maybe you think improved CPFP policy is not compelling; that is fine and we can debate and perhaps disagree. Or, perhaps the CPFP signal can still be in place on a network where the non-replacement is not enforced; that is hard for me to say without a lot more thought.

If you think about the policy regimes we have now or are currently discussing: (1) we have a way to opt-out of RBF (but still have lots of child transactions), (2) we have a way to opt-in to RBF (and still have lots of child transactions), and after the v3 proposal is deployed we will also have (3) a way to be able to opt-in to RBF and have just 1 child transaction (which itself will be limited). In some ways option (2) is the least well-designed option, in that the transaction graph complexities interact with the replacement rules in ways that result in outcomes that are both suboptimal for wallets and also sometimes incentive incompatible for miners, and it would make more sense to have option (2) restrict the child transaction graph further than it does today (but perhaps not so limited as option (3) will be, which is really tailored to a single use-case). If that were a road we go down, then that would make (1) a more compelling option to have available as a policy on the network to facilitate use cases that involve lots of child transactions, and if we eliminate (1) now I fear that any attempt to bring it back in the future will fail due to concerns about zeroconf merchants – something outside our control as protocol developers.

And as I said on the mailing list, I also fear that eliminating (1) for the reasons given could also apply to eliminating (3) in the future. If you look at the reasons you gave above for not having (1) as a policy, I think at least two of them easily apply to the v3 transaction proposal – the consequences of users making mistakes and inadvertently using version 3 transactions when they intended to chain multiple children, and the privacy implications of another distinguished transaction type. Now I think we have good answers to those objections – namely that no one should be using v3 transactions unless they mean to – but if your arguments are found to be compelling in this case, I fear that the v3 policy design may be subverted in the future and the efforts we’re putting in now are a waste of time.

—

Perhaps it would be helpful to reviewers for me to explain what I think some of the best reasons to oppose this PR would be, in the interests of advancing the debate:

Complexity. If it’s more complex for our mempool policy code to support both rbf and non-rbf transactions, then that might be a good reason to merge the two into one regime (this could be either due to code complexity or logical complexity of the policies interact).
If you believe that this PR will be seen as a misguided endorsement of zeroconf then you might find that to be a compelling reason to reject this, even if you understood what I’m trying to actually get at here with my arguments about relay design philosophy. Even though endorsing zeroconf is not my intention, I guess this is probably an issue that would need to be addressed, given the apparent lack of understanding here already and the lengthy explanations I have had to write up!
The level of user/miner interest in having a knob to enable fullrbf may be so high that it’s absurd we don’t make the simple option available given what a simple code change it is to support, and the non-consensus nature of the option. (I think knobs should have intended use cases, so while I’m sympathetic to the principle here, I think weighing this comes down to deciding how important the fullrbf use case is compared to the alternative.)
The relative weighting of the other use cases that have been given for fullrbf, compared with my argument for the CPFP use case being strongest with non-replacement – reviewers can decide for themselves which use cases seem more compelling.

petertodd commented at 7:27 PM on November 3, 2022: contributor

FYI: https://twitter.com/lightcoin/status/1588249269412057088

I am planning to run [full-RBF] with that option enabled once 24.0 is released, assuming that's an option. If it is removed, I may just run the rc with the option enabled until mempoolfullrbf either gets into a main release or I find a reason to switch away from the rc version.

Not the same person as the friend I mentioned above.

jonatack commented at 7:56 PM on November 3, 2022: member

Tentative concept ACK.

Am still forming my opinion on this pull relative to #26287, but given the controversial nature of adding the option, the most prudent course may be not to include it for v24, while continuing discussion for future releases.

ajtowns commented at 7:23 AM on November 4, 2022: member

ACK 4a4086603e6b548a21c85eb4cad9e4d4e02573fd - code review, basic testing

When #25353 was proposed and merged, the rationale provided was that this would aid in "the security of multi-party funded transactions against pinning attacks", and in addition, objections were relatively minimal and non-specific [ref]. But, it turns out that this approach to full RBF isn't sufficient to make multi-party funded transactions secure against the types of pinning attacks. Furthermore specific objections have since been raised [ref] [ref]. If we'd known that this approach didn't work for its intended use case earlier, I don't think #25353 would (or should) have been merged; so until we can fix the code/policy to achieve what it was originally intended to achieve, I think we should revert the change.

BitcoinErrorLog commented at 9:25 AM on November 4, 2022: none

If I understand correctly, the meta topic here seems to be templating for preference/curation/censorship of mempool txns, which templates to provide, and which to have as default.

Replacement is just one option a node might prefer, of, probably, many, and first-seen or any other preference or dropping of txns are no less valid as potential settings. It is up to the individual node to decide.

So, it seems this kind of thing should be part of a superficial config for both nodes and wallets, not something that requires recompiling, as such a setting may need to be changed automatically or responsively.

We should not inject bias for any particular preference as default, but we probably do need to start with some sort of policy, so this should be set to current status quo consensus, not a new agenda for rbf to become a first-class default type.

All of this is not to mention the many things I could say that are great about merchants being able to opt in to 0conf, and that the risks there are currently very manageable and exposure can easily be limited to provide great value to merchants and consumers.

We can have rbf and 0conf coexisting, well, we already do! So let's be thoughtful and address the overall design intelligently and without passively aggressing or deciding for users in conflict with current consensus. Thanks!

Bwahharharrr commented at 10:17 AM on November 4, 2022: none

Strong NACK As someone who has had transactions get stuck before, being able to RBF easily is the best experience for users.

michaelfolkson commented at 10:38 AM on November 4, 2022: contributor

I generally like to leave subtle things to the experts (of which I consider the PR author and some of the ACKers of this PR to be) who have spent more time thinking about optimal default policy for Lightning and Layer 2 protocols longer term.

I'm very uncomfortable though by talk of "divergent mempool policies" and how to prevent them. I'm of the view that if a significant subset of users want to change from default policy there shouldn't be roadblocks (e.g. already merged options removed) constructed to prevent them from doing so. Especially when there doesn't appear to perfect consensus on what the default should be. I get that the PR author doesn't think the current choice presented to users is the optimal one long term.

Despite some opposition (that I personally consider short term thinking) I thought turning this option on by default and considering removing it afterwards was the best long term roadmap. This seems like a last minute screeching u-turn to me but I guess I have to have faith that this truly does hinder plans for the long term v3 transaction proposal. So I guess it is a faith based Concept ACK.

jsarenik commented at 11:11 AM on November 4, 2022: none

Among all these strong and soft feelings, let me say I understand and respect both points of view.

Story: Another thing is that Bitcoin was for me always very peaceful because I could sleep well knowing that when a valid final (non-RBF) transaction is in the network, it will get mined eventually. And I had transactions in the mempool for half a year in 2020, all sent from my own UTXOs and 1sat/vB fee. I do not want to have a future option to take back my word which I gave (to my younger self) in the past. My nodes keep rebroadcasting all transactions from the mempool and have much higher limits, so none of them will be forgotten.

Those transactions which opt-in to RBF can be replaced. That's fine. But that is opt-in. Something I learned is very important for Bitcoin. Making all the wallets opt-in to RBF by default is fine for me as long as I can still turn it off and it will be understood by network.

Opt-in RBF is there already. No coding needed. Possibility to send a non-RBF transaction is there as well. But when it gets removed, I think that sooner or later there may arise a need for this functionality and it will need to be implemented anew.

So my point is we are showing something (some best-practices) to further generations. And despite something is technically possible (the miner can run a full-RBF node which I do not know about for example, but that's fine, it's not what everyone does and what I would expect since my transactions are not sent straight to any particular miner, but rather the peers), e.g. replacing the transaction although it does not opt-in to RBF, it does not have to be advertised. And when someone mentions sybil attacks — you only need one node which tells the truth.

Keep your opinions. Please do not use "strong" words. Let the discussion flow.

Thanks for reading.

As for me and my household, ACK 4a40866 (this was my original thinking)

UPDATE: After reading it all and thinking it through, I have changed my opinion to NACK and setting my nodes to run mempoolfullrbf=1 in order to test if it does not pose any performance issues (old hardware here).

RCasatta commented at 11:42 AM on November 4, 2022: none

NACK

mempoolfullrbf is default off
0-conf is not safe, now we also have alternatives

jwilkins commented at 11:58 AM on November 4, 2022: none

NACK

zeroconf isn't a safe, making it a tiny bit harder to RBF is delusional.

afilini commented at 11:58 AM on November 4, 2022: none

I tend to stay out of the drama, but this PR somewhat affects me personally, so I have to NACK it.

I was also waiting for v24 to be released so that I could enable the option on my node. Removing it would force me to manually patch and compile, which is something I can definitely do, but it's not ideal for me.

I don't think there's really much left to bring to the discussion, but what I'd like to say is that it's baffling to me how many people here are arguing we should just trust miners not to follow their economic incentive just because a BIP somewhere said so. I hope all the people acking here are fully aware of the implications of their arguments, because to me it just doesn't make any sense.

And replying to @jsarenik:

... knowing that when a valid final (non-RBF) transaction is in the network, it will get mined eventually.

This is not true at all. Even if it only spends your inputs it might get dropped from the mempool at some point. And if it comes from somebody else they could always try to replace it and potentially succeed.

phyro commented at 12:16 PM on November 4, 2022: none

Hi, first time commenting here. I'll first admit I have not invested nearly as much time thinking about this as most people here. I respect both views and I think I understand why some people ACK this as it seems the change doesn't solve the problem that was advertised when it was merged. I'd like to express why my general view of this direction is a NACK. As I see it, the root issue isn't the rbf itself, but the merchants/users doing 0-conf transactions. The 0-conf transaction pattern, as I understand it, works under a new assumption that lives on the social layer rather than in the system itself. I don't believe the Bitcoin node software should add layers of protection for the users that are based on social assumptions. A switch to fullrbf may be inevitable because the nature of the protocol is such that double-spending a transaction that's not protected by energy requires no energy cost. We can't change this by hiding the UI buttons/commands or by adding new social assumptions i.e. the person has no communication channel with the miners. Eventually, these communication channels will exist because it will be profitable. As I see it, a transaction floating in the mempool is just a sequence of bits like any other bits on the internet. It's only when these become a part of the ledger that they get any kind of meaningful double-spend protection - until then, they're unsafe by definition. While I see the temptation to remove the UI buttons, I fear the use of such unsafe methods (or rather, relying on unsafe social assumptions) will become more adopted and eventually exploited on a great scale.

That said, it may be a good practice to ACK this and revert the code if the main objective was not fulfilled. Making rbf as simple as possible is in my opinion slightly more important than good practices because it comes with great benefits already mentioned above and, more importantly, it keeps the Bitcoin software free from encouraging the use of unsafe social assumptions. Full-rbf seems like a natural fit for the protocol because it's inline with its objective/trustless nature. Thanks for reading.

TLDR; NACK unless rbf is re-added due to other benefits.

BitcoinErrorLog commented at 12:17 PM on November 4, 2022: none

I don't think there's really much left to bring to the discussion, but what I'd like to say is that it's baffling to me how many people here are arguing we should just trust miners not to follow their economic incentive just because a BIP somewhere said so.

Miner incentive is for total fees per block, not per txn. It is entirely possible that more fees are generated by respecting first-seen (0conf world) than by respecting replacements. Just saying!

petertodd commented at 12:21 PM on November 4, 2022: contributor

On November 4, 2022 1:17:47 PM GMT+01:00, Bitcoin Error Log @.***> wrote:

I don't think there's really much left to bring to the discussion, but what I'd like to say is that it's baffling to me how many people here are arguing we should just trust miners not to follow their economic incentive just because a BIP somewhere said so.

Miner incentive is for total fees per block, not per txn. It is entirely possible that more fees are generated by respecting first-seen (0conf world) than by respecting replacements. Just saying!

If you're so confident of that, why are we removing an option to choose otherwise? Nothing wrong with letting miners decide one way or the other.

I personally have been recently contacted by miners asking how they can turn fullrbf on. Obviously, pointing them to a config option is simplest for them.

afilini commented at 12:22 PM on November 4, 2022: none

Miner incentive is for total fees per block, not per txn. It is entirely possible that more fees are generated by respecting first-seen (0conf world) than by respecting replacements. Just saying!

Absolutely, I'm not saying miners must replace transactions (that would be unreasonable, just like saying that they must not is).

All I'm saying is that I believe miners will replace transactions if it's more convenient for them, and there's not much we can do about it. Surely we can't trusted them not to.

danielabrozzoni commented at 12:22 PM on November 4, 2022: contributor

NACK

removing the mempoolfullrbf option doesn't make accepting unconfirmed transactions safe
some people want to run nodes with mempoolfullrbf=true (such as me), and just removing the option won't stop them from doing so (it's literally a one-line patch...)

BitcoinErrorLog commented at 12:37 PM on November 4, 2022: none

0conf proponents are not saying they are "safe" they are saying they can manage the risk by establishing thresholds to cap their total exposure, and select txns with next-block-viable fee rates. A lot of people really do benefit from this behavior, including Bitcoiners, merchants and miners.

petertodd commented at 1:19 PM on November 4, 2022: contributor

On November 4, 2022 1:37:33 PM GMT+01:00, Bitcoin Error Log @.***> wrote:

0conf proponents are not saying they are "safe" they are saying they can manage the risk by establishing thresholds to cap their total exposure, and select txns with next-block-viable fee rates.

Is part of risk management trying to get incentive compatible, default off, options removed?

There's obviously demand for this option. Seems that the motivation to remove it comes from attempting to make zero conf safer.

greenaddress commented at 1:53 PM on November 4, 2022: contributor

NACK.

I planned to use this feature both personally as well as on production for example on esplora/blockstream.info and Green wallet

As others have said we can also compile Bitcoin core but it would be an inconvenience and in general I think the rbf flag provides a false sense of security especially as we seen recently even non standard transactions can find their ways to miners.

Mostly agree with afilini/ptodd/dbrozzoni's points

litch commented at 1:59 PM on November 4, 2022: contributor

NACK

Transactions in the mempool will be replaced for any number of reasons. Removing this option, and suggesting to users that some transactions won't be replaced by fee creates a false sense of security.

I see that this effort will further the bifurcation between the "people's mempool"* where it's "hard" to RBF, and a "miner's mempool" where RBF is routine. Thus reducing the value of "regular users" having the mempool at all.

Removing this would be strictly a reduction in value for the software I'm running, since it is guaranteed to diverge from what's coming in the next blocks.

I understand that there's no "one mempool", etc, and some unsophisticated miners may run "vanilla core" with the more sophisticated ones running out of band transaction collection mechanisms

ajtowns commented at 2:18 PM on November 4, 2022: member

@greenaddress

I planned to use this feature both personally as well as on production for example on esplora/blockstream.info and Green wallet

For what purpose? I haven't seen an answer to "Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?" yet; does the above imply you have one?

FixTheMoney21 commented at 2:27 PM on November 4, 2022: none

I support this Pull Request!

mariodian commented at 2:29 PM on November 4, 2022: contributor

NACK for the simple reason that 0-confs are insecure and sooner we disincentivize its usage the better. I think it's a disservice to newcomers pretending that unconfirmed transactions are something other than just that - unconfirmed - and I'd like to see that notion gone once and for all. If a vendor is willing to accept the risk that's on them but I'm running mempoolfullrbf on my node (and patching the node even if this PR gets accepted). Everything else has been said already.

Giszmo commented at 2:33 PM on November 4, 2022: none

NACK

I was in the full RBF camp many years ago and am surprised how zero conf did survive this long but almost all agree it will not survive on the long run, so better merge the strong hint at its time being over than to pretend first-seen-safe was still a thing. @petertodd you mention big losses from trust in zero conf. Is there a list of cases somewhere? I think it would help the argument.

Mining pools already have the technical expertise to do full RBF.
Most relays wouldn't enable full RBF until it's on by default. If anything, zero-conf accepting merchants probably would be better off, not worse if relays were all full-RBF. Then they wouldn't need to jump through hoops and loops to detect tx replacement all over the world for their risk assessment.

greenaddress commented at 2:36 PM on November 4, 2022: contributor

@ajtowns

For what purpose?

Multiple purposes I would say. As a wallet sender I want to be able to replace a transaction made in error as well as bump the fee in a congested situation regardless of if the user had the hindsight to set the rbf flag (especially since nodes and miners can ignore said flag, pretty useless)

As a receiver in a wallet I don't want the false sense of security, unconfirmed transactions can't be relied upon until they have a number of confirmations and if there is a transaction replacement with more incentives for miners to mine I want to be able to see it.

For the explorer similar arguments.

Miners can ignore the rbf flag and have economic incentives to do so.

petertodd commented at 2:43 PM on November 4, 2022: contributor

@Giszmo The losses I've been told about were many years ago, and the ATM operators in question didn't want it to be made public (I got the sense that they might not have been entirely honest to investors...). I might be able to dig up the messages in question and ask. But I wouldn't be surprised if this is all long gone.

Anyway, based on the fact that ATM's and in-person exchanges don't offer zeroconf anymore without good AML/KYC (at least in my experience across many countries), I think we can be confident that trying has lead to losses. It's obviously convenient to do so. But the fact is, they don't.

ecdsa commented at 2:46 PM on November 4, 2022: none

Strong NACK

If this is merged, any successful attack on a business/wallet accepting zeroconf will be regarded as a successful attack on Bitcoin itself. Bitcoin developers should not been held responsible for the bad practices of certain actors.

aureleoules commented at 2:52 PM on November 4, 2022: member

NACK on eliminating the option

It has long been understood that the mempool should not be used to confirm transactions, and this is precisely the problem that proof of work in Bitcoin addresses.

Full RBF being enabled by default will make this clear.

Though there seems to be a lot of disagreement regarding full RBF being enabled by default. It appears that businesses that rely on 0conf need more time to adapt, hence I am not in favor of full RBF being enabled by default for v24.

It could be a good idea to leave the option -mempoolfullrbf available for those who want to use it rather than fully eliminating it, and to review the results for v25 to see whether the option should be enabled by default later. Maybe even make it mandatory to choose between -mempoolfullrbf=0 and -mempoolfullrbf=1 with no default value to let the community decide without default bias. Though in practice this would not be practical for the average node operators.

I concur with @petertodd that it's a great feature and enhances user experience and privacy to be able to cancel transactions while in mempool, regardless of opt-in status. Similarly, fee-bumping during mempool congestion is great.

petertodd commented at 2:59 PM on November 4, 2022: contributor

Full RBF being enabled by default will make this clear.

Note that at the moment, the mempoolfullrbf option defaults to off. So all this pull-req would do is to remove the option entirely, for the subset of users who choose to turn it on.

aureleoules commented at 3:02 PM on November 4, 2022: member

Note that at the moment, the mempoolfullrbf option defaults to off. So all this pull-req would do is to remove the option entirely, for the subset of users who choose to turn it on.

Yes, there are a lot of PRs linked to the Full RBF discussion, so I wanted to give my full perspective on it.

petertodd commented at 3:02 PM on November 4, 2022: contributor

Yes, there are a lot of PRs linked to the Full RBF discussion, so I wanted to give my full perspective on it.

No worries! Just wanted to provide context for others.

noobedo commented at 3:07 PM on November 4, 2022: none

NACK We should not incentivize the use of unconfirmed transactions. Having a mempoolfullrbf option is a great feature. We should imo consider making it default. Bitcoin is about censorship resistance and the easier it is to bump fees the more secure is the network imo.

earonesty commented at 3:12 PM on November 4, 2022: none

NACK Miners can do this anyway without permission. Security by "hiding a config option" is not how bitcoin should work. Profit maximization and selfish mining is the very foundation of bitcoin. Let use some logic here. This is an option. It's default OFF, so it doesn't matter anyway. The last thing we want is for it to be normalized that miners run a fork!

sdaftuar commented at 3:31 PM on November 4, 2022: member

I just want to observe that very few of the reviewers on this PR have left comments that are responsive to the arguments I've laid out in my mailing list post or in my responses above. (I do still owe Antoine a response to his review comments, which I plan to get to, but I don't plan to respond to any of the reviewer feedback that ignores my reasoning.)

miladmostavi commented at 3:44 PM on November 4, 2022: none

NACK. Accepting 0conf creates bad incentives and managing the risk becomes impossible if one of the businesses doing it becomes big.

michaelfolkson commented at 3:52 PM on November 4, 2022: contributor

It appears some reviewers/commenters aren't reading the context and earlier comments from the PR author.

e.g.

I have no intent with this PR to keep zeroconf going, and I am not arguing that zeroconf is a good or safe practice for the network.

The PR author is not seeking to support zero conf, it is more subtle than that. There are valid reasons to NACK this PR though (imo) as others have capably explained.

earonesty commented at 3:56 PM on November 4, 2022: none

I think the real problem is the lack of distinction between miners vs routing and "transaction tracking" nodes.

Miners: should use this option. it's in their best interests. the idea of them not doing it is dangerous for bitcoin's stability

Nodes: should probably just route and maintain conflicts (subject to memory limits). No reason otherwise not to maintain conflicts. After all either might be valid. What's the point in a mempool if not to enumerate unconfirmed, but possibly valid transactions?

But since the latter is a hard, unsolved problem, lets leave the option in so miners can do the right thing.

petertodd commented at 4:13 PM on November 4, 2022: contributor

As a wallet sender I want to be able to replace a transaction made in error as well as bump the fee in a congested situation regardless of if the user had the hindsight to set the rbf flag (especially since nodes and miners can ignore said flag, pretty useless)

FYI I just opened #26454 to allow the bumpfee RPC command to work regardless of BIP-125.

petertodd commented at 4:16 PM on November 4, 2022: contributor

@michaelfolkson

The PR author is not seeking to support zero conf, it is more subtle than that.

What the PR author claims is the goal of this pull-req is not relevant. We can all see that removing a disabled-by-default option to do full-rbf is a change that - intent or not - supports zeroconf continuing in the face of significant interest in running full-rbf. The many NACKs this is getting come from people who can plainly see that.

BitcoinErrorLog commented at 4:29 PM on November 4, 2022: none

Strong ACK - now that I understand the implications better, I think the way things are right now already give everyone the tools they need and the highest value Bitcoin outcome. The changes are not worth the compromise. Wallets can turn RBF on by default, miners can select what they want, etc. No need to specifically kill 0conf use cases.

achow101 locked this on Nov 4, 2022

achow101 commented at 4:43 PM on November 4, 2022: member

I'm locking this conversation as "too heated" temporarily. There have been many comments from many users that have not read the motivations and are drive-by commenting that frankly are not adding much to the conversation. It will be unlocked in a few hours.

achow101 unlocked this on Nov 4, 2022

jonatack commented at 11:02 PM on November 4, 2022: member

ACK 4a4086603e6b548a21c85eb4cad9e4d4e02573fd

This option is as-yet released in Bitcoin Core, and given that the current nuance of the discussion had not yet fully taken place when the decision to merge the option was made, the reasonable and prudent course seems to be not to add it for now, while continuing discussion for future releases.

petertodd commented at 11:21 PM on November 4, 2022: contributor

The mempoolfullrbf pull-req was created June 13th, almost five months ago. There's been tons of time to have discussion about an option that isn't even enabled by default.

Shipping such an option simply gives people a chance to use it, and as has been made clear h the discussion above, lots of people intend to do just that. Taking this option away makes it clear that we are so concerned about zeroconf, that we can't even allow people to choose to run fullrbf.

Tell me, what other options do you want us to take away? Because there's a lot of other ones that make it trivial to double spend.

On November 5, 2022 12:02:43 AM GMT+01:00, Jon Atack @.> wrote: ***@.*** commented on this pull request.

ACK 4a4086603e6b548a21c85eb4cad9e4d4e02573fd

This option has never been released, and given that the current nuance of the discussion had not yet fully taken place when the decision to merge the option was made, the reasonable and prudent course seems to be not to add it for now, while continuing discussion for future releases.

-- Reply to this email directly or view it on GitHub: https://github.com/bitcoin/bitcoin/pull/26438#pullrequestreview-1169257069 You are receiving this because you were mentioned.

Message ID: @.***>

jonatack commented at 11:45 PM on November 4, 2022: member

Friendly suggestion from this article:

When you disagree, state your point of view once and move on. Don't flood the comments section, browbeat others or overreact. Be patient, never aggressive, pushy or bullying.

juestr commented at 1:06 AM on November 5, 2022: none

NACK

Removing this default off option is a bigger change than setting it to default on would be, it takes away choice for many users.

It also creates incentives to rely on zero conf. It doesn't take away the capability of criminals to exploit zero conf, and even creates a target rich environment to do that.

It creates incentives to create new authorities which decide which transactions are valid and which aren't ("risk management"). We don't need nor want middlemen to do that.

It's false that this is absolutely required to onboard new users instantly for businesses, a simple (not necessarily advisable) alternative would be transferring secret keys of preloaded UTXOs.

I believe this will make Bitcoin less safe in the long run.

SomberNight commented at 1:38 AM on November 5, 2022: contributor

I have been running patched bitcoinds with a one-line change to enable full-RBF for multiple years. I operate several electrum servers, one of which is very popular and has hundreds of txs broadcast/day for years (actually 8000 tx/day for the past few months, some large service probably started using it to broadcast their txs). All servers are backed by patched bitcoinds (with the rbf opt-out commented out in validation.cpp).

If any reviewers are unconvinced by my rationale, I'd appreciate answers to the questions I laid out at the end of that email:

Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?

The reason I started to patch out the opt-out, is essentially what @greenaddress describes:

Multiple purposes I would say. As a wallet sender I want to be able to replace a transaction made in error as well as bump the fee in a congested situation regardless of if the user had the hindsight to set the rbf flag (especially since nodes and miners can ignore said flag, pretty useless)

Every time there is a mempool congestion for a significant amount of time (several weeks or more), users start flooding the forums/reddit/github/irc asking about stuck transactions. They are told they can use CPFP or RBF, or "transaction accelerators" in the past, or just to wait it out. If there is no change output, they might not be able to CPFP, and they can only RBF bump the fee if opt-in RBF was signalled. We had lots and lots of these "support requests" in the #electrum IRC channel, and on github (also on other platforms but these are where I spend my time). End of 2017, and more recently spring of 2021 there were remarkably long periods of mempool congestions.

We've had many users who created a tx using some other wallet which did not signal RBF, paying a low fee, freaking out about their tx being "stuck", finding guides how to CPFP/RBF in electrum, who then downloaded electrum only to find CPFP needs an is-mine output and RBF requires having opted-in to signalling originally. The only option seemingly left to these users were to wait it out. After experiencing these mempool congestions a few times, I figured I could just patch my node and give users a chance.

In particular, during the spring 2021 mempool congestion I was ready. To the self-selecting few who went to the trouble of coming to IRC and asking why the Electrum GUI does not let them "increase the fee" of their tx, I would explain how to configure Electrum to create a conflicting tx and to connect to my electrum server and broadcast there. At the time that server was using the default maxconnections=125, always saturating it. These days I set higher limits and have more incoming connections. I think I helped four users to attempt an RBF, to either increase the fee or double-spend - merchant invoices have timeouts often not designed with a full mempool and the user lowballing the fee in mind. In two of the four cases, the replacement transaction got mined.

Also during spring 2021, I opened multiple lightning channels and was trying not to overpay in fees. One of my funding txs, which lacked a change output, got "stuck" in the mempool, and it did not signal RBF. I had to double-spend it or risk losing the coins: if too much time passes the remote node deletes the channel and if it gets mined after that, the money can only be recovered using manual cooperation between the channel counterparties. [0] This was around the time https://github.com/lightning/bolts/pull/839 got merged into the BOLTs - not yet implemented on most nodes so I could not even be certain how much time I have. I created a replacement tx to double-spend the funding and broadcast it on the patched electrum server. It worked: one of the replacement txs got mined (I kept bumping the fee).

Overall IIRC my score is 3 out of 5 successful replacements of non-signalling txs. The ones that got mined on average took ~100 blocks - they were paying high next block fees and were at the top of the mempool (at least in the mempools they were accepted into!).

So that's my use case. Helping unlucky users to get their tx confirmed before an invoice expires, or double-spend after the invoice had already expired, or to fix my own mistakes.

Whether the -mempoolfullrbf config parameter is kept or removed, it has little effect on me, the difference is whether or not I have to keep rebasing my trivial one-liner patch on every release. I already updated one of my nodes to run 24.0rc2 and to use -mempoolfullrbf, but I can always just go back to applying a patch.

[0]: (EDIT) to clarify, the first commitment transaction can be used to unilaterally close the channel after the funding tx gets mined, however without anchor outputs that still assumes the feerate had been set to high enough to get mined

petertodd commented at 1:52 AM on November 5, 2022: contributor

@SomberNight Thanks! I think those examples of stuck non-opt-in transactions are very clear use-cases of mempoolfullrbf. While not that common, during high fees this is an obvious lifesaver and we can't assume fees will always be low.

Whether the -mempoolfullrbf config parameter is kept or removed, it has little effect on me, the difference is whether or not I have to keep rebasing my trivial one-liner patch on every release. I already updated one of my nodes to run 24.0rc2 and to use -mempoolfullrbf, but I can always just go back to applying a patch.

I'll also point out that in this type of use-case - a use-case that may happen in an emergency - it's definitely safer when those that need full-rbf can simply enable a flag rather than have to figure out what line to modify and how to recompile.

dr-orlovsky commented at 2:10 AM on November 5, 2022: none

Strong NACK.

Mempool policies are not consensus rules. They can’t even split the p2p network.
Policies must be definable and controllable by node (i.e. full wallet=bitcoin) users.
Devs providing option for user policy control is not “enforcing new rules”, but are just helping the users to execute their right.
Not allowing users executing their rights with Bitcoin Core codebase/binary would provoke appearance of forks (since users will cherry-pick & compile themselves), reducing overall consensus security due to a possible mistakes on the user end

petertodd commented at 2:15 AM on November 5, 2022: contributor

@sdaftuar

What you are doing here with this pull-req is politics: you're trying to take away an option in an attempt to make it inconvenient enough for miners and other users that this dangerous feature continues to live on.

This is either poor phrasing or you have misunderstood my motivation: I have no intent with this PR to keep zeroconf going, and I am not arguing that zeroconf is a good or safe practice for the network. It appears from Antoine’s recent mailing list post that he may also misunderstand me as somehow trying to protect zeroconf merchants or their business practices, so I will try to clarify my views here because that is not my interest at all.

What your intent happens to be is not relevant. What is relevant is the effect of this proposed pull-req. This PR is a step towards keeping zeroconf going. As is the v24.0-only alternative: #26456

The status quo is v24.0 is getting released with mempoolfullrbf. Changing that status quo keeps zeroconf going. Which as I point out in the discussion around #26456, is particularly harmful now that John Carvallo is claiming his company is working to provide new zeroconf acceptance tools.

Since I perhaps didn’t explain this thoroughly in my mailing list post, I will expand on it here. The RBF rules we have today are clumsy, and I have personally had countless discussions over the years with other developers about how to improve on them, primarily due to the pinning issues that exist and the incentive compatibility considerations that come with it. On top of that, transaction chains themselves are hard to reason about in the context of coming up with optimal transaction selection for a miner and preventing mempool DoS in various ways (these are also topics I have substantial experience working on for this project). In thinking about all this, one conclusion I’ve reached is that Bitcoin’s transaction relay protocol and user behavior would likely have been far better from the start if the expectation was that the network wouldn’t generally relay transactions with unconfirmed parents – a position that I think is easy to justify if you look at the various anti-DoS rules we’ve put in place over the last 10 years which have generally served to limit the transactions which get relayed on the network, in comparison to what early versions of the software supported.

None of this discussion is relevant to full-rbf.

From a blank slate/initial policy of never allowing transactions with unconfirmed parents, relaxing that policy in specific situations to support narrow use cases which can be easily reasoned about would have been much better than the free-for-all we have now (where arbitrary transaction chains topologies are permitted, with the only limits being on certain kinds of package size and count).

...and again, none of this is relevant to full-rbf. Indeed, talking about all this as "rules" gives the misleading impression that there are hard and fast rules that govern transaction acceptance. Exactly what got us into trouble with zeroconf in the first place!

In such a world, there would be a meaningful difference between transactions that can be replaced via feerate considerations, and transactions that the sender does not intend to replace. In the latter case, CPFP is a far more reasonable strategy than in the former, because the recipient who chains a transaction has greater reason to believe that they are doing something helpful rather than harmful by creating the chained transaction which feebumps the parent (and not just pinning the transaction and irritating the sender, or creating a transaction that will disappear when the parent is replaced and need to be reissued, or creating a DoS vector for the network by relaying something that will need to be evicted later – which is the central issue that creates the pinning consideration in the first place).

Now I get that we aren’t starting from a blank slate; I wanted to take everyone through that thought experiment to try to understand that non-replacement is not intrinsically evil. Note that the CPFP use case I describe is completely benign, provided that the wallet engaging in it can reasonably handle the (likely rare, but certainly possible) situations where the first transaction doesn’t confirm for some reason (eg by being able to reissue a payment if needed).

Discussions about how edge cases of RBF and CPFP policy are not relevant to full-rbf. Except in the obvious way that full-rbf in the future could simplify some of these interactions by removing a category from the mempool: non-replaceable transactions.

As we've repeatedly discussed, over literally years, non-replaceable transactions certainly can't be relied on as such in normal wallets, so evil or not, they are not a feature that we have any reason to support.

Maybe you think improved CPFP policy is not compelling; that is fine and we can debate and perhaps disagree. Or, perhaps the CPFP signal can still be in place on a network where the non-replacement is not enforced; that is hard for me to say without a lot more thought.

If you think about the policy regimes we have now or are currently discussing: (1) we have a way to opt-out of RBF (but still have lots of child transactions), (2) we have a way to opt-in to RBF (and still have lots of child transactions), and after the v3 proposal is deployed we will also have (3) a way to be able to opt-in to RBF and have just 1 child transaction (which itself will be limited). In some ways option (2) is the least well-designed option, in that the transaction graph complexities interact with the replacement rules in ways that result in outcomes that are both suboptimal for wallets and also sometimes incentive incompatible for miners, and it would make more sense to have option (2) restrict the child transaction graph further than it does today (but perhaps not so limited as option (3) will be, which is really tailored to a single use-case). If that were a road we go down, then that would make (1) a more compelling option to have available as a policy on the network to facilitate use cases that involve lots of child transactions, and if we eliminate (1) now I fear that any attempt to bring it back in the future will fail due to concerns about zeroconf merchants – something outside our control as protocol developers.

...and in all this proposed complexity, it's clear that eliminating the category of non-replaceable transactions would simplify things.

And as I said on the mailing list, I also fear that eliminating (1) for the reasons given could also apply to eliminating (3) in the future. If you look at the reasons you gave above for not having (1) as a policy, I think at least two of them easily apply to the v3 transaction proposal – the consequences of users making mistakes and inadvertently using version 3 transactions when they intended to chain multiple children, and the privacy implications of another distinguished transaction type. Now I think we have good answers to those objections – namely that no one should be using v3 transactions unless they mean to – but if your arguments are found to be compelling in this case, I fear that the v3 policy design may be subverted in the future and the efforts we’re putting in now are a waste of time.

Let's look at a counter-argument: why are we proposing to remove an option to experiment with full-rbf policy, at the same time that we're also trying to design another possibly-incentive-incompatible transaction policy, v3?

Why wouldn't we try to get some data on what incentives actually look like when experimenting with policy is easier? Why are we trying to artificially restrict that process of experimentation?

Perhaps it would be helpful to reviewers for me to explain what I think some of the best reasons to oppose this PR would be, in the interests of advancing the debate:
1. Complexity. If it’s more complex for our mempool policy code to support both rbf and non-rbf transactions, then that might be a good reason to merge the two into one regime (this could be either due to code complexity or logical complexity of the policies interact).

It's clear that full-rbf can make the mempool simpler in the future. One policy vs two.

2. If you believe that this PR will be seen as a misguided endorsement of zeroconf then you might find that to be a compelling reason to reject this, even if you understood what I’m trying to actually get at here with my arguments about relay design philosophy.  Even though endorsing zeroconf is not my intention, I guess this is probably an issue that would need to be addressed, given the apparent lack of understanding here already and the lengthy explanations I have had to write up!

Your arguments around relay design policy aren't relevant: yes, getting incentives right can be tricky. But full-rbf is a very clear case of incentives. And again, why are we diminishing our chances of getting data around incentives by trying to artificially prevent people from experimenting with them?

Remember that we can't do that experimentation on testnet: testnet is worthless. The only place for experimentation with incentives is on mainnet, because incentives have to actually matter to be meaningful.

3. The level of user/miner interest in having a knob to enable fullrbf may be so high that it’s absurd we don’t make the simple option available given what a simple code change it is to support, and the non-consensus nature of the option. (I think knobs should have intended use cases, so while I’m sympathetic to the principle here, I think weighing this comes down to deciding how important the fullrbf use case is compared to the alternative.)

We've got multiple wallet authors who have posted above saying they intend to make use of full-rbf. And there are extremely clear use-cases with recovering from stuck transactions. Which wallet authors have mentioned.

4. The relative weighting of the other use cases that have been given for fullrbf, compared with my argument for the CPFP use case being strongest with non-replacement – reviewers can decide for themselves which use cases seem more compelling.

With the one exception of attempting to categorically prevent a transaction from being mined - the zeroconf "use-case" - having both CPFP and full-RBF simply means there is more than one way to get a transaction mined.

All pinning is about preventing transactions from getting mined at all. Full-RBF clearly can't make that problem worse. And it's clear that while it doesn't entirely prevent it in the multi-party scenario, it gets us closer to a solution.

jaybny commented at 2:17 AM on November 5, 2022: none

im still confused. is this revert to "Remove mempoolfullrbf option" wanting to remove the (-mempoolfullrbf) flag and make it so there is no more fullrbf at all?

so NACK on #25353

and ACK on this.. which reverts 25353

ajtowns commented at 3:33 AM on November 5, 2022: member

@greenaddress

For what purpose?

Multiple purposes I would say. As a wallet sender I want to be able to replace a transaction made in error as well as bump the fee in a congested situation

Those things are much more reliable if you set the rbf flag.

regardless of if the user had the hindsight to set the rbf flag (especially since nodes and miners can ignore said flag, pretty useless)

And that is not reliable unless many nodes and miners support fullrbf, which is not something that a default false option achieves. It's also not relevant currently, as the mempool clears regularly.

This goal can also alternatively be achieved by lowering the mempool expiry time: just as a connected subgraph of fullrbf peers would allow you to broadcast a replacement tx; a connected subgraph of peers with a low expiry time, would also allow you to broadcast a replacement, and would allow you to do so without the other BIP 125 restrictions, such as having to increase the absolute fee, or not being able to conflict more than 100 descendants or not being able to introduce new unconfirmed parents.

As a receiver in a wallet I don't want the false sense of security, unconfirmed transactions can't be relied upon until they have a number of confirmations

It's always been your choice whether you give yourself a false sense of security in that case.

and if there is a transaction replacement with more incentives for miners to mine I want to be able to see it.

And without or without full rbf it's quite easy to provide different nodes with different views of the mempool: simply announce conflicting txs at the same fee rate to different peers. That can be done in advance and used as an input to a transaction that you want to have finer propagation control over. It's also something that appears to be routinely occurring on the network currently.

UkolovaOlga commented at 7:05 AM on November 5, 2022: none

NACK

petertodd commented at 8:01 AM on November 5, 2022: contributor

On November 5, 2022 4:33:21 AM GMT+01:00, Anthony Towns @.> wrote: ***@.***

For what purpose?

Multiple purposes I would say. As a wallet sender I want to be able to replace a transaction made in error as well as bump the fee in a congested situation

Those things are much more reliable if you set the rbf flag.

Since services like Bitrerill treat opt-in-rbf transactions differently, wallets like Green and Electrum have reasons to send transactions without the flag.

Those transactions can still however become stuck. And in that circumstance full-rbf is essential to get them unstuck. @SomberNight already explained this use-case above in detail and they personally would use a mempoolfullrbf flag for it.

regardless of if the user had the hindsight to set the rbf flag (especially since nodes and miners can ignore said flag, pretty useless)

And that is not reliable unless many nodes and miners support full-rbf, which is not something that a default false option achieves. It's also not relevant currently, as the mempool clears regularly.

The mempool is not guaranteed to clear regularly. Mempool pressure can, and does, change all the time. It's quite common for minimum fee OpenTimestamps transactions to take dozens of blocks to get mined due to mempool usage spikes. It is completely reasonable to want this feature for the times when pressure is high.

Node support is not relevant to Green, as they can easily run their own. And of course, with the mempoolfullrbf option we can expect a lot more people to run full-rbf nodes. In testing I've found that there are enough of them out there now that a node with the usual 125 connections has a decent chance of having a full-rbf peer.

This goal can also alternatively be achieved by lowering the mempool expiry time: just as a connected subgraph of fullrbf peers would allow you to broadcast a replacement tx; a connected subgraph of peers with a low expiry time, would also allow you to broadcast a replacement, and would allow you to do so without the other BIP 125 restrictions, such as having to increase thec absolute fee, or not being able to conflict more than 100 descendants or not being able to introduce new unconfirmed parents.

Why are we discussing a convoluted alternative with clearly worse outcomes than just running full-rbf? That's not a like-for-like alternative to what these users want to do.

Is it because you are concerned that providing the option at all would harm zeroconf? Let's be clear here.

and if there is a transaction replacement with more incentives for miners to mine I want to be able to see it.

And without or without full rbf it's quite easy to provide different nodes with different views of the mempool: simply announce conflicting txs at the same fee rate to different peers. That can be done in advance and used as an input to a transaction that you want to have finer propagation control over. It's also something that appears to be routinely occurring on the network currently.

Your point? Why should the existence of another mechanism of double spending discourage the running of full-rbf nodes? Green (and others) have perfectly valid reasons to run them.

Is it because you're trying to protect zeroconf by taking away an option?

prusnak commented at 10:52 AM on November 5, 2022: contributor

NACK

satalik commented at 11:11 AM on November 5, 2022: none

NACK

jaonoctus commented at 11:12 AM on November 5, 2022: none

NACK

willcl-ark commented at 11:15 AM on November 5, 2022: member

My NACK on this comes mainly from a privacy perspective, whereby I do not like having to flag transactions as "special" in any way wherever possible, as its a key tool in wallet/transaction fingerprinting.

However I also strongly believe that nobody should be (or should have been) treating entry to the mempool as a confirmation of a transaction. We have other tools available for these use cases now.

Admittedly I don't run a business which accepts unconfirmed transactions, but I certainly don't accept them as valid for payments received to myself.

blockchainjag commented at 12:24 PM on November 5, 2022: none

NACK

sdaftuar commented at 1:19 PM on November 5, 2022: member

Reading the reasoning on the mailing post, I think there are more long-term implications in the suggestion of adopting the transaction-relay policy of non-interfering use cases to what the network supposes. Especially the concern we might create mining income asymmetries due to unequal access to transactions flows bypassing policies, in a future where fees should substantially contribute to block reward. @ariard I don't understand what you mean here. The philosophy I'm articulating is one where, at least in this software project, we try to keep policy rules in sync with one another, generally only deviating in situations where node resources become an issue (so keeping flags for minimum relay fee, maximum mempool size, and perhaps transaction chain limits are probably fine -- as those only serve to interfere with relay for transactions that would have poor relay propagation properties anyway).

If the policy rules are largely in sync, and the policies are designed to accommodate all the use cases we can come up with, then no one has a strong incentive to relay transactions privately to miners. I think this is the best outcome.

Answering on the direct questions, as argued multiple times, full-rbf removes one DoS concern for multi-party transactions flows. The solution is satisfying in the sense, if full-rbf is deployed on the majority of the network, a participant to a coinjoin transaction can expect the target transaction, if it offers a better fee in the definition of BIP125 to propagate to the miners. The coinjoin protocol execution doesn't have to halt, and it can keep progressing forward as long as you have fee-bumping reserves available to compete with blockspace demand >

Thanks for responding to this point, because I didn't expect after the recent discussion on ways to pin a coinjoin transaction that you would still think fullrbf is a solution. (Furthermore, if we adopted a relay policy like I proposed in #26451, I think the pinning problem would get much, much worse, which further emphasizes the issue that pinning problems are hard to solve.)

For clarity to everyone else, I think it's helpful if we can establish what the coinjoin/multiparty funding transaction model is that you're working with, because it took me a few iterations of offline discussion with @instagibbs to understand the issue. (Also, if my understanding is different than what you're thinking about, this would be a good time to get on the same page.)

I believe the example of the coinjoin protocol you're thinking about is one where anonymous users express interest in a coinjoin, and start with some sort of signing round where they coordinate a transaction with inputs and outputs they each provide. (I'm guessing there are various timeouts and such so that if one party stalls, they get dropped from the transaction, and the rest of the group continues.)

Once the inputs and signatures have been verified to look good to everyone -- perhaps there's a requirement that all the inputs are confirmed, and that all the inputs pass the standardness rules of Bitcoin Core -- then a coordinator will broadcast the transaction on the network. The coordinator wants to be able to do this, and have it succeed, without having to inspect its own mempool, and without connecting to other nodes to see what is in their mempools. As time passes, the coordinator may rbf this transaction to try to increase the likelihood that it confirms (it's not clear to me how this happens, do the other participants pre-sign feebumped versions initially?).

If the transaction becomes conflicted with what is in the chain, then the coordinator's wallet can detect this, determine which party's input was double-spent, and remove that party from the protocol and restart with the remaining users. Eventually, the protocol will terminate when an honest subset of users are able to complete their coinjoin.

The problem arises when a user or group of users (or a user pretending to be a group of users, and able to take multiple inputs of the transaction) can indefinitely stall the protocol, by relaying a low feerate transaction that cannot be feebumped away, and which doesn't confirm on its own for a long time. The question is, how might an adversary achieve this (and at what cost), and what tools are available to protocol designers to prevent it?

The limitation of not inspecting your own or any other node's mempool to see what transactions are relayed on the public network is a significant limitation, and while I can understand why it would be very useful if we could design a protocol in this way, I'm not sure that limitation is one that the network should accept as a reason to enforce a policy on everyone. But putting that aside, the pinning issues that exist even in a fullrbf world would make this protocol vulnerable to stalling all the same, albeit at greater cost than in a non-fullrbf world. The example I gave on the mailing list was that an adversary could pin the coinjoin transaction by relaying a double-spend that was large and had low feerate. If we adopted something like #26451, then this pin would be extremely effective if the adversary chained a single low fee, but high feerate transaction to their double-spend (under the rules I proposed in #26451, the replacement transaction would both need to be high fee to pay for all the fees being removed from the mempool, and high feerate to satisfy the incentive compatibility heuristic - even though the transactions generated by the adversary might not be mined for a long time, because the ancestor feerate of the child might be very low).

I also mentioned on the mailing list that an adversary with multiple inputs into the coinjoin could use the 100 transaction limit to prevent any rbf from occurring at all. The cost of this might not be very high, because the 100 transactions used can all be small.

And finally just recently I thought of a new DoS attack that you could run into, which is that an adversary can use RBF to keep your coinjoin transaction out of the network's mempools, without ever actually having their coinjoin input double-spent in a mined block. Imagine the coinjoin transaction is relayed on the network and in everyone's mempool. The adversary can double-spend their input in a higher fee/higher feerate transaction that includes multiple inputs. Then they double-spend that transaction by conflicting with one of the other inputs that were spending, resulting in a situation where the coinjoin transaction would be eligible for inclusion in node mempools (because there are no longer any conflicting spends of its inputs), but due to relay optimizations no node would reannounce or relay the transaction for a while, at least until the next block is mined. For an adversary that is willing to issue one transaction at the prevailing block fees each block, the coinjoin could be stalled indefinitely with no ability to detect which user is causing the problem, assuming your model is that you don't inspect the mempool.

I don't know if you think that all these attacks are too costly for anyone to bother with, but it seems to me that it would make more sense to iron out the deficiencies in our RBF policies and try to improve them first and decide what level of attacker costs you're concerned with, before asking the network to try to address this use case, since it seems like fullrbf is not going to be enough. Moreover, based on recent mailing list discussion, it seems like the existing coinjoin implementations out there don't even signal for RBF on their transactions and aren't worried about these kinds of DoS vectors at the moment [1] [2].

One concrete problem with deploying fullrbf before these problems are worked out is that I would expect it to be even harder to change the replacement rules we have when more users' transactions would be affected.

Another more distant concern (yet to be backed up by more research), full-rbf removes a mass mempool-partitioning vector, where an attacker could partition all the network mempools states and from then alter fee-estimation or leverage this as a building block for advanced pinning (cf. the last attack scenario [0]). Of course, partitioning mempools states due to policy difference is always a potential outcome (e.g Taproot transactions between upgraded and non-upgrade nodes), however I wonder what level of difficulty and cost we should burden on an attacker, or consider this as inevitable.

I think this is already possible in a fullrbf world, due to the ease with which someone can construct two transactions that are mutually non-replaceable under our current rules (if transaction A has higher feerate than B, but B has greater fees than A, then neither one will displace the other.). Perhaps I don't follow what your security model is for this concern?

On the second question, if we had a stronger model of what we mean by incentive-compatible, rather to evaluate any rule on a binary approach (i.e either compatible or not), we could adopt a quantitative approach.

I completely agree that a problem we are running into is that the notion of what "incentive compatible" means is not well-defined. I have attempted to articulate a philosophy of how I think we can work to meet different use cases that may not be incentive compatible if they were the only deployed policy on the network, but that I think are incentive compatible in the sense that users should have no need to subvert their flaws in the presence of policy options that meet their use case. If people don't think my philosophy works, I'd be interested to hear of other philosophies that are workable to meet use cases on the network. Thus far I'm not aware of any single set of policy rules that we could apply to all transactions that would satisfy the current use cases on the network (and please note that I'm ignoring the idea of "zeroconf" as a use case, which I think is a business practice that is not directly tied to any particular set of relay policies -- I'm thinking instead of CPFP, coinjoins, lightning, fee bumping, etc).

On the last question, if someone proposes a command line option that breaks v3 transaction relay in the future, I think I would be in favor of such change. I could understand a full-node operator which is not partaking in Lightning neither interested to have compelling fee-estimation, and willing to reduce the CPU DoS surface of a low-performance node. Any transaction-relay rules or mempool acceptance rules, even wisely designed and reviewed, increases the bug surface of a full-node, a risk a node operator could choose to not burden with.

I completely disagree with this! If you want to limit CPU DoS surface, don't relay transactions. It would be absurd to me that censoring a type of transaction would be a culture we'd want to promote in our project, for such a flimsy reason.

cryptoquick commented at 2:24 PM on November 5, 2022: none

My NACK on this comes mainly from a privacy perspective, whereby I do not like having to flag transactions as "special" in any way wherever possible, as its a key tool in wallet/transaction fingerprinting.

However I also strongly believe that nobody should be (or should have been) treating entry to the mempool as a confirmation of a transaction. We have other tools available for these use cases now.

Admittedly I don't run a business which accepts unconfirmed transactions, but I certainly don't accept them as valid for payments received to myself.

I agree with all these reasons, and as a node operator, both commercially and personally, I intend to enable mempoolfullrbf on my nodes once bitcoin-core 24.0 is released. I don't want to participate in activity that could be used to deanonymize transactions, and I also don't want to be complicit in theft. Lack of RBF signalling is not a strong enough guarantee a user is incapable of double-spending their coins since it relies upon the node implementation. Since people are doing this anyway with modified nodes, and now that there are good solutions that enable instant payments, I don't see a good reason for this use-case to be relied upon.

For these reasons, you have my NACK.

petertodd commented at 2:27 PM on November 5, 2022: contributor

@sdaftuar

If the policy rules are largely in sync, and the policies are designed to accommodate all the use cases we can come up with, then no one has a strong incentive to relay transactions privately to miners. I think this is the best outcome.

As made clear by multiple wallet authors (Green, Electrum, and Trezor) and others, bumping fees and canceling non-opt-in-rbf transactions is an important usecase. All three of those wallets went to the trouble of NACKing this pull-req. This use-case is so important that one person who spoke up above has even been running a full-rbf patch for years to help users get their transactions unstuck in times of congestion.

That's a very strong incentive to relay transactions privately, and yet your pull-req here aims to put up road-blocks to that use-case.

The problem arises when a user or group of users (or a user pretending to be a group of users, and able to take multiple inputs of the transaction) can indefinitely stall the protocol, by relaying a low feerate transaction that cannot be feebumped away, and which doesn't confirm on its own for a long time. The question is, how might an adversary achieve this (and at what cost), and what tools are available to protocol designers to prevent it?

(snip irrelevant coinjoin discussion)

I don't know if you think that all these attacks are too costly for anyone to bother with, but it seems to me that it would make more sense to iron out the deficiencies in our RBF policies and try to improve them first and decide what level of attacker costs you're concerned with, before asking the network to try to address this use case, since it seems like fullrbf is not going to be enough. Moreover, based on recent mailing list discussion, it seems like the existing coinjoin implementations out there don't even signal for RBF on their transactions and aren't worried about these kinds of DoS vectors at the moment [1] [2].

As per the mailing list discussion, both Wasabi wallet and Joinmarket are well aware of full-rbf. In both cases RBF transactions and pinning are just another kind of DoS attack that they have to deal with - a much more common type of DoS attack is to simply fail to complete the signing round. Both wallets have mechanisms to assign blame, and they expect to have to improve those mechanisms over time for a variety of reasons.

Bringing up Coinjoin in this discussion is irrelevant: full-rbf certainly does not make the problem any worse, as it simply provides one more way that inputs can get mined. Bringing this up as a blocker to merging a default-off full-rbf patch is disingenuous.

Let me repeat that: full-rbf fundamentally can't make pinning worse. Pinning attacks attempt to prevent a transaction from being mined. Full-RBF provides an additional way that a transaction can be mined.

One concrete problem with deploying fullrbf before these problems are worked out is that I would expect it to be even harder to change the replacement rules we have when more users' transactions would be affected.

Full-RBF is identical to our current replacement rules, with the one difference that it applies to more transactions. That's it. There is no way some users/miners choosing to run it will make the pinning problem even worse.

I think this is already possible in a fullrbf world, due to the ease with which someone can construct two transactions that are mutually non-replaceable under our current rules (if transaction A has higher feerate than B, but B has greater fees than A, then neither one will displace the other.). Perhaps I don't follow what your security model is for this concern?

...and again, bringing up this issue is an irrelevant distraction, as full-rbf clearly can't make the situation worse.

I completely agree that a problem we are running into is that the notion of what "incentive compatible" means is not well-defined. I have attempted to articulate a philosophy of how I think we can work to meet different use cases that may not be incentive compatible if they were the only deployed policy on the network, but that I think are incentive compatible in the sense that users should have no need to subvert their flaws in the presence of policy options that meet their use case. If people don't think my philosophy works, I'd be interested to hear of other philosophies that are workable to meet use cases on the network. Thus far I'm not aware of any single set of policy rules that we could apply to all transactions that would satisfy the current use cases on the network (and please note that I'm ignoring the idea of "zeroconf" as a use case, which I think is a business practice that is not directly tied to any particular set of relay policies -- I'm thinking instead of CPFP, coinjoins, lightning, fee bumping, etc).

Talking about perfect notions of incentive compatibility is irrelevant to the discussion at hand. There are always going to be different notions of incentives and different mempool implementations. That's ok. The only users who can't accept diversity in mempool policy are those trying to make zeroconf work.

Meanwhile, by taking away an option, you are restricting the ability of users and miners to experiment and learn about incentive compatible policies in favor of a top-down design approach. Why are you trying to restrict this experimentation?

I completely disagree with this! If you want to limit CPU DoS surface, don't relay transactions. It would be absurd to me that censoring a type of transaction would be a culture we'd want to promote in our project, for such a flimsy reason.

You are trying to censor a type of transaction right now: full-rbf replacements. The status quo for v24.0 would be to provide an option for users and minors to relay/mine those transactions, and you are trying to take that option away.

zndtoshi commented at 2:52 PM on November 5, 2022: none

NACK

Just as I have the option to decide the level of fees of a tx for my node to relay I want to have the option to choose if I want to make my mempool fullRBF or not.

BitcoinErrorLog commented at 3:04 PM on November 5, 2022: none

The decision here is whether to facilitate a change in mempool policy to force txns that did not opt in to RBF to do so. This would change years of Bitcoin standard behavior and impose qualities on people that did not know or choose to opt in to those qualities. It is not Core's place to decide such a change for users.

This will just force us to have to create a RFS respect-first-seen flag, and go in circles. What does Core do when RFS people come in droves to comment on the issue of changing mempool policy to RFS by default? The whole thing is silly, no?

Right now, people that want to RBF already can, and any wallet that wants to offer RBF-by-default as a differentiating service can (and some do). Meanwhile, merchants can accept and limit their risk and provide 0conf acceptance as a service to Bitcoiners that meet next-block-likely-inclusion fee requirements. We have a nice incentive-compatible stasis with maximum utility already. Do not break this.

I do not want to have to lobby for RFS, and I do not want merchants to have to KYC me to provide fast service.

acwildride commented at 3:08 PM on November 5, 2022: none

NACK

Security and freedom matter

juestr commented at 3:28 PM on November 5, 2022: none

This will just force us to have to create a RFS respect-first-seen flag, and go in circles. What does Core do when RFS people come in droves to comment on the issue of changing mempool policy to RFS by default? The whole thing is silly, no?

I wouldn't even mind a (full-)RFS flag which effectively disables opt-in RBF, but I doubt you'd find many nodes running that.

prusnak commented at 3:31 PM on November 5, 2022: contributor

RFS flag makes no sense because there is no way to guarantee that the transaction will be seen "first" by all nodes in the network. RFS is just an illusion.

juestr commented at 3:38 PM on November 5, 2022: none

RFS flag makes no sense because there is no way to guarantee that the transaction will be seen "first" by all nodes in the network. RFS is just an illusion.

Right, full-RFS isn't going to work (save 0conf), but if node owners really want that local policy option I see little harm in it.

michaelfolkson commented at 3:41 PM on November 5, 2022: contributor

The decision here is whether to facilitate a change in mempool policy to force txns that did not opt in to RBF to do so. This would change years of Bitcoin standard behavior and impose qualities on people that did not know or choose to opt in to those qualities.

You can repeat this statement if and when the mempool policy default is attempted to be changed in Core. Giving full nodes an option to change it (when some users are already doing this) without changing the default is not imposing anything. Leave it to the maintainers now please, I'll support whatever they do for this release. We can rehash all these same arguments when there are attempts to actually change the mempool policy default which even this PR author has said he would like to do.

taoeffect commented at 3:52 PM on November 5, 2022: none

Hi @sdaftuar!

Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?

Yes, it offers many benefits, and without it Bitcoin won't work properly! Users will lose $!

See here for details: https://fixingtao.com/2016/03/bitcoin-keep-calm-and-rbfcpfp-on/

Retailers should not be relying on 0-conf, or L1 at all. Please use Lightning Network (or similar L2) instead.

(NACK this proposal)

ghost commented at 4:31 PM on November 5, 2022: none

Leave it to the maintainers now please, I'll support whatever they do for this release.

This is not Bitcoin

ghost commented at 4:40 PM on November 5, 2022: none

Posting this on behalf of Tony:

"NACK.

I appreciate all the considerations laid out in the mailing list post but I don't see it as a reason to revert the move toward progress that has already been in motion with the RC.

However, it seems to me that similar problems exist for such a protocol
even in a fullrbf world...

Just because other problems and DoS vectors exist with Lightning and transaction pining, doesn't mean that we can't try to remove them one step at a time. I think you make a great argument about why it helps but doesn't stop it. Are you proposing that we delay until we fix those problems, which you later make the argument that we'll never really fix because user policies can always disable things like transaction v3? It gets to the point where you're arguing that policies don't really matter in the end and that any effort toward progress is pointless. To be fair, I don't know if transaction v3 is good enough to solve our problems or not and I think your thought experiments around -disable_v3_transaction_enforcement is interesting and should be a consideration as devs make progress towards it but I feel like it's a different issue that full RBF shouldn't be pinned to as a reason for not going forward with it.

Does fullrbf offer any benefits other than breaking zeroconf business
practices?  If so, what are they?

You can't break what is already broken. These were terrible assumptions to begin with. If you want instant transactions with delayed settlement, there's a thing called Lightning.

To start with, let's chat about why this doesn't change anything for them. If their arguments for why it's necessary revolve around the fact that they can do proper risk assessment and manage it appropriately, that doesn't change. Let them continue to do their risk assessment. There's still going to be a % of users that may decide to double spend given X amount of sats.

Secondly, I find the original motivation for this change coming from Muun to be overstated and a use case that should not be advocated for. There's a difference between a merchant delivering a product on 0 conf and Muun providing 0 conf submarine swaps with the illusion that they are atomic. They are not and they are lying to their users, there's nothing atomic about them when it's 0 conf. Furthermore, there are many ways that their model is both unsustainable and insecure. There are plenty of ways to steal money from them regardless of 0 conf. I bring it up as an example of why users and use cases may be wrong in general, so why not rip the bandaid off now? You agree it should happen eventually, why not happen when it's at the top of everyone's minds and also to prevent other businesses in the future from falling trap to the illusions given by non-RBF and non-RBF companies?"

https://stacker.news/items/89456

luke-jr commented at 7:58 PM on November 5, 2022: member

Concept NACK.

Your reasoning assumes mempool policy is a global decision made by the Bitcoin Core developers. This premise is incorrect. It is a per-node decision, subject to the whims of the owner of the individual node. While we don't have to actively support all options, there is no justification for denying users the right to use a policy when the development team is willing to maintain that support.

All arguments against full RBF are worth consideration only by those end users and when developers decide if we want to spend time maintaining/supporting the policy. To an extent, it may make sense to influence the default policy setting as well. But it is not an excuse to obstruct wilful developers from providing wilful users with a policy they wish to use.

Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?

Exposing reality for what it is, at least. Unconfirmed transactions are no safer than RBF-enabled transactions, and the false idea that they are is apparently leading to businesses to harm not only themselves, but also users. For example, it was related to me that users paying for a product with a RBF-signalling transaction, are defrauded of their product, and have to jump through hoops to even get their bitcoins returned to them.

(I'm assuming others have already offered more obvious use cases)

Is it reasonable to enforce BIP 125's rbf rules on all transactions, if those rules themselves are not always incentive compatible?

To begin with, "incentive compatible" is just one ideology/policy, not the only one. Most nodes gain nothing from fees, so have no direct incentive to favour higher fees in the first place. Some developers attempting to force only a limited ideology on Bitcoin Core is part of the project's problems, and not a valid justification.

If someone were to propose a command line option that breaks v3 transaction relay in the future, is there a logical basis for opposing that which is consistent with moving towards fullrbf now?

Not comparable at all.

mjdietzx commented at 9:39 PM on November 5, 2022: contributor

NACK

Enough people obviously want this option, and they can always run something other than bitcoin core if we don't give it to them. I would be strongly against changing the default of -mempoolfullrbf to true, but that is not what the commit you're reverting does. I think your reasoning @sdaftuar provided in bitcoin-dev misses the crux of this issue - mempool policy is up to each individual full node. I am very inline w/ @luke-jr's comment above.

As a general principle I am very in favor of Linux/Torvalds "WE DO NOT BREAK USERSPACE" golden rule. And I would hate to see anything that breaks zerconf as long as it is being used (regardless of how I or anyone else feels about it). So if that's how you feel it is your responsibility to campaign with full node operators to not take the time/effort to exercise this option (which is already their right, with or without this option being formalized).

ghost commented at 11:04 PM on November 5, 2022: none

or this is just politics

That is the whole point .

Nothing should be easier to include in Bitcoin Core nor remove important things that are being being included and reviewed.

Everything else is politics

jaybny commented at 11:37 PM on November 5, 2022: none

@ryanofsky #26438 (comment) thanks for this. is clears up a lot

I think if this option flag has not been released yet, then it should not be released. because of the can of worms with competing options. -disable_v3_transaction_enforcement @ryanofsky

if removing this option is symbolic, the option itself is also symbolic @ryanofsky

going farther, those power-users that would set this flag, already have this code merged into bitcoin core and can copy/fork maintain their own special full rbf implementation. pretty straight forward for power users.

Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?

still no good response here.

breaking external (zeroconf) assumptions based on core devs assumptions as to what are best practices is not a reason to push code changes through bitcoin core

can see a better future solutions here with signaling

ACK
remove it

ghost commented at 11:46 PM on November 5, 2022: none

if removing this option is symbolic

nothing in this software is symbolic, almost everything works and has a function

this has become political now

going farther, those power-users that would set this flag, already have this code merged into bitcoin core and can copy/fork maintain their own special full rbf implementation. pretty straight forward for power users.

nobody is saving power users and devs as this option is already available in knots

there is no flag as well

sdaftuar commented at 12:07 AM on November 6, 2022: member

Multiple purposes I would say. As a wallet sender I want to be able to replace a transaction made in error as well as bump the fee in a congested situation regardless of if the user had the hindsight to set the rbf flag (especially since nodes and miners can ignore said flag, pretty useless) @greenaddress I'm not familiar with your wallet/service -- is there a reason that transactions your software generates (or that you generate as a user) don't always opt-in to RBF? If there's an issue with why that can't be a default to avoid these kinds of problems, I'd be interested to hear it. @SomberNight Same question for you, I'm not familiar with what wallets the users you're referring to might be using; is there an issue with having opt-in be the default for these wallets?

petertodd commented at 12:49 AM on November 6, 2022: contributor

On November 5, 2022 4:04:46 PM GMT+01:00, Bitcoin Error Log @.***> wrote:

The decision here is whether to facilitate a change in mempool policy to force txns that did not opt in to RBF to do so. This would change years of Bitcoin standard behavior and impose qualities on people that did not know or choose to opt in to those qualities. It is not Core's place to decide such a change for users.

Core is not deciding such a change for users. Core is allowing users to choose such a change.

This will just force us to have to create a RFS respect-first-seen flag, and go in circles. What does Core do when RFS people come in droves to comment on the issue of changing mempool policy to RFS by default? The whole thing is silly, no?

We ignore them, because what they're asking for would be foolish.

...and with a RFS flag, some merchants would demand it be set, those transactions would get stuck on occasion, and there would be reasons to override the flag.

The fact is in a decentralized system like Bitcoin, unconfirmed transactions are second class citizens and no amount of hope can change that.

What's silly is trying to force the issue. Doubly silly when we're trying to force it by removing options!

Right now, people that want to RBF already can, and any wallet that wants to offer RBF-by-default as a differentiating service can (and some do). Meanwhile, merchants can accept and limit their risk and provide 0conf acceptance as a service to Bitcoiners that meet next-block-likely-inclusion fee requirements. We have a nice incentive-compatible stasis with maximum utility already. Do not break this.

Multiple wallet authors have spoken up on this topic. The fact is for good user experience, the ability to double spend unconfirmed transactions of all types is important.

Meanwhile, in the real world, the vast majority of Bitcoin users don't even try to rely on zeroconf because attempting to do so results in getting defrauded.

If by some dark magic zeroconf somehow worked reliably, merchants certainly would accept it widely. They don't, because decentralized wallets certainly can't safely rely on zeroconf.

sdaftuar commented at 1:02 AM on November 6, 2022: member

@sdaftuar

What you are doing here with this pull-req is politics: you're trying to take away an option in an attempt to make it inconvenient enough for miners and other users that this dangerous feature continues to live on.

This is either poor phrasing or you have misunderstood my motivation: I have no intent with this PR to keep zeroconf going, and I am not arguing that zeroconf is a good or safe practice for the network. It appears from Antoine’s recent mailing list post that he may also misunderstand me as somehow trying to protect zeroconf merchants or their business practices, so I will try to clarify my views here because that is not my interest at all.

What your intent happens to be is not relevant. What is relevant is the effect of this proposed pull-req. This PR is a step towards keeping zeroconf going. As is the v24.0-only alternative: #26456 @petertodd Please -- if you read the quoted text, I am responding to a claim that you made about my intent. If my intent doesn't matter, then don't assign intent to me, especially if it's wrong. If my response to your comment is irrelevant, then perhaps you should rethink what you post in this forum, because you're wasting our time. (Perhaps you could go further and apologize for bringing my intent into this, when as you say, it doesn't seem to matter?)

None of this discussion is relevant to full-rbf.

It absolutely is relevant to consider the ways in which "fullrbf" is incentive incompatible. Can anyone commenting on this PR even describe a set of policy rules that would implement fullrbf which are always incentive compatible? (If so, someone should open a PR -- I attempted to yesterday and then discovered a mistake; these things are hard.) If you're going to suggest that we apply rules to all transactions that we didn't previously apply for the sake of incentive compatibility, it's worth understanding what that really means.

(Maybe that is not precisely your argument, but I think that idea captures the vast majority of how people reading this thread think about "fullrbf" in the abstract, and things are far more subtle than that.)

Let's look at a counter-argument: why are we proposing to remove an option to experiment with full-rbf policy, at the same time that we're also trying to design another possibly-incentive-incompatible transaction policy, v3?

There are two sets of options going on here that are counter to each other. I am arguing for giving Bitcoin users more options, by trying to establish that neutral policies that give them more choices for how their transactions are handled should not be subverted in our software. You are trying to give node operators more choices that could lead to subverting those policies, removing use cases from our software for users.

I think my view of the world is more consistent with making the Bitcoin network work well, not just in this case but for future policy work (such as v3 transactions).

Why wouldn't we try to get some data on what incentives actually look like when experimenting with policy is easier? Why are we trying to artificially restrict that process of experimentation?

We could also give everyone a knob to control every policy rule we have, and I think that is foolish. A principle that I have heard articulated by others that I agree with is that we should only offer a knob/command line option if we can also advise users when they might want to use it. Which is why I have been consistently asking for the use case of fullrbf, given that we already have opt-in rbf. It seems that the main use case for fullrbf that you and I both agree on is users making mistakes, perhaps from using wallet software that doesn't opt-in, and then regretting it later when they can't feebump/replace their transaction. (There is also the privacy angle that you mentioned, but I think that argument is weak, as wallets have a coordination problem for how they set these fields in a transaction regardless of whether opt-in is a policy or not, and I don't think the opt-in flag meaningfully changes that.)

While I agree that such mistakes are bad, they seem avoidable and I think it's a weak rationale; the whole point of opt-in RBF is to give users a way to avoid the situation.

Your repeated comments about the irrelevancy of what I'm saying makes me think you're not really interested in engaging with my thinking. If you disagree with that assessment and are willing to spend the time, I'd ask you to try to steelman the argument you think I'm making and summarize it here, so that I can assess whether we're getting anywhere or, indeed, this is a waste of time for both of us to continue discussing.

For my part, I'm open to being wrong about this -- I'm specifically looking for a development philosophy that is consistent with both moving towards fullrbf that is consistent with where I see future policy development going, such as for the v3 case that I've brought up several times. I've yet to see such a philosophy articulated in a way that I can understand, and almost every commenter has ignored this line of thinking, which all makes me believe that no one thus far is thinking about this adequately.

Alternatively, if you have a philosophy that would suggest that all the work we're doing on package rbf/package relay/v3 transactions is a big waste of time, then that would be helpful to hear as well, because maybe all that work is misguided and we should give up coordinating policy on a decentralized network. (It would be good think about the implications of this for layer 2 projects like lightning.) If you predict, for instance, that user mistakes and privacy concerns in the future will mean that people will work to subvert alternate policies like v3 once they're deployed, it'd be better to establish that now and save a lot of dev work.

All pinning is about preventing transactions from getting mined at all. Full-RBF clearly can't make that problem worse. And it's clear that while it doesn't entirely prevent it in the multi-party scenario, it gets us closer to a solution.

Pinning arises in two ways: an adversary trying to abuse our rules, or a coordination problem between sender and recipient. I believe the latter is far more common than the former, as people have been complaining about pinning since we deployed opt-in RBF. Offering tools to help prevent pinning with better coordination seems obviously useful.

SomberNight commented at 1:34 AM on November 6, 2022: contributor

@sdaftuar

Same question for you, I'm not familiar with what wallets the users you're referring to might be using; is there an issue with having opt-in be the default for these wallets?

Well, ideally all wallets should be defaulting to RBF having opted in, sure, but they don't. I expect there would always be wallets that not only not default to RBF but don't even expose an option to set it.

I don't know what weird wallets people are using these days, but I distinctly remember e.g. users of the blockchain.info webwallet complaining about not signalling (and then finding Electrum, restoring from seed and trying to RBF) - but I don't even know if blockchain.info still has a wallet these days. The Electrum wallet has defaulted to creating RBF transactions for many years, and had options to replace txs for even longer, so there are lots of guides that come up in search engines that people find in times of mempool congestion.

Also, BitPay in particular, but IIRC maybe other merchants too, have been actively telling users to disable RBF in their wallets (to facilitate zeroconf). Just now I've tried donating to a service that uses BitPay using Electrum and got the following "warning" on the website: bitpay ("Show me how" links to this) If a user disables RBF, then it stays disabled, unless they remember to re-enable it after -- although I guess Electrum could be more aggressive and make the setting per-transaction. I guess same might apply to other wallets (?).

is there a reason that transactions your software generates (or that you generate as a user) don't always opt-in to RBF? If there's an issue with why that can't be a default to avoid these kinds of problems, I'd be interested to hear it.

I guess only somewhat related but still might be interesting to some: Electrum never opts-in to RBF for lightning funding txs (see https://github.com/spesmilo/electrum/issues/7072). This is specifically to make it very difficult for users to destructively RBF the funding tx. Double-spend-to-self would be fine but "increase fee" would burn the coins: the 2of2 multisig output would stay but the txid would change, so the funding outpoint (so the channel id) would change, leaving the other node not finding the channel and deleting it. The wallet opening the channel ofc knows it is a funding tx and use that knowledge, but if a user restored from seed and bumped the fee on a different wallet, well that would be problematic (note that channel v2 has the same issue). So we don't opt-in to RBF for funding txs, and the GUI only offers bumping the fee if RBF is signalled. (and as I explained in my previous comment, I personally was sort of burned by this once, when I was trying to RBF-cancel a funding tx)

greenaddress commented at 1:36 AM on November 6, 2022: contributor

@sdaftuar

From a UX prospective asking users if they want replaceability or not is not very good. Damned if it's always on and damned if it isn't. All for a flag that is a gimmick that can be bypassed at will.

We experimented with a few options and generally users are confused with replaceability and it makes no sense to confuse or complicate users life. At the moment it is always on but we were recently planning to bring back the option in advanced settings. Still not very useful for your average users.

Ultimately with or without the flag transactions can be replaced. The only difference with this PR being merged or not is users being forced to use knots or a release candidate (possibly with bugs) or self compile.

Either way, all our green and electrum/esplora/blocksyream.info servers were planned to run with the flag on as of release. Luckily we already self compile so it's only limited extra effort and extra delay on our side for each release but that is not true of all users. And even for us the extra pain and delay in adopting releases is not ideal.

It seems a few electrum servers are already using this policy and we are likely to attempt direct peering.

Other than that, I tend to agree with @petertodd explaining above

TonyGiorgio commented at 2:08 AM on November 6, 2022: none

NACK.

I appreciate all the considerations laid out in the mailing list post but I don't see it as a reason to revert the move toward progress that has already been in motion with the RC.

However, it seems to me that similar problems exist for such a protocol even in a fullrbf world...

Just because other problems and DoS vectors exist with Lightning and transaction pining, doesn't mean that we can't try to remove them one step at a time. I think you make a great argument about why it helps but doesn't stop it. Are you proposing that we delay until we fix those problems, which you later make the argument that we'll never really fix because user policies can always disable things like transaction v3? It gets to the point where you're arguing that policies don't really matter in the end and that any effort toward progress is pointless. To be fair, I don't know if transaction v3 is good enough to solve our problems or not and I think your thought experiments around -disable_v3_transaction_enforcement is interesting and should be a consideration as devs make progress towards it but I feel like it's a different issue that full RBF shouldn't be pinned to as a reason for not going forward with it.

Does fullrbf offer any benefits other than breaking zeroconf business practices? If so, what are they?

You can't break what is already broken. These were terrible assumptions to begin with. If you want instant transactions with delayed settlement, there's a thing called Lightning.

To start with, let's chat about why this doesn't change anything for them. If their arguments for why it's necessary revolve around the fact that they can do proper risk assessment and manage it appropriately, that doesn't change. Let them continue to do their risk assessment. There's still going to be a % of users that may decide to double spend given X amount of sats.

Secondly, I find the original motivation for this change coming from Muun to be overstated and a use case that should not be advocated for. There's a difference between a merchant delivering a product on 0 conf and Muun providing 0 conf submarine swaps with the illusion that they are atomic. They are not and they are lying to their users, there's nothing atomic about them when it's 0 conf. Furthermore, there are many ways that their model is both unsustainable and insecure. There are plenty of ways to steal money from them regardless of 0 conf. I bring it up as an example of why users and use cases may be wrong in general, so why not rip the bandaid off now? You agree it should happen eventually, why not happen when it's at the top of everyone's minds and also to prevent other businesses in the future from falling trap to the illusions given by non-RBF and non-RBF companies?

BitcoinErrorLog commented at 9:02 AM on November 6, 2022: none

Our wallet app has RBF off by default because we believe mobile users are more likely to be spending than fee-minimizing or unsticking txns.

We even have a plan for turning on RBF by default if we can succeed with extending BIP21 in a way that merchants can include their required feerate, rbfoff, and invoice timeout in the URI. This would allow our app to turn off RBF only when merchants specify to... but none of this will work if mempools force optin rbf.

This really is something totally manageable at the app layer with a first-seen mempool policy! Bitcoin is just better and can do more if we leave the current mempool culture intact. If you want to join our effort to update BIP21, everyone is welcome.

Lightning also benefits from 0conf and first-seen as it allows wallets/LSPs to offer 0conf spending in new channels, and seamless UX once splicing is in place.

We should not promote features that impose new txn qualities onto users that did not opt in to those qualities. Some people here even admit wanting to kill first-seen because it makes people "face reality" and prevents them from accepting 0conf. Hurting a useful mutual use case is an insane goal.

If all txns were always easy to doublespend, and merchants are delusional, then RBF would not exist in the first place!

Merchant adoption is an important part of Bitcoin economy and growth, removing 0conf viability incentivizes more forms of KYC and more reliance on payment processors, increasing costs, censorship, centralization, and reducing privacy.

If your rbf experience is not ideal, choose a better wallet, lobby your wallet to change, or improve your wallet UX like we do. Do not impose it on the whole network (as only ~15% of fullrbf pools would effect into 100% rbf behavior) by modifying Core tools.

gruve-p commented at 10:57 AM on November 6, 2022: contributor

NACK

Same reasons as stated on #26287 (comment)

sdaftuar commented at 12:46 PM on November 6, 2022: member

@SomberNight @greenaddress Thanks for the additional information. One question -- what does BitPay do if you send the transaction with rbf enabled anyway? Do they just wait for it to confirm and then everything works, or does something more terrible happen?

A couple things occur to me:

It's surprising to me that some lightning-enabled wallets have found a use for non-replacement (avoiding that footgun with LN funding transactions).
The "damned if you do, damned if you don't" comment relating to opt-in RBF makes it seem like there is at least some demand for non-RBF transactions. Otherwise there would be no problem with defaulting everyone in to RBF and not worry about exposing an option to disable it, right? Is the only demand for non-RBF transactions coming from people who run into things like the bitpay notification and then try to disable it in their wallet, or do users have other reasons as well?

It seems to me like there are two ways of looking at this: removing the ability to opt-out of RBF would be some simplification, because it would be a force towards eliminating a distinction that is confusing to users. On the other hand, the existence of wallets that do not signal opt-in RBF in all situations implies some benefits to a multi-policy regime, particularly if we're seeing those behaviors in wallets that aren't just legacy software from pre-2016.

Also it's worth pointing out that it's not entirely clear the opt-in "signal" would go away if we get rid of non-replacement. Signaling is just a convention, and imparting meaning to it even in the absence of a validation rule enforcing it is something that wallets will be free to do. Given the history of BIP 125 signaling, and the current wallet software on the network, I'd think that it's entirely possible a new standard might emerge where wallets may still signal for non-replacement intent, particularly because older wallets presumably are coded up to not try to double-spend BIP 125 signaling transactions and it will require deploying a software change to enable such behavior. This might have the benefit of continuing to provide footgun relief for the LN commitment transaction case, for instance (the signal becomes a signal that is internal to the wallet, maybe that is the best outcome?) -- but of course could also be used by the likes of bitpay. Were this to happen, I think this type of outcome would come at the expense of becoming an even more confusing situation for users trying to understand the options and what they really mean.

I don't know the best way to weigh these concerns. My thought would have been that by now, modern wallets (certainly ones developed in the last few years, such as lightning wallets) would have just opted all transactions in to RBF to achieve largely the same effect as deploying fullrbf on the network for their own users. If there are reasons for that to not happen -- seemingly because non-replaceability has some benefits -- I don't think we should ignore that.

I also think the simplicity from having a single policy regime is a serious consideration and should not be dismissed. While I still tend to lean towards a multiple-policy approach, I can see how others (particularly those with user-facing applications) might disagree.

All in all, while I think there's merit to the argument in favor of a fullrbf deployment to simplify how users understand transaction types on the network, I think it still raises a few questions that we should try answer:

Are the RBF rules we have so complex that it will create even more questions for users? (As one example, will intentional pinning become a use case for anyone? That seems awful for the network, but I would imagine it could arise if there's a use case to prevent replacements, whether for UX reasons or economic incentive reasons or otherwise.)
Will modifying our RBF rules to try to be more incentive compatible for miners, eliminate DoS risks or pinning, etc be more difficult if all users' transactions are subject to those rules, and more software is dependent on the particular policy rules we currently have in place? For instance, I've previously held off on advocating for changes like #26451 because I imagined they would be quite disruptive -- am I mistaken? If it is disruptive, that might imply that future improvements should instead be made in an opt-in basis only, which would be contradictory to what we're doing now.
What implications does this line of reasoning (about simplicity of our ruleset) have on other work that is going on, such as package relay/package rbf/v3 transactions?

petertodd commented at 2:08 PM on November 6, 2022: contributor

What your intent happens to be is not relevant. What is relevant is the effect of this proposed pull-req. This PR is a step towards keeping zeroconf going. As is the v24.0-only alternative: #26456

@petertodd Please -- if you read the quoted text, I am responding to a claim that you made about my intent. If my intent doesn't matter, then don't assign intent to me, especially if it's wrong. If my response to your comment is irrelevant, then perhaps you should rethink what you post in this forum, because you're wasting our time. (Perhaps you could go further and apologize for bringing my intent into this, when as you say, it doesn't seem to matter?)

Sorry, I had misremembered who I had written that in response too. Anyway, my point is still correct: regardless of what your intent is, the impact is the same regardless of what you intend to do.

None of this discussion is relevant to full-rbf.

It absolutely is relevant to consider the ways in which "fullrbf" is incentive incompatible. Can anyone commenting on this PR even describe a set of policy rules that would implement fullrbf which are always incentive compatible? (If so, someone should open a PR -- I attempted to yesterday and then discovered a mistake; these things are hard.) If you're going to suggest that we apply rules to all transactions that we didn't previously apply for the sake of incentive compatibility, it's worth understanding what that really means.

Opt-in-RBF was incentive compatible enough to merge into Bitcoin. It's exactly the same thing as Full-RBF, with one constraint added to it: opt-in. That constraint makes it obviously less incentive compatible for the obvious reason that sometimes high-fee-paying transactions exist that could be mined with full-rbf.

Whether or not full-rbf is "incentive incompatible" in some hypothetical situations is entirely irrelevant to this discussion! It's fine to bring this up when discussing how we're going to modify replace-by-fee behaviors in the future. But you bringing up here is just a distraction from the actual debate at hand.

(Maybe that is not precisely your argument, but I think that idea captures the vast majority of how people reading this thread think about "fullrbf" in the abstract, and things are far more subtle than that.)

You keep trying to argue that this is a subtle issue. It is not. The subtleties around replace-by-fee are irrelevant to the discussion about the opt-in flag.

There are two sets of options going on here that are counter to each other. I am arguing for giving Bitcoin users more options, by trying to establish that neutral policies that give them more choices for how their transactions are handled should not be subverted in our software. You are trying to give node operators more choices that could lead to subverting those policies, removing use cases from our software for users.

(emphasis mine) You are not arguing for giving Bitcoin users more options. You are trying to take away an option that many people wish to use. Some who have wished to use it so much that they've been running patched versions of Bitcoin Core for years.

I think my view of the world is more consistent with making the Bitcoin network work well, not just in this case but for future policy work (such as v3 transactions).

Above we have three different wallet authors with years of experience running wallets that have NACKed this pull-req, and they have made clear that there are numerous problems introduced by non-rbf transactions that full-rbf helps fix for them. Your view of the world is clearly not widely supported by actual wallet authors.

Why wouldn't we try to get some data on what incentives actually look like when experimenting with policy is easier? Why are we trying to artificially restrict that process of experimentation?

We could also give everyone a knob to control every policy rule we have, and I think that is foolish.

We do give everyone policy knobs for tons of policies, including ones that are much less commonly relevant to wallets than this one like the datacarrier and datacarriersize, the dust relay fee, four different options for descendant limits, etc.

Why is mempoolfullrbf - a rare knob with clear use-cases outlined by three different wallet authors special in this regard?

Which is why I have been consistently asking for the use case of fullrbf, given that we already have opt-in rbf. It seems that the main use case for fullrbf that you and I both agree on is users making mistakes, perhaps from using wallet software that doesn't opt-in, and then regretting it later when they can't feebump/replace their transaction.

You have clear use-cases for full-rbf from three different wallet authors. One use-case so clear that someone above mentioned how they are already running a full-rbf patch to help get transactions unstuck. It's obvious that being able to double spend and cancel all types of transactions is extremely useful, given that there is no guarantee that a low-fee transation will ever be mined.

(There is also the privacy angle that you mentioned, but I think that argument is weak, as wallets have a coordination problem for how they set these fields in a transaction regardless of whether opt-in is a policy or not, and I don't think the opt-in flag meaningfully changes that.)

Why are we harming everyone's privacy for a use-case that literally just two wallets spoke up in support of, one of which effectively argued that replacement should be removed entirely?

While I agree that such mistakes are bad, they seem avoidable and I think it's a weak rationale; the whole point of opt-in RBF is to give users a way to avoid the situation.

...and as multiple wallet authors told you, as long as opt-in RBF is a second class citizen we have to deal with those mistakes.

Furthermore, let's be clear: there is a significant component of the community that looks at that discrimination and thinks it's good to end it entirely by simply adopting full-rbf widely. Your pull-req is trying to take an option away from that part of the community.

Why are you so interested in denying them the ability to choose?

For my part, I'm open to being wrong about this -- I'm specifically looking for a development philosophy that is consistent with both moving towards fullrbf that is consistent with where I see future policy development going, such as for the v3 case that I've brought up several times. I've yet to see such a philosophy articulated in a way that I can understand, and almost every commenter has ignored this line of thinking, which all makes me believe that no one thus far is thinking about this adequately.

The v3 use-case may not be incentive compatible. That's ok. We can better have that discussion by learning more about full-rbf in practice.

v3 is not set in stone at this point, and there's probably lots of ways to achieve that goal. You seem to be dead set on v3 existing, so much that you're trying to take options away from people that may show that v3 as currently described isn't a good idea.

Alternatively, if you have a philosophy that would suggest that all the work we're doing on package rbf/package relay/v3 transactions is a big waste of time, then that would be helpful to hear as well, because maybe all that work is misguided and we should give up coordinating policy on a decentralized network. (It would be good think about the implications of this for layer 2 projects like lightning.) If you predict, for instance, that user mistakes and privacy concerns in the future will mean that people will work to subvert alternate policies like v3 once they're deployed, it'd be better to establish that now and save a lot of dev work.

Indeed, it would be better to establish that now! I highly doubt that all work on package relay is pointless, as it obviously is a useful idea regardless of whether or not there is an RBF flag. But yes, the v3 rules might not work, and full-rbf might prove that to us. Better to find out now.

Conversely, v3 - unlike zeroconf - does not depend on everyone following the same set rigid of rules. So from that perspective, it's obvious that v3 may be able to work in situations where zeroconf does not.

All pinning is about preventing transactions from getting mined at all. Full-RBF clearly can't make that problem worse. And it's clear that while it doesn't entirely prevent it in the multi-party scenario, it gets us closer to a solution.

Pinning arises in two ways: an adversary trying to abuse our rules, or a coordination problem between sender and recipient. I believe the latter is far more common than the former, as people have been complaining about pinning since we deployed opt-in RBF. Offering tools to help prevent pinning with better coordination seems obviously useful.

You still haven't explained how offering more ways to get a transaction mined will make pinning worse.

Equally, if - as an example - everyone switched to full-rbf tomorrow, we'd be in exactly the same situation as we are now re: pinning as opt-in-rbf and full-rbf are identical except for that flag. (relevant: it's impossible to spend certain types of outputs without enabling the opt-in-rbf flag, as we overloading the meaning of nSequence)

petertodd commented at 2:16 PM on November 6, 2022: contributor

@sdaftuar This is a very weak argument:

It's surprising to me that some lightning-enabled wallets have found a use for non-replacement (avoiding that footgun with LN funding transactions).

LN is filled with footguns around state. I don't think one LN wallet using or not using the opt-in flag means much. I just checked and all my lightning channels on my LND nodes were opened with opt-in-rbf.

Clearly the better way to fix this footgun is to fix the underlying problem of making sure it's possible to properly cancel/feebump a LN funding tx, as we fundamentally can't prevent users who go to the trouble of importing a seed in a different wallet (!!) to try to cancel a transaction, from getting their transaction broadcast.

Full-RBF nodes exist. We can't stop that now.

petertodd commented at 2:26 PM on November 6, 2022: contributor

@BitcoinErrorLog

We even have a plan for turning on RBF by default if we can succeed with extending BIP21 in a way that merchants can include their required feerate, rbfoff, and invoice timeout in the URI. This would allow our app to turn off RBF only when merchants specify to... but none of this will work if mempools force optin rbf.

The authors of Green, Electrum, and Trezor all NACKed this pull-req for the obvious reason that no amount of "required feerate", etc. can change the fact that transactions get stuck sometimes due to high demand and need to be feebumped/cancelled. Full-RBF is the only reasonable way to do this. Which is exactly why one of the people who NACKed this pull-req has been running full-rbf nodes for years, to help people get their transactions unstuck.

...and let's be clear: you're supporting this pull-req because you have a future product that will rely on zeroconf. Pulling fullrbf from v24.0 will simply delay this discussion even further, creating a bigger mess of a debate in the future when you have yet another dangerous service trying to risk-manage zeroconf. Most likely a failing service because the idea is inherently unreliable.

promag commented at 2:44 PM on November 6, 2022: member

NACK, I share the opinions of @petertodd and @luke-jr.

jaybny commented at 3:29 PM on November 6, 2022: none

using terms like "dangerous service" or "failing" or "unreliable", makes this whole thing less credible. I'm calling bullshit.

zeroconf is just a valid Bitcoin transaction with double-spend risk.. many times this risk is easily mitigated.

bitcoin core should not be providing behavior to replace users valid transactions with a double spend unless they opted in.

ACK revert

On Sun, Nov 6, 2022, 6:26 AM Peter Todd @.***> wrote:.

...and let's be clear: you're supporting this pull-req because you have a future product that will rely on zeroconf. Pulling fullrbf from v24.0 will simply delay this discussion even further, creating a bigger mess of a debate in the future when you have yet another dangerous service trying to risk-manage zeroconf. Most likely a failing service because the idea is inherently unreliable.

— Reply to this email directly, view it on GitHub https://github.com/bitcoin/bitcoin/pull/26438#issuecomment-1304813875, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5Y3VPUYQWFWK2ISVZWQTWG6WZ7ANCNFSM6AAAAAARUCUSLI . You are receiving this because you commented. Message ID: @.***>

juestr commented at 4:58 PM on November 6, 2022: none

bitcoin core should not be providing behavior to replace users valid transactions with a double spend unless they opted in.

Are you saying bitcoin core should not provide a means to help users with stuck transactions? Should wallets refuse to import seeds if they detect a stuck tx, or even more drastic steps?

luke-jr commented at 5:26 PM on November 6, 2022: member

We even have a plan for turning on RBF by default if we can succeed with extending BIP21 in a way that merchants can include their required feerate, rbfoff, and invoice timeout in the URI. This would allow our app to turn off RBF only when merchants specify to... but none of this will work if mempools force optin rbf.

This just reaffirms the need for full RBF to put an end to this absurd and irrational discrimination. There is no justification for telling users they have to set the "RBF" bit off. If there was, it would be part of the address format already.

If all txns were always easy to doublespend, and merchants are delusional, then RBF would not exist in the first place!

It's fraudulent doublespending that is easy, when it is known in advance you want to double spend - even if the merchant cannot see that it was planned in advance.

What's difficult/unreliable is double spending after the fact, due to unforeseeable circumstances (like fee bumping, adding additional recipients, etc). The honest use cases.

petertodd commented at 5:58 PM on November 6, 2022: contributor

On November 6, 2022 11:29:59 AM AST, Jay Berg @.***> wrote:

using terms like "dangerous service" or "failing" or "unreliable", makes

this whole thing less credible. I'm calling bullshit.

On a slightly different version of this same pull-req, @harding said the following:

I think a warning that future releases may include a mempoolfullrbf option, along with a reminder that zeroconf is unsafe, sends a clear message that nobody should be relying on it.

#26456 (comment)

Do you disagree with him?

zeroconf is just a valid Bitcoin transaction with double-spend risk.. many

times this risk is easily mitigated.

@harding Seems to me that we do have users who think zeroconf isn't a big deal.

SkanderHelali commented at 6:36 PM on November 6, 2022: none

NACK.

0-conf transactions are notifications of a valid transaction that is in the mempool, nothing more. Even the naming is disingenuous, it isn't "zero conf" - it's unconfirmed. Period. If you think the fantasy of 'managing' 0-conf risk is something that can be accomplished then continue to do so at your own peril. Miner incentive is to include the highest fee paying tx.

Rambling aside, NACK because this change takes away user choice and would imply/affirm that '0-conf' is expected/dependable behavior for merchants.

taoeffect commented at 8:27 PM on November 6, 2022: none

Rambling aside, NACK because this change takes away user choice and would imply/affirm that '0-conf' is expected/dependable behavior for merchants.

It doesn't just take away user choice - it puts them in danger.

This is not a user experience:

Otherwise, you may just have to wait either until the transaction confirms or until the bitcoins reappear in your wallet. It’s important to note that until a transaction confirms, the bitcoins are technically still in your wallet — it’s just that it often doesn’t appear that way. The bitcoins are not literally “stuck” on the network and cannot get lost.

It is a user nightmare.

Let me quote that one more time:

you may just have to wait either until the transaction confirms or until the bitcoins reappear in your wallet.

Which is it?

Will you get your bitcoins back or will they confirm? And when will they know?

I have not seen or found the answer to these questions anywhere.

Bitcoin cannot not have an answer to that. Any payment or settlement network that has this type of behavior is broken.

RBF should be enabled by default to prevent this brokenness - the exact opposite of this proposal.

taoeffect commented at 9:00 PM on November 6, 2022: none

RBF should be enabled by default to prevent this brokenness - the exact opposite of this proposal.

Adding default RBF wouldn't be enough though. Users who don't use the feature (for whatever reason) would still have uncertainty about whether or not their transaction would go through, and when.

To fix this, transactions should have a max-age in the mempool. It's my understanding that currently such a thing doesn't exist. It should be added via a BIP. I'd create the BIP myself if I had time to do so. Hopefully someone else can do it.

taoeffect commented at 9:12 PM on November 6, 2022: none

Sorry for the multiple comments (this should be my last comment on the subject): according to this link there's a 14 day max-age for the mempool.

However, it's unclear whether this is a globally enforced value or a miner-specific value. It needs to be globally enforced so that some type of guarantee can be given to users, and transactions themselves should specify the max-age.

In other words, clients should allow users to tweak the max-age of their transactions, up to the global value that miners enforce. And it'd be cool if merchants can also suggest values themselves to clients, so that a ~~store~~ seller could tell ~~shoppers~~ buyers, "we'll wait up to 24hours for your txn to confirm". EDIT: but stores should still use LN instead. 😁

sipa commented at 9:14 PM on November 6, 2022: member

Nothing can prevent someone from rebroadcasting a valid, expired transaction, as long as its inputs are unspent. The mempool expiration policy is necessarily local, based on when a node first saw the transaction.

petertodd commented at 9:15 PM on November 6, 2022: contributor

On November 6, 2022 5:00:16 PM AST, Greg Slepak @.***> wrote:

RBF should be enabled by default to prevent this brokenness - the exact opposite of this proposal.

Adding default RBF wouldn't be enough though. Users who don't use the feature (for whatever reason) would still have uncertainty about whether or not their transaction would go through, and when.

To fix this, transactions should have a max-age in the mempool. It's my understanding that currently such a thing doesn't exist. It should be added via a BIP. I'd create the BIP myself if I had time to do so. Hopefully someone else can do it.

Transactions already expire from the mempool. But that doesn't prevent them from being readded, and anyone can re-add a transaction. And trying to keep a list of evicted transactions would be a DoS vector...

Hence why the supermajority support full-rbf. By far the simplest and cleanest solution to these issues.

taoeffect commented at 9:16 PM on November 6, 2022: none

Reply to both @sipa & @petertodd: if the txn includes a date in itself (set by the client), that cannot be rebroadcast past the expiry, since the date is absolute time, and signed.

sipa commented at 9:19 PM on November 6, 2022: member

@taoeffect That creates a trivial reorg risk, where transactions included just before their expiration times can arbitrarily becomes invalid without any double-spending, but just due to randomness, if they get reorged out. That implies that transaction receivers need to start doing risk analysis based on how close a transaction is to its expiration when confirmed.

Transaction validity is, by design, monotonous.

I'd suggest staying on topic, and not trying to do armchain protocol development in the comments of an unrelated PR.

petertodd commented at 9:19 PM on November 6, 2022: contributor

On November 6, 2022 5:17:02 PM AST, Greg Slepak @.***> wrote:

Reply to both @sipa & @petertodd: if the txn includes a date in itself (set by the client), that cannot be rebroadcast past the expiry, since the date is absolute time, and signed.

A transaction expiry date causes problems with reorgs if consensus enforced; if not, not enforceable. Getting a fee bump or cancel tx mined is the only reliable solution, and that needs full-rbf.

Anyway, this discussion is both very stale - we've discussed these issues ad-nauseum over the years - and off topic for this pull-req.

taoeffect commented at 9:24 PM on November 6, 2022: none

Ok, fair point, you're right this is off-topic. I don't understand @sipa's reply about re-orgs, but I get that this isn't the place to hash that out, my bad!

SomberNight commented at 9:57 PM on November 6, 2022: contributor

@sdaftuar

One question -- what does BitPay do if you send the transaction with rbf enabled anyway? Do they just wait for it to confirm and then everything works, or does something more terrible happen?

They also show you on the web interface what minimum feerate you should use. If you send a tx that does not signal bip125 and pays at least the given feerate, they accept it as zeroconf. Otherwise they wait for one confirmation.

jaybny commented at 11:49 PM on November 6, 2022: none

What? why would you not be allowed to start core with signed transactions in the mempool? what?

what is a "stuck transaction?" - who says it's "stuck"? There are many solutions to deal with spending zero-conf for badly written wallets - or finality /time issues.

"stuck transactions" is part of the scaling problem. users can opt-in for the complexity protection.

think about money. im saying a double spend is a spend of a utxo output that is already spent. and we don't replace spends with double spends. the exact opposite. We race the spends into the block and start mining to beat the fraud into the block. if its my own incoming utxo, i dont care about the relative fee. there are many users of bitcoin core. some actual end users of bitcoin.

On Sun, Nov 6, 2022 at 8:58 AM Jürgen Strobel @.***> wrote:

bitcoin core should not be providing behavior to replace users valid transactions with a double spend unless they opted in.

Are you saying bitcoin core should not provide a means to help users with stuck transactions? Should wallets refuse to import seeds if they detect a stuck tx, or even more drastic steps?

— Reply to this email directly, view it on GitHub https://github.com/bitcoin/bitcoin/pull/26438#issuecomment-1304844878, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5Y3U2OA7ODLOVJVTEY6TWG7PSTANCNFSM6AAAAAARUCUSLI . You are receiving this because you commented.Message ID: @.***>

ajtowns commented at 1:36 AM on November 7, 2022: member

This goal can also alternatively be achieved by lowering the mempool expiry time: just as a connected subgraph of fullrbf peers would allow you to broadcast a replacement tx; a connected subgraph of peers with a low expiry time, would also allow you to broadcast a replacement, and would allow you to do so without the other BIP 125 restrictions, such as having to increase the absolute fee, or not being able to conflict more than 100 descendants or not being able to introduce new unconfirmed parents.

Why are we discussing a convoluted alternative with clearly worse outcomes than just running full-rbf? That's not a like-for-like alternative to what these users want to do.

In a technical discussion, you discuss possible alternatives so that you better understand both the goal that people are trying to achieve, and the different tradeoffs different approaches offer.

An obvious tradeoff between bip125-rbf whether signalled or not and shorter expiry, is that shorter expiry allows a much broader range of replacement options (ie, anything that would violate rules 2-5 despite being more attractive to miners) at a cost of replacement still not being available for some days. When opt-in RBF was first supported, the mempool expiry time was 3 days; it was changed two releases later to 2 weeks.

Is it because you are concerned that providing the option at all would harm zeroconf? Let's be clear here.

I don't think questioning contributors' motives is a constructive use of github, and nor is brigading of this PR with content free NACKs or ACKs. The thing we should be being clear about here is the technical consequences of proposed changes, and those approaches aren't conducive to improved technical understanding. If you don't want to have a technical conversation, there are plenty of other more suitable places. Of course, that's just my view; if the folks merging PRs do somehow consider this useful information, that's a different matter.

jaybny commented at 1:53 AM on November 7, 2022: none

On Sun, Nov 6, 2022 at 9:59 AM Peter Todd @.***> wrote:

On a slightly different version of this same pull-req, @harding said the following:

I think a warning that future releases may include a mempoolfullrbf option, along with a reminder that zeroconf is unsafe, sends a clear message that nobody should be relying on it.

#26456 (comment)

Do you disagree with him?

yes. I disagree with him. zeroconf is not safe nor unsafe. its just zeroconf. how is it unsafe? because it can fall out of mempool? so can a 1conf.

Message ID: @.***>

ariard commented at 3:17 AM on November 7, 2022: member

@ariard I don't understand what you mean here. The philosophy I'm articulating is one where, at least in this software project, we try to keep policy rules in sync with one another, generally only deviating in situations where node resources become an issue (so keeping flags for minimum relay fee, maximum mempool size, and perhaps transaction chain limits are probably fine -- as those only serve to interfere with relay for transactions that would have poor relay propagation properties anyway). @sdaftuar To give illustrations to the aforementioned intuition of "transactions flows bypassing policies", let's say you have a non replaceable batch transaction paying to N third-party outputs and 1 change output. The transaction issuer opted out from replacement according to BIP 125 rules, as such allowing chain of transactions with some degree of reliability. After few minutes of propagation, the issuer observes a sudden change in Lightning public channels congestion, realizing it would be more economically profitable to replace the batch transaction with a new version of the batch, splitting the change output in M channel funding outputs to earn LN routing fees. This new version won't propagate on the p2p network due to the previous version opting out from BIP 125 replacement, therefore the batch issuer calls on private transaction-relay communication channels directly towards miners offering this service, attaching a compelling mining fee to evict out potential chain of transaction (where the overhead mining fee is still under the expected gain of LN routing fees).

This is a first illustration. A second one coming to my mind, current MAX_OP_RETURN_RELAY payload allows 83 bytes. OpenTimestamps-like, timestamping proof use-cases could target to experiment commitment with accumulator proof membership more efficient than the one proposed with Merkle Tree, i.e with a lengthier root state, while still conserving the publicity property of the chain. Those use-cases would again calls on private-transaction-relay communication channels directly towards miners known accepting non-bounded OP_RETURN payload, attaching a compelling mining fee, or offering an adequate out-of-band fees.

Those two illustrations aim to make the point that some policy rules design could create viable incentive to relay transactions privately to miners, and as such generate long-term asymmetries between miners. I still wonder if the mining asymmetries we're creating are only present in the case when we're introducing special replacement/relay regime (e.g BIP125 or nVersion=3) or also when we're allowing operators to manage node resources (e.g minimum relay fee, maximum mempool size), if such conceptual distinction holds. I don't think this is because we wouldn't be able to design policies to accommodate all the use cases we can come up with, rather the fact that some Bitcoin transaction issuers might have incentives (defined differently than miners ones) to bypass policy rules.

Thanks for responding to this point, because I didn't expect after the recent discussion on ways to pin a coinjoin transaction that you would still think fullrbf is a solution. (Furthermore, if we adopted a relay policy like I proposed in #26451, I think the pinning problem would get much, much worse, which further emphasizes the issue that pinning problems are hard to solve.)

To bring clarity, I never expected fullrbf to be a complete solution to the pinning of coinjoin transaction, as I noted in my initial fullrbf proposal a year ago [0]. I rather saw it as one step in a broader systematic elimination of pinning vectors (BIP125 rule 3, BIP125 rule 5) annoying coinjoin and contracting protocols. After the recent discussions, sounds we need more thoughts about our transaction-relay philosophy, and if crafting policy to each use-case is a more sustainable paradigm.

For clarity to everyone else, I think it's helpful if we can establish what the coinjoin/multiparty funding transaction model is that you're working with, because it took me a few iterations of offline dis> cussion with @instagibbs to understand the issue. (Also, if my understanding is different than what you're thinking about, this would be a good time to get on the same page.)

The model I've in mind is the interactive construction protocol as specified out by the Lightning development community: https://github.com/lightning/bolts/pull/851. It can be used as a building block for p2p coinjoin, Lightning dual-funded/splicing.

I believe the example of the coinjoin protocol you're thinking about is one where anonymous users express interest in a coinjoin, and start with some sort of signing round where they coordinate a transaction with inputs and outputs they each provide. (I'm guessing there are various timeouts and such so that if one party stalls, they get dropped from the transaction, and the rest of the group continues.)

Once the inputs and signatures have been verified to look good to everyone -- perhaps there's a requirement that all the inputs are confirmed, and that all the inputs pass the standardness rules of Bitcoin Core -- then a coordinator will broadcast the transaction on the network. The coordinator wants to be able to do this, and have it succeed, without having to inspect its own mempool, and without connecting to other nodes to see what it is in their mempools. As time passes, the coordinator may rbf this transaction to try to increase the likelihood that it confirms (it's not clear to me how this happens, do the other participants pre-sign feebumped versions initially?).

This is a clear model, I think until then we agree on. Effectively, all the inputs (and outputs) should pass the standardness rules of Bitcoin Core, otherwise you have a DoS vector where a participant can contribute a non-standard element, failing the transaction broadcast. Current interactive construction protocol v0.1 as specified out by Lightning relies on RBF as a fee-bumping technique. However, the addition of an anchor output spendable by the coordinator can be done discretionary, with all the shenanigans of the carve-out rule coming in. Pre-signing multiple feebumped versions is not part of the current spec, though it has been envisioned by Ligthning developers in recent discussions about second-stage HTLC transactions, and it could be adopted here.

The problem arises when a user or group of users (or a user pretending to be a group of users, and able to take multiple inputs of the transaction) can indefinitely stall the protocol, by relaying a low feerate transaction that cannot be feebumped away, and which doesn't confirm on its own for a long time. The question is, how might an adversary achieve this (and at what cost), and what tools are available to protocol designers to prevent it?

For a recent discussion on how an adversary can achieve this, and what tools are available to protocol designers/application developers to prevent this, see https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-November/021142.html and https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-November/021170.html. Especially, between the choice of defensive architecture/mempools inspecting protocols, at the end of the day it's likely a question of economic affordability to the end user.

The limitation of not inspecting your own or any other node's mempool to see what transactions are relayed on the public network is a significant limitation, and while I can understand why it would be very useful if we could design a protocol in this way, I'm not sure that limitation is one that the network should accept as a reason to enforce a policy on everyone. But putting that aside, the pinning issues that exist even in a fullrbf world would make this protocol vulnerable to stalling all the same, albeit at greater cost than in a non-fullrbf world. The example I gave on the mailing list was that an adversary could pin the coinjoin transaction by relaying a double-spend that was large and had low feerate. If we adopted something like #26451, then this pin would be extremely effective if the adversary chained a single low fee, but high feerate transaction to their double-spend (under the rules I proposed in #26451, the replacement transaction would both need to be high fee to pay for all the fees being removed from the mempool, and high feerate to satisfy the incentive compatibility heuristic - even though the transactions generated by the adversary might not be mined for a long time, because the ancestor feerate of the child might be very low).

This pinning vector due to high-fee/low-feerate double-spend was noted in my initial fullrbf proposal, as an additional concern to address on top of fullrbf. Effectively, the rules as proposed in #26451 would modify the cost for an adversary.

I also mentioned on the mailing list that an adversary with multiple inputs into the coinjoin could use the 100 transaction limit to prevent any rbf from occurring at all. The cost of this might not be very high, because the 100 transactions used can all be small.

While I think we have considered the 100 transaction limit numerous time on how it could affect the LN commitment transaction as a pinning, from my knowledge it has never been considered in the funding flow of coinjoin transactions. Thanks to bring it in to awareness!

And finally just recently I thought of a new DoS attack that you could run into, which is that an adversary can use RBF to keep your coinjoin transaction out of the network's mempools, without ever actuall> y having their coinjoin input double-spent in a mined block. Imagine the coinjoin transaction is relayed on the network and in everyone's mempool. The adversary can double-spend their input in a higher fee> /higher feerate transaction that includes multiple inputs. Then they double-spend that transaction by conflicting with one of the other inputs that were spending, resulting in a situation where the coinjoi> n transaction would be eligible for inclusion in node mempools (because there are no longer any conflicting spends of its inputs), but due to relay optimizations no node would reannounce or relay the trans> action for a while, at least until the next block is mined. For an adversary that is willing to issue one transaction at the prevailing block fees each block, the coinjoin could be stalled indefinitely wit> h no ability to detect which user is causing the problem, assuming your model is that you don't inspect the mempool.

My understanding of this one is the coinjoin double-spend is itself double-spent thanks to an unconfirmed input. To achieve its pinning effect, the coinjoin double-spend only have to be present in the network mempools at propagation time of the honest coinjoin by the coordinator. The propagation time of the honest coinjoin can be likely predicted by the adversary, and the usual flooding time across the network as much, therefore giving strong knowledge when the unconfirmed input of the pinning transaction must be evicted from network mempools. There is a non-null odd a block is mined during this interval, though Ibelieve the edge is in favor of the adversary to keep the pinning "silent" from mined blocks. Note, I think even if you assume a model where the local mempool is inspect, I think you can partition its view of the rest of the network by creating conflicting versions of the unconfirmed input of the coinjoin double-spend itself.

I don't know if you think that all these attacks are too costly for anyone to bother with, but it seems to me that it would make more sense to iron out the deficiencies in our RBF policies and try to improve them first and decide what level of attacker costs you're concerned with, before asking the network to try to address this use case, since it seems like fullrbf is not going to be enough. Moreover, based on recent mailing list discussion, it seems like the existing coinjoin implementations out there don't even signal for RBF on their transactions and aren't worried about these kinds of DoS vectors at the moment [1] [2].

In a future world where p2p coinjoin/multi-party Lightning funding transactions are offered in a peer-to-peer/permissionless fashion by services nodes (in the same way we have a LN routing node market with its own risks of channel jammings), I think those attacks have reasonable odds to happen, as the attacker might be able to earn a substituting service fee. I think it might make more sense to craft out the deficiencies in our RBF policies, though I think the fullrbf as solution itself to remove one pinning vector should stay an open question. It sounds to me laying out better what could be a transaction-relay policy design philosophy and its robustness in face of a consistent modeling of mining incentives are likely some of the next steps in the discussion.

While it seems the existing coinjoin implementations operators are not worried of these kinds of DoS vectors, reading the reports it sounds more because they have existent DoS at the application-level. Mentalities could evolve quickly in the light of sustained pinning attacks against the services. Of course, this kind of DoS vector isn't the most urgent safety issue affecting multi-party applications and contracting protocols, though we had to pick up one and start somewhere, initially it sounded a low-hanging fruit one. I'm still personally concerned that multi-party transaction being speced out by the Lightning community and expected to be supported by LN wallets in a middle-term timeline could make the DoS vectors far more sensible to solve.

I think this is already possible in a fullrbf world, due to the ease with which someone can construct two transactions that are mutually non-replaceable under our current rules (if transaction A has higher feerate than B, but B has greater fees than A, then neither one will displace the other.). Perhaps I don't follow what your security model is for this concern?

I don't deny it would be possible in a fullrbf world due to the exact same trick you'e raising. There has been recent research of fee-estimator more efficient than the Bitcoin Core based on real-time mempools trends, however without relying on buckets updated each blocks]. Such class of fee-estimators would be indeed exposed to deliberate partitioning, how to protect them is an open question. Also how it could be a building block for advanced pinnings. This could be solvable with some "mempool-sync" where fullrbf could be a prerequisite, and this might not be the best approach. This is low priority to me, though still a potential long-term concern.

I completely agree that a problem we are running into is that the notion of what "incentive compatible" means is not well-defined. I have attempted to articulate a philosophy of how I think we can work to > meet different use cases that may not be incentive compatible if they were the only deployed policy on the network, but that I think are incentive compatible in the sense that users should have no need to > subvert their flaws in the presence of policy options that meet their use case. If people don't think my philosophy works, I'd be interested to hear of other philosophies that are workable to meet use case> s on the network. Thus far I'm not aware of any single set of policy rules that we could apply to all transactions that would satisfy the current use cases on the network (and please note that I'm ignoring> the idea of "zeroconf" as a use case, which I think is a business practice that is not directly tied to any particular set of relay policies -- I'm thinking instead of CPFP, coinjoins, lightning, fee bumping, etc).

I think I start to understand better the idea to have isolated policy regime for use-case, well-designed enough that users of let's say policy A does not have an incentive to subvert policy B to meet their use case. Where I still wonder, and where I might diverge is about the "intra-policy" usage itself. I.e a user of policy A competing against another user of policy A (e.g a LN channel counterparty exploiting the artificial nVersion=3 package limits), to gain an economic or security advantage. This is a subject definitely warrant of more thoughts. And I think I'm converging on dissociating structured patterns of transactions (like CPFP/coinjoin/lightning) from the application processing of those chains of transactions (zeroconf). It should be noted Lightning also introduced 0conf opening channel this year (instead of the default of 6 blocks).

I completely disagree with this! If you want to limit CPU DoS surface, don't relay transactions. It would be absurd to me that censoring a type of transaction would be a culture we'd want to promote in our> project, for such a flimsy reason.

Let's explain more the type of L1/L2 node architecture I'm thinking of. In terms of second-layers protocols like Lightning, we have a concern of cross-layer link, a type of privacy leak allowing an adversary to map your LN node to your Bitcoin Core full-node, and from then escalate to more sophisticated attacks like pinning. One potential mitigation is to deploy multiple full-nodes, one full-node used for your fee-estimation only, another for your transaction broadcast, another for your block-relay only, as such leveling the bar for an attacker to cross-layer your infrastructure. In that deployment paradigm, you would like your transaction broadcast full-node to be as much robust as you would like, as the transaction are time-sensitive (e.g in Lightning, the justice transaction should confirm before expiration of the CSV delay encumbering the counterparty funds). Reducing the DoS surface by taking all the more conservative policy options sounds a reasonable choice to me from a node operator risk-management perspective.

I can understand this is an engineering culture different than the current Bitcoin Core one, though note I'm coming from the Lightning ecosystem where enforcing stricter routing policies on HTLC forward is a standard reasoning.

ariard commented at 3:48 AM on November 7, 2022: member

While being the author of mempoolfullrbf and the preferential peering patchset, I'm Concept ACK on removing this option from Bitcoin Core, as motivated here.

I think Bitcoin protocol development is to build and iron out consensus, and in the present case there is a clear lack of it. While the feature has already been merged, there have been since then far more wonders raised. Some questioning deeply what should be our transaction-relay design philosophy in face of Bitcoin use-cases gaining in complexity and diversity. From my perspective, this is far more valuable to re-build a common understanding on how to address pinning vectors, what is the miners incentives model, what are the Bitcoin merchants practices and risks models, what should be the Bitcoin Core communication process in case of significant policy upgrades, what the flow of contracting protocols and the type of DoS vectors they're exposed, what should be the level of mempool policy rules setting flexibility offered by Bitcoin Core towards node operators and few more questions. This is really true, that any part of the mining ecosystem could still turn on mempoolfullrbf by default, however I think this would bring new information in the qualification of miner incentives, not settle many more questions raised above.

I can't speak as an expert of Bitcoin Core release process, though I don't think there is a "solidfied" right to a feature once being merged on the main branch, and only included in the RCs. I think from now on the conversation is far more productive to be held on the mailing list, with concrete, historical examples of why removing the option would be contrary to Bitcoin Core release process, or even if it's a precedent on all the short-term/long-term implications it would constitute for the project decision-making process.

luke-jr commented at 4:56 AM on November 7, 2022: member

RBF is not Bitcoin protocol development, and is not a matter of consensus. It is a matter of every node [deciding] for himself

glozow commented at 8:19 AM on November 7, 2022: member

Thank you to Bitcoin community members for voicing your opinions, particularly when providing examples of your usage or application and how it would be affected by this change. For example, I found #26438 (comment) a helpful data point about full RBF in the wild.

As a reminder or if you have not read Bitcoin Core's contributing guidelines before commenting, regardless of who the reviewer is, an ACK/NACK with no explanation can be disregarded. Decisions are not based on 1-account-1-vote, which is trivially sybil-able. This is a place for technical discussion about the proposed code changes, and it is difficult to do so if the discussion page is flooded with unrelated comments.

newbiesolominer commented at 10:49 AM on November 7, 2022: none

NACK

I am a large miner looking at doing solo mining for the first time. I agree with the philosophy of full-rbf and am considering mining it. I don't know much about v3 transactions. But trying to push for this even when so much of the community is against taking away this option feels like there's something shady going on. Are v3 transactions broken? Why can't the community decide? It's a philosophy question on how Bitcoin relies on trust. Not technical. @glozow "Decisions are not based on 1-account-1-vote, which is trivially sybil-able."

I don't see brigading above. Most NACKs have good explanations and I see Core devs ignoring those explanations. If an option has so much debate let people try the option and let that decide. That process is not "sybil-able".

Didn't Satoshi say "1 CPU 1 VOTE"? Let nodes and miner hashing power decide.

juestr commented at 11:26 AM on November 7, 2022: none

Otherwise, you may just have to wait either until the transaction confirms or until the bitcoins reappear in your wallet. It’s important to note that until a transaction confirms, the bitcoins are technically still in your wallet — it’s just that it often doesn’t appear that way. The bitcoins are not literally “stuck” on the network and cannot get lost.

This is a total disaster which may well lead to lost funds.

So I have an open invoice or other receiver demanding timely payment, my non-RBF payment is not confirming, what do I do? Some users will opt to pay another time thinking they'll get their stuck money back eventually, the alternative being to let the invoice expire?

juestr commented at 12:17 PM on November 7, 2022: none

think about money. im saying a double spend is a spend of a utxo output that is already spent. and we don't replace spends with double spends. the exact opposite. We race the spends into the block and start mining to beat the fraud into the block. if its my own incoming utxo, i dont care about the relative fee. there are many users of bitcoin core. some actual end users of bitcoin.

Unconfirmed transactions are not spent yet, and no one can tell which are fraudulent. In case of mutually exclusive transaction the mined transaction becomes the valid one, and others still aren't necessarily fraudulent, just invalid.

You are trying to justify a moral super authority over consensus rules by drawing in the specter of fraud.

Sjors commented at 4:47 PM on November 7, 2022: member

@dr-orlovsky wrote:

Mempool policies are not consensus rules. They can’t even split the p2p network.

No, but they can cause (very brief) chain splits. A compact block that includes fullrbf transactions will propagate faster though the subset of nodes with -mempoolfullrbf enabled.

This probably isn't an issue at the current high propagation speeds, but it might be in a more censorship heavy world where blocks are relayed the hard way (Tor, satellite).

In other words, the concern about mempool policy bifurcation in this case could be addressed by a commitment to changing the default to 1 in the not-too-far future. But such a commitment can't be made.

reardencode commented at 4:47 PM on November 7, 2022: none

NACK - Not adding an option to bitcoin core when that option is already deployed in ~1% of the network increases risks of various sorts that arise from node operators patching their source or running forked releases. This is an option that has been proven to be demanded by the community, and it is exactly that community of node runners that defines bitcoin. Core maintainers' job is to follow the will of the community and maintain the software that facilitates that will.

If certain businesses desire a bitcoin which is strictly opt-in RBF: that thing does not currently exist, but they are free to create it.

petertodd commented at 5:25 PM on November 7, 2022: contributor

@Sjors

Mempool policies are not consensus rules. They can’t even split the p2p network.

No, but they can cause (very brief) chain splits. A compact block that includes fullrbf transactions will propagate faster though the subset of nodes with -mempoolfullrbf enabled.

Given the number of people commenting here who are not familiar with the details of the P2P protocol, you should be making alarmist claims like this. @dr-orlovsky is correct. By your definition every time a block is found there is a "chain split", as not all miners have the block instantaneously. That is not "causing" a chain split. That's just making the inevitable one-block difference in consensus take slightly longer.

Miners regularly mine blocks containing transactions not already seen by the rest of the network. Indeed, because the mempool doesn't have consensus, and it's common for conflicting transactions to exist that can't be resolved by fee, this is inevitable. Particularly if someone chooses to broadcast a lot of divergent transactions.

What full-rbf does do is reduce the chance of a divergence in a subset of the problem, when one transaction pays a sufficiently higher fee than the other. It also discourages zeroconf attackers from doing that, by simply making zeroconf not a thing.

luke-jr commented at 10:15 PM on November 7, 2022: member

No, but they can cause (very brief) chain splits. A compact block that includes fullrbf transactions will propagate faster though the subset of nodes with -mempoolfullrbf enabled.

This is FUD, not a serious rationale. If block propagation is an issue, miners should think more carefully about what blocks they mine. Reduce the size [limit?] if necessary.

Sjors commented at 1:44 PM on November 8, 2022: member

From the original mailinglist post "On mempool policy consistency":

There are a few efficiencies to be gained from similar mempool policies as well: more reliable compact block reconstruction (if you're not missing any transactions, you avoid a round-trip) and presumably more efficient set reconstruction with erlay. You'll also waste less bandwidth sending transactions that the other node is only going to reject. Both those depend on how many transactions are going to rely on unusual mempool policies in the first place though.

What I said follows directly from this.

miners should think more carefully about what blocks they mine

When there's two different policies, the rational thing might be to exclude bumped transactions that don't signal RBF, as well as the original. The trade-off there is between the increased risk of mining a stale block and the fee revenue lost.

alarmist claims

This is FUD

I said:

This probably isn't an issue at the current high propagation speeds

I'm not going to account for people being alarmed because they're not reading what I'm saying. This is not a safe-space for people who don't like thinking through edge cases :-)

vnprc commented at 2:22 PM on November 8, 2022: none

NACK

In my amateur opinion we should not hold back bitcoind development to support ill-advised application layer design decisions. Full RBF simplifies mempool policy and aids the development of multiparty contract protocols. These protocols will be essential in spreading adoption and usability of bitcoin. Zero-conf txs are insecure and do nothing to increase bitcoin adoption. It's 2022, if you need instant bitcoin transactions you need to run a lightning node.

Furthermore, removing the full RBF flag permits the continued concealment of a consensus-compatible local capability that node runners already have. This strikes me as a deceptive, user-unfriendly dark pattern that should not be permitted in a free and open protocol. This is the kind of behavior I expect from monopolistic software service companies. Bitcoin should do better.

petertodd commented at 3:30 PM on November 8, 2022: contributor

@Sjors

When there's two different policies, the rational thing might be to exclude bumped transactions that don't signal RBF, as well as the original. The trade-off there is between the increased risk of mining a stale block and the fee revenue lost.

No, the hyper-rational thing to do is to work out the increase in stale rate per byte of transaction, and ensure that the bump in fees is high enough to be worth the tiny increase in stale rate. The easiest way to accomplish that would be to just increase the minimum mining fee for your node to something that actually makes a difference to revenue.

Of course, doing that can make double-spending quite easy...

petertodd commented at 3:43 PM on November 8, 2022: contributor

@ajtowns

Why are we discussing a convoluted alternative with clearly worse outcomes than just running full-rbf? That's not a like-for-like alternative to what these users want to do.

In a technical discussion, you discuss possible alternatives so that you better understand both the goal that people are trying to achieve, and the different tradeoffs different approaches offer.

An obvious tradeoff between bip125-rbf whether signalled or not and shorter expiry, is that shorter expiry allows a much broader range of replacement options (ie, anything that would violate rules 2-5 despite being more attractive to miners) at a cost of replacement still not being available for some days. When opt-in RBF was first supported, the mempool expiry time was 3 days; it was changed two releases later to 2 weeks.

As we all know, you can't rely on transaction expiration because anyone else can just rebroadcast the transaction. Indeed, we've seen this happen before! That's why I said it's a convoluted alternative with clearly worse outcomes. Even if it did work, you're suggesting that users wait multiple days at minimum for a chance at getting their transaction mined; two weeks with the current expiration time.

No-one has raised any technological reasons why wallets like @greenaddress shouldn't simply use full-rbf. They and other wallets clearly want to offer their users this capability, and the capability has helped users in the past.

Is it because you are concerned that providing the option at all would harm zeroconf? Let's be clear here.

I don't think questioning contributors' motives is a constructive use of github, and nor is brigading of this PR with content free NACKs or ACKs. The thing we should be being clear about here is the technical consequences of proposed changes, and those approaches aren't conducive to improved technical understanding. If you don't want to have a technical conversation, there are plenty of other more suitable places. Of course, that's just my view; if the folks merging PRs do somehow consider this useful information, that's a different matter.

When someone proposes what appears to be clearly worse solution, without any technical reasons as to why, it helps the conversation along if we discuss what are the actual reasons for proposing that solution. Now, if you want to protect zeroconf, go ahead and say it. But if you don't, you have not brought to the table any reason why we should put up arbitrary roadblocks in the way of wallets like @greenaddress to use full-rbf.

BitcoinErrorLog commented at 4:08 PM on November 8, 2022: none

Green wallet already offers users full rbf. Even our wallet has the setting. Core does not need to be involved.

In fact, any wallet could switch to default-rbf-on at specific fee rate ranges or block capacity thresholds, when congestion is detected. It could even warn the user if they turn it off during such times.

UX is an app concern. Policy is a user concern. Neutrality is a Core value. Core is not here to anticipate or steer how Bitcoin is used, it is here to be a maximally useful tool while being as simple as possible.

Just because code is able to do something does not make it necessary or appropriate at the protocol level. Bitcoin is for users, not academic proof.

0conf is useful in an inverse way to rbf, yet neither can be a guarantee. Both are valid use cases and thankfully both already work fine.

ghost commented at 4:40 PM on November 8, 2022: none

I am not sure at this point which users or use cases are important:

0 conf - insecure always
More than 30 different developers and users NACKing this

Full RBF can never be stopped, 0 conf is non sense and we can improve full rbf later or make it default. If maintainers are unable to take the basic decision after 30+ NACKs with valid reasons, then maybe I am missing something.

Path of least resistance after all the drama :

Close this PR and release v24.0 for users that want full rbf (the users who run this software)

Sjors commented at 4:44 PM on November 8, 2022: member

@petertodd:

work out the increase in stale rate per byte of transaction, and ensure that the bump in fees is high enough to be worth the tiny increase in stale rate

That is necessary, but not sufficient. You don't know if other miners and nodes in between do the same. If they pick the higher fee and you don't, your block propagates more slowly. If you stick to the lower fee and they don't, same problem. If you leave the transaction out entirely you're safer (at an opportunity cost). The choice is between picking the higher fee version, picking the lower fee version and avoiding both. (this sounds completely impractical to implement though)

Note that this is not an argument for or against -fullrbf. It's an argument for having a single choice emerge.

BitcoinErrorLog commented at 5:04 PM on November 8, 2022: none

Full RBF can never be stopped, 0 conf is non sense and we can improve full rbf later or make it default. If maintainers are unable to take the basic decision after 30+ NACKs with valid reasons, then maybe I am missing something.

This is not a voting system, such is sybilable anyway, and none of the affected users are actually in this forum or understanding the nuances.

Core cannot and should not attempt to change Bitcoin or pretend they know what users want, nor what they "should" want.

The last thing we need is me or Bitrefill lobbying for 0conf users to come here and "vote" against devs.

I prefer the basic argument of: please stop meddling with Bitcoin, we have no way to get proper user input.

If you ask users if they want merchants accepting 0conf, they will say yes. If you ask them if they would like the ability to undo/bump a txn they will say yes. If you ask them if it is okay to take away 0conf they will say no. Simple.

ghost commented at 5:11 PM on November 8, 2022: none

Full RBF can never be stopped, 0 conf is non sense and we can improve full rbf later or make it default. If maintainers are unable to take the basic decision after 30+ NACKs with valid reasons, then maybe I am missing something.

This is not a voting system, such is sybilable anyway, and none of the affected users are actually in this forum or understanding the nuances.

Core cannot and should not attempt to change Bitcoin or pretend they know what users want, not what they "should" want.

The last thing we need is me or Bitrefill lobbying for 0conf users to come here and "vote" against devs.

I prefer the basic argument of: please stop meddling with Bitcoin, we have no way to get proper user input.

I have been contributing to bitcoin core for sometime, founder of joinstr (coinjoin implementation) that is affected by opt-in rbf, write a blog for privacy (rbf using by setting nSequence < maxint-1 on at least one input affects privacy) , work as dev for an exchange (non-kyc, we don't need opt-in rbf) and proud regular bitcoin core contributor.

greenaddress commented at 5:21 PM on November 8, 2022: contributor

@BitcoinErrorLog

Green wallet already offers users full rbf.

Green wallet does not currently offer full rbf. It supports opt-in rbf. we plan to support full rbf soon with both esplora/blocksteam info as well as green wallet.

Even our wallet has the setting. Core does not need to be involved.

With that mentality we wouldn't even have even opt in rbf. However opt in rbf is far from optimal and while better than nothing (and an inherit UX problem) it perpetrates this idea that more infrastructure around unconfirmed transactions could be probabilistically relied upon when it comes to block inclusion which will likely result in systematic risks when inevitably it doesn't.

Without fullrbf users are forced to wait potentially indefinitely for transactions to get unstuck when things go wrong with forgetting to set the flag, or likewise they may set it and get denied service potentially for hours or more.

I agree this could be slightly improved with better software/mempool state detection/ bip21 flags but it really is a house of cards, potentially a big weakness in Bitcoin waiting to be exploited

ajtowns commented at 5:48 PM on November 8, 2022: member

When someone proposes what appears to be clearly worse solution, without any technical reasons as to why, it helps the conversation along if we discuss what are the actual reasons for proposing that solution. Now, if you want to protect zeroconf, go ahead and say it.

Given I was the proposer of #26323, I think it's pretty absurd to cast me as a protector of zeroconf.

My goals are:

I think bitcoin core should be making changes that enable new, useful use cases. Disappointingly, full rbf as currently specced doesn't do that: the pinning attacks that this was aiming to solve remain possible, and ordinary replacement is already solved by opt-in rbf.
If a PR was merged without sufficient review -- that is, it turns out later that it doesn't do what it was described to do in the PR, and that wasn't caught in review -- then I think it should either be fixed (so that it does solve the problem it was designed to) or, if that's not possible, reverted. There shouldn't be a benefit to sneaking a PR in without proper review.
I think bitcoin core shouldn't be making changes that come as a surprise to users and risk causing them significant financial harm, absent some emergency, even if the way they're using bitcoin is stupid. There's no emergency here, so I think if we were to go ahead with this change, we should hold off for 6 months (or some other non-trivial timeframe).
I think bitcoin core should be aiming towards consistent mempool policies: if we think fullrbf is a good thing, it should be the default in the software we release; if we don't think it's a good thing, we shouldn't support it at all. There's other node software out there with a different philosophy, and that's fine -- people can use that if they want. I expect that to generally provide a worse experience, if the difference is noticeable at all.
If bitcoin core doesn't want to aim towards consistent mempool policies, and does want to have knobs so people can tune choose their own relay policy, then I think we should be forthright about that, as it's significantly different to past practice.

All of these things seem fairly straightforwardly desirable to me, even if you take "zeroconf delenda est" as your guiding principle.

Of course, one reason to be against any or all of them is if you care more about some competitor to bitcoin: bitcoin handling new use cases is just more competition for you; review missing design problems and bugs is already a PR win and if devs don't even do anything about them that's even better; making surprising changes that stops businesses from using bitcoin is both new potential customers for you, and an opportunity to discourage unrelated people from considering bitcoin; inconsistent mempool policies is a great way to make tx relay even more unreliable; and not being forthright about policy changes is just an opportunity to stir up more drama later.

But if you don't, you have not brought to the table any reason why we should put up arbitrary roadblocks [...]

I've written at length about all of the above both on the list and in various PRs on this topic.

petertodd commented at 5:49 PM on November 8, 2022: contributor

@BitcoinErrorLog

Full RBF can never be stopped, 0 conf is non sense and we can improve full rbf later or make it default. If maintainers are unable to take the basic decision after 30+ NACKs with valid reasons, then maybe I am missing something.

This is not a voting system, such is sybilable anyway, and none of the affected users are actually in this forum or understanding the nuances.

GitHub is not sybilable. The NACKs are associated with long-standing, real-world, identities. I personally know the people behind the following NACKs, all of which have given clear rationals for their NACK here and elsewhere, including but not limited to: @RCasatta - Blockstream dev with a long history (also contributes to OpenTimestamps) @jwilkins - Blockstream co-founder, now Chief Security Officer of River Financial @afilini - Brink grantee, working on Bitcoin Dev Kit and Rust Bitcoin @danielabrozzoni - Another Bitcoin Dev Kit dev, also worked for Revault @greenaddress - Author of Green Wallet @ecdsa - Author of Electrum Wallet @dr-orlovsky - Author of the RGB protocol and wallet @UkolovaOlga - Also RGB @Giszmo - Wallet security expert, https://walletscrutiny.com @prusnak - Founder of Trezor Wallet @luke-jr - Long standing Bitcoin Core contributor (longer than me!) and Bitcoin Knots maintainer

It is not productive to this conversation to imply that the NACKs may be a sybil attack.

ghost commented at 5:57 PM on November 8, 2022: none

GitHub is not sybilable. The NACKs are associated with long-standing, real-world, identities. I personally know the people behind the following NACKs, all of which have given clear rationals for their NACK here and elsewhere, including but not limited to:

Anyone trying this is ignoring the feedback and depends on a small number of maintainers who should not be able to decide everything including config of mempool or freedom of bitcoin users.

This is not sybil attack or brigading but similar to people going against governments in their country.

Example: Truckers and Farmers in different countries. Even they were told about rules and regulations.

This option exists for users and let them decide.

BitcoinErrorLog commented at 6:21 PM on November 8, 2022: none

If you want this to be a voting system, it becomes sybilable, because we need to get actual users in here, not Core devs, etc.

It is not a vote afaik, and I doubt you really want it to be either.

This conversation is getting noisy, a chore.

The author of the work ACK'd its removal. The topic is controversial. The user space could be violated. Etc etc etc.

This thread probably should be closed, and the feature removed.

Please stop meddling with Bitcoin.

Love,

All the users that are not Core devs.

ghost commented at 6:34 PM on November 8, 2022: none

If you want this to be a voting system, it becomes sybilable, because we need to get actual users in here, not Core devs, etc.

It is not a vote afaik, and I doubt you really want it to be either.

This conversation is getting noisy, a chore.

The author of the work ACK'd its removal. The topic is controversial. The user space could be violated. Etc etc etc.

This thread probably should be closed, and the feature removed.

Please stop meddling with Bitcoin.

Love,

All the users that are not Core devs.

You should secure applications. I won't blame you alone as most of the LN devs live in some fantasy and this is what I got answered for sharing something about turbo channels. I cannot search but there was a dev and it was a spec approved by lot of devs. I was answered " it works on trust" so not vulnerable.

Please continue that trust and this was done months back.

reardencode commented at 6:37 PM on November 8, 2022: none

@BitcoinErrorLog

I don't think anyone is claiming it's a voting system. As you say, Bitcoin's operation should be up to primarily the users, not the bitcoin core developers. Bitcoin core v23 doesn't facilitate the users' will because it forces users to run patched or forked versions in order to express their will. The proposed v24 with -mempoolfullrbf better reflects the users' will, but still biases (through its default) in the direction that you prefer.

You yourself said that core should be neutral. This PR (removing -mempoolfullrbf) increases core's bias. Do we have your NACK on this PR to prevent that increase in bias?

luke-jr commented at 6:53 PM on November 8, 2022: member

No, the hyper-rational thing to do is to work out the increase in stale rate per byte of transaction, and ensure that the bump in fees is high enough to be worth the tiny increase in stale rate.

Actually, the miner can't know which transaction other nodes have accepted into their mempool (hypothetically it could even be both, though I don't know if any nodes implement that at present).

It's also possible that nodes have rejected both transactions entirely.

Therefore, the stale rate must always be considered in all fee considerations, replacement or not.

michaelfolkson commented at 6:57 PM on November 8, 2022: contributor

I think a lot of this general higher level discussion can move elsewhere (e.g. mailing list, IRC). All the information seems to be available to the maintainers at this point. They are ultimately responsible for merging pull requests and signing off on Bitcoin Core releases (that's just the reality). If you feel strongly enough on decision(s) they make there are consensus compatible forks (e.g. Knots) and alternative implementations. By all means hold them accountable and request information on why certain decisions were made after the fact though.

As I've said elsewhere I personally will support whatever decision is made on this issue, I think it is a tricky decision for this particular release given the makeup of the comments. I would be very disappointed though if there wasn't full RBF support in either this release or a future release though. The vast majority of comments not only seem to want this but also think supporting the zero conf use case shouldn't be a major consideration.

ghost commented at 7:06 PM on November 8, 2022: none

I think a lot of this general higher level discussion can move elsewhere (e.g. mailing list, IRC). All the information seems to be available to the maintainers at this point. They are ultimately responsible for merging pull requests and signing off on Bitcoin Core releases (that's just the reality). If you feel strongly enough on decision(s) they make there are consensus compatible forks (e.g. Knots) and alternative implementations. By all means hold them accountable and request information on why certain decisions were made after the fact though.

I think this PR should not have been created here before some users and devs agreeing on other platforms.

michaelfolkson commented at 7:11 PM on November 8, 2022: contributor

@1440000bytes: I'm happy to discuss this anywhere online you want (within reason). Just not this PR, this repo. Decisions have to be made and we move on.

ghost commented at 7:12 PM on November 8, 2022: none

There is a line in https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md#communication-channels:

The developer mailing list should be used to discuss complicated or controversial consensus or P2P protocol changes before working on a patch set.

I think we can change to any controversial changes and until settled on other platforms

petertodd commented at 7:16 PM on November 8, 2022: contributor

@ajtowns

When someone proposes what appears to be clearly worse solution, without any technical reasons as to why, it helps the conversation along if we discuss what are the actual reasons for proposing that solution. Now, if you want to protect zeroconf, go ahead and say it.

Given I was the proposer of #26323, I think it's pretty absurd to cast me as a protector of zeroconf.

People change their minds. You're actions right now are consistent with trying to protect continued uses of zeroconf, and indeed, you even say so below:

My goals are:

* I think bitcoin core should be making changes that enable new, useful use cases. Disappointingly, full rbf as currently specced doesn't do that: the pinning attacks that this was aiming to solve remain possible, and ordinary replacement is already solved by opt-in rbf.

I find it extremely disrespectful to the wallet authors that have spoken up with clear usecases for full-rbf that you and others continue to say that replacement is solved by opt-in RBF. It obviously is not for the precise reason that opt-in RBF transactions get discriminated against, people have a reasonable desire to send without that bit, and some of the time those transactions get stuck and need fixing.

Also, re: the pinning attacks, as we all know full-rbf does in fact eliminate large classes of them, and thus forces attackers to use more expensive attacks that tie up more funds. It also of course eliminates unintentional attacks where transactions are double-spent by accident (eg by people importing seeds into different wallets, something that has happened regularly with Wasabi).

Finally, the default-off mempoolfullrbfoption was of course not intended to solve these attacks directly! Obviously a default-off option doesn't itself solve attacks. What it does do - in the context of multi-party protocols - is give easy ways for devs and users to experiment with full-rbf behavior.

That there are still some questions around that specific use case does not justify the removal of that widely desired option.

* If a PR was merged without sufficient review -- that is, it turns out later that it doesn't do what it was described to do in the PR, and that wasn't caught in review -- then I think it should either be fixed (so that it does solve the problem it was designed to) or, if that's not possible, reverted. There shouldn't be a benefit to sneaking a PR in without proper review.

Again, the default-off mempoolfullrbf flag has very well understood behavior that people have been reviewing for years. The four different wallet authors who have NACKed this pull-req are completely familiar with what it does; we all know the (lack of) impact it will have on the network as a whole from a technical perspective. Questions around a specific use-case do not constitute insufficient review.

You can't seriously argue that full-rbf needs to be "fixed" to achieve the goals people want to use it for.

* I think bitcoin core shouldn't be making changes that come as a surprise to users and risk causing them significant financial harm, absent some emergency, even if the way they're using bitcoin is stupid. There's no emergency here, so I think if we were to go ahead with this change, we should hold off for 6 months (or some other non-trivial timeframe).

The effect of your actions in NACKing this pull-req are to protect continued use of zeroconf. As is easy to see on Twitter, Reddit, etc. the fact that zeroconf is insecure is widely known. But a small minority of people like @BitcoinErrorLog are trying to argue that it is in fact secure. By pulling this default off option, you are inevitably sending a message that Bitcoin Core is in fact in control of mempool policies that make zeroconf secure.

If you wanted to minimize this risk, we'd simply ship with full-rbf enabled, and make the fact that we're doing that very public to eliminate usage of zeroconf as fast as possible. The fact is you are doing the exact opposite, increasing the total exposure to this risk.

* I think bitcoin core should be aiming towards consistent mempool policies: if we think fullrbf is a good thing, it should be the default in the software we release; if we don't think it's a good thing, we shouldn't support it at all. There's other node software out there with a different philosophy, and that's fine -- people can use that if they want. I expect that to generally provide a worse experience, if the difference is noticeable at all.

This is clearly a case where getting consensus over mempool policies is difficult for political reasons, not technical. And far from providing a worse experience, multiple wallet authors have spoken up with clear use-cases where it provides a better user experience.

The obvious thing to do with that kind of political disagreement is provide the option for people to use. With more use, we'll probably be able to resolve this disagreement with actual usage rather than endless theoretical discussions on GitHub.

* If bitcoin core doesn't want to aim towards consistent mempool policies, and does want to have knobs so people can tune choose their own relay policy, then I think we should be forthright about that, as it's significantly different to past practice.

The fact is Core has lots of knobs to control relay policy, including many knobs controlling things with a much smaller interested audience than full-rbf. Past practice on Bitcoin Core clearly allows people to pick mempool policies.

Where is this discussion of knobs going anyway? As you well know, we already ship with settings like the maximum mempool size and min relay fees that, if set, make zeroconf unreliable. Are you planning on removing those settings?

Of course, one reason to be against any or all of them is if you care more about some competitor to bitcoin:

bitcoin handling new use cases is just more competition for you;

You are clearly interfering with desirable use-cases by taking away this option.

review missing design problems and bugs is already a PR win and if devs don't even do anything about them that's even better;

What are you trying to insinuate here?

making surprising changes that stops businesses from using bitcoin is both new potential customers for you, and an opportunity to discourage unrelated people from considering bitcoin;

Surprising? We've been telling people for years that zeroconf is insecure. Full-RBF is not surprising for anyone who isn't in the tiny minority trying to accept it.

Are you saying you expect more people to start relying on zeroconf? And that it's a good thing? Because that's how this reads.

inconsistent mempool policies is a great way to make tx relay even more unreliable;

Full-RBF nodes are entirely compatible with opt-in-rbf nodes. How specifically does full-rbf make tx relay "even more unreliable"?

and not being forthright about policy changes is just an opportunity to stir up more drama later.

We're talking about a default-off option. That is not a policy change.

sdaftuar commented at 7:18 PM on November 8, 2022: member

I’ll start with a summary of how I see the review feedback here. I’d broadly categorize the most common responses expressing support for -mempoolfullrbf in the following ways:

Zeroconf is bad or unsafe, and we must send that message and/or give users and miners tools to exploit it because it’s the rational thing to do. This was the overwhelming majority of the reasons given (and it seems most commenters are unable to grasp that defending zeroconf business practices is not an argument I was making).
Even though users can choose to enable RBF for their transactions already, something about the current ecosystem results in users making mistakes and not making this choice, and then later regretting it. This could be because: wallets don’t have good defaults; this whole topic of replaceability vs. non-replaceability is confusing to users and they choose the wrong thing; or some services encourage users to set their wallets this way. It also could be because zeroconf merchants incentivize users to signal non-replacement (to receive favorable treatment from the merchant) and then use RBF to reclaim their funds, but only a few commenters acknowledged this use case.
It’s privacy enhancing to get rid of a distinguishing characteristic of transactions.

There were also circular answers given for why we should offer this as a flag, namely that users of our software demand it (why do they?), or that fullrbf is inevitable (why?), and therefore we must support it eventually.

There were also several comments that seemed to indicate clear misconceptions as the justification for why we should have fullrbf, which I’ll gloss over quickly: some thought this PR would eliminate all forms of RBF, or that this PR implies we are trusting miners to not follow their incentives (perhaps worth debating, but not an assumption built into this), or that the only possible utility from a non-replacement signal is if a user trusted that it would be confirmed (or give users a “false sense of security”) – this ignores the points I raised on why non-replacement can be useful, and also ignores evidence that came up indicating that some modern wallets and protocols use non-signaling transactions in certain limited situations (such as for LN funding transactions or coinjoin protocols deployed today). Another user thought that RBF should be opt-out only, and not opt-in, misunderstanding that our RBF policy today can equivalently be thought about as an opt-in or opt-out policy, because every transaction that is authored has to pick, via the choice of sequence values, whether it will be subject to RBF or not.

Another commenter mentioned that we should not incentivize the use of unconfirmed transactions. I thoroughly appreciate this comment, because I would be the first to agree that our software would be far simpler, and the network would work much better in many ways, if users were on board with us eliminating relay of transactions that spend unconfirmed outputs – the most visible form of “use of unconfirmed transactions” on the network today. I have avoided proposing this because I believe that Bitcoin users would find this too restricting, but I would welcome others’ efforts to move Bitcoin’s culture in this way if it’s practical for the ecosystem. This would make our RBF policy code much simpler, too, and would eliminate reasons to have a non-replacement policy – unless we then wanted to support some limited transaction chains, which would work much better for transactions that weren’t subject to replacement.

Finally, I’ll get to what I think are the best reasons to oppose this PR. I laid some of these out already in an earlier post, but I’ll restate them here more concisely. It seems to me that we have not reached consensus about this PR, so in the interests of not holding up this project with more debate I thought I’d acknowledge what I believe to the best reasons to continue ahead with the -mempoolfullrbf flag in our code and continue with the 24.0 release with no further change on this front.

First, this PR unfortunately has conflated two separate debates: whether we should be moving towards a fullrbf policy on the network, and regardless of that, whether it’s reasonable to offer this default-off command line option to our users. My view on this is that if fullrbf as our default policy doesn’t make sense, then it also doesn’t make sense for us to offer it as an option, but I know that others might disagree with that logic. I won’t repeat my arguments for why I think encouraging a more homogenous network policy and promoting broad standards for transaction makes sense for users to have a better experience transacting on the network, but I will acknowledge that the existence of debate on this point is a reason that (from a maintainer’s perspective) I’d expect this PR to have a high threshold to clear in order to be merged.

Second, I think there are logical reasons to want the network to be fullrbf:

If you believe that miners will eventually mine non-signaling transactions due to the existence of zeroconf merchants whose business practices incentivize theft, then it’s rational to both want to support those miners now, and (if you’re a node operator) to also eventually run with the same policy the miners are running with, so that your mempool more closely matches what will happen on the network.
The simplicity for Bitcoin users to not have to distinguish what it means for a transaction to be replaceable or not might be a real win for the Bitcoin user experience. The idea that a transaction is “non-replaceable” as a factual matter is counter to Bitcoin’s consensus design, so wallets that present that choice are indeed communicating an incorrect message to users.

Moreover, the objections I have to the concept of fullrbf on the network are philosophical in nature. I opened this PR to try to discuss the higher level protocol development concerns that come with decisions like this – such as how we evaluate incentive compatibility or meet disparate use cases (but disappointingly, reviewer feedback has largely avoided engaging on these points). And in thinking about the alternative, I don’t expect that offering a default-off flag is going to materially alter the network in the near term, as a practical matter. I do still plan to raise these protocol development concerns in the future, if and when they are applicable again to work happening in this project.

So even though I remain unconvinced by the NACKs here, and despite what I see as deplorable behavior by some of the participants, I think it makes sense to close this PR due to lack of consensus, and my recommendation to maintainers would be to move forward with the 24.0 release without reverting #25353.

sdaftuar closed this on Nov 8, 2022

MarcoFalke locked this on Nov 9, 2022