BIP 78: Add payjoin proposal #923

Previous discussion at https://github.com/NicolasDorier/bips/pull/3

Submitting to mailing list.

Ping @instagibbs I added some commit clarifying what you sent me by mail.

About your point:

There is no global “minimum relay fee policy”. My node my have 5 sat/byte floor, yours 1. The receiver may think something is fine but sender can’t actually get it into its own mempool. Seems pertinent to have the sender choose feerate and receiver adhere to this up to the maxfeebumpcontribution if specified.

While true in theory, this is not true in practice. The network adopt the default of Bitcoin core every single time. (Got interesting problems with colored coins back in the days because of this)

I suggest we add an additional parameter so the sender can specify what is the minfeerate for him?

I can see a problem happening in case the sender’s wallet stop being able to make payjoin with receivers because receivers bump the fee rate not knowing the new network’s default.

the other case is where the receiver piggy backs and accomplishes another goal like payouts. Seems to me I’m that case the receiver should pay for everything aside from maybe the original deporting input? The receiver is getting a free ride from the input and possibly denying the sender inclusion in their analysis cluster. Basically just doing cut-through.

The receiver is bounded by the additionalfeecontribution. So he is free to add more than the fee contribution, but he would have to pay from his own pocket.

Now that is true that he can make the sender pay for a few output, even if the receiver has an upper limit. That said, I think this is quite difficult to calculate properly. For example, the receiver may have changed his payment output from P2WPKH to P2SH which would change the outputs length (top of my head calculation +2 bytes), but the sender should pay for it.

Another solution is to drop address substitution completely. But I wanted to keep doubt in the analyst calculation that their heuristic may be poisoned.

Re minimum feerate: I was more saying that one mempool could have slightly higher min feerate in times of large congestion/spam once it starts removing txns from the mempool due to lack of space.

On Sun, May 17, 2020, 7:14 AM Nicolas Dorier notifications@github.com wrote:

Ping @instagibbs https://github.com/instagibbs I added some commit clarifying what you sent me by mail.

About your point:

There is no global “minimum relay fee policy”. My node my have 5 sat/byte floor, yours 1. The receiver may think something is fine but sender can’t actually get it into its own mempool. Seems pertinent to have the sender choose feerate and receiver adhere to this up to the maxfeebumpcontribution if specified.

While true in theory, this is not true in practice. The network adopt the default of Bitcoin core every single time. (Got interesting problems with colored coins back in the days because of this)

I suggest we add an additional parameter so the sender can specify what is the minfeerate for him?

I can see a problem happening in case the sender’s wallet stop being able to make payjoin with receivers because receivers bump the fee rate.

the other case is where the receiver piggy backs and accomplishes another goal like payouts. Seems to me I’m that case the receiver should pay for everything aside from maybe the original deporting input? The receiver is getting a free ride from the input and possibly denying the sender inclusion in their analysis cluster. Basically just doing cut-through.

The receiver is bounded by the additionalfeecontribution. So he is free to add more than the fee contribution, but he would have to pay from his own pocket.

Now that is true that he can make the sender pay for a few output, even if the receiver has an upper limit. That said, I think this is quite difficult to calculate properly. For example, the receiver may have changed his payment output from P2WPKH to P2SH which would change the outputs length (top of my head calculation +2 bytes), but the sender should pay for it.

Another solution is to drop address substitution completely. But I wanted to keep doubt in the analyst calculation that their heuristic may be poisoned.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-629780536, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFU5MXMGJE4PVWVCZMMTRR7BIZANCNFSM4NDBRK6Q .

Correct. I think that makes it clear and clears up another set of questions is sent you about the receiver lowering feerate too much.

On Sun, May 17, 2020, 8:30 AM Nicolas Dorier notifications@github.com wrote:

Indeed. And BTCPayServer is using bitcoin’s core one we get from RPC… So I think the best solution is to let the sender define it, and if not defined, use the receiver’s one?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-629789342, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFU7YKWUAZ4AAMYO2SVDRR7KEPANCNFSM4NDBRK6Q .

Well practically I was thinking sender would set the floor at whatever priority they wanted which could be much higher as well.

I guess receiver should signal their absolute lowest rate in opening, and then sender responds with something both can agree with in the end.

On Sun, May 17, 2020, 8:44 AM Nicolas Dorier notifications@github.com wrote:

I was thinking though: If the feerate of the sender is lower than the minfeerate of the receiver, the payjoin will still fail because the receiver is using testmempoolaccept before doing the proposal.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-629791056, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFUZJOZP4MR3UTMDB23DRR7L4RANCNFSM4NDBRK6Q .

I relaxed the endpoint requirement.

Also, authorizing unauthenticated channel, because the payment output can effectively be used to make sure that the sender is sending to the right person, even over unauthenticated channel.

I took it a the sender setting it since they may try to send a new body message type for example a set of PSBT to be modified.

On Fri, May 22, 2020, 6:21 AM Adam Gibson notifications@github.com wrote:

@AdamISZ commented on this pull request.

In bip-xxxx.mediawiki https://github.com/bitcoin/bips/pull/923#discussion_r429165630:

+* The sender should allow the payment output to be modified by the receiver (The receiver may substitute a P2WPKH payment to P2SH payment to increase privacy) +* The sender must allow the receiver to add outputs. +* The sender must allow the receiver to not add any input. Useful for the receiver to change the paymout output scriptPubKey type. +* If no input have been added, the sender’s wallet implementation should accept the payjoin proposal, but not mark the transaction as an actual payjoin in the user interface.

+Our method of checking the fee allows the receiver and the sender to batch payments in the payjoin transaction. +It also allows the receiver to pay the fee for batching adding his own outputs. + +On top of those check, it is recommended, but not required for the sender to check that: +* If the sender is making a payjoin with a change (ie, not in the [[#spare-change|spare change]] case), make sure the receiver is paying for any change in the output list. + +===Optional parameters=== + +When the payjoin sender posts the original PSBT to the receiver, he can optionally specify the following HTTP query string parameters: + +* v=, the version number of the payjoin protocol that the sender is using. The current version is 1.

Do we think v= in the BIP21 uri actually makes sense? This is a version specific to the payjoin feature; what if the URI contains other features which themselves might be versioned? I mean pjv= could kinda work but it almost seems like the versioning should be embedded within the pj= field. Don’t want to open a can of worms there, but v= seems .. “off” somehow. No?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#pullrequestreview-416804985, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFU62KQYRPFC4ZB2FKBLRSZG3BANCNFSM4NDBRK6Q .

I took it a the sender setting it since they may try to send a new body message type for example a set of PSBT to be modified.

Did you intend to reply outside the thread? Well, anyway: we discussed this quite a bit in the earlier version of this document (see here and various other strands of discussion around that), from what I can glean you’re just talking about the motivation for versioning/version negotiation? I don’t think that’s so much in question any more (it seems like I convinced people it’s necessary, or more, they convinced themselves :)), I’m just talking here about the syntax of the URI, not whether to do it or why.

@AdamISZ the v= is not part of the BIP21. It is a parameter passed by the sender to the receiver in the POST request, not from receiver to sender.

Basically receiver say “here is my negociation endpoint”, then sender say “Ok, let’s do this! I am running v2”, then receiver say “Ah sorry, I don’t support 2, but 1 and 3”, then the sender can say “Ok let’s use 1 then”

@AdamISZ the v= is not part of the BIP21. It is a parameter passed by the sender to the receiver in the POST request, not from receiver to sender.

Basically receiver say “here is my negociation endpoint”, then sender say “Ok, let’s do this! I am running v2”, then receiver say “Ah sorry, I don’t support 2, but 1 and 3”, then the sender can say “Ok let’s use 1 then”

Doh! Sorry for wasting time there. No wonder it seemed weird, it’s because it was completely wrong, lol.

Backwards compatibility section is required.

(In this case, it should probably document interaction with existing BIP21 wallets, at least.)

I am thinking about restricting the receiver to make it easier to code the sender. #923 (review)

I am not really thrilled by it, as it decrease the number of suspected payjoin, but I feel the sender would be way too tricky to code properly, making it prone to loss of fund if the receiver is malicious.

I am thinking changing the following:

• No output substitution
• The receiver can’t change outputs
• No reordering of inputs and outputs
• The receiver can’t reorder outputs or inputs (it can only insert new inputs)
• No spare change case to handle
• Only exception is change of value to the change output

This would make it way more easy for the sender to check that the receiver is not malicious.

A sender could just remove the added input, and verify that the resulting PSBT is identical to the original PSBT.

It also greatly simplify fee verification (ping @instagibbs ) @Kukks @junderw @lukechilds @lontivero @AdamISZ @ncoelho @nopara73 @RHavar @andrewkozlik

Having coded both receiver and sender, I found out that the sender is actually harder to code and review than the receiver, this should not be the case.

I am also concerned that those edge cases are not correctly tested by senders, and as soon as the receiver take advantage of it, it will break senders, losing the opportunity to coinjoin. (https://github.com/bitcoin/bips/pull/923#discussion_r434685523)

Note: We could always allow support for the other corner cases later by introducing more optional parameter where the client can signal what it supports. But I think at this stage, this may be premature.

• No output substitution
• The receiver can’t change outputs
• No reordering of inputs and outputs
• The receiver can’t reorder outputs or inputs (it can only insert new inputs)
• No spare change case to handle

This would make it way more easy for the sender to check that the receiver is not malicious.

A sender could just remove the added input, and verify that the resulting PSBT is identical to the original PSBT.

To avoid the new inputs going to fees purely, the receiver also must be allowed to change the outputs in some fashion. The simplest thing would be to allow increasing the value of existing outputs but nothing else. Is this what you have in mind?

I agree with the additional restrictions. Although:

The receiver can’t change outputs The receiver can’t reorder outputs or inputs (it can only insert new inputs)

Might be going a little too far? For obvious privacy reasons the receiver can’t simply add his input to the end; so he has to do insertion at randomized locations. That’s probably not too hard to do, but requires slightly more thinking than receiver adding their inputs, and simply shuffle the entire list.

I’ve helped with several implementations, and I’d still advocate the same bip79 (bustpay) restrictions which I think are pretty straight forward for a sender to verify

Although to be honest, I can’t say I feel too strongly about it. I think requiring the original order is fine too

But:

(it can only insert new inputs)

I think is overly restrictive. I prefer the BIP79 rule:

All outputs from the template transaction exist in the partial transaction, except they are allowed to be reordered and have their amounts increased (but never decreased)

Which I don’t think puts any extra burden on the sender to verify while allowing interesting use cases. A couple that come to mind: the receiver adds a extra output to pay a 3rd party in the same transaction the receiver is getting paid. Or a merchant outsources the payjoin to a 3rd party who can return some of the contributed funds back to themselves.

@RHavar I think randomizing the input/output order does not improve anything, while making it a bit harder for someone to verify the transaction is correct.

That said, for adding output I think this is fine indeed. (outside the change of value in the change output to pay for the additional fee)

I wanted to remove this ability, because we don’t want the receiver to bundle output for free, paid by the sender. We mention that in the BIP, but it is actually quite difficult to verify for the sender.

I wanted to remove this ability, because we don’t want the receiver to bundle output for free, paid by the sender. We mention that in the BIP, but it is actually quite difficult to verify for the sender.

I don’t think this makes it any more simple or more complicated. Even without making any changes to the outputs … the receiver can lower the fee-rate by just simply adding (dust) inputs. So feerate related stuff is something that always needs to be done, regardless of any output related stuff.

@RHavar I think randomizing the input/output order does not improve anything,

Agree

while making it a bit harder for someone to verify the transaction is correct.

I disagree. In pretty much every mainstream programming language, it’s simpler to do shuffle(concat(listA, listB)) than randomlyIntercalate(listA, shuffle(listB)) with the later being complicated enough I’d have to stop and think to implement correctly (as I couldn’t just call a couple builtin functions). So from the receiver side, shuffling is clearly simpler.

And from the sender side, the verification is actually simpler too, for the shuffle approach: “for each input in original transaction, check it exists in the Payjoin Proposal” [1] while the preserving order requires an additional thing to check.

That said, this is probably the most trivial detail. Don’t really think this matters at all.

[1] This is generally going to be accidentally-quadratic time. But it’s impossible to ever be big enough to matter. And it’s pretty easy to turn into linear time by using a hash lookup.

Actually I am thinking: The payment output’s value can be decreased to pay for fee as well. (if that’s overpaid)

So @RHavar and @andrewkozlik, if I understand, the only complain you have is on the address substitution?

About adding output, I am really unsure about it now though. While it is useful for receiver, it is hard for the sender to calculate that the receiver is not making him pay for the additional outputs…

• No output substitution
• The receiver can’t change outputs
• No reordering of inputs and outputs
• The receiver can’t reorder outputs or inputs (it can only insert new inputs)
• No spare change case to handle
• Only exception is change of value to the change output

Although this would simplify the integration, I think this will reduce the privacy benefits and incentives to use PayJoin drastically. This means…

• Receiver cannot change output address to be same address type [bad for privacy]
• No reordering can lead to fingerprinting that this might be a PayJoin, compared to value ordering or shuffling
• It entirely prevents use cases like batching other transactions in this PayJoin. This prevents a great trinity of blockspace efficiency improvement, a privacy improvement, and usability improvement.

To remove this numerous benefits would be a shame indeed.

No reordering can lead to fingerprinting that this might be a PayJoin, compared to value ordering or shuffling

I don’t think so, on the contrary, it makes sure the payjoin adopt the fingerprint of the sender’s wallet. But yeah @RHavar think that it does not make the sender’s implementation too much more complicated, so maybe I am overshooting.

It entirely prevents use cases like batching other transactions in this PayJoin. This prevents a great trinity of blockspace efficiency improvement, a privacy improvement, and usability improvement.

It does not prevent the sender from batching though.

It does not prevent the sender from batching though.

But what about No output substitution. The receiver can't change outputs.? Doesn’t this exclude adding new outputs that pay the batched transaction output?

I was thinking about excluding adding output, but you and @RHavar does not seem to think it is a good idea and make things necessarily easier.

I am open to restrict some freedom of the receiver in order to make the sender easier to implement, also I am worried that some freedom that are not being taken advantage of now end up breaking senders who did not tested it.

Actually I am thinking: The payment output’s value can be decreased to pay for fee as well. (if that’s overpaid) So @RHavar and @andrewkozlik, if I understand, the only complain you have is on the address substitution?

I find that changing an output’s address, decreasing an output’s amount or removing an output are all problematic in terms of user experience. See https://github.com/bitcoin/bips/pull/923/files#r435831538.

About adding output, I am really unsure about it now though. While it is useful for receiver, it is hard for the sender to calculate that the receiver is not making him pay for the additional outputs…

I see no problem with the receiver adding outputs, it seems like a useful feature. As long as the original outputs are not decreased or removed and the sender doesn’t have to sign for any additional UTXO’s than those he already signed for in the original transaction, he can rest assured that he is not paying more than he agreed to.

@RHavar I think randomizing the input/output order does not improve anything, while making it a bit harder for someone to verify the transaction is correct.

I am inclined to agree. Maintaining the order (while allowing insertions) would make the verification I mentioned easier to implement in hardware wallets, because transactions are not loaded into the wallet’s RAM, but are streamed. Without going into details, verifying that the conditions are satisfied in case of randomly reordered inputs and outputs can be solved in hardware wallets by implementing some kind of unordered hash function. So it’s not an obstacle, but makes the implementation more complicated. (The verification would need to be done by the hardware wallet itself if we want to make PayJoin both user friendly and secure for the sender.)

I find that changing an output’s address, decreasing an output’s amount or removing an output are all problematic in terms of user experience.

About changing the output’s amount, we have no choice, as the sender need to pay for the input of the receiver.

Removing output is not possible in the protocol.

Changing output address is only limited to the paid output, so easy to check. But I agree, if the signature is on a hardware wallet, this would be confusing to the user…

I am inclined to agree. Maintaining the order (while allowing insertions) would make the verification I mentioned easier to implement in hardware wallets, because transactions are not loaded into the wallet’s RAM, but are streamed.

I think it is true as well, because you only have to check within a simple loop. I am tempted to rewrite my client for this as this is less error prone.

I think for v1 this type of simplification can make a lot of sense. There are use cases for more extensive transaction changes but may hinder adoption due to sender complexity.

On Fri, Jun 5, 2020, 8:45 AM Nicolas Dorier notifications@github.com wrote:

I am inclined to agree. Maintaining the order (while allowing insertions) would make the verification I mentioned easier to implement in hardware wallets, because transactions are not loaded into the wallet’s RAM, but are streamed.

I think it is true as well, because you only have to check within a simple loop. I am tempted to rewrite my client for this as this is less error prone.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-639458593, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFU6LAFDK4S25QWONQMDRVDSGPANCNFSM4NDBRK6Q .

I think @andrewkozlik makes a good point about hardware wallets wanting to be able to stream transactions. So my refined opinion is the restrictions should be:

• All inputs from the original must be in the payjoin proposal, with the exact same sequence numbers. The inputs MUST be in the same order, and the sender must verify this.

• The receiver should randomly intercalate their added inputs, and not simply append to the end. The sender is not able to verify this.

• All outputs from the original transaction must be in the payjoin proposal. The receiver is allowed to insert new outputs (at any location) but like inputs, must preserve order. The receiver must leave the existing output amounts the same or increase them, but NEVER decrease.

On a different note, I also disagree with everything about “spare change”. I think it should be completely purged from the spec. I don’t believe the premise that it’s better for a receiver to turn a changeless-transaction into one that looks like it has change. I would even go as far as argue it’s harmful, because if you require it then any transaction with 1 output cannot be a payjoin.

Under the rules I proposed in my last comment, creating a fake-change output is completely allowed if a receiver feels it’s useful (although personally I would be inclined to mildly advise against). But it’s no longer a special case, just a receiver implementation detail.

P.S. The reason “change-less transactions” are so uncommon is actually because wallet implementations (currently) suck at coin-selection, not because they are inherently rare. I wrote the payment processing system for bustabit.com which supports instant withdrawals and (partial) batched (user choice) and fires it through a commercial constraint solver. They report that >80% of the transactions they make have no change output. And the solver gives no extra weighting to a transaction with no change. It is creating changeless transactions purely because it is more economical to do so. If other wallets upped their game, we would see a staggeringly higher amount of changless transactions.

All outputs from the original transaction must be in the payjoin proposal. The receiver is allowed to insert new outputs (at any location) but like inputs, must preserve order. The receiver must leave the existing output amounts the same or increase them, but NEVER decrease. @RHavar, I actually have to rectify my statement about not decreasing any output’s amount. As @NicolasDorier correctly pointed out, if the sender offers to pay for the receiver’s added inputs, then indeed that has to be taken out of additionalfeeoutputindex. So that is the one exception, but ALL other outputs must not decrease.

All outputs from the original transaction must be in the payjoin proposal. The receiver is allowed to insert new outputs (at any location) but like inputs, must preserve order. The receiver must leave the existing output amounts the same or increase them, but NEVER decrease.

We can’t do this, as we need to decrease for paying fee.

I would even go as far as argue it’s harmful, because if you require it then any transaction with 1 output cannot be a payjoin.

No, this is not deterministic, the receiver don’t have to add an output. Note that if you allow the addition of new output you allow the spare change behavior. As you say But it's no longer a special case, just a receiver implementation detail..

So that is the one exception, but ALL other outputs must not decrease.

So there is only two output which can actually decrease: The change output and the payment output. All the rest should stay the same.

So that is the one exception, but ALL other outputs must not decrease.

So there is only two output which can actually decrease: The change output and the payment output. All the rest should stay the same.

The payment output should not decrease. Once we allow that, we might as well allow removing outputs or changing the output address, because there would be nothing stopping the receiver from decreasing the output’s amount to 1 satoshi and introducing a new output with a different address. This would again mean that the user would have to reconfirm a completely different transaction, which is what I am trying to avoid.

I advocate that as a basis this specification should maintain the rules given in BIP-0079, because these were well thought out to allow for simple verification of the transaction details in the final signing stage. The new features provided by this specification should be extending BIP-0079 on an opt-in basis. I would also like to see some consideration given to shifting the responsibility for choosing the destination script type to the sender as I mentioned here.

@andrewkozlik the case of giving away dust from your wallet to prevent further tainting is an important one. If we do so, there is a single output, so the protocol should be allowed to bump fee.

Also it is important that the merchant can incur the fee bump cost to the sender, as if the original PSBT is equal to the min relay tx fee, the payjoin proposal is unbroadcastable. We can’t either to take the money from the merchant: If the merchant ask for 0.1 BTC, he expects receiving 0.1 BTC (or more).

If a sender does not want to support it, should we add a optional parameter?

The payment output should not decrease. […]

I concur.

Also it is important that the merchant can incur the fee bump cost to the sender, as if the original PSBT is equal to the min relay tx fee, the payjoin proposal is unbroadcastable. We can’t either to take the money from the merchant: If the merchant ask for 0.1 BTC, he expects receiving 0.1 BTC (or more).

Why can’t the merchant pay part of the fee, for the newly added inputs/outputs? If the merchant wants to add a new input, the overall size of the tx will increase. I don’t see why it must be the customer who pays the fee for the merchant’s input. The merchant can pay the fee by increasing whichever output they want to increase by a smaller value than the value of the input they added. (unless they added an input that costs more to spend than its value… do we really care about this edge-case?)

@SomberNight PayJoin for the most part is a privacy boon for sender, not receiver.

I don’t really wanna derail this discussion, but I don’t think this is true. I’d assume in normal usage it benefits the receiver more than the sender. Just the receiver might not actually care about privacy and just want consolidation benefits. And the receiver also does expose themselves to a privacy risk (i.e. a malicious entity aborting the coinjoin to learn a receivers utxo) that they would otherwise not be exposed to.

Well even if you’re right and they simply don’t care about privacy the downstream incentives are effected similarly with respect to allowing sender to pay for the privacy. Not trying to derail it’s just an important consideration.

On Wed, Jun 10, 2020, 2:57 PM Ryan Havar notifications@github.com wrote:

@SomberNight https://github.com/SomberNight PayJoin for the most part is a privacy boon for sender, not receiver.

I don’t really wanna derail this discussion, but I don’t think this is true. I’d assume in normal usage it benefits the receiver more than the sender. Just the receiver might not actually care about privacy and just want consolidation benefits. And the receiver also does expose themselves to a privacy risk (i.e. a malicious entity aborting the coinjoin to learn a receivers utxo) that they would otherwise not be exposed to.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-642196218, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFU3P6HYUJA5HBXWG2ALRV7JQVANCNFSM4NDBRK6Q .

Well even if you’re right and they simply don’t care about privacy the downstream incentives are effected similarly with respect to allowing sender to pay for the privacy.

I agree with you in theory

/tip 0.01 dollars

but the whole idea of trying to allow the sender to pay the receiver for a coinjoin is over-engineering and silly. It’s akin to allowing people to add a micropayments to tinder profile to incentivize the person to swipe right. Either the person wants to fuck you or not, and you’re not going to move the needle with some micropayment. Instead you devalue a simple mutually beneficial interaction by adding pennies. Unless you want to turn it into a full blown prostitution service (i.e. an actual mixing service) you are adding cognitive overhead and engineering complexity for nothing.

–

That said, it’s so bad that I don’t really care if it’s left in the proposal. Pretty much no wallet is ever going to want to give a user this UX lol:

“I want to send $X to $Y, and I’m willing to pay $Z in fees and pay up to an extra $P for it to be a payjoin”

so I think all implementations will just ignore it and we can safely pretend it doesn’t exist. If it was better designed (e.g. the receiver encoded a “payjoinfee=$P” in the BIP 21, and then wallets could say: “The receiver wants $P to do this payjoin, wanna do it?” I might have an objection as that’s a passable UX that people might actually use.

UX lol

That’s not how anything else in wallets works, the hyperbole doesn’t help. edit: to be clear I’m saying that hiding fee amounts in defaults is completely normalized in bitcoin wallets. Feerate, dust amounts, not to mention the various constants in LN wallets. It’s not a new UX burden at all.

That said, if you think merchants will do it for free/privacy, great sign me up. I’m definitely not an expert in that respect. I will stop polluting this thread since I at least understand the position now.

That’s not how anything else in wallets works, the hyperbole doesn’t help.

Well nothing in bitcoin works like “I am willing to pay up to $X”, so, no surprise. The only thing I can think works like that is ethereums gas limits. And it’s universal UX disaster when it needs to be exposed. But this is even worse, cause the only way you can sanely hide it is if maxadditionalfeecontribution=0 (which everyone will do, lol).

But nah, this is even worse because the receiver has to communicate OOB what their expectation for “incentives” are. It simply makes no sense. ( although maxadditionalfeecontribution could still have utility in the receiver carefully engineering their output-amounts to create confusing analysis. But let’s get realistic, no one will do that either).

That said, if you think merchants will do it for free/privacy, great sign me up.

I guess my point is that merchants will either do it, or not do it. No one’s behavior is going to be modified by senders willing to chip in a few cents. And no senders are going to be willing to pay real amounts of money, as you don’t even know what guarantees you’re getting.

edit: to be clear I’m saying that hiding fee amounts in defaults is completely normalized in bitcoin wallets.

If the amounts involved are so small that you can sanely just hide it (e.g. lightning actual routing fee) then you’re not going to incentivize shit.

I feel like maxadditionalfeecontribution has some pretty elegant properties, but it’s doomed in practice. ( I also believe the same thing about micro-tipping in online communities)

I will stop polluting this thread since I at least understand the position now.

Me too. I don’t really care much, because it’s easy to ignore. As a receiver you can always safely just ignore it. And as a sender, you can just not support setting it, which then reverts the harmful side-effects of it (no longer need to worry about output amounts dropping).

From UX perspective of the merchant, the merchant says: The invoice is 0.1 BTC he expects the sender to pay for the fees.

I explained the reason why in the BIP, and I don’t see any reason to come back on this decision.

For BTCpay Server point of view, there is a way to set a “Network fee” if the user is paying on-chain (those network fee are 0 for lightning), this is so the merchant can make the customer pay for dealing with on-chain UTXO.

See the “Network cost here”.

Lightning on the other hand don’t propose it.

If the receiver was not bumping the fee, it means that the merchant would need to have a different “Network cost” for sending with or without payjoin to get money back. There is two problems with this:

1. We don’t know the payjoin network cost in advance, we know it only when the sender actually pay.
2. It means that we would not be able to expose the pj endpoint as BIP21, because we would need another “payment method” so we can reflect this different “network cost” to the sender.

Last, if the receiver was paying the fees. It means he would have to manually approve it, which is not possible as he might not be online when he actually receive the payment. On the other hand, the sender is online.

I don’t want to come back on this decision to make the sender pays for the fees.

If you really don’t want, as a sender, to not pay fees, just use maxadditionalfeecontribution=0, this is perfectly fine, we support it. It just means the receiver will add his inputs and not bump the fees. The fee rate will drop, if it goes below min relay fee the proposal will fail. The only thing to make sure for the sender is to disable RBF.

However, it means that anytimes you want to attempt a payjoin, you will have to pay fees slightly above the min relay tx fee. Also, the transaction will be slightly slower to confirm. Note that a nice receiver can still decide to pay from his pocket in this situation. (maybe we will add a setting for this in btpcay)

I can’t help feel maxadditionalfeecontribution is poorly designed, even for it stated purpose. As a sender wtf should I even pick? No seriously; I’m not even being facetious; I actually don’t know what value to use..

If I was writing a sender and someone put a gun-to-my-head (or paid me by the hour) and really wanted to support this, I’d probably set it to 99999999999999 and then when the receiver returned a proposal prompt the user: “Receiver wants to charge you $X and do $Y to your fee rate. You cool with that?”. If the user wasn’t cool with it, I’d send the original transaction. If the user was, I’d send the payjoin

But the reason it feels silly, is that you already have a “round” of communication. Namely the bip21 string. The receiver can encode it’s fee policy / expectation in it (i.e. I’m going to add N weight and M fee) and the sender could just plan around that without the goofy interaction.

Probably the best way to find out what is best, is just ask the green-wallet people for their opinion and then do the exact opposite. I am confident we will arrive at the correct decision, even if they know what we’re doing.

Regarding sender complexity:

If you really don’t want, as a sender, to not pay fees, just use maxadditionalfeecontribution=0, this is perfectly fine, we support it. It just means the receiver will add his inputs and not bump the fees. The fee rate will drop, if it goes below min relay fee the proposal will fail. The only thing to make sure for the sender is to disable RBF.

However, it means that anytimes you want to attempt a payjoin, you will have to pay fees slightly above the min relay tx fee. Also, the transaction will be slightly slower to confirm. Note that a nice receiver can still decide to pay from his pocket in this situation. (maybe we will add a setting for this in btpcay)

To resolve the unpredictable confirmation time issue the sender could just set both maxadditionalfeecontribution=0 and minfeerate=originaltxfeerate.

That way any receivers that support bumping fees from their own funds can propose valid payjoins and any that don’t will just reject the proposal and broadcast the original transaction.

No need for the sender wallet to implement complex maxadditionalfeecontribution stuff, and they can also be sure the transaction will confirm in a predictable time frame that the user expects.

The downside is you’ll be limiting the amount of receivers you can successfully payjoin with to the amount of receivers willing to bump fees which could be pretty small.

Also regarding payjoin being mostly a benefit to the sender, I’m not sure I agree with that.

I think the direct privacy gain to the actual sender is relatively small. The big privacy benefit of payjoin is that with enough usage it breaks the common-input-ownership heuristic, which is a general privacy gain to all Bitcoin users, of which both the sender and receiver benefit from equally.

However, from the sender perspective, if payjoin is not used carefully it has the potential to result in significant privacy loss.

Consider the following scenario:

• User visits clearnet merchant webite
• User uses Tor Browser to hide their IP from merchant
• User makes a purchase by scanning a QR code with their mobile wallet
• Mobile wallet is not using Tor
• User DOES NOT opt in to payjoin
• Mobile wallet broadcasts a transaction

The merchant learns nothing about the sender’s identity.

Now using payjoin:

• User visits clearnet merchant webite
• User uses Tor Browser to hide their IP from merchant
• User makes a purchase by scanning a QR code with their mobile wallet
• Mobile wallet is not using Tor
• User DOES opt in to payjoin
• Mobile wallet requests clearnet payjoin endpoint
• Mobile wallet broadcasts a payjoin transaction

The merchant learns the sender’s mobile device’s IP from the clearnet payjoin request.

In this scenario the user experienced a significant reduction in privacy when using payjoin compared to when not using payjoin.

@RHavar before maxadditionalfeecontribution I was explicitly saying “The sender can’t pay more than 2 times the fee”. I think this is a good rule of thumb, the sender now can decide if that’s a good rule or not by setting this parameter.

On top of this, the sender is actually checking that the fee rate did not increase already. So maxadditionalfeecontribution is only controlling the max number of inputs that the merchant can add. What about instead of maxadditionalfeecontribution we should have maxadditionalinputs?

The downside is that this prevent the merchant adding them at his own cost.

@RHavar before maxadditionalfeecontribution I was explicitly saying “The sender can’t pay more than 2 times the fee”. I think this is a good rule of thumb, the sender now can decide if that’s a good rule or not by setting this parameter.

Yeah, I get what you’re doing; and it makes sense for sure, I just don’t get how it works nicely. Like say the transaction fee is $1, if you double that it’s now $2 – which is a significant enough amount of money that wallets can’t hide it. So the sender is like “ok, I’m fine willing to pay $2 for a payjoin” and then it turns out it only costs $1.50 (say because the receiver added only 1 input, and the original transaction had 2).

I mean it all works fine, but picking upper-bounds is always going to be a little clumsy. I would just get rid of all the “max” stuff, and provide the actual value as a hint. Like the bip21 URL the receiver tells the sender it’s “fee policy”. (i.e. how much weight it’ll add to a transaction, and how much fees it’ll add to a transaction).

I understand it’s a bit less flexible as now the receiver can’t dynamically adjust it on a per-transaction level. But it’s flexibility that realistically no one will use.

And now the sender can plan around that exactly, and give that information to the user.

I think that’s all you really need to do; but it does come with a significant drawback; that in the (normal) sender-pays case, the sender might end up paying (in tx fees) for a payjoin but ends up falling back to the original transaction.

But you can pretty much fully mitigate this by just saying that a well-behaved receiver should never propagate the original transaction unless the sender does something wrong (e.g. not go through with the payjoin). And say a receiver is not in a position to do payjoins (e.g. has no inputs, or some other internal problem) it just gives an error to the sender and the sender is responsible for sending a normal payment (which may or may not be the original transaction)

@RHavar the problem is that the receiver don’t really know the actual value until he actually receive the original transaction.

I think from UX experience, in a nutshell of all you said, this is the flow you would want:

0Broadcast via Payjoin: 2.5$ of fee
1Normal broadcast: 2.0$ of fee

versus what we have now (without maxadditionalfeecontribution):

0Broadcast via Payjoin: Continue.
1Normal broadcast: 2.0$ of fee

Then a second screen

0Broadcast via Payjoin: 2.5$ of fee
1Normal broadcast: 2.0$ of fee

@lukechilds also wanted to remove this second confirmation. Which is why I created maxadditionalfeecontribution.

With maxadditionalfeecontribution you can have the exact same experience with a single screen

0Broadcast via Payjoin: Maximum 2.5$ of fee
1Normal broadcast: 2.0$ of fee

The main problem you are pointing out is that this estimated value is hard coded in the wallet via maxadditionalfeecontribution and that there is no way to know which value would be right.

But the problem is that from the receiver side, the problem is the same. We have no way to know what is the actual fee unless we receive the original transaction. (This is not mentioning part of the fee may be paid by receiver)

Which is why, I think a better way of dealing completely with this problem is to have a maxadditionalinputs. If you set that to 1, you know exactly how the weight will be affected in advance. You know the weigth, you know the rate BTC/USD, you can show exactly the fee that the user should expect if he tried payjoin without any surprise. @lukechilds what do you think about maxadditionalinputs ?

maxadditionalinputs certainly seems like a simpler solution, solves all the confusing UX implications, we just show users two options, with two different fees, and they pick one.

Plus I imagine maxadditionalinputs=1 is unlikely to be rejected by most merchants so shouldn’t have much impact on the success rate of the payjoin.

Although aren’t we still allowing the receiver to add additional outputs for things like payment batching? This could also effect the fee if we’re trying to maintain a specific fee rate.

@lukechilds the BIP is specifying that the receiver should not free ride on this. As such, for a sender here are the only point to check:

• maxadditionalinputs is respected.
• The fee rate is not increasing.
• The amount of additional money sent is less or equal to the Actual Additional Inputs size * fee rate.

@RHavar the problem is that the receiver don’t really know the actual value until he actually receive the original transaction.

I think from UX experience, in a nutshell of all you said, this is the flow you would want:

0Broadcast via Payjoin: 2.5$ of fee
1Normal broadcast: 2.0$ of fee

versus what we have now (without maxadditionalfeecontribution):

0Broadcast via Payjoin: Continue.
1Normal broadcast: 2.0$ of fee

Then a second screen

0Broadcast via Payjoin: 2.5$ of fee
1Normal broadcast: 2.0$ of fee

@lukechilds also wanted to remove this second confirmation. Which is why I created maxadditionalfeecontribution.

With maxadditionalfeecontribution you can have the exact same experience with a single screen

0Broadcast via Payjoin: Maximum 2.5$ of fee
1Normal broadcast: 2.0$ of fee

The main problem you are pointing out is that this estimated value is hard coded in the wallet via maxadditionalfeecontribution and that there is no way to know which value would be right.

But the problem is that from the receiver side, the problem is the same. We have no way to know what is the actual fee unless we receive the original transaction. (This is not mentioning part of the fee may be paid by receiver)

Which is why, I think a better way of dealing completely with this problem is to have a maxadditionalinputs. If you set that to 1, you know exactly how the weight will be affected in advance. You know the weigth, you know the rate BTC/USD, you can show exactly the fee that the user should expect if he tried payjoin without any surprise. @lukechilds what do you think about maxadditionalinputs ?

If you are changing and adding outputs, the fee rate would still vary though.

Also, if maxadditionalinputs=5, but the receiver adds only 3, the fee rate would be different from what the sender was shown. Could a receiver attempt to game the difference in input to themselves?

If maxadditionalinputs is used, I would still recommend to allow additional inputs, only paid for by the receiver (eg maxadditionalinputs is only to compute the sender max fee). Without this, payment batching could be severely restricted as you could only batch payments when sufficiently larger utxos are available.

@RHavar the problem is that the receiver don’t really know the actual value until he actually receive the original transaction.

I think we’re talking past each other. My idea is that the receiver communicates to the sender its policy on how it modifies the transaction fee and weight.. The most general way of doing this (which to be clear, i’m not advocating), would be the the receiver gives (via bip21 url) an arbitrary function, like:

modificationPolicy :: transaction -> (feeChange, weightChange)

Given this function (modificationPolicy) the sender knows what the receiver plans on doing, and can plan around it.

So let’s say I’m a sender. And I know the receiver is going to add $N of weight, and $F of fees. I want to send a transaction with $FEERATE. I know a payjoin is going to cost me: ($BASEWEIGHT + $N) * $FEERATE - $F and a normal transaction will cost $BASEWEIGHT * $FEERATE

Now I can tell the user exactly how much they’re going to have to pay for a normal transaction and how much for a payjoin. If the user picks a payjoin, I create an “original transaction” with a fee of ($BASEWEIGHT + $N) * $FEERATE - $F and send it to the payjoin endpoint. If a user doesn’t want to create a payjoin, I create a transaction with a fee of $BASEWEIGHT * $FEERATE

By doing this, you can both have a nice UX and sidestep a lot of complexity. The only downside I really see, is that a sender might pay extra for a payjoin but end up not getting it. But you can pretty much fully mitigate this by just saying that a well-behaved receiver should never propagate the original transaction unless the sender does something wrong (e.g. not go through with the payjoin).

And for obvious reasons, you don’t want the receiver to pass an arbitrary function modificationPolicy. You kind of want to “Whitelist” some functions that make sense. I’m going to go on a limb here, and say the “modificationPolicy” of 99.9%+ of all senders can be expressed with two constants: how much weight they will add to a transaction. And how much fees they will contribute to a transaction (which will generally be zero).

@Kukks

Also, if maxadditionalinputs=5, but the receiver adds only 3, the fee rate would be different from what the sender was shown. Could a receiver attempt to game the difference in input to themselves?

So what? There is no reason for the sender to refuse a transaction in which he pays less than the expected fee.

If you are changing and adding outputs, the fee rate would still vary though.

No, the fee rate should not vary, and only contributed inputs should be payable by the sender, not the rest.

Also, if maxadditionalinputs=5, but the receiver adds only 3, the fee rate would be different from what the sender was shown. Could a receiver attempt to game the difference in input to themselves?

This is an interesting point, it messes up the incentives.

If a sender sets maxadditionalinputs to a value more than one, why would a receiver ever bother adding more than one input if they can just stick to one and then take the extra sender fees for themselves.

If receivers do this, then all sender implementations would probably just limit to maxadditionalinputs=1 to prevent it, essentially making it a hard limit.

why would a receiver ever bother adding more than one input if they can just stick to one and then take the extra sender fees for themselves.

They can’t. The fees are on the actual added input count.

@RHavar I am not too queen to rewrite a whole new BIP on this idea, reimplementing from everything scratch when the maxadditionalinputs I suggest allow you to have the exact same UX you want.

Think from the user’s perspective. What you want is to give him a choice of 2 fee amount that he will pay and let him choose, and not ask him to confirm a second time. The maxadditionalinputs achieve exactly this.

In case the receiver contribute less than the maxadditionalinputs, this is not a problem. The user should only care about paying the actual added inputs. He would just pay less fee than he would have expected, and no need to ask for a second confirmation.

They can’t. The fees are on the actual added input count.

Ahh, yes! I was confusing myself thinking the sender needs to pay the fees for the potential max number of inputs on the initial TX for some reason.

So:

• Sender sets desired fee rate on initial TX and sets maxadditionalinputs=n.
• Receiver adds <= n inputs and takes required funds from senders change output to maintain the same fee rate.
• Sender checks, signs, and broadcasts.

If receiver adds n inputs the tx will require the maximum amount fee the sender confirmed was ok for a payjoin tx. If the receiver adds <n inputs the sender pays less fees than they confirmed but still gets the same fee rate they confirmed.

Is my understanding correct now?

@lukechilds

Correct. Basically, as far as the sender is concerned, he can say: “Ok, 1 input = 300 weight, so I must pay at most for 900 weight if my maxadditionalinputs=3” now if the actual inputs is 2, the max the sender will agree to pay for 600 weight maximum.

He can check that those 600 weight has actually been sent to fee by making sure the absolute fee at least increase from 600 weight.

By doing this, he can make sure that the receiver don’t abuse by consolidating too much or bundling his own outputs. And it gives the freedom to the receiver to increase fee as he see fit.

I’m probably missing something, but doesn’t maxadditionalinputs makes things worse than before?

Firstly it’s a bit confusing because it’s actually “maxadditionalinputsthatthesenderwillpayfor” and then secondly, the first thing a receiver then just needs to do some maths with the minfeerate to figure out how much the sender is willing to pay, then use that. The receiver then also needs to do the maths to verify the receiver did it correctly.

So at that point, why not just send “payjoinExtraFee” (along with minfeerate?) directly? It’s actually less confusing (as the sender is directly saying how much extra they will pay for a payjoin, which will generally be enough for the receiver to add 1 extra input at minfeerate) and allows more precision.

@NicolasDorier would maxadditionalinputs replace maxadditionalfeecontribution?

Yes I don’t see any reason to keep maxadditionalfeecontribution as I added it to solve the UX issue @lukechilds was complaining about, but which can be better solved via maxadditionalinputs.

So at that point, why not just send “payjoinExtraFee” (along with minfeerate?) directly?

That was what the maxadditionalfeecontribution was about, but as you said, it has a difficult UX. At least maxadditionalinputs is pretty clear: The receiver can add x inputs at the fee rate of the original transaction. This also better explains that the sender is only supposed to pay for additional inputs.

Actually we should allow the receiver to add inputs as long as he pays for the fee. I think it is easy to verify.

But what if the transaction has no output??

This is the spare change case. The receiver can decrease the amount on the only output. This basically mean the receiver pays for fee. (In BTCPay we allow it, if the sender pays 0.11 for a 0.1 invoice, then the receiver will use the 0.01 surplus to pay for the fee) If there is no surplus, then we still create the payjoin proposal, as long as it meet the min relay fee rate.

Are you really sure you don’t just want to do the bip79 way?

This is really crucial for us that the senders can pay for it.

I will reimplement our receiver/sender based on all this feedback. My goal is to make a reference implementation I can copy/paste in this BIP to demonstrate that making the sender is easy.

That was what the maxadditionalfeecontribution was about, but as you said, it has a difficult UX.

I still don’t love it, but it’s not so bad if you provide a sensible suggested value (“use minFeeRate * expected input weight). And I’d also prefer not having “max” but providing additionalfeecontribution

At least maxadditionalinputs is pretty clear: The receiver can add x inputs at the fee rate of the original transaction. This also better explains that the sender is only supposed to pay for additional inputs.

I don’t think it’s very clear, it’s more “im willing to pay for X inputs” an indirect way. And that’s even less clear what it means for for mixed input cases etc.

This is the spare change case. The receiver can decrease the amount on the only output. This basically mean the receiver pays for fee. (In BTCPay we allow it, if the sender pays 0.11 for a 0.1 invoice, then the receiver will use the 0.01 surplus to pay for the fee)

Woah, Hang on……. This is starting to get awful. You are now having wildly different behavior and fee-burden depending on if the transaction happens to have change or not. And you’re giving extra incentive for the sender to now use a changeless transaction, cause they know the extra fees will instead be paid for by the other party. I know of two coinselection algorithms that would allow simply encoding this and would allow picking changeless solutions with even higher frequency.

BTW what happens if I send a 1-input-1-output payjoin to BTCPay and pay $500 in fees. Is BTCPay going to chip in another ~$500 to match the fee rate? Or do you also now need some sort of maxfeerate?

I like what you’re doing with this, but I feel like you’ve generalized stuff that doesn’t need to be generalized. And somehow managed to not handle common cases elegantly (like changeless transactions, and the ability for the receiver to communicate he’ll chip in a fixed amount because he either likes payjoins or because he won’t have to consolidate)

Are you really sure you don’t just want to do the bip79 way?

This is really crucial for us that the senders can pay for it.

Hm? I think I’ve mentioned quite a few times now that bip79 allows this, and it’s the common case and in fact how even the reference implementation works. bip79 just also allows the flexibility for the receiver to chip in if they want

Another annoying case: You payjoin 1 BTC to X. Your wallet was able to send it without change. The receiver replies back with the coinjoin proposal with you paying 1 satoshi to X and the rest to tx fees. Should the sender sign it?

If yes, from a practical point of view that’s annoying because if a payjoin receiver can be malicious it’s harder to outsource and now part of the critical infrastructure to be secured. If the payjoin (receiver) server is not capable of being malicious, a business is able to deploy it in production a lot easier setting (and provide it with some imported funds it can mix into payments).

–

bip79 is dead, so I very much support your proposal because it seems to likely have momentum to be implemented; but I think you’d probably be better off just taking bip79 and make the following changes:

• stop using hex-encoded transactions, and use PSBT (this is a nice improvement that makes a lot more sense considering its support now)
• enforce that the receiver doesn’t shuffle inputs/output (as per earlier discussion for hardware wallets) and only randomly insert
• maybe standardize error codes (if that’s useful, then go for it)

And I think it’s pretty solid. I think the rest of the stuff in this bip is kinda useless, worse or over-engineered. A good example is trying to introduce versioning before you have anything to version, when if you find there are things you want to version it’s just trivial to add “&v=2” to the bip21 URL if you want backwards compatibility, or change “pj” to “pj2” if you want backwards incompatibility depending on the exact change.

And then if you want to hit a stretch-goal, I’d add a feature I regret not adding to bip79: The bip21 URL should encode the “modificationPolicy” (as I discussed earlier, which I’d do just via 2 constants)

And you’re giving extra incentive for the sender to now use a changeless transaction,

No we don’t. What happen is: If you need to pay an invoice 0.1 BTC, and don’t want change back, you will probably pay 0.11 BTC. The receiver substract the fee from the single output, up to 0.1 BTC. Which is basically the receiver paying for the fee up to 0.01 BTC. There is no incentive for the sender as it is possible only because the sender is over paying in the first place.

stop using hex-encoded transactions, and use PSBT (this is a nice improvement that makes a lot more sense considering its support now)

Yes, I agree, this is already the case. We support hex in BTCPayServer, but not in the BIP.

enforce that the receiver doesn’t shuffle inputs/output (as per earlier discussion for hardware wallets) and only randomly insert

Agree

maybe standardize error codes (if that’s useful, then go for it)

I did, I tried to keep only errors which are relevant for the sender.

A good example is trying to introduce versioning before you have anything to version, when if you find there are things you want to version it’s just trivial to add “&v=2”

I introduced that just so supporter of v1 can show a proper error message instead if there is an incompatible version. IMHO, it can be ignored by implementer for now.

About modificationPolicy, I understand your point, but you can already reach the same UX with the maxAdditionalInputs. (Where you show the user two choices with precisely how much fee he is expected to pay, with pretty good accuracy)

Another annoying case: You payjoin 1 BTC to X. Your wallet was able to send it without change. The receiver replies back with the coinjoin proposal with you paying 1 satoshi to X and the rest to tx fees. Should the sender sign it?

I would say: As long as the sender sends the amount of money, or less, than what he wanted to, he should accept it.

I really need to implement all the suggestions we talked about to see if this is easily implementable. And if not, see what are the pain point.

I think maxadditionalinputs is misleading in its naming: I would expect that the receiver can only add x amount of inputs. As @RHavar said, it’s more like maxadditionalinputsthatthesenderwillpayfor.

I also cannot see any incentive for a sender to set maxadditionalinputs to anything over 1. It basically tells the receiver “I don’t want to pay for anything beyond the bare essential to create a payjoin (add one input)”.

What happen is: If you need to pay an invoice 0.1 BTC, and don’t want change back, you will probably pay 0.11 BTC. The receiver substract the fee from the single output, up to 0.1 BTC. Which is basically the receiver paying for the fee up to 0.01 BTC. There is no incentive for the sender as it is possible only because the sender is over paying in the first place.

Ok, this makes sense. But (AFAICT) strictly worse than BIP79, unless I’m missing something (which is quite likely) I actually don’t single a single advantage.

Let’s imagine greenaddress wanted to (or more likely: was forced to) return the 0.1 BTC of mine that they first blacklisted and then assumed de facto custodial control of and decided the best way to do this was with a 0.1 bitcoin changeless payjoin.

In your scheme they would send me an original transaction for 0.11 BTC, I would add my input of $N amount and then modify the output to something like 0.11 + $N - 0.01. But in BIP79: they would send me an original transaction for 0.1 BTC with an extra (on top of the normal feerate) 0.01 BTC in fees. I would add my input, and modify the output to 0.1 + $N

So why is BIP79 vastly better at handling this?

• The receiver has less incentive to screw the sender. In your proposal if the receiver broadcasts the original-transaction (i.e. doesn’t complete the payjoin) they will literally get to pocket more money. While in BIP79 if the receiver broadcasts the original transaction (i.e. doesn’t complete the payjoin) it will get the same amount of money (just faster, as the txfee is higher).

• It is simpler for the sender. In this proposal the sender the sender needs to use a customized coinselection to say: “I want to pay 0.11 if the transaction is changeless, but 0.1 if it has change” vs bip79 which you run coinselection as normal

• It is far simpler for the receiver, not requiring access to information it quite likely doesn’t have. For this to work well, the receiver (’s payjoin server) needs aware of the invoice-amount to know the “extra” the sender added. In practice, this is very awkward to do. Because normally in bitcoin wallets you just add/create addresses, but don’t attach an expected-invoice amount to them. So where I have deployed bip79 it just sits as a little layer above the wallet and needs no information the wallet doesn’t already have.

But in BIP79: they would send me an original transaction for 0.1 BTC with an extra (on top of the normal feerate) 0.01 BTC in fees. I would add my input, and modify the output to 0.1 + $N

The issue is that if the original transaction is RBF, the payjoin transaction will be replaced by the original.

Also, this scenario is actually supported by this BIP. As a sender, you can do what you just said. The way the receiver behave is: If it is not possible to bump the fees because of some constraints (say maxadditionalfeecontribution set to 0, or, as you in this case, not having change outputs to substract money from), the receiver will not bump the fee and attempt the payjoin if it has fee above min relay tx fee.

This is a decision from the sender.

I also cannot see any incentive for a sender to set maxadditionalinputs to anything over 1. It basically tells the receiver “I don’t want to pay for anything beyond the bare essential to create a payjoin (add one input)”.

I don’t see either, but if you force the BIP by saying “there is only 1 input” this is an important information that can be used by chain analysis.

The issue is that if the original transaction is RBF, the payjoin transaction will be replaced by the original.

This isn’t a problem. In more ways than one:

Firstly, the only reason the original transaction should be broadcasted after the payjoin is broadcasted is if one of the parties is a dickface. But if one of the parties is a dickface they can always broadcast the original transaction without completing the payjoin. So, really, who cares?

Secondly, I don’t believe your reading of bip125 is correct. The original transaction and payjoin transaction pay an identical amount of fees, and the bip125 rules say that a replacement transaction must pay (minFeeRate * sizeOf(replacementTransaction)) + originalTransactionFee and minFeeRate is never zero, so it’ll never work.

Thirdly, designing around a protocol around bip125 is probably practical but not ideal or extremely future-proof. bip125 is kind of like a “minimal viable product” imo, and could radically be improved to allow vastly more transactions be eligible for replacement.

I don’t see either, but if you force the BIP by saying “there is only 1 input” this is an important information that can be used by chain analysis.

You might as well just remove the (confusing) param and specify something like: “If the receiver adds more than MAX(inputWeights) weight to a transaction, it is expected to pay for it”.

It really makes no difference from a blockchain analysis point of view, as the receiver is still allowed to add more than 1 input.

BTW probably the only reason blockchain analysis even works is because of the assumptions they make based on observing how people actually use bitcoin vs. what is allowed. Merely allowing a behavior that no one uses isn’t going to make a difference.

Also, this scenario is actually supported by this BIP.

It’s not that it isn’t supported, it’s just that it’s strictly worse at it than the BIP it replaces. I outlined some reasons it’s worse, and I still haven’t been able to see a single advantage.

You are doing great work, and I support your BIP regardless of the outcome of this point. And I have enough self-awareness to know I sound like a dick, but not enough self-control to stop myself: but I wonder if your reason for making these changes were due to misunderstanding of the original BIP? If so, you shouldn’t feel bad – I am pretty bad at the English – but you shouldn’t let that stop you just reverting to how BIP79 does it. I saw it deployed in a production environment, and how it handles fees/who-pays really quite well. There’s really no point complicating it.

“If the receiver adds more than MAX(inputWeights) weight to a transaction, it is expected to pay for it”.

No, because the sender is then unable to know the fees before making the payjoin. Which is what we want to fix in the first place.

It’s not that it isn’t supported, it’s just that it’s strictly worse at it than the BIP it replaces.

I do not understand your point. What is possible in BIP79 is still possible in BIP78 with the right parameters (which can be hard coded in the sender). The sender has the choice. ~The only complication is on the receiver side, but frankly the receiver is not that complicated to do right compared to the sender~ (EDIT: Actually let me think about this point, because I receiver seems to be able to behave like BIP79 without breaking the sender either.)

Secondly, I don’t believe your reading of bip125 is correct. The original transaction and payjoin transaction pay an identical amount of fees, and the bip125 rules say that a replacement transaction must pay (minFeeRate * sizeOf(replacementTransaction)) + originalTransactionFee and minFeeRate is never zero, so it’ll never work.

That is interesting, I need to look more how the code behave in Bitcoin Core. If you are right, it means that the policy is not enforcing the best interest of the miner! In the case where the block space is scare, the miner want to maximize the fee rate, not the absolute fee.

No, because the sender is then unable to know the fees before making the payjoin. Which is what we want to fix in the first place.

Yeah it does. The sender should be able to assume the receiver will add: MAX(inputWeights) of weight and 0 of fees, and plan according. If the receiver does add more, it should pay additional fees to keep the fee rate the same as if it just added MAX(inputWeights) of weight and 0 in fees.

If you are right, it means that the policy is not enforcing the best interest of the miner! In the case where the block space is scare, the miner want to maximize the fee rate, not the absolute fee.

That’s correct. Hence why I think of bip125 as a “minimal viable product”, rather than a finished one. It’s a set of rules that is obviously robust against relay spam, but it does so in an overly restrictive way. Although it’s not particularly easy to solve well. Like you don’t really want someone replacing a 100KB transaction paying N feerate, with a 100 byte transaction paying N+1 feerate. It might be in miners interests to do so, but the relay network will go to hell.

A few weeks ago I was discussing an idea with some people that drastically changing how the bitcoin relay network works, would be able to obsolete all the hardcoded rules like min-relay-fee, bip125 replacement rules, etc. into something that would always be optimal for miners. But even if the idea is solid, the amount of engineering effort required to implement it would be staggering (which I don’t think anyone is remotely volunteering ) .

That is interesting, I need to look more how the code behave in Bitcoin Core. If you are right, it means that the policy is not enforcing the best interest of the miner! In the case where the block space is scare, the miner want to maximize the fee rate, not the absolute fee.

I have already pointed this out above… :P The BIP is about the absolute fee, and that paragraph in this text is redundant and confusing.

I also cannot see any incentive for a sender to set maxadditionalinputs to anything over 1. It basically tells the receiver “I don’t want to pay for anything beyond the bare essential to create a payjoin (add one input)”.

I don’t see either, but if you force the BIP by saying “there is only 1 input” this is an important information that can be used by chain analysis.

"there is only 1 input" The property is only to tell the receiver of how many inputs they can “freeload” off the sender, if a receiver wanted to add more inputs, they can just pay the feerate difference.

It becomes a well known assumption that no sender would set this to >1 then.

If I’m understanding this right, maxadditionalinputs is a useless configuration and the codebase should just assume that the sender will ALWAYS only pay for 1 contributed input and the receiver pays the rest. This makes it easy enough for the sender to show the user what they can expect to pay in total with no hassle.

~So in summary we remove maxadditionalinputs and maxfeecontribution and hardcode in the BIP the fact that the sender expect at least 1 input and will pay for it. And can pay only for that. That’s fine to me.~ ~Actually @Kukks the sender may want to put at maxadditionalinputs 0 to say to the receiver he won’t pay for anything. Which is also fine.~ @RHavar @SomberNight you are right concerning BIP125, and it seems the only reason to bump the fee is actually to be able to reach minrelayfee! It is painful to handle this case. We can’t pay on the receiver case without massive pain in the butt UX wise either.

Their is also the case that some wallet (wasabi) have round fee rate and thus, not bumping fee would create fee that stand out.

Given that now, with maxfeecontribution it is already possible for a sender to order the receiver to not bump the fee, I wonder if we need change the BIP at all. Maybe just specify that if maxfeecontribution is not specified, it should be considered 0. This would make everybody happy.

You expect to pay for 0 inputs? just set maxfeecontribution to 0. If you do that today in btcpayserver, the receiver would still add 1 inputs, not bumping the fee.

If you create a receiver, you can also just ignore maxfeecontribution completely without breaking senders: Senders do not care about paying less.

There is reason to set maxfeecontribution to something other than zero: The minrelaytxfee case and the “round fee rate” case. This should really satisfy all cases.

Something unrelated to current discussion but would this scenario be possible:

• Person A requests a payment from Merchant of 0.05BTC
• Person A requests a payment from Merchant of 0.05BTC
• Customer C payjoin with merchant of 0.1BTC payment
• Merchant coordinates payjoin to pay for payment requests of Person A and Person B in payjoin, without even needing to add an input!

This effectively ends up being one transaction with 4 parties involved, where the merchant basically becomes a coordinator and never touches the money.

I think that maxfeecontribution is unnecessary and overkill. I agree it has some (marginal) upsides, so I don’t mind too much either way, but I think you should carefully consider if you really want to complicate the protocol with it.

I’m not sure if you’re planning on keeping it in the revised protocol, but what I am strongly against is if the invoice is for $X, allowing, or expecting the sender to send $Y (e.g. changeless case) with the expectation that ($Y-$X) is used as a fee contribution.

While it seems innocuous, this has pretty drastic consequences to the protocol. And the exact same thing can be done by the sender by using a higher-fee-original-transaction.

–

P.S. I think the protocol should make an explicit assumption that the sender will pay for 1 (MAX(inputWeights)) worth of extra weight, and that the receiver should be allowed to add more weight but there’s an assumption that if he does so, he’ll chip in the fees required to not lower the feerate. This will add a lot of predictability for everyone

@RHavar I don’t see how it has drastic consequence as both receiver implementation AND sender implementation can completely ignore it without being incompatible with those who does not ignore it.

I think the protocol should make an explicit assumption that the sender will pay for 1 (MAX(inputWeights)) worth of extra weight,

I disagree. A sender might wants to pay for 0 of extra weight, and that’s fine. There is several reason where he agree to pay for more:

• Min fee rate
• Spare change
• Obfuscation of round fee rate

But outside those, with BIP125 only caring about absolute fee, it is perfectly fine to not want to pay any additional fee (I would say advised to!), does not mean the receiver has to pay either.

@RHavar I don’t see how it has drastic consequence as both receiver implementation AND sender implementation can completely ignore it without being incompatible with those who does not ignore it.

I think I’ll let you first revise your spec and solidify these things, and these things should become a bit more clear and I can be clear I’m understanding what you’re proposing.

But giving the sender two very different, but equivalent, ways to “pay for” a payjoin is a disaster. There’s no realistic way to sanely handle this as a receiver without knowing the “invoice amount” (or if there is one) and I’ve explained why that’s very annoying. And in the most general cases, the sender will actually need to figure out which way the sender is “paying for” the payjoin to process it. That is just a bizarre burden to shove on people.

And I don’t want to repeat myself by enumerating all the other disadvantages for both the sender and receiver (even if I concede it works and is possible).

But let me put it another way: Unless you can tell me a single advantage (which I don’t believe one exists) this proposal is objectively worse than the BIP79 way for changeless transactions.

I disagree. A sender might wants to pay for 0 of extra weight, and that’s fine.

Sure, I agree. But this is a pretty niche case, and in theory already supported without any protocol changes. Basically just the sender looks at the payjoin proposal and based on that decided to go ahead with it or not.

It’s pretty barebones, but I think it’s good enough for now. If it’s something you really want to support, you should do it properly … which I imagine means the receiver first (via bip21?) communicates it’s fee policy and then the sender decides if they want to do a payjoin or normal transaction.

But outside those, with BIP125 only caring about absolute fee, it is perfectly fine to not want to pay any additional fee (I would say advised to!), does not mean the receiver has to pay either.

In the BIP79 implementations I was involved in, the receiver always adds 1 input and increase the payment output the value of that input. (i.e. leaves the absolute fee the same, but lowers the fee rate by a predictable amount) and the sender knows how much the receiver is going to lower the feerate by, so it plans accordingly. [But that’s not required by the spec]

It’s not perfect, but it’s good and works well. It’s also stupidly simple and has very nice properties. I really urge you to use it as the base for your work, as frankly it’s just better and simpler :D

Yeah, looking a lot better for sure 👍 I’ll stop commenting on the fee stuff.

But circling back to “output substitution” – can we please just get rid of it? It has some obvious upsides, but it does contain some very significant downsides. And not just from the sender, but the receiver too. If the payment output can be substituted, now the payjoin server is far more security sensitive and going to be far harder to get deployed.

My strong preference would be for for the rule to say the only thing the receiver is allowed to do to outputs is increase or leave the amount the same.

But if this is really feature you really want, why not add it as a “feature flag” sort of thing? So say the receiver (might) do output substitution, in the bip21 URL it adds “os=1” and know the sender knows the reciever supports it. And if the sender also supports it (e.g. it’s not a hardware wallet) it also adds “os=1” when sending the request.

That I think pretty much has all the advantages of output substitution, but also allows both sender and receivers who don’t want it, to sort of opt out.

Yeah, looking a lot better for sure 👍 I’ll stop commenting on the fee stuff.

But circling back to “output substitution” – can we please just get rid of it? It has some obvious upsides, but it does contain some very significant downsides. And not just from the sender, but the receiver too. If the payment output can be substituted, now the payjoin server is far more security sensitive and going to be far harder to get deployed.

My strong preference would be for for the rule to say the only thing the receiver is allowed to do to outputs is increase or leave the amount the same.

But if this is really feature you really want, why not add it as a “feature flag” sort of thing? So say the receiver (might) do output substitution, in the bip21 URL it adds “os=1” and know the sender knows the reciever supports it. And if the sender also supports it (e.g. it’s not a hardware wallet) it also adds “os=1” when sending the request.

That I think pretty much has all the advantages of output substitution, but also allows both sender and receivers who don’t want it, to sort of opt out.

Why do you feel that output substitution is bad? I fail to see the downsides around it.

Why do you feel that output substitution is bad? I fail to see the downsides around it.

There are certainly pros and cons.

(1) If output substitution is allowed, a merchant/receiver can’t really give you signed invoices. Let’s say a merchant signs a message saying they will deliver goods upon receiving X bitcoins to address A, and they put a bip21 URI in the signed message too. If the URI uses payjoin, and payjoin allows replacing output addresses then the user would need to be prompted to verify the replaced address as part of the flow.

E.g. the user starts a payment, verifies the address, it matches with the signed address, clicks “send”, and the payjoin flow starts. The receiver replaces the address, and either

• the user now needs to be prompted again to manually verify the payjoin tx
• or if the software accepts any substitution then the merchant can trick the user by having them pay to a different address that they did not sign

(2) Without output substitution, and assuming the receiver is not allowed to decrease output amounts, I imagine it would be possible for hardware wallets to auto-sign (without prompts) the payjoin tx – only prompting the user once, at the beginning, to confirm the original tx.

@RHavar for output substitution:

• If the receiver think it has downside, he should not use it. This is not a mandatory feature.

if the sender also supports it (e.g. it’s not a hardware wallet) it also adds “os=1” when sending the request.

I think that is a good idea. Would fix the concerns enumerated by @SomberNight. This is very easy to implement on both side.

@NicolasDorier Not quite. The receiver suffers the disadvantages regardless if they use the feature. They suffer merely because it’s possible.

Trivial example: Receiver wants payments to their cold wallet. BIP79 makes this possible, because there’s nothing bad/hacked/outsourced/malicious payjoin server can really do. But this BIP makes it impossible, because say the payjoin server was hacked it could see a large payment came in and just steal it. (So it is no longer sending to a cold-wallet really. It is more like sending to a hot wallet that immediately forwards it to a cold wallet).

And that’s not even that contrived. By making it impossible for the payjoin server to steal money, you make it more likely people will want to use payoin because implementing/supporting it is not as risky.

Address reuse can sometimes stop substitution…

ducks

On Wed, Jun 17, 2020, 9:19 PM Nicolas Dorier notifications@github.com wrote:

If the payment server is compromised, the malicious actor can already substitute any information he want in the BIP21, there is nothing the spec specifically enable him to do.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-645711406, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFUZ7LD74G65HUBOR6MLRXFTS3ANCNFSM4NDBRK6Q .

Substitution attacks*

On Wed, Jun 17, 2020, 9:22 PM Greg Sanders gsanders87@gmail.com wrote:

Address reuse can sometimes stop substitution…

ducks

On Wed, Jun 17, 2020, 9:19 PM Nicolas Dorier notifications@github.com wrote:

If the payment server is compromised, the malicious actor can already substitute any information he want in the BIP21, there is nothing the spec specifically enable him to do.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-645711406, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFUZ7LD74G65HUBOR6MLRXFTS3ANCNFSM4NDBRK6Q .

If the payment server is compromised, the malicious actor can already substitute any information he want in the BIP21, there is nothing the spec specifically enable him to do.

The systems responsible for giving users a bip21 invoices is often radically different than the part of the system responsible for running a payjoin server. And securely generating/showing/giving bip21 invoices an existing burden that all bitcoin services have.

Imagine you were approaching X company to support payjoin. You will get a lot better reception by saying: “This is sandbox’d such that the worst case is it loses funds in the wallet which it has access to. This can just be some random shit you want to consolidate anyway” vs “Oh yeah, consider this part of your critical infrastructure. And don’t even try use this with cold storage, cause it largely breaks that”

I can also do stuff like “Don’t pay the invoice, unless you receive it in a pgp signed message from me” with BIP79, but not really possible here..

And even if they were theoretically the same system, and an attacker completely pwnd it – this bip would make the problem worse, because the attacker could selectively only steal large amounts of money, instead of blowing his load and being immediately noticed by doing bip21 substitutions.

I get that there are upsides to output substitution. But you really need to allow both the sender and receiver opt out. Or consider if it’s even worth the complexity of supporting in the first place, for a pretty niche feature.

@RHavar I disagree, this is just overkill.

Maybe, but overkill is better than half-assed. You’re adding a pretty niche feature (output substitution) which has a considerable amount of downsides for both the sender and receiver. And then providing only a way for the sender to mitigate those downsides, but not the receiver. You should either do it properly, or not at all. (My vote would be for: not-at-all).

And it’s not just a theoretical concern: I am aware of two different services that would be unable to deploy this proposal if output substitution is allowed. One is a privacy-oriented service which provides a signed, non-repudiable message that says: “If you send X bitcoin to Y address by Z time, we promise will do A”. Output substitution would completely break this. The other is an exchange with a pretty serious and well defined security model where all deposits are first sent to cold-storage where they are later processed. They are not going to change their security models because you feel its not worth planning for.

And of the people who are able to implement it, you will significantly increase the operational burden. When I operated a high-volume site (i.e. processed literally hundreds of millions USD worth of bitcoin deposits+withdrawals), I saw some pretty nasty attacks (including an AWS employee violating company protocol, which I suspect he was probably bribed into doing so). There’s no way I’d considering trying to support this BIP in its current form. It wouldn’t be because it’s impossible (it would be) but because I don’t want yet-another-critically-important part of infrastructure to secure and maintain. It’s even harder for me to pay people to work on, because now I have to be ultra-careful of (intentional) backdoors etc.

BIP79 on the other hand is pretty easy to support, and you can even outsource it to someone who is just going to consolidate some dust lol.

On a side note, the output-substitution almost feels like you’re trying to shoe-horn in an orthogonal idea, which could be better addressed by a BIP covering something like “alternate addresses” in which a sender can pick an alternate address for compatibility or privacy reasons.

@SomberNight

Why do you feel that output substitution is bad? I fail to see the downsides around it.

There are certainly pros and cons.

(1) If output substitution is allowed, a merchant/receiver can’t really give you signed invoices. Let’s say a merchant signs a message saying they will deliver goods upon receiving X bitcoins to address A, and they put a bip21 URI in the signed message too. If the URI uses payjoin, and payjoin allows replacing output addresses then the user would need to be prompted to verify the replaced address as part of the flow.

E.g. the user starts a payment, verifies the address, it matches with the signed address, clicks “send”, and the payjoin flow starts. The receiver replaces the address, and either

* the user now needs to be prompted again to manually verify the payjoin tx

* or if the software accepts any substitution then the merchant can trick the user by having them pay to a different address that they did not sign

(2) Without output substitution, and assuming the receiver is not allowed to decrease output amounts, I imagine it would be possible for hardware wallets to auto-sign (without prompts) the payjoin tx – only prompting the user once, at the beginning, to confirm the original tx.

For 2, I don’t think that is the case at all, all HWs I tried with payjoin (no output substitution) required 2 confirmations.

For 1, I agree, it is a harder flow for the end user to verify against the original payment request ( send X to Y). What if we add a header value with the payjoin proposal response to the sender: a message (payjoin proposal tx hash + the bip21 sender was given) signed by the private key of the original payment address?

• Server creates payjoin BIP21 bitcoin:AddressA?amount=1&pj=https://gozo.com
• Sender reads BIP21, creates original PSBT paying AddressA 1BTC, sends to http://gozo.com
• Receives creates a payjoin proposal with output substitution where it nows pays AddressB 0.5BTC and AddressC 0.5BTC.
• Receiver sends payjoin proposal to sender BUT also sends a message signed with AddressA and its content being the payjoin proposal transaction hash + the BIP21 sender was given

This should allow you to verify that the output substitution was not a malicious action from merchant to customer and is verifiably linked to the original BIP21 payment request.

@RHavar

And it’s not just a theoretical concern: I am aware of two different services that would be unable to deploy this proposal if output substitution is allowed. One is a privacy-oriented service which provides a signed, non-repudiable message that says: “If you send X bitcoin to Y address by Z time, we promise will do A”. Output substitution would completely break this. The other is an exchange with a pretty serious and well defined security model where all deposits are first sent to cold-storage where they are later processed. They are not going to change their security models because you feel its not worth planning for.

This is not really a good argument IMO: this is not part of any specification used by this BIP: of course very specific workflows may need to adapt to use Payjoin.

If there is a signed message stating “If you send X bitcoin to Y address by Z time, we promise will do A”, you can add " If Y address signed alternative destinations B,C for X bitcoin, also do A on X bitcoin sent to B,C "

Services where the money is sent to cold storage directly cannot use Payjoin anyway as you need the BIP21 destination address private key to partially sign payjoin proposal. And let’s face it, I doubt most exchanges would even consider using Payjoin unless it could benefit them in some way (privacy is not a benefit to them since they are required to deal with analysis companies anyway). One benefit of payjoin for exchanges actually comes from output substitution, where the exchange would show a p2sh address (for compatibility) and substitute it with native segwit for savings.

And of the people who are able to implement it, you will significantly increase the operational burden. When I operated a high-volume site (i.e. processed literally hundreds of millions USD worth of bitcoin deposits+withdrawals), I saw some pretty nasty attacks (including an AWS employee violating company protocol, which I suspect he was probably bribed into doing so). There’s no way I’d considering trying to support this BIP in its current form. It wouldn’t be because it’s impossible (it would be) but because I don’t want yet-another-critically-important part of infrastructure to secure and maintain. It’s even harder for me to pay people to work on, because now I have to be ultra-careful of (intentional) backdoors etc.

I still fail to see the operation burden increase when compared to BIP79. The biggest difference in implementation is the fee burden decision (to which I think I am more on your side) and the forced upgrade to PSBT (good for everyone anyway).

For me, output substitution creates a whole array of use-cases, both for privacy and for block space efficiency. It would be a huge downgrade to cripple this BIP on such an niche case.

(2) Without output substitution, and assuming the receiver is not allowed to decrease output amounts, I imagine it would be possible for hardware wallets to auto-sign (without prompts) the payjoin tx – only prompting the user once, at the beginning, to confirm the original tx.

For 2, I don’t think that is the case at all, all HWs I tried with payjoin (no output substitution) required 2 confirmations.

What @SomberNight is saying is that it would be possible, not that it’s currently implemented in any hardware wallet. I agree that it’s possible to implement the automated signing relatively easily if the receiver respects the BIP79 rules (no output substitution, no output removal, no decreasing of output amounts). We can allow decreasing the output amount for additionalfeeoutputindex, but that’s about it.

I think that this is quite important, because implementing automated signing of the PayJoin proposal is key to making it user-friendly and secure, which in turn is critical to achieve any sort of wider adoption.

(2) Without output substitution, and assuming the receiver is not allowed to decrease output amounts, I imagine it would be possible for hardware wallets to auto-sign (without prompts) the payjoin tx – only prompting the user once, at the beginning, to confirm the original tx.

For 2, I don’t think that is the case at all, all HWs I tried with payjoin (no output substitution) required 2 confirmations.

What @SomberNight is saying is that it would be possible, not that it’s currently implemented in any hardware wallet. I agree that it’s possible to implement the automated signing relatively easily if the receiver respects the BIP79 rules (no output substitution, no output removal, no decreasing of output amounts). We can allow decreasing the output amount for additionalfeeoutputindex, but that’s about it.

I think that this is quite important, because implementing automated signing of the PayJoin proposal is key to making it user-friendly and secure, which in turn is critical to achieve any sort of wider adoption.

The biggest challenge is having the user know they need to use the device twice which means the sender needs to show a UI telling the user what needs to happen. Telling the user they will sign twice is fine as long as the sender can verify that the payjoin proposal is not malicious (if payjoin proposal is signed by BIP21 original destination, it can only be as malicious as the BIP21 creator is). Even if they could get it automatically signed the second time, you would need to communicate to the user to leave the device plugged in and most likely would want to show to user that it is being signed automatically for a special reason. I don’t think the small UX benefit (on probably a very small minority of HW devices that would add such a feature) is worth crippling Payjoin functionality.

I don’t understand any of the arguments right now…

1. The receiver don’t have to use output substitution if he does not want to.
2. Nor does the sender. @RHavar please explain

And it’s not just a theoretical concern: I am aware of two different services that would be unable to deploy this proposal if output substitution is allowed. One is a privacy-oriented service which provides a signed, non-repudiable message that says: “If you send X bitcoin to Y address by Z time, we promise will do A”. Output substitution would completely break this. The other is an exchange with a pretty serious and well defined security model where all deposits are first sent to cold-storage where they are later processed. They are not going to change their security models because you feel its not worth planning for.

This is addressed by disableoutputsubstitution=true set on the sender. @andrewkozlik, the point raised by @SomberNight is solved by this flag as well.

Or consider if it’s even worth the complexity of supporting in the first place, for a pretty niche feature.

Actually this is not a niche feature. We plan to integrate it next next release in BTCPay. Imagine the merchant need to pay Bob. Alice sends a payjoin to merchant. Merchant can decide to use Alice’s payment output to pay Bob. The payjoin transaction becomes a three party coinjoin without Alice even knowing. @RHavar your only point, if I understand is that this feature prevent the delegation of payjoin to an untrusted payjoin server.

I think having an untrusted payjoin server is actually an interesting usecase. As it would mean that a payjoin can be a three party coinjoin where the untrusted payjoin server is the merchant but somebody else wanting to mix his coins. It may create an interesting market where the merchant get paid out of band for providing such service to the third party.

If such is the goal, we can indeed think about a way for the merchant to signal the payjoin server is untrusted and thus, output substituion is not possible. Would it suit you?

@Kukks

For 1, I agree, it is a harder flow for the end user to verify against the original payment request ( send X to Y). What if we add a header value with the payjoin proposal response to the sender: a message (payjoin proposal tx hash + the bip21 sender was given) signed by the private key of the original payment address? […] This should allow you to verify that the output substitution was not a malicious action from merchant to customer and is verifiably linked to the original BIP21 payment request.

Yes, this could mitigate the issue for the sender. Although note that now the sender client UI will have to expose this new signed message to the user, and also the user needs to be aware that this kind of substitution can happen, and they need to export and store this signed message alongside the original signed invoice they got out of band from the receiver.

Services where the money is sent to cold storage directly cannot use Payjoin anyway as you need the BIP21 destination address private key to partially sign payjoin proposal

Why would the receiver need to sign anything with the key used for one of the outputs? (apart from your proposal in same comment to try to solve my output substitution concern)

@andrewkozlik @SomberNight can you confirm your issue is solved by disableoutputsubstitution=true at the client level? @RHavar by thinking more about it, the only useful point for an untrusted payjoin server is for having another party than the receiver mixing his inputs.

But by thinking more about it, this is already possible with the current protocol!

1. The receiver send bip21 with pj going to his own trusted server
2. The sender send a original PSBT
3. The receiver forward the original PSBT as-is to a mixing party with disableoutputsubstitution=true.
4. The receiver send back the proposal to the sender.

Why would the receiver need to sign anything with the key used for one of the outputs? (apart from your proposal in same comment to try to solve my output substitution concern) @SomberNight Brainfart from my end. I understand it a bit more now

I don’t understand any of the arguments right now…

1. The receiver don't have to use output substitution if he does not want to.

2. Nor does the sender.

@RHavar please explain

And it’s not just a theoretical concern: I am aware of two different services that would be unable to deploy this proposal if output substitution is allowed. One is a privacy-oriented service which provides a signed, non-repudiable message that says: “If you send X bitcoin to Y address by Z time, we promise will do A”. Output substitution would completely break this. The other is an exchange with a pretty serious and well defined security model where all deposits are first sent to cold-storage where they are later processed. They are not going to change their security models because you feel its not worth planning for.

This is addressed by disableoutputsubstitution=true set on the sender. @andrewkozlik, the point raised by @SomberNight is solved by this flag as well.

Or consider if it’s even worth the complexity of supporting in the first place, for a pretty niche feature.

Actually this is not a niche feature. We plan to integrate it next next release in BTCPay. Imagine the merchant need to pay Bob. Alice sends a payjoin to merchant. Merchant can decide to use Alice’s payment output to pay Bob. The payjoin transaction becomes a three party coinjoin without Alice even knowing.

@RHavar your only point, if I understand is that this feature prevent the delegation of payjoin to an untrusted payjoin server.

I think having an untrusted payjoin server is actually an interesting usecase. As it would mean that a payjoin can be a three party coinjoin where the untrusted payjoin server is the merchant but somebody else wanting to mix his coins. It may create an interesting market where the merchant get paid out of band for providing such service to the third party.

If such is the goal, we can indeed think about a way for the merchant to signal the payjoin server is untrusted and thus, output substituion is not possible. Would it suit you?

I don’t think this is what he meant, It’s not that his payjoin endpoint is untrusted but that it may or may not be compromised and starts routing selective payments to a malicious destination.

@RHavar sorry to spam the conversation here, by thinking about it more (actually @nopara73 pointed it out to me), having an untrusted party contributing inputs make no sense as the output back to him would be equal to the sum of the input he contributed…

Knowing no other party than the receiver can contribute input, such “untrusted payjoin server” would need the private key of the receiver to sign inputs… not so much untrusted then.

Knowing no other party than the receiver can contribute input, such “untrusted payjoin server” would need the private key of the receiver to sign inputs… not so much untrusted then.

I don’t believe the “Only payjoin server is compromised, not the payment server” is a very realistic worry. Even if it was, the payjoin server has access to the private keys of the merchant to be able to contribute inputs so if it is compromised, output substitution or not, you lost money.

As @RHavar pointed out above, some receivers might want to receive payments to their cold storage, and only use a small value hot wallet that the PJ server has access to. With a trusted invoicing server, and a less-trusted PJ server, the idea is that the invoicing server creates the bip21 URIs with cold storage addresses, and the PJ server uses the hot wallet to contribute inputs. Compromise of PJ server would not compromise the cold storage, nor the future received payments (they would still go to the cold storage, or worst case rejected: DOS), only the hot wallet would be lost.

@Kukks I don’t believe the “Only payjoin server is compromised, not the payment server” is a very realistic worry. Even if it was, the payjoin server has access to the private keys of the merchant to be able to contribute inputs so if it is compromised, output substitution or not, you lost money.

I agree. Even if it is the case, the hacker would have to decide between:

• emptying out available hot wallet funds
• taking the next payjoin payment but risk detection before then
• waiting for a big payjoin payment but risk detection before then x2

@NicolasDorier

can you confirm your issue is solved by disableoutputsubstitution=true at the client level?

I believe that solves the hardware wallet issue (referred above as (2)).

Re issue (1), the merchant giving out a signed invoice, the user might still need to know that the receiver might be able to substitute output addresses if it is allowed in the spec at all. Still, for a sender point of view, if the client defaults to disableoutputsubstitution=true that seems good enough for me. However, ideally, the receiver should be able to disable output substitution and sign the fact they disabled it. One way to do that would be to signal this in the bip21 URI. If it was signalled in the URI, the client could also enforce that regardless of what the client might do by default. I guess this is similar to @RHavar’s pj-/pj+ or os=1 idea.

sender forbids the receiver to modify his own output,

I understand what it’s saying but I think it’s slightly ambiguous who “his” is.

Suggestion:

sender forbids the receiver to modify sender’s output,

@NicolasDorier

can you confirm your issue is solved by disableoutputsubstitution=true at the client level?

Yes, this solution is good enough and it is what we would use in Trezor if we decide to implement this BIP.

There is however the broader issue that the concept of output substitution as defined in the spec is half-baked. So far I see two uses for output substitution:

1. For the receiver’s output to match the sender’s input script types in order to improve privacy.
2. To allow the receiver to use the PayJoin transaction to make a payment to a third party.

Using output substitution for (1) is unreasonably complex. The proper solution is for the BIP21 URI to contain an alt-address field so that the sender may choose an address with a script type matching his inputs.

As for (2), I have my doubts as to how realistic and practical this scenario is, not to mention the issues with sending the payment to a different address than the one given in the invoice. Nevertheless, assuming it’s practical, then the BIP should also define how to construct the message signed using the private key to the original destination address as @kukks proposed.

To summarize, disableoutputsubstitution makes the BIP acceptable, but there is room for improvement. @Kukks

Even if they could get it automatically signed the second time, you would need to communicate to the user to leave the device plugged in and most likely would want to show to user that it is being signed automatically for a special reason.

In standard situations I would expect the communication with the receiver to be pretty swift, so the second signing would take place within seconds and the user would perceive the whole process as one uninterrupted flow. If there are situations where it takes longer, then indeed the desktop software would prompt the user to replug the HW wallet if it had been unplugged and give an explanation as to why.

I don’t think the small UX benefit (on probably a very small minority of HW devices that would add such a feature) is worth crippling Payjoin functionality.

I suppose this is just a matter of opinion, because I would say that a niche feature is not worth crippling the UX and security (given that matching the script types can be solved by an alt-address field in the BIP21 URI).

BTW the automated signing is in no way special to HW wallets, it can be implemented in any wallet. All I was saying is that we can do it in HW wallets too.

I don’t think the small UX benefit (on probably a very small minority of HW devices that would add such a feature) is worth crippling Payjoin functionality.

I suppose this is just a matter of opinion, because I would say that a niche feature is not worth crippling the UX and security (given that matching the script types can be solved by an alt-address field in the BIP21 URI).

BTW the automated signing is in no way special to HW wallets, it can be implemented in any wallet. All I was saying is that we can do it in HW wallets too.

Yes I agree with @andrewkozlik, if we implement the sender in Electrum, I would want to set all params in such a way by default that the user would not need to be prompted a second time. This implies disableoutputsubstitution=true, only sane modifications to output amounts, and only sane changes in overall fee/feerate.

then the BIP should also define how to construct the message signed using the private key to the original destination address as @Kukks proposed.

This cannot be done in a number of interesting use-cases:

1. Paying to a cold address, and the deposit being re-routed to a withdrawing party.
2. Directly opening an LN channel from a user deposit
3. Any use-case that isn’t single-signature

I think output substitution is a pretty powerful primitive and would be a huge shame if removed as an optional extension for senders who bother to support it.

I would want to set all params in such a way by default that the user would not need to be prompted a second time. This implies disableoutputsubstitution=true, only sane modifications to output amounts, and only sane changes in overall fee/feerate.

Certainly your prerogative(complexity has cost and I’ll likely do an implentation as you suggest to start) but there’s no real reason the user has to be confronted twice and allow output substitution(well, outside of “current hww firmware limitations!). An updated hww could certainly handle both cleanly(doing the same analysis as listed in the BIP).

@NicolasDorier

can you confirm your issue is solved by disableoutputsubstitution=true at the client level?

Yes, this solution is good enough and it is what we would use in Trezor if we decide to implement this BIP.

There is however the broader issue that the concept of output substitution as defined in the spec is half-baked. So far I see two uses for output substitution:

1. For the receiver's output to match the sender's input script types in order to improve privacy.

2. To allow the receiver to use the PayJoin transaction to make a payment to a third party.

Using output substitution for (1) is unreasonably complex. The proper solution is for the BIP21 URI to contain an alt-address field so that the sender may choose an address with a script type matching his inputs.

As for (2), I have my doubts as to how realistic and practical this scenario is, not to mention the issues with sending the payment to a different address than the one given in the invoice. Nevertheless, assuming it’s practical, then the BIP should also define how to construct the message signed using the private key to the original destination address as @Kukks proposed.

To summarize, disableoutputsubstitution makes the BIP acceptable, but there is room for improvement.

@Kukks

Even if they could get it automatically signed the second time, you would need to communicate to the user to leave the device plugged in and most likely would want to show to user that it is being signed automatically for a special reason.

In standard situations I would expect the communication with the receiver to be pretty swift, so the second signing would take place within seconds and the user would perceive the whole process as one uninterrupted flow. If there are situations where it takes longer, then indeed the desktop software would prompt the user to replug the HW wallet if it had been unplugged and give an explanation as to why.

I don’t think the small UX benefit (on probably a very small minority of HW devices that would add such a feature) is worth crippling Payjoin functionality.

I suppose this is just a matter of opinion, because I would say that a niche feature is not worth crippling the UX and security (given that matching the script types can be solved by an alt-address field in the BIP21 URI).

BTW the automated signing is in no way special to HW wallets, it can be implemented in any wallet. All I was saying is that we can do it in HW wallets too.

One more use-case, similar to 1), is to have a store use p2sh-segwit as a default for compatibility’s sake and the payjoin would substitute it to native segwit. As a merchant, this is awesome, as older, unwilling clients can stay using p2sh while allowing newer clients with payjoin support switch to a more efficient format.

@instagibbs

Certainly your prerogative(complexity has cost and I’ll likely do an implentation as you suggest to start) but there’s no real reason the user has to be confronted twice and allow output substitution(well, outside of “current hww firmware limitations!).

I gave an example above for a valid reason. The merchant might have given the sender a signed invoice including the address, out of band. The wallet software has no way of knowing this. Hence, the only sane default is to either not allow output substitution or to prompt the user to confirm the updated output.

Sounds like a reason to add the optional params and support both under different modes of operation?

On Thu, Jun 18, 2020, 10:41 AM ghost43 notifications@github.com wrote:

@instagibbs https://github.com/instagibbs

Certainly your prerogative(complexity has cost and I’ll likely do an implentation as you suggest to start) but there’s no real reason the user has to be confronted twice and allow output substitution(well, outside of “current hww firmware limitations!).

I gave an example above for a valid reason. The merchant might have given the sender a signed invoice including the address, out of band. The wallet software has no way of knowing this. Hence, the only sane default is to either not allow output substitution or to prompt the user to confirm the updated output.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/923#issuecomment-646063587, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFUY3SDDQTJWT5XLFGODRXIRQXANCNFSM4NDBRK6Q .

@RHavar sorry to spam the conversation here, by thinking about it more (actually @nopara73 pointed it out to me), having an untrusted party contributing inputs make no sense as the output back to him would be equal to the sum of the input he contributed…

Nah, it’s pretty realistic. I can think of a few examples:

a) Dust elimination. i.e. my wallet has >1000 dust inputs in it (as I modified my wallet to ignore dust). I’d be happy to donate it to someone via payjoin, cause I know it’ll fuck up analysis rofl

b) Limited Trust. You outsource the payjoin server to someone who trusts you (but you don’t trust them) and then after X time they ask you to comp them the amount extra they sent you

c) Coinjoin Puzzle. (i wrote something about this and wrote a minizinc solver for it) but you could out source the payjoin server to someone who needs to consolidate. They add 2 inputs and 1 output. There’s a technique (which is a bit of out scope) where the person can search for input amounts (if they have any) that would be ambigious so even if an analytics engine knows what they’re doing, it’s ambigious which inputs are which. It works extremely well [1] even with a small amount of inputs to select from.

[1] In theory it should work amazing when you have even more amount of inputs, but my solver isn’t very production worthy and can’t handle a wallet with more than ~20 utxos. I’m kind of out of ideas of how to solve it, but an optimization expert probably has some.

I think output substitution is a pretty powerful primitive and would be a huge shame if removed as an optional extension for senders who bother to support it.

I agree it’s actually pretty cool. Like making a deposit pay out a pending withdrawal is actually super neat. That’s something I’d actually very much consider doing.

But I’ll just restate my earlier objection: It comes with some significant downsides for both the sender and the receiver. I think either both the sender and receiver should be able to opt out (or opt into) this feature. Or it’s simply just not included in the spec, and address adding it with a v2 (or even a different BIP like “alt-addresses” in the bip21 url)

Anyway, I’m beating a bit of a dead horse and it’s not my BIP. So I’ll leave it :D

I’d also like the receiver to be allowed to change any output amount (even if it’s the senders) as long as the value goes up.

Looks ok to me, as long as there is some sanity check for the fee; at minimum, the absolute fee should not decrease.

Looks ok to me, as long as there is some sanity check for the fee; at minimum, the absolute fee should not decrease.

Is there something that (currently) stops the receiver lowering the absolute fee? (I very quickly scanned the BIP, but didn’t see the rule that restricted it)

@RHavar @SomberNight the sender can use minFeeRate to put bounds on how low the fee rate can be. Why putting a limit on absolute fee? If the objective is to prevent the receiver to direct fees into his own pocket, this is solved by using minFeeRate == originalFeeRate. (or minFeeRate slighlty below originalFeeRate if no fee output)

The absolute fee can realistically be lower, if there is an address substitution to a smaller output. (say going from P2SH-P2WPKH to P2WPKH, which save 12 vbytes)

@RHavar @SomberNight I think it is a good idea to prevent the receiver from pocketing fees. While it is possible via a good value of minFeeRate, it should actually be enforced at protocol level.

I was thinking we could just ignore the case where an output become smaller because of substitution, but that may show up if the sender’s fee are round. :/

The check need to be simple, thinking about it…

What about this: Allow the absolute fee to fall as much as the size of the payment output at the original transaction fee rate? (As in the best case scenario, receiver might decide to just completely remove his output)

I would like to make it easy to review and check. That’s not perfect, but good enough.