New BIP 351: Private Payments

alfred-hodler commented at 11:49 am on August 8, 2022: contributor

I am submitting a new BIP following a discussion on the mailing list.

Link: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-July/020812.html

Private payments BIP ae247aa6cf

Rework Motivation section ab90b8e787

Address type flags in a table 9c57035d5b

Improve notifications 134120166c

Italic symbol notation instead of <code> fb18d17106

Update wording 947916e5ba

Two potential third-party services 9caa27ccd4

Update authors list df9830a855

Improve wording 063d70219a

Incorporate feedback 7330484680

Leaner notifications 50e707bd8c

added reference to BIP 47 c053962759

clarkmoody commented at 5:15 pm on August 16, 2022: contributor

@luke-jr @kallewoof We’re ready for a BIP number assignment.

in bip-alfredhodler-privatepayments.mediawiki:96 in c053962759 outdated

91+Currently defined flags:
92+
93+{| class="wikitable"
94+! Address Type !! Flag !! Flag Value !! Ordinal Value
95+|-
96+| P2PKH || <code>1 << 0</code> || <code>0x0001</code> || 0

junderw commented at 0:53 am on August 17, 2022:

nit:

Gleaning from the fact that BIP32 requires it and the payment code also requires it, most people will assume that this is a P2PKH of the compressed key.

However, in past BIPs I have seen many people completely miss things that aren’t explicitly spelled out for them, so I would feel better if “compressed” was somewhere around here (a note or something)

junderw commented at 0:55 am on August 17, 2022:

Maybe P2PKH<sub>compressed</sub>

alfred-hodler commented at 10:29 am on August 17, 2022:

It is actually not possible to have an uncompressed key at any point because BIP32 is utilized, which works exclusively with compressed keys.

But I agree that we can make this more explicit in order to prevent confusion :+1:

alfred-hodler commented at 10:18 am on August 19, 2022:

This has been addressed.

in bip-alfredhodler-privatepayments.mediawiki:37 in c053962759 outdated

32+'''Sharing a BIP47 payment code''' addresses most of the above shortcomings. However, it introduces the following problems:
33+
34+* The BIP uses a notification mechanism that relies on publicly known per-recipient notification addresses. If Alice wants to send funds to Bob, she has to use the same notification address that everyone else uses to notify Bob. If Alice is not careful with coin selection, i.e. ensuring that her notification UTXO is not linked to her, she will publicly expose herself as someone who is trying to send funds to Bob and their relationship becomes permanently visible on the blockchain.
35+
36+* The BIP does not say anything about address types. Receiving wallets therefore have to watch all address types that can be created from a single public key. Even then, a sender could send to a script that a receipient cannot spend from.
37+

luke-jr commented at 6:12 am on August 17, 2022:

How about sharing a BOLT12?

alfred-hodler commented at 10:30 am on August 17, 2022:

Could you expand on this?

luke-jr commented at 7:50 pm on August 19, 2022:

What are the drawbacks of BOLT12, benefits of this over BOLT12, etc?

alfred-hodler commented at 10:09 am on August 22, 2022:

My understanding is that BOLT12 is specific to Lightning and lives one layer above this, hence I didn’t consider it worthwhile to try to compare it with static addresses and payment codes, which all work on-chain.

luke-jr commented at 10:37 pm on September 29, 2022:

Perhaps it is debatable in the case of static addresses, but payment codes are definitely a layer above the chain.

Regardless, they serve the same goals of the BIP, so IMO should be addressed here.

in bip-alfredhodler-privatepayments.mediawiki:68 in c053962759 outdated

63+* ''|'': string concatenation
64+* ''[a..b]'': string slicing (inclusive of ''a'', exclusive of ''b'')
65+
66+===Public Key Derivation Path===
67+
68+The derivation path for this BIP follows BIP44. The following BIP32 path levels are defined:

luke-jr commented at 6:13 am on August 17, 2022:

Can’t it be derivation-neutral so non-BIP44 wallets can use it?

alfred-hodler commented at 10:34 am on August 17, 2022:

They can still use it, they just have to follow the prescribed BIP32 derivation path. It’s simply a matter of convention for the sake of wallet compatibility.

We could relax this requirement by stating that one can start the process with any extended private key, but then each wallet may end up implementing its own incompatible derivation path that other wallets won’t know how to import (see Electrum derivation path mess).

luke-jr commented at 3:21 pm on August 17, 2022:

Wallets are not intended to be compatible between different software.

alfred-hodler commented at 6:38 pm on August 18, 2022:

The fact that BIP39 and BIP44/49/84 exist and are ubiquitous attests to the fact that wallets indeed try to keep a degree of compatibility.

luke-jr commented at 8:26 pm on August 18, 2022:

They’re not ubiquitous, and even wallets that do use them, do not claim to be compatible with others that do.

clarkmoody commented at 8:54 pm on August 18, 2022:

Survey of wallet derivation paths for reference: https://walletsrecovery.org/

in bip-alfredhodler-privatepayments.mediawiki:109 in c053962759 outdated

104+
105+While payment codes use 2-byte bitflag arrays, notifications use ordinal values in the form of a single byte.
106+
107+===Notifications===
108+
109+Notifications are performed by publishing transactions that contain a 40-byte <code>OP_RETURN</code> output. The value of the <code>OP_RETURN</code> is constructed using the following formula:

luke-jr commented at 6:15 am on August 17, 2022:

I thought part of the point of this BIP was to avoid such spam?

Why can’t this stay out of band?

alfred-hodler commented at 10:41 am on August 17, 2022:

Notifications can indeed be performed out of band (the original BIP was trying to use Bitmessage), but then we lose the ability to deterministically restore a wallet from seed simply by rescanning the blockchain.

So if we were to get rid of OP_RETURNs, we’d either need a centralized directory of notifications that’s vulnerable to takedowns, or each user would have to back up their set of notifications (losing it would mean losing funds forever).

I tried my best to cut the size of these notifications from 80 bytes in BIP47 to half that amount.

luke-jr commented at 3:29 pm on August 17, 2022:

This older draft didn’t require notifications at all?

https://gist.github.com/RubenSomsen/c43b79517e7cb701ebf77eec6dbb46b8

alfred-hodler commented at 6:36 pm on August 18, 2022:

Silent Payments is a different standard by different people that I had no part in making. It requires a full node and doesn’t work with light clients such as mobile wallets.

Private Payments requires these notifications in order to be compatible with light wallets.

clarkmoody commented at 9:01 pm on August 18, 2022:

I’m showing OP_RETURN at 2.8 GB right now, which is 0.6% of the total chain size and is prunable if full nodes need that space.

The history of total OP_RETURN size looks like a very large user came online at one point but then stopped after a while.

nulldata-totals-graph

This protocol does not need to continually place anchors into the chain like other protocols, so the usage should have much less impact.

luke-jr commented at 9:31 pm on August 18, 2022:

Bitcoin’s blockchain is not for storing private backups, or distributing information between users.

Instead of lazily abusing OP_RETURN, it would be preferable to have an external central directory, but there’s no reason the directory couldn’t be p2p either. It doesn’t require any of the properties of the blockchain (double spend protection).

alfred-hodler commented at 10:19 am on August 19, 2022:

I expanded on out of band notifications.

luke-jr commented at 7:53 pm on August 19, 2022:

Far better to just require a full node, than to spam Bitcoin for the benefit of light wallets (ie, non-Bitcoin users).

(FWIW, I would discourage implementation of this BIP it it keeps OP_RETURN abuse.)

alfred-hodler commented at 6:41 pm on August 18, 2022: contributor

Feedback is always welcome but ideally we’d get a BIP number at this point because it’ll change the test vectors. Not having one also blocks us from publishing the reference implementation.

luke-jr commented at 9:34 pm on August 18, 2022: member

For a number assignment, this needs a section addressing backward compatibility.

Updates to the BIP

- fix out of range indexing
- add a note about compressed keys
- add a note about of-of-band notifications
- add a backward compatibility section

187135c4f6

alfred-hodler commented at 10:19 am on August 19, 2022: contributor

Backward compatibility has been addressed.

in bip-alfredhodler-privatepayments.mediawiki:89 in 187135c4f6 outdated

84+
85+Payment codes are encoded in bech32m and the human readable part is "pay" for mainnet and "payt" for testnet (all types), resulting in payment codes that look like "pay1cqqq8d29g0a7m8ghmycqk5yv24mfh3xg8ptzqcn8xz6d2tjl8ccdnfkpjl7p84".
86+
87+===Address Types===
88+
89+Address type flags determine which address types a payment code accepts. This is represented by big-endian ordered 16 bits. For instance, a hypothetical payment code that handles all address types will have all defined bits set to 1 (<code>0xffff</code>).

luke-jr commented at 7:44 pm on August 19, 2022:

The “payment code” here is the address. Script templates and scripts are not addresses.

alfred-hodler commented at 9:03 am on August 22, 2022:

Strictly speaking you’re correct, but “address type” is a commonly used and understood term. Calling “p2wpkh” an address type is easier to understand than calling it a “script template”. Example from a prominent library: https://docs.rs/bitcoin/latest/bitcoin/util/address/enum.AddressType.html

luke-jr added the label New BIP on Aug 19, 2022

luke-jr renamed this:
~~New BIP: Private Payments~~
New BIP 351: Private Payments
on Aug 19, 2022

luke-jr commented at 7:49 pm on August 19, 2022: member

Assigned BIP 351

Update BIP number and test vectors 75a6f21e3a

Awilliams76 approved

Update the index page e56fb64292

Add reference implementation 1385d65187

alfred-hodler commented at 9:55 am on August 22, 2022: contributor

Added a link to reference implementation.

Add Comments-URI; fix author list ac7b6c1f50

clarkmoody commented at 1:57 pm on August 25, 2022: contributor

It looks like this is ready for merge.

alfred-hodler commented at 7:47 am on August 29, 2022: contributor

@luke-jr we are ready for merge. We’ll keep the draft status for now.

michaelfolkson commented at 9:51 am on August 29, 2022: contributor

I’m not sure if @RubenSomsen’s feedback in his mailing list response has been addressed. He is listed as a co-author so just checking he’s happy with this.

RubenSomsen commented at 10:31 am on August 29, 2022: none

Briefly, my thoughts:

I think the Prague discussion and its authors deserve mentioning, seeing as the protocol is the same except for one change (blinding the payment code)
I’m not convinced this was a good change, given that it introduces a scanning requirement (albeit more modest than Silent Payments) which complicates light client integration, though on the other hand it does remove the complexity of outsourcing the on-chain notification
I don’t think any of this matters for BIP assignment

Add a refernce to BIP351 bffc84f63a

alfred-hodler commented at 10:59 am on August 30, 2022: contributor

Added a reference to the Prague discussion.

clarkmoody commented at 10:35 pm on September 7, 2022: contributor

ACK bffc84f

junderw commented at 0:48 am on September 8, 2022: contributor

I’m not convinced this was a good change, given that it introduces a scanning requirement

I agree here. This definitely does not prevent this becoming a BIP… but the reason why Dark Wallet’s stealth address protocol and BIP47 never really caught on was the various scanning requirements increasing the implementation difficulty.

Not saying that there’s an “easily implemented alternative”… they all have their pros and cons from a wallet dev perspective.

ACK bffc84f

clarkmoody commented at 2:06 am on September 8, 2022: contributor

scanning requirement

The required data is the OP_RETURN payload, which can be outsourced in a privacy-preserving way to the service described in Appendix B. Such a service would publish bulk OP_RETURNs matching the format described here, and wallets would scan against them (ECDH + SHA256 for each) to find matches and discover the stealth addresses. The reference implementation function detect_notification helps here as well.

So this is a lighter weight alternative to Silent Payments (no full node required) and a more private approach than BIP47 (no reused address).

RubenSomsen commented at 9:44 am on September 8, 2022: none

The required data is the OP_RETURN payload, which can be outsourced in a privacy-preserving way […] this is a lighter weight alternative to Silent Payments (no full node required)

Note that a very similar mitigation strategy (with a few more steps) exists for Silent Payments. Once every e.g. month or so you’d perform a scan over wifi by receiving a list with one pubkey per as-of-yet unspent transaction (fully spent transactions can be pruned). Senders are expected to send an out-of-band notification of a payment (e.g. email), allowing you to detect the payment immediately, so the monthly scanning only serves as a backup strategy for failed notifications.

While this is a theoretically sound light client implementation, the complexity involved makes me hesitant to make the strong claim that Silent Payments are suitable for light clients.

It’s also important to note that these non-interactive key generation protocols (all of them) only make full sense for wallets that don’t hand their xpub to a server (i.e. you’d need compact block filter support). If you do hand out the xpub, the server can do the scanning for you, or even better, they can hand out fresh keys for you.

this is a lighter weight alternative to Silent Payments (no full node required) and a more private approach than BIP47

Perhaps, but you should also compare it to the Prague protocol, and this is where I think it compares less favorably. In comparison, the Prague protocol basically trades off the scanning complexity for outsourced notifications. You send a 32 byte key + a 4 byte (non-blinded) identifier to an outsourcer and they publish it on-chain on your behalf. This ensures the sender’s inputs won’t be correlated with the recipient. The recipient only needs to monitor keys that are associated with their identifier (no scanning).

And as I mentioned on bitcoin-dev, I think it should also be compared to this protocol by @RobinLinus. See the bitcoin-dev post for my description of its benefits.

clarkmoody commented at 4:24 pm on September 8, 2022: contributor

It’s also important to note that these non-interactive key generation protocols (all of them) only make full sense for wallets that don’t hand their xpub to a server (i.e. you’d need compact block filter support). If you do hand out the xpub, the server can do the scanning for you, or even better, they can hand out fresh keys for you.

Agreed. The assumption for all of these proposals is that we are attempting to have good privacy without having to run a server. Other proposals like Lightning Address and those that use the .well-known path on servers are able to achieve great privacy once a server enters into the mix.

Perhaps, but you should also compare it to the Prague protocol, and this is where I think it compares less favorably. In comparison, the Prague protocol basically trades off the scanning complexity for outsourced notifications. You send a 32 byte key + a 4 byte (non-blinded) identifier to an outsourcer and they publish it on-chain on your behalf. This ensures the sender’s inputs won’t be correlated with the recipient. The recipient only needs to monitor keys that are associated with their identifier (no scanning).

Such a notification service is also described in Appendix C, but it only helps to break the wallet clustering analysis around notification. This BIP blinds the recipient key, which is a strict privacy improvement over the Prague Protocol. Given that every light wallet will need to source OP_RETURN data from somewhere (anyone want to write a new block filter for that data - BIP159? 😄 ), a light scanning requirement on a few chunks of data each block should not be too much of a burden, especially for the privacy benefit.

And as I mentioned on bitcoin-dev, I think it should also be compared to this protocol by @RobinLinus. See the bitcoin-dev post for my description of its benefits.

That is also an interesting idea, but it’s not really a spec that can be scrutinized at the level of this BIP.

Overall, I don’t think we need to include an exhaustive comparison to all other proposals & protocols into this BIP in order to merge. That sort of analysis should be left up to mailing list posts or blog posts that help wallets decide which protocols to implement. Indeed, we are not required to amend previous BIPs in light of newer proposals.

RubenSomsen commented at 2:41 pm on September 9, 2022: none

strict privacy improvement over the Prague Protocol

This seems incorrect to me. The Prague protocol doesn’t leak any meaningful information because the sender pubkey is completely fresh and unlinkable. All that is revealed is that some completely anonymous person probably intends to pay a known person (or pseudonym) at some point in the future, which is fairly harmless information. The end result is the same, but the Prague protocol replaces scanning with outsourcing.

While this is not in the write-up, in Prague I even argued that it might be acceptable for the sender pubkey to be known as well (which could save more bytes) because anyone could have published the notification (i.e. Carol could link Alice and Bob without their consent), meaning there is full plausible deniability.

a new block filter for that data

I’m not sure if you mean this BIP or the Prague protocol, but afaict in neither case a block filter is required. You just need the OP_RETURN data that is relevant to you, which is every sender/recipient key pair in the case of this BIP and only the sender keys of your specific recipient key in the case of the Prague protocol.

interesting idea, but it’s not really a spec that can be scrutinized at the level of this BIP

Yes, in my opinion ideally it should have been scrutinized before this BIP was written (i.e. when I made the post on bitcoin-dev), but we’re past that point now.

I don’t think we need to include an exhaustive comparison to all other proposals & protocols into this BIP in order to merge

I agree, it’s not relevant to the question of whether this should be merged. I am merely responding to statements that seemed incomplete/inaccurate to me.

michaelfolkson commented at 3:42 pm on September 9, 2022: contributor

You’re still happy to be a co-author on this BIP though @RubenSomsen despite the above disagreements? An alternative would be you are listed in the acknowledgements rather than as a co-author if you think this is heading in a direction where you disagree with some of what is included in the BIP. Ideally we’d avoid future disputes between co-authors on a BIP if we can.

RubenSomsen commented at 5:18 pm on September 9, 2022: none

An alternative would be you are listed in the acknowledgements

That makes more sense to me as well. On bitcoin-dev I suggested that the authors of the Prague protocol (namely myself, @afilini, and @Kixunil) be acknowledged, since it seems to have served as an inspiration (as well as Silent Payments). In response I was made co-author.

Update authors aa4f030041

clarkmoody commented at 3:26 am on September 15, 2022: contributor

ACK aa4f030

alfred-hodler commented at 1:48 pm on September 15, 2022: contributor

@luke-jr this BIP is completed, we have nothing to add to it.

Update BIP351 reference implementation link 2cabfe167e

luke-jr merged this on Sep 29, 2022

luke-jr closed this on Sep 29, 2022

luke-jr commented at 10:40 pm on September 29, 2022: member

Merged.

Nevertheless, please do address the review comments. I would discourage implementation of anything with on-chain signalling.

clarkmoody commented at 2:55 pm on October 4, 2022: contributor

Nevertheless, please do address the review comments.

Reading back through the comments, it seems like the feedback that remains to be addressed is:

What are the drawbacks of BOLT12, benefits of this over BOLT12, etc?
Objection to using OP_RETURN
Some points from this mailing list post

Pantamis commented at 3:07 pm on October 5, 2022: none

Is there something preventing Alice to add an output in the notification transaction so that she can pay Bob at the same time she notifies him ? (by sending her first payment to P + H(S|0)G in the notification transaction)

I think it’s worth specifying in the BIP whether this is possible or not.

clarkmoody commented at 9:19 pm on October 5, 2022: contributor

Is there something preventing Alice to add an output in the notification transaction so that she can pay Bob at the same time she notifies him ? (by sending her first payment to P + H(S|0)G in the notification transaction)

I think it’s worth specifying in the BIP whether this is possible or not.

There is nothing preventing this behavior, but I would recommend against it since it leaks information about the stealth address. I agree that an additional section on UTXO handling and notification would be helpful. Appendix C describes a notification service that would completely de-link the two parties (while trusting the third a bit).

New BIP 351: Private Payments #1349