RHavar commented at 8:00 pm on February 16, 2018: contributor

BIP69 is a pretty under appreciated bip that defines a standard ordering for transactions inputs / outputs.

There’s quite a few obvious advantages to the approach, like making it easy to verify correctness, or making it easier to write unit tests. But I think the most important aspect is it helps users verify their wallet isn’t maliciously leaking information. Granted a malicious wallet could try leak information via the nonces, deterministic signing helps make that detectible with access to the private keys.

As some wallets have already moved to bip69 (and unlikely to consider moving back) it would be very beneficial if core moves in the same direction resulting in better transaction uniformity for everyone.

(And a bounty for fun: if this is merged in the next 3 months I’ll offer 0.03 BTC to the author :D)

fanquake added the label Consensus on Feb 17, 2018

jl2012 commented at 5:49 pm on March 5, 2018: contributor

@fanquake I think BIP69 is merely an informational BIP, not consensus. Making such kind of thing consensus will invalidate pre-signed txs and limit flexibility for future upgrade

MarcoFalke added the label Privacy on Mar 5, 2018

MarcoFalke removed the label Consensus on Mar 5, 2018

Sjors commented at 7:45 pm on March 15, 2018: member

@jl2012 I don’t think @RHavar is proposing to make this a consensus rule, merely to use this when creating transactions.

The standard is currently used by several wallets, e.g. blockchain.info, Electrum and probably everything that uses BitcoinJS, so it would help blending in.

According to the BIP std::lexicographical_compare should do the trick.

Do we know of any downside?

Sjors commented at 3:18 pm on March 17, 2018: member

@luke-jr pointed out elsewhere that there might be an issue with SIGHASH_SINGLE. From the BIP:

While out of scope of this BIP, protocols that do require a specified order of inputs/outputs (e.g. due to use of SIGHASH_SINGLE) should consider the goals of this BIP and how best to adapt them to the specific needs of those protocols.

the most important aspect is it helps users verify their wallet isn’t maliciously leaking information

Compromised wallets can do all sorts of damage. I think a more important aspect is blending in with other wallets.

MarcoFalke added the label Brainstorming on Mar 17, 2018

MarcoFalke added the label Wallet on Mar 17, 2018

RHavar commented at 3:56 pm on March 17, 2018: contributor

Compromised wallets can do all sorts of damage. I think a more important aspect is blending in with other wallets.

Well, not that it really matters what is more important but they go a bit hand-in-hand. Like in the context of how I use bitcoin core, I don’t really care – but there other wallets where bip69 is important to me (with my trezor) and I wouldn’t want to give it up. But the best of both worlds is that every wallet uses bip69 for maximal transaction conformity (hence this issue)

nopara73 commented at 3:40 pm on April 22, 2018: none

As long as most wallets don’t comply with BIP69, (which will be the case for like forever,) input and output shuffling results in superior privacy.

Of course testing it requires a bit more effort, but should you make your software worse so to save 10 lines of testing code?

RHavar commented at 4:57 pm on April 22, 2018: contributor

As long as most wallets don’t comply with BIP69, (which will be the case for like forever,) input and output shuffling results in superior privacy.

I’m not sure I can agree. I hope we can agree on two basic ideas:

The status-quo is harmful – it allows for (messy) clustering of transaction origins. Which among other things can be used to further increase the accuracy of estimates if outputs are change or not
Certain wallets require the determinism that bip69 offers, while for other wallets it doesn’t matter

Which I think leads to the logical conclusion that moving to (something like) bip69 is the smart thing – even for wallets (perhaps like core) that don’t really care too much about it.

If you are worried that core switching to bip69 could reduce privacy (which I don’t think is a big concern, as several big wallets already are using bip69), there is an easy solution: it can slowly be phased in (e.g. seed a random number generator with the hash of the first priv key, then with a X% chance use bip69 or shuffle with inputs/outputs with the rng).

Of course testing it requires a bit more effort, but should you make your software worse so to save 10 lines of testing code?

Testing improvements really is one of the smallest benefits of bip69, but you are really underselling it. For a wallet that uses bip69, if a transaction is ever created that doesn’t conform – then it’s immediately obvious. You don’t nearly have the same benefit with shuffled inputs/outputs – where it would be very easy to have a wallet or code path that doesn’t shuffle without anyone realizing.

nopara73 commented at 6:05 pm on April 22, 2018: none

I assume if you take a look at the blockchain how many transactions are BIP69 compliant today you will find it’s less than 5%. I think our differences come from that you assume otherwise.

If my assumption is correct, with BIP69, if blockchain analysis looks at a transaction it would know if the transaction can be from Core or not. If it’s shuffled then it would not have that information.

I think this change requires some statistics: “What percentage of blockchain transactions are BIP69 compliant today?”
And if the answer is “tiny”, then BIP69 integration is just flagging the transaction as a Bitcoin Core transaction to make the job of Blockchain analysis easier.

achow101 commented at 6:14 pm on April 22, 2018: member

I think this change requires some statistics: “What percentage of blockchain transactions are BIP69 compliant today?”

According to https://p2sh.info/dashboard/db/bip-69-stats?orgId=1, approximately 55% of transactions comply with BIP69.

nopara73 commented at 7:00 pm on April 22, 2018: none

According to https://p2sh.info/dashboard/db/bip-69-stats?orgId=1, approximately 55% of transactions comply with BIP69.

Let’s try to estimate the false positives. It was approximately 40% until 2015, so the creation of the BIP may resulted in 15% additional BIP compliance. A more accurate estimate would be 15*0.4 + 15 = 21% of the network is actively BIP69 compliant today.

nopara73 commented at 7:18 pm on April 22, 2018: none

Our threat model is: a blockchain observer wants to know if a transaction was done with a Bitcoin Core wallet:

	BIP69	Shuffling
55% of the times tx is BIP69 compliant	maybe	maybe
45% of the times tx is NOT BIP69 compliant	no	maybe

RHavar commented at 7:20 pm on April 22, 2018: contributor

Let’s try to estimate the false positives.

It’s not really a false-positive per se, it’s more just accidentally compliance. Which raises an interesting point: if you generate a transaction that is bip69-compliant it’s only probabilistic (depending on the amount of inputs/outputs) that you were using a bip69 wallet. But if you generate a transaction that isn’t bip69 compliant, then you know for sure they’re using a non-bip69 wallet.

So everything being equal, you’re still better off using bip69

nopara73 commented at 7:29 pm on April 22, 2018: none

@RHavar That would only make sense if the Bitcoin Core would be close sourced.

RHavar commented at 8:04 pm on April 22, 2018: contributor

@RHavar That would only make sense if the Bitcoin Core would be close sourced.

???

nopara73 commented at 8:49 pm on April 22, 2018: none

@RHavar I misunderstood what you said:

But if you generate a transaction that isn’t bip69 compliant, then you know for sure they’re using a non-bip69 wallet.

So your threat model is: a blockchain observer wants to know if a transaction was done with a BIP69 compliant wallet or not.

My threat model was: a blockchain observer wants to know if a transaction was done with a specific wallet or not.

The first threat model asks the wrong question, because blockchain analysis doesn’t care if a wallet BIP69 compliant or not.

But we could even take a step back and consider all possible 2 output clustering combinations: you’ve done a transaction with your wallet and blockchain analysis wants to know which was your change.

A

	BIP69	Shuffling
Which is the change?	dontknow	dontknow

B

	BIP69	Shuffling
Which is the change?	1.	dontknow

C

	BIP69	Shuffling
Which is the change?	canthappen	dontknow

D

	BIP69	Shuffling
Which is the change?	canthappen	dontknow

E

	BIP69	Shuffling
Which is the change?	canthappen	dontknow

F

	BIP69	Shuffling
Which is the change?	canthappen	dontknow

Question asked during clustering

Which one was the change?

Blockchain observer deducted the answer to the question by examining if a specific transaction was done with a specific wallet or not, ergo applying the second threat model directly to every single transaction.

Can you come up with a scenario where blockchain analysis is interested if the wallet is BIP69 or not?

nopara73 commented at 8:55 pm on April 22, 2018: none

@RHavar That would only make sense if the Bitcoin Core would be close sourced.

???

If a wallet is closed sourced and BIP69 compliant, but blockchain analysis doesn’t know that, then it could generate BIP69 compliant txs all day long, blockchain analysis would not be able to rule out on a non-BIP69 transaction that it wasn’t done by the wallet.

At least originally I thought that’s what you meant.

RHavar commented at 9:41 pm on April 22, 2018: contributor

Your analysis of scenario B isn’t correct, it’s not generally clear if the bip69 use is accidental or on purpose – so it’s not correct to assume output 1 is change.

An analysis service can definitely say it probably is – but actually the exact same thing in reverse is happening scenario D which in practice is going to lead to more information leak.

But I really think you’re overthinking it. The solution here is to get as many wallets and transactions using bip69 as possible, and suddenly there’s no difference and everyone wins.

nopara73 commented at 7:05 am on April 23, 2018: none

Your analysis of scenario B isn’t correct, it’s not generally clear if the bip69 use is accidental or on purpose

Of course the assumption was Blockchain analysis knows the wallet the first transaction was taken with. That’s often the case, since there are a bunch of wallet identifier metadata left behind: 1. what fees are added, 2. rbf usage patterns, 3. types of scripts used, etc…

but actually the exact same thing in reverse is happening scenario D which in practice is going to lead to more information leak.

???

But I really think you’re overthinking it.

A deterministic algorithm, until everyone uses it will leak more information than a nondeterministic one. The intuition is quite clear here.

The solution here is to get as many wallets and transactions using bip69 as possible, and suddenly there’s no difference and everyone wins.

Agreed, yet it’s not happening since 2015. Good luck getting CoinBase/other banks getting into Bitcoin to use BIP69. You need an SF for that, not a BIP. Which by the way, if you would want to propose this SF, I would support.

RHavar commented at 1:23 pm on April 23, 2018: contributor

Of course the assumption was Blockchain analysis knows the wallet the first transaction was taken with.

Ok, sure. In that case scenario B leaks which is the change. But knowing which wallet a transaction uses is hardly the common case, and the main purpose of this issue is to make it harder to find out ;D

Good luck getting CoinBase/other banks getting into Bitcoin to use BIP69

It doesn’t requires 100% use to be greatly beneficial. And it’s a lot more reasonable and easy to go “everyone use bip69” direction, then “everyone use shuffling” direction.

You need an SF for that, not a BIP. Which by the way, if you would want to propose this SF, I would support.

I don’t think a SF for this would be reasonable, especially without months (or years?!) notice. You could pretty easily force adoption though by making non-bip69 transactions non-standard. Of course the first step before that, is first achieving as wide-spread use of bip69 as possible.

Rspigler commented at 8:26 pm on October 31, 2018: contributor

Should this be updated to deterministic sorting based on a semi-secret discussed by Ryan on the bitcoin-dev ML and bitcoinops newsletter?

RHavar commented at 8:52 pm on October 31, 2018: contributor

Yeah, I guess I’ll just close this. It’s clear that “Let’s get everyone using bip69” is probably insurmountable, so probably the best approach is to come up with a new standard for deterministic sorting to replace bip69 (based on a “semi-secret”) and see if the current bip69 users are amenable to.

I don’t really have the time for it, but if anyone want to pick up the mantle..

RHavar closed this on Oct 31, 2018

benma commented at 10:07 pm on November 13, 2018: none

For future reference: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-October/016457.html

shesek referenced this in commit 8292324b37 on Mar 5, 2019

greenaddress referenced this in commit d592eacfe5 on Mar 6, 2019

DrahtBot locked this on Sep 8, 2021

Implement bip69 (Lexicographical Indexing of Transaction Inputs and Outputs) #12457

A

B

C

D

E

F

Question asked during clustering