Wallet: Support not reusing addresses #10233

Implements -freezeaddress (also via the bitcoin.conf): a way to skip address(es) during coin selection. Fixes #10065 Requested to be similar to the lock coins functionality, but with addresses & persistence.

This seems entirely unrelated to #10065 or address reuse...

Not sure if this is the right direction... I agree with @luke-jr, blocking "addresses" from coin selection seems to be a different thing the "address reuse". @jet0: can you explain what kind of a use case your PR does allow/solve?

doing something like automatically freezing addresses once used would be interesting; but we'd need to have an extra 'frozen balance' displayed everywhere.

@jonasschnelli , all - perhaps the title of this commit is wrong (it does not prevent reuse globally) - I was trying to solve the issue in #10065 and would like to get @RHavar 's opinion on this approach, since his use case was: People send tiny amounts to well known addresses in his wallet. The coin selection mechanism continually spends those outputs, tainting most outgoing transactions. He was looking for a (persistent) way to exclude certain well known addresses from the coin selection algorithm. (RPC calls to display amounts/addresses TBD if the overall idea works).

An advanced wallet feature should be the ability to disable all address reuse. The attack this is designed to stop is if I know that the address 1BPmZt3kjnFffANNs6MCBBBKRYwRGGrXQZ is part of Alice's wallet, it shouldn't be possible to continually pepper it with dust and learn more about her as her wallet automatically keeps spend-linking it with her other outgoing transactions.

@jet0 I'm not sure if this right granularity to disable address reuse. For instance, let's say I request someone sends me money to address X and they do so.

I can't immediately use freezeaddress because doing so would stop me spending the coins. I'd have to wait until the coins left address X and then freeze it. But that's rather annoying to do, and leaves open the reuse problem until I've spent all the money.

Also I don't think using the configuration is appropriate for a list of frozen addresses. It's highly dynamic and often huge (I personally have half a million addresses I'd like to freeze).

I think a better way of solving the problem would be to extend the locked outputs. I think making them persistent would be a good first step (it's rather annoying that locked outputs aren't already persistent). And then the configuration option to disable address reuse could automatically lock any outputs you got, if the associated address has already been used.

... tainting most outgoing transaction`

Yes. This is a problem.

But is blacklisting addresses from coin-selection a valid solution? I would prefer if I could block certain UTXOs by txid/vout. Wouldn't blocking addresses lead to the problem that I cannot distinguish between taint-inputs and a real inputs (both could be sent to the same address)?

Both is already solvable: -> lockunspent (just lock the taint utxo) -> Qt coin controll

But your approach may be useful. But I'd recommend to add it as RPC command rather then a startup argument (otherwise you need to restart Core in order to get this thing working).

Wouldn't blocking addresses lead to the problem that I cannot distinguish between taint-inputs and a real inputs

I believe the only time one needs to worry about taint inputs is after a spend from that address happens, so long as the spends make sure to spend all (non-dust) coins paid to that address so far.

What we should prefer doing is when we spend any coin connected to an address we should spend all (or all non-dust) coins for that address. Then any new(/remaining) coins should be considered tainted, and ideally spent via coinjoins or given away as donations.

I believe the only time one needs to worry about taint inputs is after a spend from that address happens, so long as the spends make sure to spend all (non-dust) coins paid to that address so far.

That's a good point, but from a pragmatic point of view I think it's not worth it. It comes at a terrible cost to coin selection (instead of picking from a set of outputs, it's going to have to pick from a set of sets). And coin selection in core is already one of the weakest parts, I don't favor making it any harder to deal with.

And keep in mind that if we're trying to prevent someone dusting the wallet that dust (even when it's standard) is a lot more expensive to spend than it contains. (As dust is based on the tiny relayfee, but the wallet typically uses a feerate of >100x more than that when creating transactions).

So I think for simplicity, it makes a lot more sense to just lock the output immediately instead of delaying it at the cost of putting fancy logic in the coin selection as well.

And then the configuration option to disable address reuse could automatically lock any outputs you got, if the associated address has already been used. @RHavar - to clarify - "associated address has already been used" - do you mean to automatically lock all (new or existing) outputs for an address that matches any in the already locked set?

@jonasschnelli what would you recommend the feature looks like?

But your approach may be useful. But I'd recommend to add it as RPC command rather then a startup argument (otherwise you need to restart Core in order to get this thing working).

A set of RPC commands to freeze/unfreeze a wallet address from being used in coin selection (similar to the IsFrozenCoin() I have)?

@RHavar - to clarify - "associated address has already been used" - do you mean to automatically lock all (new or existing) outputs for an address that matches any in the already locked set?

I mean if you're running with -dont-allow-address-reuse when the wallet sees a new inbound output, it checks to see if that address has already previously been used. If it's been used, it locks the output (so it's not spendable).

Maybe other people will disagree with that approach, but it seems the simplest to implement.

it checks to see if that address has already previously been used

How do you define "previously been used"? I don't understand its relationship with "lockunspent" feature in your comments. (1) funds were received to it, or (2) its funds were spent, or (3) that some or all of its txouts are locked via lockunspent RPC or (4) something else? On the other hand - if we are working with persistent lockunspent outputs -- it sounds simple to automatically lock all other outputs with the associated address (via a config option).

How do you define "previously been used"? I don't understand its relationship with "lockunspent" feature in your comments. (1) funds were received to it, or (2) its funds were spent, or (3) that some or all of its txouts are locked via lockunspent RPC or (4) something else?

I mean (1). If you're getting money to a previously used address (i.e. You've previously received money to this address before) it would just lock it, so it doesn't get automatically spent.

@RHavar I think I understand now - in your original request you were not intending to execute a bunch of "lockunspent" calls or "freezeaddress" with specific txouts or addresses in mind, but wanted a system-wide switch to 'spend once from each address'. I.e. no manual txout or address selection, correct?

@jet0 yeah, ideally it would just be an option that I enable and the wallet does all the hard work for me =)

I mean, if there was an rpcinterface freezeaddress I'd happily use it. But it's more of a pain in the ass than I'd like (especially if I had to first make sure there was no money left in it).

That's a good point, but from a pragmatic point of view I think it's not worth it. It comes at a terrible cost to coin selection (instead of picking from a set of outputs, it's going to have to pick from a set of sets).

With due respect, I think you're handwaving here -- and that isn't a way to accomplish anything. Doing anything less then treating all spendable coins paid to an address as a single unit is a guaranteed privacy failure... and I don't believe this would add_any_ complexity to coin selection-- the "coin" you select over is just the effect of spending the whole group... though it would make corner cases where we mishandle dust payments a bit more obvious, but those need to be addressed regardless.

And coin selection in core is already one of the weakest parts,

Can you suggest ANY piece of software that does better? (A lot of other things do provably worse, see e.g. the study published at the last scaling bitcoin...)

And keep in mind that if we're trying to prevent someone dusting the wallet that dust (even when it's standard) is a lot more expensive to spend than it contains.

Not considering outputs which have a marginal size greater than their fee-rate cost is pretty trivial (one loc), and independently interesting.

So I think for simplicity, it makes a lot more sense to just lock the output immediately instead of delaying it at the cost of putting fancy logic in the coin selection as well.

You mean to 'lock' subsequent payments after you've had a first? This would create a trivial attack if widely used-- I see payments to new address and I quickly pay dust. Now the actual payment is 'locked' and all you'll spend is the dust.

This vulnerability is avoided if the horizon for locking is immediately prior to spending, since an attacker cannot race that. I think it would also turn the feature from an exotic and difficult to support thing to something we could reasonably make a default, at least once there were the right UI features for handling the held balance.

and I don't believe this would add any complexity to coin selection-- the "coin" you select over is just the effect of spending the whole group

Well, that's only because the current algorithm is naive and doesn't even consider the size of the transaction. But if it's smarter, then there's a different cost to adding different amounts of inputs (i.e. adding two 5 BTC inputs is going to cost more than adding 10 BTC input).

Also how do you propose it handles an address that also has uneconomical to spend outputs?

Can you suggest ANY piece of software that does better? (A lot of other things do provably worse, see e.g. the study published at the last scaling bitcoin...)

I haven't seen the study. But core's just tries to minimize the amount of the change output, which strikes me as a little bizarre and a privacy nightmare (>99% of the time, the change is trivially identifiable). It also conflates utxo consolidation with coin selection (which I agree is what most normal users wallet want), and it routinely adds inputs that cost more to spend than they add.

But anyway, I've been able to save well over 70% on transaction fees with a trivial patch to core and a utxo consolidation cron job, which translates to over a thousand dollars a month. Not that it is suitable for normal users, but it clearly shows bitcoin's coin selection is not nearly "optimal" and that one size doesn't fit all.

That said, you make a convincing case about avoiding the attack vector. I just hope it's not one of those cases where the perfect solution is so much work that no one bothers doing it, while they would've been happy doing the good one. =)

Oh by no means do I think the behavior is anywhere near optimal, but to me it sounded you were opposing fixing it in favor of more special purpose options. The amount of configurables is already a major burden in improving the software already. Damn, I really wish your answer had been "I think X does it perfectly." :P

But core's just tries to minimize the amount of the change output,

Heh. If it just did that it would be a lot simpler. Instead its actual objective probably takes a page of text to explain. The main goal, such as it is, is best understood as privacy optimizing under a strong assumption that addresses are not reused-- this assumption is just outright wrong, but if you look at it in that light its behavior makes more sense. It tries to move as little of your coins at a time as it can to avoid linking the payments. Beyond that, it tries to make change outputs (if there is change) large enough to be meaningfully useful in the future. (0.01 btc-ish is the current target).

I think It's far too hesitant to throw a bit of dust away when doing so would really improve the result (e.g. avoiding a change output at all)... Maybe the upcoming improvements in the fee estimator will make it easier to come up with a reliable estimate of "effectively dust" rather than just using the relay fee.

But if it's smarter, then there's a different cost to adding different amounts of inputs (i.e. adding two 5 BTC inputs is going to cost more than adding 10 BTC input)

Depends on what fees are now relative to the future, absent other information (like knowing you're going to make consolidation transactions during lower feerate times) it's probably generally the case that we should prefer to spend coins eagerly at least when doing so won't gum up privacy-- simply because generally we expect fees to not be lower in the future; but right now it's not eager because it assumes all addresses are independent.

More eager policies are also good for the long term because w/ aggregation in the future it will be cheaper to spend more inputs at a time, even ignoring the commons benefit.

Also how do you propose it handles an address that also has uneconomical to spend outputs?

Spend a small amount of them anyways (I doubt many users would mind being a little civically minded-- and paying a tiny bit more in their transactions to remove dust, especially if they're break-even to spend vs a loss), and if there are many ignore them like received after the fact coins-- so that they don't become attractive attack vector.

E.g. specific implementation would be to go through the available scriptpubkeys (or taint groups, though that functionality is less useful now that coinjoin obfuscates them somewhat) which haven't been marked frozen and gather up all their economical to spend outputs, plus up to one or two extra outputs which aren't economical from that spk and treat it as a single input for the purpose of the rest of the selection. Then when a transaction is sent, mark all the involved SPKs as frozen (probably need a better term than frozen, since it sounds like a third party adverse action rather than a privacy feature).

It tries to move as little of your coins at a time as it can to avoid linking the payments.

I am familiar with the coin selection algorithm, and don't understand how you come to this conclusion.

The algorithm is a little tricky, because it's kind of two unrelated algorithms but it basically is:

Stage 1: If sending N, and you have an input of exactly N ... use that. Stage 2: If the sum of all inputs < N is greater than N, pick from them randomly until you you're >= N. Stage 3: Otherwise, go to ApproximateBestSubset and do a partial-ish randomish selection from biggest to smallest with the goal of picking the subset that that has the least amount of change.

I really struggle seeing how this is "trying to move as little of your coins at a time", when Stage 2 almost does the exact opposite.

Sorry for the long discussion - I'd like to bring this to some closure. I think @jonasschnelli had some use for "freezeaddress" type functionality, perhaps with RPC calls instead of the config - I will deliver that if it can be quantified. Comments please. :) @RHavar - is there a minimum subset of manual lockunspent + some automation for new txouts appearing you'd like to see? (The deluxe/ideal research GMaxwell & yourself are going over is bigger than I can do now). We can take the detailed discussion offline if you prefer.

This seems to have reached an impasse, closing for now, feel free to tag me to reopen it when there is more agreement that this specific change is a good idea.