wallet: Deniability API (Unilateral Transaction Meta-Privacy) #27792

This PR is the wallet API and implementation portion of the GUI PR ( https://github.com/bitcoin-core/gui/pull/733 ) which is an implementation of the ideas in Paul Sztorc's blog post "Deniability - Unilateral Transaction Meta-Privacy"(https://www.truthcoin.info/blog/deniability/).

The GUI PR has all the details and screenshots of the GUI additions. Here I'll just copy the relevant context for the wallet API changes:

"In short, Paul's idea is to periodically split coins and send them to yourself, making it look like common "spend" transactions, such that blockchain ownership analysis becomes more difficult, and thus improving the user's privacy. We've implemented this as an additional "Deniability" wallet view. The majority of the code is in a new deniabilitydialog.cpp/h source files containing a new DeniabilityDialog class, hooked up to the WalletView class. "

While the Deniability dialog can be implemented entirely with the existing API, adding the core "deniabilization" functions to the interfaces::Wallet API allows us to implement the GUI portion with much less code, and more importantly allows us to add RPC support and more thorough unit tests, which is what this PR attempts to do.

The main "deniabilization" functionality is implemented in spend.cpp/h and feebumper.cpp/h and exposed via the interfaces::Wallet API. We've added RPC support via a new RPC call "walletdeniabilizecoin". We also extended the Wallet tests with coverage for the functionality.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage

For detailed information about the code coverage, see the test coverage report.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process.

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #27865 (wallet: Track no-longer-spendable TXOs separately by achow101)
• #27286 (wallet: Keep track of the wallet's own transaction outputs in memory by achow101)
• #24963 (RPC/Wallet: Convert walletprocesspsbt to use options parameter by luke-jr)
• #21283 (Implement BIP 370 PSBTv2 by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@achow101 - per our discussion, I've created a PR with the core "deniability" functionality in the main repo here. The GUI portion is rebased over this branch and is in the GUI repo. I've updated the PRs' descriptions to point to each other. Please let me know what you think. Thanks!

Not sure how much this achieves. Bitcoin Core's wallet has a "unique" fingerprint on transactions that is shared by way less than 50% of transactions in the network, so creating additional transactions where it is likely (latest when the outputs are spent) that it was a self-transfer seems only to be bloating the utxo set and chain space? I haven't thought about this, but it would be good to have a more in-depth analysis of this, considering using a different "fingerprint vector".

NACK

It doesn't improve privacy. Not considering fingerprints. A better way to improve privacy in bitcoin payments using bitcoin core could be payjoin.

False sense of privacy implemented in this pull request only helps miners with fees.

Bitcoin Core's wallet has a "unique" fingerprint on transactions that is shared by way less than 50% of transactions in the network ...

By fingerprint, do you mean Bitcoin Core's transactions can be identified on the block chain? If that's the case that's unfortunate for sure. What's the nature of the difference? Is it on purpose and if not, would it be possible to fix?

See for example https://github.com/achow101/wallet-fingerprinting/blob/main/fingerprints.md

See for example https://github.com/achow101/wallet-fingerprinting/blob/main/fingerprints.md

Thanks! At a quick glance, it looks like Core's transactions are somewhat close to Electrum's. I'm not familiar with some of the items, but assuming it's possible, I can look into injecting some variety in the "deniabilized" transactions to make them resemble a variety of wallets (eg Electrum as a start). Is there prior work in this by any chance? That would help me for sure. Also, should I pursue it for this PR, or would a follow up PR be acceptable as well?

It doesn't improve privacy.

Can you please elaborate why you think so? Paul's blog post had some pretty compelling arguments why this scheme should work and I think it has a lot of merit.

False sense of privacy implemented in this pull request only helps miners with fees.

It's important to point out that this PR doesn't add functionality that couldn't be achieved manually before. If a user wanted to split a coin and send it to themselves, they could do it with the existing Bitcoin Core (either via RPC/Console or the GUI). It's just cumbersome and error prone.

The API we're adding in this PR makes this a simple and robust single operation (C++ API call or RPC call). It's entirely opt-in and the user can decide to use it or not, up to their own judgement on how effective it is in improving their privacy. I would assume anyone using the C++/RPC interface understands the implications and can be trusted to know what they are doing.

The GUI PR (https://github.com/bitcoin-core/gui/pull/733) implements automation on top of this API, which is also entirely opt-in. We provide an explanation of how it works, so the user could make an educated decision on whether to use it or not. A lot of new Bitcoin users (myself including), weren't aware early on that it's important to use new addresses when receiving BTC. Many exchanges also don't make it easy either (eg by requiring time-locked whitelisting etc). As a result I suspect many users may wish to improve their privacy by "deniabilizing" their UTXOs after the fact. This has been my main motivation to pursue creating this tool. I personally wanted to use it and I suspected others may appreciate it too.

It's important to point out that this PR doesn't add functionality that couldn't be achieved manually before. If a user wanted to split a coin and send it to themselves, they could do it with the existing Bitcoin Core (either via RPC/Console or the GUI). It's just cumbersome and error prone.

Good point and most of the people who tried this with coins involved in any incident regret doing it. Privacy in bitcoin is not as simple as doing some transactions with your own addresses automatically. There are lot of things you need to take care of. One mistake and your privacy is breached, publicly available for the whole world.

Payjoin will improve privacy without wasting fees on these automated transactions by hiding the amounts involved in transaction. Presently, payjoin is less used because of the need for a server but this can be improved.

Good point and most of the people who tried this with coins involved in any incident regret doing it. Privacy in bitcoin is not as simple as doing some transactions with your own addresses automatically. There are lot of things you need to take care of. One mistake and your privacy is breached, publicly available for the whole world.

What kind of incidents do you mean? Do you have some concrete examples? I don't see how one's privacy would be any worse after sending transactions to oneself. Each transaction moves further away from a potentially known UTXO, to new UTXOs that only the user's wallet knows are theirs. To a blockchain analysis that would look like a "spend" and they'll have to guess which one is potentially a different user and which one is a change address. With each split, the guesses become less and less accurate, thus leading to increased privacy, no? If you see a flaw in this argument, please share you explanation, because I'm not seeing an issue.

Hey @denavila, welcome to Bitcoin Core and thanks for working on this! While I think that this idea could be useful to some extent if implemented correctly, I have a few concerns about the current implementation:

• Currently there is no precaution against a user spending two outputs of a deniabilization transaction together. IMO I think that this is quite problematic and could lead to these deniabilization transactions being rendered ineffective. Not only that, but then it also becomes obvious that a user is trying to deniabilize their funds, which is something that should be avoided.
• As mentioned by @MarcoFalke, currently it is simple to identify Bitcoin Core transactions. Given that the Bitcoin Core wallet does not make up a large portion of on-chain transactions, if a blockchain observer sees that there is a long chain of transactions sending to and receiving from Bitcoin Core wallets, it could become clear that these are deniabilization transactions.
• Additionally, in the current implementation all of the inputs of a deniabilization transaction must share the same address. It is not immediately obvious to me why this was done. This fact, combined with the previous two issues, could make it trivial to identify deniabilization transactions created by BItcoin Core.

Solving some of these problems could be tricky and might require significantly more code being added to the Bitcoin Core wallet. Given this, I’m unsure whether it would make sense for an implementation of something like this to be built directly into Bitcoin Core. I'd be curious to hear what others think.

Good point and most of the people who tried this with coins involved in any incident regret doing it. Privacy in bitcoin is not as simple as doing some transactions with your own addresses automatically. There are lot of things you need to take care of. One mistake and your privacy is breached, publicly available for the whole world.

What kind of incidents do you mean? Do you have some concrete examples? I don't see how one's privacy would be any worse after sending transactions to oneself.

Sorry. I cannot share exact incidents that involve "crime" on GitHub. However, you can look at this explanation from bitcoin privacy wiki to understand basics:

"In a peeling chain, a single address begins with a relatively large amount of bitcoins. A smaller amount is then peeled off this larger amount, creating a transaction in which a small amount is transferred to one address, and the remainder is transferred to a one-time change address. This process is repeated - potentially for hundreds or thousands of hops - until the larger amount is pared down, at which point (in one usage) the amount remaining in the address might be aggregated with other such addresses to again yield a large amount in a single address, and the peeling process begins again[5]."

https://en.bitcoin.it/wiki/Privacy

Hi @ishaanam and thank you for looking at my PR. I do agree the implementation can be improved and your feedback helps me greatly.

Here are my thoughts regarding your concerns:

1. I thought about this problem too and indeed spending deniabilized outputs is somewhat challenging. Obviously, if the user just HODLs, or spends less than a deniabilized UTXO, then all is good. However if they spend larger amounts, more than one UTXOs would get picked up and that would reveal that they belong to the same user. I don't think this destroys the entire privacy gained from deniabilization, since in a regular "spend" transaction with no-reuse tag, would look similar to deniabilized transaction, and blockchain analysis would've probably assigned some probability that some UTXOs are change, and our spending would confirm those ... But yes, the larger the spend the more would be revealed, so it's not ideal. Perhaps we could augment the UTXO selection algorithm to be deniabilization-aware, and perhaps there's ways to pick UTXO that minimize the damage? I'm not sure what heuristic we should use for that, so I'm open to suggestions. Another option would be to provide a user parameter for "likely spendable size", which we use in the chunking decision and make strictly bigger chunks if possible. There's already logic that avoids "dust" chunks so it won't be difficult to extend in that direction.

2. Yes, the fingerprint concern is valid and unfortunate (and sadly I wasn't aware of it when I started this work). However it's also a pre-existing problem and while I do agree we should fix it, I'm hoping we could do this in a separate PR. Also I'm not yet knowledgable enough to tackle it, so if you guys want me to pursue a solution I could use some help.

3. The reason for grouping UTXOs that share the same address is that (and correct me if I'm wrong) if we can spend one UTXO from a given address, then we have ownership of all UTXOs that share that address. In other words, we don't lose any privacy by deniabilizing the whole group. In my limited experience, large amount of shared-address UTXOs tends to happen when transferring from an exchange (eg periodic buys) - often people don't update their withdrawal address, and many exchanges make it difficult to update (eg by requiring whitelisting and so on). This grouping allows the deniabilization process to actually reduce the UTXO count if used for just few cycles. At any rate, it would be trivial do disable the grouping if there's a flaw in this reasoning, or otherwise have it on a user toggle or some such.

Given this, I’m unsure whether it would make sense for an implementation of something like this to be built directly into Bitcoin Core

I thought about if this could be implemented in some kind of a plugin, but as far as I know, there's no support for plugins yet? Originally my PR was entirely in the GUI, since I wanted to avoid API changes, but @achow101 correctly pointed out it would be safer and easier to test the main functions as part of the wallet API, so that's why I split the PR into a core and GUI parts. If there's a path to implement this in a more "plugin" way, I'd be interested to hear of course.

Thank you for your feedback. I greatly appreciate it.

1. But yes, the larger the spend the more would be revealed, so it's not ideal.

Not only that, but if you spend a UTXO that’s the change from the previous transaction with the output of the next, that’s one thing. But if you spend an output with the change from fifteen transactions prior, you may heavily imply that all payments in between were also self-sends.

• Another option would be to provide a user parameter for "likely spendable size", which we use in the chunking decision and make strictly bigger chunks if possible. There's already logic that avoids "dust" chunks so it won't be difficult to extend in that direction.

Yeah, that sounds like a reasonable direction of development. It would e.g. be possible to pick from two otherwise similar inputs sets the one that creates a change output whose value is most different from other UTXOs already in the wallet. Otherwise, it is a bit difficult to predict what a “likely spendable size” is, since it depends on many different factors such as future exchange rate, and unknown future motivations for the user to create transactions. I would surmise that past transactions would only have a limited utility in predicting future spending sizes. In fact in my master-thesis, I explored a similar idea by attempting to always create change of a similar amount as the recipient output. Unfortunately, the resulting UTXO pool performed subpar.

In a tx graph ( tree really ) with large and randomized fan out, eventual merging of some terminal UTXOs does not necessarily indicate that the later clustered entity is the same entity that was KYC’ed on an exchange prior to the deniabilization fan out. Some probability maybe assigned to both entities being the same but there’s also a combinatorial explosion of other possible clusterings in between.

Would it not be better to postpone removing the Core tx fingerprints to Deniabilization 1.1 or 2.0 version and get this safe, isolated UI centric feature merged in sooner rather than later ?

Needs rebase if still relevant

Needs rebase if still relevant

Ah, I didn't notice, sorry. I'll rebase it now.

I hit an issue in my wallet tests after rebasing. Looks like a bug was introduced in ExtractDestination. I left a comment in #28246

Maybe mark as draft while CI is red?

@maflcko - The CI has been failing on the "fuzzer" test with a "time out", and I re-tried it few times, but to no avail. I'm not sure if this is a global CI issue, or if it is indeed something wrong with my code. Is there a chat/forum I can inquire about CI issues?

The fuzz timeout can be ignored for now. It will be fixed next week.

Concept ACK

I cant comment on the approach before testing and going through everything. I had tried to implement similar feature in electrum: https://github.com/spesmilo/electrum/pull/8159

Fingerprinting issue could be resolved by using random fingerprints for transactions in this feature or add spoofing feature in core: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2023-October/021998.html

BTCPay Server also randomizes wallet fingerprint for transactions: https://github.com/btcpayserver/btcpayserver/blob/a921504bcf619c5e845813b8f994b39147694a97/Changelog.md?plain=1#L2079

@1440000bytes , @maflcko, @achow101
In commit c6e1d30 I implemented a first pass "fingerprint spoofing" for deniabilization transactions, based on the pointers you guys gave me. Please let me know what you think. Thanks!

@denavila

If there's a path to implement this in a more "plugin" way, I'd be interested to hear of course.

Technically this could be written as a python script that uses Bitcoin Core RPCs to create and sign transactions, but does the deniabilization steps and anti-fingerprinting measures in python.

That being said, I also have a few reservations about the fingerprint imitation implemented here:

• It seems like it introduces a big maintenance burden because whenever another wallet changes the fingerprints they exhibit, this will need to updated correspondingly.
• Additionally, I'm not sure how much this actually fixes. Because this does not eliminate all of Bitcoin Core's fingerprints (which would be quite a challenge), combining another wallet's fingerprints with Bitcoin Core's could just produce a more obvious fingerprint which will make it obvious that a given transaction is a deniability transaction.
• Lastly, some of the fingerprints that are removed when certain wallets are imitated (like standardVersion and antiFeeSniping) are actually useful. antiFeeSniping is something that we want to be doing because it prevents fee-sniping. In fact, we should be encouraging other wallets to do this instead of removing it from our own transactions. Additionally, the nVersion of a transaction may need to have a certain value depending on which time locks are being used (this is not implemented yet, but now that Bitcoin Core has miniscript, they likely will be soon).

@ishaanam - Thank you for your feedback!

Yes, it's true Deniability can be implemented without changing the API. Indeed, my first implementation was entirely in the GUI source code. Probably it can also be done in Python and RPC, but Python is not my forte so I wouldn't attempt that myself.

Having said that, @achow101 suggested that we move the core deniability logic in the Core source code (eg in the CWallet implementation), as that would make it more robust, easier to test and allow exposing it to RPC. I agreed with this assessment, and indeed after splitting off the core functionality, the GUI portion became much more straightforward. At any rate, if the API or GUI PRs don't get merged, implementing this via RPC / script would be the way to go.

Regarding the fingerprinting:

1. The maintenance concern is legitimate, and while not super time sensitive (since users may use old wallets for a long time), it does mean we'd need to revisit this form time to time. Let me know if you have ideas how we can do this in a more future proof way?
2. The implementation I put together above was just a first stab to get feedback. I implemented the 3 fingerprints that were easy and I understood. I'd be happy to implement more with some help. For example what are positive/negative EV inputs?
3. Currently I'm just using the fingerprint spoofing for deniabilization transactions, which by design use the lowest fees, and so hopefully they won't encourage fee sniping even if we use the nLockTime = 0. But I do agree, for the general case, we do want it, so maybe I can add an additional control to randomize in favor of sniping more often ... Let me know if you have ideas how we can improve on that. I can also add nVersion dependence on the time lock. What should I check for?

A failure in the "Win64 native, VS 2022" CI job is unrelated. See: #28905.

This pull silently conflicts with master:

wallet/rpc/spend.cpp: In lambda function:
wallet/rpc/spend.cpp:1806:49: error: ‘transaction_identifier<has_witness>::transaction_identifier(const uint256&) [with bool has_witness = false]’ is private within this context
 1806 |                 COutPoint outpoint(txid, nOutput);
      |                                                 ^
In file included from ./primitives/transaction.h:14,
                 from ./consensus/validation.h:11,
                 from wallet/rpc/spend.cpp:5:
./util/transaction_identifier.h:16:5: note: declared private here
   16 |     transaction_identifier(const uint256& wrapped) : m_wrapped{wrapped} {}
      |     ^~~~~~~~~~~~~~~~~~~~~~

Tested it and I think it can be improved further before users can try it on mainnet: https://github.com/bitcoin-core/gui/pull/733#issuecomment-2081539227

<!--85328a0da195eb286784d51f73fa0af9-->

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin/bitcoin/runs/25124417167}

This PR does not seem to have attracted much attention from reviewers. As such, it does not seem important enough right now to keep it sitting idle in the list of open PRs.

Closing due to lack of interest.

Its true there was a lack of interest to review this pull request and its not the best tool for privacy. However, a worse way to do the same thing was used in Samourai earlier and still available in Ashigaru. Ricochet fee is 0.001 BTC for the default 4 hops, with an additional fee for each extra hop beyond 4.

http://ashicodepbnpvslzsl2bz7l2pwrjvajgumgac423pp3y2deprbnzz7id.onion/Ashigaru/Ashigaru-Mobile/src/commit/6b37843e5d1f7748836471cd1e3621a5fb1cc29f/app/src/main/java/com/samourai/wallet/ricochet/RicochetMeta.java#L65

Users have paid more than 2.8 BTC in fees for ricochet service in past and it was even mentioned in the DOJ press release.

So, it would have been great if users had a better alternative which has no service fee.

An alternative for anyone reading this pull request in the future: https://github.com/kristapsk/bitcoin-scripts/blob/master/ricochet-send.sh

It's a bash script that uses bitcoin core wallet.