Silent Payments: sending

josibake commented at 4:37 pm on August 2, 2023: member

This PR is part of integrating silent payments into Bitcoin Core. Status and tracking for the project is managed in #28536

This PR depends on #28122 and is marked as a draft until it is merged. If interested in those commits, please review on #28122

Pre-work / Refactors

The first three commits are pre-work / refactors needed for this PR. They are broken out into the following PRS:

Sending

Silent Payments logic

The main focus of this PR is:

Applying the Taptweak to a taproot internal private key (this is a copy-paste of the code for applying the taptweak in the signing process)
Getting a private key from a given scriptPubKey
Creating silent payment outputs
Applying the created scriptPubKeys back to the vector of CRecipients

The functions are then used together to create silent payment outputs during CreateTransactionInternal.

Final steps

The last commits ensure that:

Coin selection is silent payments aware and knows to exclude taproot script path spends and inputs with unknown witness when funding a transaction which pays to a silent payment address
The change output type is correctly chosen when paying to a silent payment address
Functional tests

DrahtBot commented at 4:37 pm on August 2, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/28201.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#32238 (qt, wallet: Convert uint256 to Txid by marcofleon)
#32028 (Update secp256k1 subtree to latest master by hebasto)
#31974 (Drop testnet3 by Sjors)
#31936 (rpc: Support v3 raw transactions creation by Bue-von-hon)
#29491 ([EXPERIMENTAL] Schnorr batch verification for blocks by fjahr)
#28333 (wallet: Construct ScriptPubKeyMans with all data rather than loaded progressively by achow101)
#27865 (wallet: Track no-longer-spendable TXOs separately by achow101)
#27286 (wallet: Keep track of the wallet’s own transaction outputs in memory by achow101)
#27260 (Enhanced error messages for invalid network prefix during address parsing. by portlandhodl)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot added the label CI failed on Aug 2, 2023

josibake renamed this:
~~Silent Payments: implement sending~~
Silent Payments: sending
on Aug 3, 2023

josibake force-pushed on Aug 3, 2023

DrahtBot removed the label CI failed on Aug 3, 2023

josibake commented at 9:50 am on August 3, 2023: member

Maybe add a quick summary in the description with the main implementation differences relative to #24897. It seems a big one is that this doesn’t require an index!

updated! I added the summary in #27827 and added links back to the parent PR in each of the child PRs.

in src/wallet/spend.cpp:942 in 181e329a1d outdated

899+    assert(tx_outpoints.size() > 0);
900+    CKey scalar_ecdh_input = PrepareScalarECDHInput(input_private_keys, tx_outpoints);
901+    std::vector<SilentPaymentRecipient> silent_payment_recipients = GroupSilentPaymentAddresses(silent_payment_destinations);
902+    std::vector<CRecipient> outputs;
903+    for (const auto& recipient : silent_payment_recipients) {
904+        Sender sender{scalar_ecdh_input, recipient};

josibake commented at 10:29 am on August 8, 2023:

This does not need to be a struct, it can just be a function.

in test/functional/wallet_silentpayments_sending.py:113 in e6f7458324 outdated

86+            address=SILENT_PAYMENT_ADDRESS,
87+            amount=21,
88+        )
89+        assert txid
90+
91+    def test_deterministic_send(self):

BrandonOdiwuor commented at 5:32 pm on August 9, 2023:

Incorporating logs within the test is crucial for offering transparent insight into the test’s progression, simplifying the identification of problems, and enhancing comprehension of the test’s overall behavior.

in test/functional/wallet_silentpayments_sending.py:160 in e6f7458324 outdated

122+                assert output["scriptPubKey"]["address"] == "bcrt1p5num3dvry0ffusg3s7v2j5fy025p95jtn6js65jlruwc69r9je2s6qfsj7"
123+                break
124+        else:
125+            assert False
126+
127+    def test_address_reuse(self):

BrandonOdiwuor commented at 5:34 pm on August 9, 2023:

Also consider adding logs to this test as the ones above

josibake force-pushed on Aug 30, 2023

josibake force-pushed on Aug 31, 2023

josibake force-pushed on Sep 8, 2023

josibake force-pushed on Sep 11, 2023

DrahtBot added the label CI failed on Sep 11, 2023

josibake force-pushed on Sep 11, 2023

josibake force-pushed on Sep 12, 2023

josibake force-pushed on Sep 14, 2023

DrahtBot removed the label CI failed on Sep 14, 2023

DrahtBot added the label Needs rebase on Sep 19, 2023

josibake force-pushed on Sep 21, 2023

DrahtBot added the label CI failed on Sep 21, 2023

DrahtBot removed the label Needs rebase on Sep 21, 2023

DrahtBot removed the label CI failed on Sep 21, 2023

josibake commented at 4:30 pm on September 26, 2023: member

Note: send does not work, but sendall, sendtoaddress does

josibake force-pushed on Oct 2, 2023

DrahtBot added the label CI failed on Oct 2, 2023

josibake force-pushed on Oct 3, 2023

DrahtBot removed the label CI failed on Oct 4, 2023

DrahtBot added the label Needs rebase on Oct 16, 2023

josibake force-pushed on Jan 15, 2024

DrahtBot removed the label Needs rebase on Jan 15, 2024

DrahtBot commented at 9:36 pm on January 17, 2024: contributor

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/20505330863

DrahtBot added the label CI failed on Jan 17, 2024

josibake force-pushed on Jan 19, 2024

josibake commented at 2:30 pm on January 19, 2024: member

Rebased https://github.com/bitcoin/bitcoin/commit/3f4b0f77c7f2ff52b9c84bf1c5a70281a066cfe2 -> https://github.com/bitcoin/bitcoin/commit/3b0dd79cacaf4e97489746ace5c1c8d6fe8db38d (implement-bip352-sending-01 -> implement-bip352-01-rebase, compare)

Fixed silent merge conflict with CKey now returning std::byte*

DrahtBot removed the label CI failed on Jan 19, 2024

willcl-ark added the label Wallet on Jan 24, 2024

willcl-ark added the label Privacy on Jan 24, 2024

DrahtBot added the label Needs rebase on Jan 26, 2024

josibake force-pushed on Jan 26, 2024

DrahtBot removed the label Needs rebase on Jan 26, 2024

DrahtBot added the label CI failed on Feb 2, 2024

DrahtBot removed the label CI failed on Feb 7, 2024

DrahtBot added the label Needs rebase on Feb 20, 2024

josibake force-pushed on Apr 22, 2024

DrahtBot removed the label Needs rebase on Apr 22, 2024

DrahtBot added the label CI failed on Apr 22, 2024

DrahtBot commented at 9:52 pm on April 22, 2024: contributor

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/24114299414

josibake force-pushed on Apr 27, 2024

josibake force-pushed on May 5, 2024

DrahtBot removed the label CI failed on May 5, 2024

DrahtBot added the label Needs rebase on May 20, 2024

achow101 referenced this in commit f0745d028e on Jun 27, 2024

josibake force-pushed on Jul 15, 2024

DrahtBot removed the label Needs rebase on Jul 15, 2024

josibake force-pushed on Jul 23, 2024

DrahtBot added the label CI failed on Jul 23, 2024

DrahtBot commented at 10:45 am on July 23, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/27794809832

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

DrahtBot added the label Needs rebase on Aug 2, 2024

ryanofsky referenced this in commit b38fb19b7e on Aug 7, 2024

Squashed 'src/secp256k1/' changes from 0cdc758a56..324ba7d9cc

324ba7d9cc docs: update README
1719790169 ci: enable silentpayments module
f9aa16e715 tests: add constant time tests
094fe7bfa4 tests: add BIP-352 test vectors
bfa4e18a48 silentpayments: add benchmarks for scanning
36f818b993 silentpayments: add examples/silentpayments.c
793a1a42c0 silentpayments: receiving
3aaaa5a135 silentpayments: recipient label support
eb2e69584d silentpayments: sending
8f69297215 build: add skeleton for new silentpayments (BIP352) module
70f149b9a1 Merge bitcoin-core/secp256k1#1662: bench: add ellswift to bench help output
6b3fe51fb6 bench: add ellswift to bench help output
d84bb83e26 Merge bitcoin-core/secp256k1#1661: configure: Show exhaustive tests in summary
3f54ed8c1b Merge bitcoin-core/secp256k1#1659: include: remove WARN_UNUSED_RESULT for functions always returning 1
20b05c9d3f configure: Show exhaustive tests in summary
e56716a3bc Merge bitcoin-core/secp256k1#1660: ci: Fix exiting from ci.sh on error
d87c3bc58f ci: Fix exiting from ci.sh on error
1b6e081538 include: remove WARN_UNUSED_RESULT for functions always returning 1
2abb35b034 Merge bitcoin-core/secp256k1#1657: tests: remove unused uncounting_illegal_callback_fn
51907fa918 tests: remove unused uncounting_illegal_callback_fn
a7a5117144 Merge bitcoin-core/secp256k1#1359: Fix symbol visibility issues, add test for it
13ed6f65dc Merge bitcoin-core/secp256k1#1593: Remove deprecated `_ec_privkey_{negate,tweak_add,tweak_mul}` aliases from API
d1478763a5 build: Drop no longer needed  `-fvisibility=hidden` compiler option
8ed1d83d92 ci: Run `tools/symbol-check.py`
41d32ab2de test: Add `tools/symbol-check.py`
88548058b3 Introduce `SECP256K1_LOCAL_VAR` macro
03bbe8c615 Merge bitcoin-core/secp256k1#1655: gha: Print all *.log files, in a separate action
59860bcc24 gha: Print all *.log files, in a separate action
4ba1ba2af9 Merge bitcoin-core/secp256k1#1647: cmake: Adjust diagnostic flags for `clang-cl`
abd25054a1 Merge bitcoin-core/secp256k1#1656: musig: Fix clearing of pubnonces
961ec25a83 musig: Fix clearing of pubnonces
3186082387 Merge bitcoin-core/secp256k1#1614: Add _ge_set_all_gej and use it in musig for own public nonces
6c2a39dafb Merge bitcoin-core/secp256k1#1639: Make static context const
37d2c60bec Remove deprecated _ec_privkey_{negate,tweak_add,tweak_mul} aliases
432ac57705 Make static context const
1b1fc09341 Merge bitcoin-core/secp256k1#1642: Verify `compressed` argument in `secp256k1_eckey_pubkey_serialize`
c0d9480fbb Merge bitcoin-core/secp256k1#1654: use `EXIT_` constants over magic numbers for indicating program execution status
13d389629a CONTRIBUTING: mention that `EXIT_` codes should be used
c855581728 test, bench, precompute_ecmult: use `EXIT_...` constants for `main` return values
965393fcea examples: use `EXIT_...` constants for `main` return values
2e3bf13653 Merge bitcoin-core/secp256k1#1646: README: add instructions for verifying GPG signatures
b682dbcf84 README: add instructions for verifying GPG signatures
00774d0723 Merge bitcoin-core/secp256k1#1650: schnorrsig: clear out masked secret key in BIP-340 nonce function
a82287fb85 schnorrsig: clear out masked secret key in BIP-340 nonce function
4c50d73dd9 ci: Add new "Windows (clang-cl)" job
84c0bd1f72 cmake: Adjust diagnostic flags for clang-cl
f79f46c703 Merge bitcoin-core/secp256k1#1641: doc: Improve cmake instructions in README
2ac9f558c4 doc: Improve cmake instructions in README
1823594761 Verify `compressed` argument in `secp256k1_eckey_pubkey_serialize`
8deef00b33 Merge bitcoin-core/secp256k1#1634: Fix some misspellings
39705450eb Fix some misspellings
ec329c2501 Merge bitcoin-core/secp256k1#1633: release cleanup: bump version after 0.6.0
c97059f594 release cleanup: bump version after 0.6.0
64228a648f musig: Use _ge_set_all_gej for own public nonces
300aab1c05 tests: Improve _ge_set_all_gej(_var) tests
365f274ce3 group: Simplify secp256k1_ge_set_all_gej
d3082ddead group: Add constant-time secp256k1_ge_set_all_gej

git-subtree-dir: src/secp256k1
git-subtree-split: 324ba7d9cc2f33a3be89e6d7c733c776f43382e2

f6cbfce0e2

Merge commit 'f6cbfce0e2dd848d1dc1620a881718b189af3ceb' into refresh-secp256k1 2d9bc5363b

crypto: add read-only method to KeyPair

Add a method for passing a KeyPair object to secp256k1 functions expecting a secp256k1_keypair.
This allows for passing a KeyPair directly to a secp256k1 function without needing to create a
temporary secp256k1_keypair object.

b97c020047

Add "sp" HRP c6bfa1d96d

Add V0SilentPaymentDestination address type 37a77100d3

josibake force-pushed on Apr 4, 2025

DrahtBot removed the label Needs rebase on Apr 4, 2025

josibake force-pushed on Apr 4, 2025

common: add bip352.{h,cpp} secp256k1 module

Wrap the silentpayments module from libsecp256k1. This is placed in
common as it is intended to be used by:

  * RPCs: for parsing addresses
  * Wallet: for sending, receiving, spending silent payment outputs
  * Node: for creating silent payment indexes for light clients

a72cac6ee1

wallet: disable sending to silent payment address

Have `IsValidDestination` return false for silent payment destinations
and set an error string when decoding a silent payment address.

This prevents anyone from sending to a silent payment address before
sending is implemented in the wallet, but also allows the functions to
be used in the unit testing famework.

1a16a7832b

tests: add BIP352 test vectors as unit tests

Use the test vectors to test sending and receiving. A few cases are not
covered here, namely anything that requires testing specific to the
wallet. For example:

* Taproot script path spending is not tested, as that is better tested in
  a wallets coin selection / signing logic
* Re-computing outputs during RBF is not tested, as that is better
  tested in a wallets RBF logic

The unit tests are written in such a way that adding new test cases is
as easy as updating the JSON file

9d78a2ce4f

build: link secp256k1 header in wallet

this feels a bit off, perhaps there is a cleaner way to do this?

27b2aff05f

wallet: get serialized size for `V0SilentPayments`

BIP352 v0 specifies that a silent payment output is a taproot output.
Taproot scriptPubKeys are a fixed size, so when calculating the
serialized size for a CRecipient with a V0SilentPayments destination,
use WitnessV1Taproot for the serialized txout size.

af8eff184e

wallet: add method for retreiving a private key

Add a method for retreiving a private key for a given scriptPubKey.
If the scriptPubKey is a taproot output, tweak the private key with the
merkle root or hash of the public key, if applicable.

cec6597e56

wallet: make coin selection silent payment aware

Add a flag to the `CoinControl` object if silent payment destinations
are provided. Before adding the flag, call a function which checks if:

* The wallet has private keys
* The wallet is unlocked

Without both of the above being true, we cannot send to a silent payment
address.

During coin selection, if this flag is set, skip taproot inputs when
script spend data is available. This is based on the assumption that if
a user provides script spend data, they don't have access to the key
path spend. As future improvement, we could instead check to see if we
have access to the key path spend, and only exclude the output when we
don't regardless of whether or not the user provides script spend data.

Also skip UTXOs of type `WITNESS_UNKNOWN`, although it is very unlikely
our wallet would ever try to spend a witness unknown output.

1353a4a592

wallet: add `IsInputForSharedSecretDerivation` function b2a1da584c

wallet: add `CreateSilentPaymentOutputs` function

`CreateSilentPaymentsOutputs` gets the correct private keys, adds them
together, groups the silent payment destinations and then generates the
taproot script pubkeys. These are then passed back to
CreateTransactionInternal, which uses these scriptPubKeys to update
vecSend before adding them to the transaction outputs.

2fb11646ba

wallet: update TransactionChangeType

If sending to a silent payment destination, the change type should be taproot

029746f944

wallet: enable sending to silent payment address d99022c451

tests: add sending functional tests 88c8d34539

DrahtBot removed the label CI failed on Apr 4, 2025

josibake force-pushed on Apr 7, 2025

DrahtBot added the label CI failed on Apr 27, 2025

Silent Payments: sending #28201

Pre-work / Refactors

Sending

Silent Payments logic

Final steps

Code Coverage & Benchmarks

Reviews

Conflicts