silentpayments: extending the sending API for verification of created outputs (using DLEQ proofs, BIP-374) #1925

Problem statement: The current silentpayments sending API consists of a single function that creates transaction (x-only public key) outputs corresponding to the recipient list, without providing a possibility to verify whether these outputs have been derived correctly. This is fine for scenarios where both transaction creation and broadcasting happens on a single device (e.g. hot wallets), but is insufficient if the transaction broadcasting happens on a different machine that doesn't have access to the secret keys (e.g. HW wallets); in this case, there is the possibility that the "SP output creation" device has derived those incorrectly (either by malicious intent or e.g. a bug), which could ultimately lead to a loss of money if it remains undetected before broadcasting. We can verify correct SP output creation without the need of secret key access by using DLEQ proofs, see also the motivation section in BIP-374 and a explaining blog post by benma from BitBox.

This issue is opened for the purpose of figuring out on how to best integrate this functionality w.r.t. API design, continuing the recent discussion started in #1802 (comment). Prior work in this direction has been done in PR #1651, where two new SP functions _sender_create_outputs_with_proof and _verify_proof are added (IIUC the outputs re-creation part is missing there). PR #1802 provides a generic DLEQ module, which might still be useful and could go in first.

Designing the API to be compatible with the strongly related BIP-375 makes a lot of sense; I don't think it's strictly necessary to fully support it at this point, the "per-input ECDH share creation/aggregation" parts seems less important than the one for the global share/proof (which solves the issue described above and already used as of today, e.g. in BitBox02), though it's reasonable to keep it in mind for API design already. Curious to hear opinions on that.

<details>

<summary>Quoting also the proposed API design by [@macgyver13](/bitcoin-core-secp256k1/contributor/macgyver13/) here, as I think it could serve as a good starting point:</summary>

The shape follows from BIP375 having two forms of share/proof, a global one over every eligible input key and a per-input one, plus a verifier holding no key for the inputs it checks and a combiner holding none at all. Listed below in order of how much key material the caller holds, starting with @stratospher's existing function for context:

/* Signer holding EVERY eligible input key, on one device:
 * derive the outputs and the global share/proof together, sharing the single ECDH
 * per unique scan key that both steps need. This is [#1651](/bitcoin-core-secp256k1/1651/)'s existing function,
 * shown with master's parameter naming. Would fill PSBT_GLOBAL_SP_ECDH_SHARE / _DLEQ,
 * from dleq_data, subject to the question below the block. */
int secp256k1_silentpayments_sender_create_outputs_with_proof(
    const secp256k1_context *ctx,
    secp256k1_xonly_pubkey **generated_outputs,
    secp256k1_silentpayments_dleq_data **dleq_data,
    size_t *n_dleq_size,
    const secp256k1_silentpayments_recipient **recipients,
    size_t n_recipients,
    const unsigned char *outpoint_smallest36,
    const secp256k1_keypair * const *keypairs,
    size_t n_keypairs,
    const unsigned char * const *seckeys,
    size_t n_seckeys
);

/* Signer creates the ECDH share and DLEQ proof for ONE scan key. Derives no outputs,
 * so a caller holding only some of the eligible input keys can still contribute its
 * share to a transaction it cannot complete alone.
 *
 * The share is computed over the sum of whatever keys are passed, and that choice
 * decides which PSBT field the result belongs in:
 *   all eligible input keys -> global share  -> PSBT_GLOBAL_SP_ECDH_SHARE / _DLEQ
 *   one input's key         -> that input's  -> PSBT_IN_SP_ECDH_SHARE / _DLEQ
 *
 * Which mode to use is the caller's decision: BIP375 allows a signer holding every key
 * to still choose per-input ("or does not want to create a global ECDH share").
 *
 * Call once per scan key in global mode; once per (input, scan key) pair in per-input
 * mode. Keys passed via keypairs are treated as taproot inputs and even-Y negated
 * before summing as in _sender_create_outputs. */
int secp256k1_silentpayments_sender_create_share_and_proof(
    const secp256k1_context *ctx,
    unsigned char *share33,
    unsigned char *proof64,
    const secp256k1_pubkey *recipient_scan_pubkey,
    const unsigned char *aux_rand32,
    const secp256k1_keypair * const *keypairs,
    size_t n_keypairs,
    const unsigned char * const *seckeys,
    size_t n_seckeys
);

/* Signer verifying shares it did NOT create, holding no key for those inputs:
 * BIP375 assigns this to the party that verifies proofs "for all inputs it does
 * not have the private keys for".
 * One pubkey for the per-input case, all of them for the global case. */
int secp256k1_silentpayments_verify_share_proof(
    const secp256k1_context *ctx,
    const unsigned char *share33,
    const unsigned char *proof64,
    const secp256k1_pubkey *recipient_scan_pubkey,
    const secp256k1_xonly_pubkey * const *xonly_pubkeys,
    size_t n_xonly_pubkeys,
    const secp256k1_pubkey * const *pubkeys,
    size_t n_pubkeys
);

/* Transaction Extractor, holding NO secret keys at all:
 * compute output scripts from shares supplied by others. This is the case
 * _sender_create_outputs_with_proof cannot cover, since it requires the seckeys
 * in order to derive the outputs itself.
 * Takes one already-summed share per scan key, keyed by share_scan_pubkeys, so a
 * caller holding per-input shares combines them first.
 * Input pubkeys for every eligible input are needed, since input_hash commits
 * to their sum. */
int secp256k1_silentpayments_sender_create_outputs_from_shares(
    const secp256k1_context *ctx,
    secp256k1_xonly_pubkey **generated_outputs,
    const secp256k1_silentpayments_recipient **recipients,
    size_t n_recipients,
    const unsigned char *outpoint_smallest36,
    const secp256k1_pubkey * const *share_scan_pubkeys,
    const unsigned char * const *shares33,
    size_t n_shares,
    const secp256k1_xonly_pubkey * const *xonly_pubkeys,
    size_t n_xonly_pubkeys,
    const secp256k1_pubkey * const *pubkeys,
    size_t n_pubkeys
);

A sender holding every key should keep using the first, unchanged in shape, since deriving outputs and proofs together shares the one ECDH per unique scan key that both need. The other three are illustrative. The last one keys each share by its scan pubkey, matching how BIP375 keys the share fields themselves.

</details>

I have a few thoughts on that already (e.g. on simplification by passing prevouts_summary objects instead of public key objects on the verification side, and maybe unifying the "verify proofs" and "re-create outputs" in a single API call), which I will try to structure and write down here within the next days.

Quickly dropping an alternative API idea here, tailored for the "host verifies SP outputs created by hardware wallet" use-case described above; I think we could achieve this with only two new functions:

typedef struct secp256k1_silentpayments_proof_entry {
    unsigned char scan_pubkey[33]; /* in BIP-375: PSBT_GLOBAL_SP_{ECDH_SHARE,DLEQ} <keydata> */
    unsigned char ecdh_share[33];  /* in BIP-375: PSBT_GLOBAL_SP_ECDH_SHARE <valuedata> */
    unsigned char dleq_proof[64];  /* in BIP-375: PSBT_GLOBAL_SP_DLEQ <valuedata> */
} secp256k1_silentpayments_proof_entry;

int secp256k1_silentpayments_sender_create_outputs_with_proofs(
    const secp256k1_context *ctx,
    secp256k1_xonly_pubkey **generated_outputs,
    secp256k1_silentpayments_proof_entry **generated_proof_entries,
    size_t *n_proof_entries,
    const secp256k1_silentpayments_recipient **recipients,
    size_t n_recipients,
    const unsigned char *outpoint_smallest36,
    const secp256k1_keypair * const *keypairs,
    size_t n_keypairs,
    const unsigned char * const *seckeys,
    size_t n_seckeys
);
/* TODO: could allow user to pass in auxrand32 for DLEQ proof creation (as per BIP-374) */

int secp256k1_silentpayments_sender_verify_created_outputs_with_proofs(
    const secp256k1_context *ctx,
    const secp256k1_xonly_pubkey * const *created_outputs,
    const secp256k1_silentpayments_proof_entry * const *created_proof_entries,
    size_t n_proof_entries,
    const secp256k1_silentpayments_recipient **recipients,
    size_t n_recipients,
    const secp256k1_silentpayments_prevouts_summary *prevouts_summary
);
/* TODO: instead of prevouts_summary, could still pass in the smallest outpoint and the 
         two (x-only) public key lists (more parameters, but symmetric to the function above
         and only one function call needed on the host side) */

That would make it quite easy to use I think. The drawback of the verify function is that it does many things under the hood (DLEQ verification, outputs re-creation, comparing expected and re-created outputs) and the result is only binary. This could be very annoying at least during development, if something fails and one has no idea what exact step went wrong.

Continuing the discussion from #1802 and replying to the comment #1802 (comment):

One question came out of reading #1651 to work out how the above would fit alongside it. In sender_create_outputs_with_proof, seckey_sum_scalar is multiplied by input_hash before create_shared_secret_with_proof is called, so the value that ends up as the DLEQ secret is input_hash·a_n rather than a_n. The proof is then a statement about input_hash·a_n, and the stored shared_secret is input_hash·a_n·B_scan.

If I'm reading BIP375 correctly, the value it wants in PSBT_GLOBAL_SP_ECDH_SHARE is a_n·B_scan, with the proof over a_n and input_hash applied afterwards during output computation, so a proof made the other way wouldn't verify against A_n. Am I missing a reason it's folded in earlier? I can see why it happens in master's sender_create_outputs, where the comment at main_impl.h:277 notes that multiplying the scalars first saves an elliptic curve multiplication and nothing observable depends on the ordering, so I assume it carried over naturally once proofs were added.

Good observation. Yes, on master the reason for preferring to calculate the ECDH shared secret (as defined in BIP-352) in the order $(inputhash \cdot a_{sum}) \cdot B_{scan}$ over $inputhash \cdot (a_{sum} \ \cdot B_{scan})$ is performance; the former needs only one EC multiplication instead of two, and the multiplication of the two scalars is very cheap. For creating the (global) BIP-375 ECDH shares and proofs, we need the bare $a_{sum} \cdot B_{scan}$ (without the input hash) result though and thus calculation using the first order unfortunately doesn't work anymore, as you correctly pointed out.

#1651 deviates from that indeed and is not BIP-375 compliant for that reason, I guess following it was not a a particular goal at that time, I'm not even sure if the BIP was in its final form when the work on that PR started (must have been end of 2024-ish, IIRC, cc @stratospher). One very simple but slightly hacky way to correct that without having to change too much in the current sending code would be to multiply the non-compliant ECDH result with the modular inverse of $inputhash$ after (should still be fine performance-wise, variable-time inversion is relatively cheap, ~1us on my machine, and we would have to do two point multiplications anyways) only for the _create_outputs_with_proofs variant before creating the proofs.

@theStack once you've had a chance to look at these signatures we can work out who's best placed to integrate them into the silent payments module. I can see a future where both the silent payments functions and the DLEQ primitives are useful.

As for you suggested API, I think one of the main questions we should answer now is we want to have a general API for BIP-375 already now (offering more flexibility, having individual proof verification and output creation) or want to focus on the main use-case first (hardware wallet output verification scenario), keeping it as simple as possible for the user. I came up with a two-function API for the latter (see suggestion above) follows the "hard to misuse" philosophy the library tries to follow better, but I'm not so sure anymore. Maybe in this case there is just a bit too much happening within in a single function, and splitting it up into "proof verification" and "output creation" (where a user would compare outputs manually after) would indeed be preferable.

Some more questions up for discussion:

• should we pass a prevouts_summary object for the _create_outputs_with_proofs function instead of the public key lists? On one hand this reduces the number of parameters, on the other hand the user now has to call two functions, and the API function naming convention would become a bit weird (the sender side would need to call a _silentpayments_recipient... function...)
• do we need opaque objects for proof data or is it fine to output them serialized them right away (already in BIP-375 compliant format), since generation and verification would happen on different machines anyways? (see struct in my suggestion above) that would make things quite a bit simpler, with no extra serialize/parse functions needed

If you meant sender_verify_created_outputs_with_proofs, then I agree that passing a prevouts_summary object cleans up that interface. The recipient_ prefix is awkward in that workflow because the summary itself is not recipient-specific. Would it be possible to use a role-neutral name such as secp256k1_silentpayments_prevouts_summary_create at this stage?

Returning proof_entries as described would be useful to consumers. A hardware signer can generate the entries, derive and validate all outputs, and only then insert the shares, proofs, and scripts into the PSBT. Without the returned entries, a consumer that wants this transactional behavior would need its own staging structure or would have to clone the PSBT.