Add BIP352 silentpayments module #1519

pull josibake wants to merge 9 commits into bitcoin-core:master from josibake:bip352-silentpayments-module changing 22 files +9602 −12
  1. josibake commented at 2:21 pm on April 19, 2024: member

    This PR adds a new Silent Payments (BIP352) module to secp256k1. It is a continuation of the work started in #1471.

    The module implements the full protocol, except for transaction input filtering and silent payment address encoding / decoding as those will be the responsibility of the wallet software. It is organized with functions for sending (prefixed with _sender) and receiving (prefixed by _recipient).

    For sending

    1. Collect private keys into two lists: taproot_seckeys and plain_seckeys Two lists are used since the taproot_seckeys may need negation. taproot_seckeys are passed as keypairs to avoid the function needing to compute the public key to determine parity. plain_seckeys are passed as just secret keys
    2. Create the _silentpayment_recipient objects These structs hold the scan and spend public key and an index for remembering the original ordering. It is expected that a caller will start with a list of silent payment addresses (with the desired amounts), convert these into an array of recipients and then match the generated outputs back to the original silent payment addresses. The index is used to return the generated outputs in the original order
    3. Call silentpayments_sender_create_outputs to generate the xonly public keys for the recipients This function can be called with one or more recipients. The same recipient may be repeated to generate multiple outputs for the same recipient

    For scanning

    1. Collect the public keys into two lists taproot_pubkeys and plain_pubeys This avoids the caller needing to convert taproot public keys into compressed public keys (and vice versa)
    2. Compute the input data needed, i.e. sum the public keys and compute the input_hash This is done as a separate step to allow the caller to reuse this output if scanning for multiple scan keys. It also allows a caller to use this function for aggregating the transaction inputs and storing them in an index to vend to light clients later (or for faster rescans when recovering a wallet)
    3. Call silentpayments_recipient_scan_outputs to scan the transaction outputs and return the tweak data (and optionally label information) needed for spending later

    In addition, a few utility functions for labels are provided for the recipient for creating a label tweak and tweaked spend public key for their address. Finally, two functions are exposed in the API for supporting light clients, _recipient_created_shared_secret and _recipient_create_output_pubkey. These functions enable incremental scanning for scenarios where the caller does not have access to the transaction outputs:

    1. Calculating a shared secret This is done as a separate step to allow the caller to reuse the shared secret result when creating outputs and avoid needing to do a costly ECDH every time they need to check for an additional output
    2. Generate an output (with k = 0)
    3. Check if the output exists in the UTXO set (using their preferred light client protocol)
    4. If the output exists, proceed by generating a new output from the shared secret with k++

    See examples/silentpayments.c for a demonstration of how the API is expected to be used.

    Note for reviewers

    My immediate goal is to get feedback on the API so that I can pull this module into https://github.com/bitcoin/bitcoin/pull/28122 (silent payments in the bitcoin core wallet). That unblocks from finishing the bitcoin core PRs while work continues on this module.

    Notable differences between this PR and the previous version

    See #1427 and #1471 for discussions on the API design. This iteration of the module attempts to be much more high level and incorporate the feedback from #1471. I also added a secp256k1_silentpayments_public_data opaque data type, which contains the summed public key and the input_hash. My motivation here was:

    1. I caught myself mixing up the order of arguments between A_sum and recipient_spend_key, which was impossible to catch with ARG_CHECKS and would result in the scanning process finishing without errors, but not finding any outputs
    2. Combining public key and input_hash into the same data type allows for completely hiding input_hash from the caller, which makes for an overall simpler API IMO

    I also removed the need for the recipient to generate a shared secret before using the secp256k1_silentpayments_recipient_scan_outputs function and instead create the shared secret inside the function.

    Outstanding work

    • clean up the testing code
    • improve test coverage (currently only using the BIP352 test vectors)
    • optimize the implementation, where possible
  2. real-or-random added the label feature on Apr 19, 2024
  3. in src/modules/silentpayments/main_impl.h:497 in f113564298 outdated
    492+         * return without labels calculation (one of the two would result in point of infinity) */
    493+        secp256k1_xonly_pubkey_save(&P_output_xonly, &P_output_ge);
    494+        found = 0;
    495+        for (i = 0; i < n_tx_outputs; i++) {
    496+            if (secp256k1_xonly_pubkey_cmp(ctx, &P_output_xonly, tx_outputs[i]) == 0) {
    497+                found_outputs[n_found]->output = *tx_outputs[i];
    


    theStack commented at 3:21 pm on April 22, 2024:

    The found_with_label boolean needs to be set here, to not leave it uninitialized (I guess we don’t want to rely on the user zeroing the _found_output struct instances before scanning):

    0                found_outputs[n_found]->output = *tx_outputs[i];
    1                found_outputs[n_found]->found_with_label = 0;
    

    Should we also set label here to something invalid for consistency, e.g. all-zero-bytes, even though the user wouldn’t evaluate it anyways? Not sure what the best practices are in cases like this and if returning uninitialized data is acceptable.

    Might be nice to update the tests to also check for the found_with_label flag.


    josibake commented at 9:46 am on April 24, 2024:
    Updated and for now am just setting the label with the output when the output is found without a label, but would be better if we have a “canonical” invalid pubkey I can use.
  4. in include/secp256k1_silentpayments.h:243 in f113564298 outdated
    238+ *                              This number represents the number of outputs found while scanning (0 if
    239+ *                              none are found)
    240+ *  In:             tx_outputs: pointer to the tx's x-only public key outputs
    241+ *                n_tx_outputs: the number of tx_outputs being scanned
    242+ *                    scan_key: pointer to the recipient's scan key
    243+ *           public_tweak_data: pointer to the input public key sum (optionaly, with the `input_hash`
    


    theStack commented at 3:30 pm on April 22, 2024:
    0 *                 public_data: pointer to the input public key sum (optionaly, with the `input_hash`
    

    josibake commented at 9:46 am on April 24, 2024:
    Fixed.
  5. in include/secp256k1_silentpayments.h:247 in f113564298 outdated
    242+ *                    scan_key: pointer to the recipient's scan key
    243+ *           public_tweak_data: pointer to the input public key sum (optionaly, with the `input_hash`
    244+ *                              multiplied in, see `_recipient_compute_public_data`).
    245+ *      recipient_spend_pubkey: pointer to the receiver's spend pubkey
    246+ *                  input_hash: pointer to the input_hash. MUST be NULL if the input_hash is already
    247+ *                              multipled into the input public key sum (see `_recipient_compute_public_data`)
    


    theStack commented at 3:30 pm on April 22, 2024:

    (parameter doesn’t exist anymore)


    josibake commented at 9:46 am on April 24, 2024:
    Fixed.
  6. in src/modules/silentpayments/main_impl.h:361 in 4fb8716f4f outdated
    356+
    357+    /* Compute input public keys sum: A_sum = A_1 + A_2 + ... + A_n */
    358+    secp256k1_gej_set_infinity(&A_sum_gej);
    359+    for (i = 0; i < n_plain_pubkeys; i++) {
    360+        secp256k1_pubkey_load(ctx, &addend, plain_pubkeys[i]);
    361+        secp256k1_gej_add_ge(&A_sum_gej, &A_sum_gej, &addend);
    


    theStack commented at 3:39 pm on April 22, 2024:

    Not sure why I chose the constant-time point addition routines back then in #1471 (probably was inspired by _ec_pubkey_combine), but I strongly suspect that they are not needed for summing up public data:

    0        secp256k1_gej_add_ge_var(&A_sum_gej, &A_sum_gej, &addend, NULL);
    

    (also for second summing loop a few lines below)


    josibake commented at 9:45 am on April 24, 2024:
    Done.
  7. in src/modules/silentpayments/tests_impl.h:216 in 85946762a5 outdated
    211+                /* At this point, we check that the utxo exists with a light client protocol.
    212+                 * For this example, we'll just iterate through the list of pubkeys */
    213+                found = 0;
    214+                secp256k1_xonly_pubkey_serialize(CTX, xonly_print, &potential_output);
    215+                printf("what we generated :");
    216+                print_hex(xonly_print, 32);
    


    theStack commented at 3:49 pm on April 22, 2024:

    Tests are usually silent, so this should be removed (or maybe put in some verbose/debug #if block, though I’m not sure if this is considered a good idea due to increased maintenance burden):


    josibake commented at 9:45 am on April 24, 2024:
    :doh: those sneaked in from my debugging, good catch!
  8. in src/modules/silentpayments/main_impl.h:518 in f113564298 outdated
    501+                k++;
    502+                break;
    503+            }
    504+
    505+            /* If desired, also calculate label candidates */
    506+            if (label_lookup != NULL) {
    


    theStack commented at 4:01 pm on April 22, 2024:
    The handling of the first and second scan label candidate look quite similar (both their calculation and the setting of the found output / advancing the counters if the label lookup in the cache was successful), seems like quite a bit of code could be duplicated here.

    josibake commented at 9:44 am on April 24, 2024:
    Agree! I’m working on a commit now to improve this section and also minimize the conversions between pubkey -> ge > gej. Specifically, there was a comment from Llyod on the original gist that mentions being able to test equality with jacobian == affine, which could save us conversions from gej to ge. I’m going to try and incorporate that feedback and remove some of the repeated code in a follow up commit.

    josibake commented at 8:30 am on May 3, 2024:
    Coming back to this, I tried a few variations but found with the benchmark that comparing the generated output to the tx outputs as serialized bytes is marginally faster than the alternatives, so there wasn’t much opportunity for minimizing conversions. I was able to clean up this code a bit, but will likely take a second pass at it.
  9. in include/secp256k1_silentpayments.h:62 in 14ca754578 outdated
    57+ *  Returns: 1 if shared secret creation was successful. 0 if an error occured.
    58+ *   Args:                 ctx: pointer to a context object
    59+ *    Out:   generated_outputs: pointer to an array of pointers to xonly pubkeys, one per recipient.
    60+ *                              The order of outputs here matches the original ordering of the
    61+ *                              recipients array.
    62+ *     In:          recipients: pointer to an array of pointers to silent payment recipients,
    


    theStack commented at 4:10 pm on April 22, 2024:
    Considering that the array is sorted in-place, is this in the category “In/Out”?

    josibake commented at 7:46 am on April 24, 2024:
    Good question. I would argue no, since the caller is passing the array in and then never using it again. The fact that the array is sorted in place is irrelevant to the caller since they shouldn’t be reading those values again, anyways. But maybe there is a more strict definition of In/Out, that if the function can modify the parameter it is an Out param?

    jonasnick commented at 8:59 am on April 24, 2024:
    I don’t think there’s a definition for this case and I don’t remember a precedent. “In” seems fine to me if the modified argument is not supposed to be read again. The doc also mentions already that the array is modified (“The recipient array will be sorted in place”).

    josibake commented at 9:56 am on April 24, 2024:
    Going to leave it as “In” for now. I think it makes it more clear to the caller that they are supposed to read the recipients array after the function call.
  10. in src/modules/silentpayments/main_impl.h:228 in 14ca754578 outdated
    223+        int ret = secp256k1_keypair_load(ctx, &addend, &addend_point, taproot_seckeys[i]);
    224+        VERIFY_CHECK(ret);
    225+        (void)ret;
    226+        /* declassify addend_point to allow using it as a branch point (this is fine because addend_point is not a secret) */
    227+        secp256k1_declassify(ctx, &addend_point, sizeof(addend_point));
    228+        secp256k1_fe_normalize_var(&addend_point.y);
    


    theStack commented at 4:22 pm on April 22, 2024:

    Now that we get the point from a keypair instance, I think these lines are not needed anymore (see e.g. a similar secret key negation code part in the schnorrsig module):


    josibake commented at 9:40 am on April 24, 2024:
    Removed.
  11. in src/modules/silentpayments/main_impl.h:224 in 14ca754578 outdated
    219+    }
    220+    /* private keys used for taproot outputs have to be negated if they resulted in an odd point */
    221+    for (i = 0; i < n_taproot_seckeys; i++) {
    222+        secp256k1_ge addend_point;
    223+        int ret = secp256k1_keypair_load(ctx, &addend, &addend_point, taproot_seckeys[i]);
    224+        VERIFY_CHECK(ret);
    


    theStack commented at 4:24 pm on April 22, 2024:
    Should we return an error (i.e. return 0) here if the user passes in an invalid keypair?

    josibake commented at 9:40 am on April 24, 2024:
    Replaced both instances of VERIFY_CHECK in this function with return 0;.
  12. theStack commented at 4:26 pm on April 22, 2024: contributor

    Concept ACK

    Left some initial feedback, especially around the scanning routine, will do an in-depth review round soon. Didn’t look closer at the public_data type routines and the examples yet.

  13. josibake force-pushed on Apr 24, 2024
  14. josibake force-pushed on Apr 24, 2024
  15. josibake commented at 9:39 am on April 24, 2024: member

    Updated https://github.com/bitcoin-core/secp256k1/commit/8b48bf19c3c020e653734f6c9d9364e6a47a30d1 -> https://github.com/bitcoin-core/secp256k1/commit/f5585d4b93606144e76e45ad3d43a797a9afefcf (bip352-silentpayments-module-rebase -> bip352-silentpayments-module-02, compare):

    • Fix function documentation for _recipient_scan_outputs
    • Replace VERIFY_CHECK with return 0; in _sender_create_outputs
    • Remove unneeded declassify code from _sender_create_outputs
    • Change _gej_add_ge to _gej_add_var in _recipient_public_data_create
    • Fix label scanning in _recipient_scan_outputs
    • Remove unneeded prints from the tests

    For the label scanning, I looked for an example of using an invalid public key but didn’t see anything except for the invalid_pubkey_bytes in the tests. For now, if the output is found without a label, I’m setting found_with_label = 0 and saving the found output in both the output and label field. Happy to change this if there is a better suggestion for communicating an invalid public key.

    I also used secp256k1_pubkey_save instead of output = *tx_outputs, as I think this makes the code more clear.

  16. in src/modules/silentpayments/main_impl.h:18 in 97220f7b29 outdated
    14+ *  ensure the correct values of k are used when creating multiple outputs for a recipient. */
    15+static int secp256k1_silentpayments_recipient_sort_cmp(const void* pk1, const void* pk2, void *cmp_data) {
    16+    return secp256k1_ec_pubkey_cmp(
    17+        ((secp256k1_ec_pubkey_sort_cmp_data*)cmp_data)->ctx,
    18+        &(*(const secp256k1_silentpayments_recipient **)pk1)->scan_pubkey,
    19+        &(*(const secp256k1_silentpayments_recipient **)pk2)->scan_pubkey
    


    theStack commented at 1:18 am on April 28, 2024:

    pk1/pk2 are addresses for _silentpayments_recipient instances, so involving double-pointers here doesn’t seem to be necessary:

    0        &((const secp256k1_silentpayments_recipient *)pk1)->scan_pubkey,
    1        &((const secp256k1_silentpayments_recipient *)pk2)->scan_pubkey
    

    josibake commented at 8:20 am on May 3, 2024:
    I’ll admit, I don’t fully understand why the double pointers are necessary, but they are: when I removed them, my tests would pass locally but the branch would fail the CI. I noticed the double pointers are also used for the secp256k1_pubkey_sort function, so leaving for now.
  17. in src/modules/silentpayments/main_impl.h:23 in 97220f7b29 outdated
    19+        &(*(const secp256k1_silentpayments_recipient **)pk2)->scan_pubkey
    20+    );
    21+}
    22 
    23-/* TODO: implement functions for receiver side. */
    24+int secp256k1_silentpayments_recipient_sort(const secp256k1_context* ctx, const secp256k1_silentpayments_recipient **recipients, size_t n_recipients) {
    


    theStack commented at 1:18 am on April 28, 2024:
    nit: could be static, since it’s not part of the public API

    josibake commented at 8:17 am on May 3, 2024:
    Done.
  18. in src/modules/silentpayments/main_impl.h:172 in 50dcc1aedd outdated
    193+    unsigned char shared_secret[33];
    194+    secp256k1_silentpayments_recipient last_recipient;
    195+
    196+    /* Sanity check inputs. */
    197+    VERIFY_CHECK(ctx != NULL);
    198+    ARG_CHECK(recipients != NULL);
    


    theStack commented at 1:21 am on April 28, 2024:

    could also ARG_CHECK for n_recipients here:

    0    ARG_CHECK(recipients != NULL);
    1    ARG_CHECK(n_recipients >= 1);
    

    josibake commented at 8:17 am on May 3, 2024:
    Added. Out of curiosity, I noticed we have VERIFY_CHECK and ARG_CHECK, do you know when one is preferred over the other? In most cases, I see VERIFY_CHECK being used to check the ctx, but there were a few places where VERIFY_CHECK was also being used to check the arguments.

    theStack commented at 5:43 pm on May 28, 2024:

    Sorry for the late reply, missed this comment. To my understanding, the differences are: VERIFY_CHECKs are not included in production builds (i.e. only done for tests), while ARG_CHECKs are always done. The latter call an illegal callback if the check fails, but need a ctx object for that, so the ctx != NULL condition can’t be checked via ARG_CHECK (if it failed, ctx->illegal_callback would be a null pointer dereference). I don’t think there is usually a good reason to do an argument check with VERIFY_CHECK though, as users wouldn’t notice immediately if they pass something wrong. I guess you saw VERIFY_CHECKs for argument checks that are not for public API functions?

    nit: could also add ARG_CHECK(generated_outputs != NULL) here

  19. in include/secp256k1_silentpayments.h:55 in 50dcc1aedd outdated
    50+ * input_hash = hash(outpoint_smallest || (a_sum * G))
    51+ * taproot_output = B_spend + hash(a_sum * input_hash * B_scan || k) * G
    52+ *
    53+ * If necessary, the private keys are negated to enforce the right y-parity.
    54+ * For that reason, the private keys have to be passed in via two different parameter
    55+ * pairs, depending on whether they seckeys correspond to x-only outputs or not.
    


    theStack commented at 1:22 am on April 28, 2024:

    typo

    0 * pairs, depending on whether the seckeys correspond to x-only outputs or not.
    

    josibake commented at 8:16 am on May 3, 2024:
    Fixed.
  20. in include/secp256k1_silentpayments.h:57 in 50dcc1aedd outdated
    52+ *
    53+ * If necessary, the private keys are negated to enforce the right y-parity.
    54+ * For that reason, the private keys have to be passed in via two different parameter
    55+ * pairs, depending on whether they seckeys correspond to x-only outputs or not.
    56+ *
    57+ *  Returns: 1 if shared secret creation was successful. 0 if an error occured.
    


    theStack commented at 1:23 am on April 28, 2024:
    0 *  Returns: 1 if creation of outputs was successful. 0 if an error occured.
    

    josibake commented at 8:15 am on May 3, 2024:
    Fixed.
  21. in include/secp256k1_silentpayments.h:136 in aeef184995 outdated
    131+ *   In: receiver_spend_pubkey: pointer to the receiver's spend pubkey
    132+ *                       label: pointer to the the receiver's label
    133+ */
    134+SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipient_create_labelled_spend_pubkey(
    135+    const secp256k1_context *ctx,
    136+    secp256k1_pubkey *labeled_spend_pubkey,
    


    theStack commented at 1:27 am on April 28, 2024:
    nit: should use either labeled or labelled consistently (both seem to be correct english) for function name vs parameter

    josibake commented at 8:15 am on May 3, 2024:
    Used labelled throughout.
  22. in include/secp256k1_silentpayments.h:130 in aeef184995 outdated
    125+ *  The result is used by the recipient to create a Silent Payment address, consisting
    126+ *  of the serialized and concatenated scan public key and (labelled) spend public key each.
    127+ *
    128+ *  Returns: 1 if labelled spend public key creation was successful. 0 if an error occured.
    129+ *  Args:                  ctx: pointer to a context object
    130+ *  Out: l_addr_spend_pubkey33: pointer to the resulting labelled spend public key
    


    theStack commented at 1:27 am on April 28, 2024:
    the out-param description for label is missing

    josibake commented at 8:15 am on May 3, 2024:
    Added.
  23. in include/secp256k1_silentpayments.h:175 in 41cd97dea3 outdated
    170+/** Serialize a silentpayments_public_data object into a 33-byte sequence.
    171+ *
    172+ *  Returns: 1 always.
    173+ *
    174+ *  Args:       ctx: pointer to a context object.
    175+ *  Out:   output33: pointer to a 32-byte array to place the serialized key in.
    


    theStack commented at 1:30 am on April 28, 2024:
    0 *  Out:   output33: pointer to a 33-byte array to place the serialized silentpayments_public_data in.
    

    josibake commented at 8:21 am on May 3, 2024:
    Fixed.
  24. in include/secp256k1_silentpayments.h:156 in 41cd97dea3 outdated
    151+} secp256k1_silentpayments_public_data;
    152+
    153+/** Parse a 33-byte sequence into a silent_payments_public_data object.
    154+ *
    155+ *  Returns: 1 if the data was able to be parsed.
    156+ *           0 if the sequence is invalid (e.g. does not represnt a valid public key).
    


    theStack commented at 1:31 am on April 28, 2024:
    0 *           0 if the sequence is invalid (e.g. does not represent a valid public key).
    

    josibake commented at 8:12 am on May 3, 2024:
    Fixed.
  25. in src/modules/silentpayments/main_impl.h:361 in 41cd97dea3 outdated
    356+        secp256k1_pubkey_load(ctx, &addend, plain_pubkeys[i]);
    357+        secp256k1_gej_add_ge_var(&A_sum_gej, &A_sum_gej, &addend, NULL);
    358+    }
    359+    for (i = 0; i < n_xonly_pubkeys; i++) {
    360+        secp256k1_xonly_pubkey_load(ctx, &addend, xonly_pubkeys[i]);
    361+        secp256k1_gej_add_ge(&A_sum_gej, &A_sum_gej, &addend);
    


    theStack commented at 1:32 am on April 28, 2024:

    can use non-constant-time ge adding for the second loop here as well:

    0        secp256k1_gej_add_ge_var(&A_sum_gej, &A_sum_gej, &addend, NULL);
    

    josibake commented at 8:12 am on May 3, 2024:
    Fixed.
  26. in src/modules/silentpayments/main_impl.h:379 in 41cd97dea3 outdated
    374+    secp256k1_ec_pubkey_serialize(ctx, &public_data->data[1], &pubkeylen, &A_sum, SECP256K1_EC_UNCOMPRESSED);
    375+    memcpy(&public_data->data[1 + pubkeylen], input_hash_local, 32);
    376+    return 1;
    377+}
    378+
    379+int secp256k1_silentpayments_recipient_public_data_load(const secp256k1_context *ctx, secp256k1_pubkey *pubkey, unsigned char *input_hash, const secp256k1_silentpayments_public_data *public_data) {
    


    theStack commented at 1:33 am on April 28, 2024:
    nit: not used in public API, could be made static

    josibake commented at 8:12 am on May 3, 2024:
    Done.
  27. in include/secp256k1_silentpayments.h:216 in 41cd97dea3 outdated
    210+ *             n_xonly_pubkeys: the number of taproot input public keys
    211+ *               plain_pubkeys: pointer to an array of pointers to non-taproot
    212+ *                              public keys (can be NULL if no non-taproot inputs are used)
    213+ *             n_plain_pubkeys: the number of non-taproot input public keys
    214+ */
    215+SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipient_public_data_create(
    


    theStack commented at 1:35 am on April 28, 2024:
    nit: could reorder that function to be above the other _public_data API functions, since that’s the first one in the lifecycle of a _public_data instance.

    josibake commented at 8:12 am on May 3, 2024:
    Done.
  28. in include/secp256k1_silentpayments.h:144 in 41cd97dea3 outdated
    137@@ -138,6 +138,90 @@ SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipien
    138     const secp256k1_pubkey *label
    139 ) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
    140 
    141+/** Opaque data structure that holds silent payments public input data.
    142+ *
    143+ *  This structure does not contain secret data. Guaranteed to be 98 bytes in size. It can be safely
    144+ *  copied/moved. Created with `secp256k1_silentpayments_compute_public_data`. Can be serialized as
    


    theStack commented at 1:36 am on April 28, 2024:
    0 *  copied/moved. Created with `secp256k1_silentpayments_public_data_create`. Can be serialized as
    

    josibake commented at 8:12 am on May 3, 2024:
    Fixed.
  29. in include/secp256k1_silentpayments.h:266 in 016c20d95f outdated
    261+    const secp256k1_context *ctx,
    262+    secp256k1_silentpayments_found_output **found_outputs,
    263+    size_t *n_found_outputs,
    264+    const secp256k1_xonly_pubkey * const *tx_outputs,
    265+    size_t n_tx_outputs,
    266+    const unsigned char *scan_key,
    


    theStack commented at 1:39 am on April 28, 2024:

    nit, to match the spend key naming:

    0    const unsigned char *recipient_scan_seckey,
    

    josibake commented at 8:12 am on May 3, 2024:
    Changed s/receiver/recipient/ consistently. Also removed references to seckey and “private,” considering the scan key is not a private key.
  30. in src/modules/silentpayments/main_impl.h:463 in 016c20d95f outdated
    450+    /* Sanity check inputs */
    451+    VERIFY_CHECK(ctx != NULL);
    452+    ARG_CHECK(found_outputs != NULL);
    453+    ARG_CHECK(tx_outputs != NULL);
    454+    ARG_CHECK(scan_key != NULL);
    455+    ARG_CHECK(public_data != NULL);
    


    theStack commented at 1:40 am on April 28, 2024:
    could add ARG_CHECKS for n_found_outputs != NULL and n_tx_outputs >= 1 here

    josibake commented at 8:10 am on May 3, 2024:
    Added.
  31. in examples/silentpayments.c:175 in 6704738b69 outdated
    162+                    assert(ret);
    163+                    break;
    164+                } else {
    165+                    printf("Failed to create keypair\n");
    166+                    return 1;
    167+                }
    


    theStack commented at 1:44 am on April 28, 2024:
    nit: i think the else-brach can be removed (or at least, the return 1; in it); if keypair creation fails, we want to continue in the loop and try with another secret key

    josibake commented at 8:10 am on May 3, 2024:
    Done.

    real-or-random commented at 9:03 am on July 12, 2024:

    Really done?

    edit: Well, we should change the other examples and perhaps also the API docs. Creating keys in a while loop is not good practice. If you hit an invalid key, you’re most probably not very lucky, but very unlucky because your randomness is broken. But hey, yeah, let’s just yolo and try again. :)

    So I guess either is fine for now: keep the loop for consistency with the other examples, or just return 1 here, but having a loop and the else branch is certainly a smell.


    josibake commented at 12:10 pm on July 12, 2024:

    Whoops! I think I had a local commit for this but forgot to squash it in.

    Regarding best practices, creating the keys in a loop seemed excessive to me but figured I’d just copy the existing examples in case there was something I wasn’t understanding.

    Considering this is new code, seems fine to me to break from the other examples and then I can open a separate PR to clean up the examples/API docs.


    josibake commented at 2:16 pm on July 12, 2024:
    Actually fixed this time to take out the while loop and return if it fails to create the key.

  32. theStack commented at 1:45 am on April 28, 2024: contributor
    Second review round through, looks good so far! Left a bunch of nits, mostly about naming and missing ARG_CHECKS etc.
  33. josibake force-pushed on May 3, 2024
  34. josibake force-pushed on May 3, 2024
  35. josibake commented at 8:25 am on May 3, 2024: member

    Thanks for the thorough review, @theStack ! I’ve addressed your feedback, along with some other changes.


    Update https://github.com/bitcoin-core/secp256k1/commit/f5585d4b93606144e76e45ad3d43a797a9afefcf -> https://github.com/bitcoin-core/secp256k1/commit/1a3a00bd0999a89e30d5dc9f927592ead72ab7a3 (bip352-silentpayments-module-02 -> bip352-silentpayments-module-03, compare)

    • Spelling and wording cleanups, notably:
      • s/receiver/recipient/, s/labeled/labelled/
      • s/scan_seckey/scan_key/
    • Reduce duplicate code in scan_outputs
    • Add ARG_CHECKs
    • Update tests
    • Add benchmark for scan_outputs

    The sending tests now check that the generated outputs match exactly one of the possible expected output sets. Previously, the sending tests were checking that the generated outputs exist in the array of all possible outputs, but this wouldn’t catch a bug where k is not being set correctly e.g. [Ak=0, Bk=0] would (incorrectly) pass [Ak=0, Bk=1, Ak=1, Bk=0] but will now (correctly) fail [[Ak=0, Bk=1], [Ak=1, Bk=0]]

  36. josibake force-pushed on May 3, 2024
  37. josibake force-pushed on May 8, 2024
  38. in include/secp256k1_silentpayments.h:33 in ac97828119 outdated
    23@@ -24,6 +24,21 @@ extern "C" {
    24  * implementation without requiring any further elliptic-curve operations from the wallet.
    25  */
    26 
    27+/* This struct serves as an In param for passing the silent payment address data.
    28+ * The index field is for when more than one address is being sent to in a transaction. Index is
    29+ * set based on the original ordering of the addresses and used to return the generated outputs
    30+ * matching the original ordering. When more than one recipient is used the recipient array
    31+ * will be sorted in place as part of generating the outputs, but the generated outputs will be
    32+ * outputs will be returned original ordering specified by the index to ensure the caller is able
    


    theStack commented at 10:08 am on May 28, 2024:

    nit, duplicated “outputs will be”:

    0 * returned in original ordering specified by the index to ensure the caller is able
    
  39. in include/secp256k1_silentpayments.h:35 in ac97828119 outdated
    29+ * set based on the original ordering of the addresses and used to return the generated outputs
    30+ * matching the original ordering. When more than one recipient is used the recipient array
    31+ * will be sorted in place as part of generating the outputs, but the generated outputs will be
    32+ * outputs will be returned original ordering specified by the index to ensure the caller is able
    33+ * to match up the generated outputs to the correct silent payment address (e.g. to be able to
    34+ * assign the correct amounts to the correct generatetd outputs in the final transaction).
    


    theStack commented at 10:08 am on May 28, 2024:
    0 * assign the correct amounts to the correct generated outputs in the final transaction).
    
  40. in src/modules/silentpayments/main_impl.h:98 in f35ab7daa4 outdated
    92+    (void)ser_ret;
    93+
    94+    return 1;
    95+}
    96+
    97+int secp256k1_silentpayments_create_shared_secret(const secp256k1_context *ctx, unsigned char *shared_secret33, const unsigned char *secret_component, const secp256k1_pubkey *public_component, unsigned char *input_hash) {
    


    theStack commented at 5:56 pm on May 28, 2024:
    nit: can be made static, as it’s not part of the public API (I was slightly confused as I wondered why the API description isn’t added in the same commit, but the API function in a later commit calls this function rather than directly exposing it).
  41. in src/modules/silentpayments/main_impl.h:151 in a4f3583091 outdated
    146+    secp256k1_sha256_write(&hash, k_serialized, sizeof(k_serialized));
    147+    secp256k1_sha256_finalize(&hash, hash_ser);
    148+    secp256k1_scalar_set_b32(t_k_scalar, hash_ser, NULL);
    149+}
    150+
    151+int secp256k1_silentpayments_create_output_pubkey(const secp256k1_context *ctx, secp256k1_xonly_pubkey *P_output_xonly, const unsigned char *shared_secret33, const secp256k1_pubkey *recipient_spend_pubkey, unsigned int k) {
    


    theStack commented at 5:56 pm on May 28, 2024:
    nit: can also be made static
  42. in include/secp256k1_silentpayments.h:245 in d43636b96d outdated
    240+ *                              none are found)
    241+ *  In:             tx_outputs: pointer to the tx's x-only public key outputs
    242+ *                n_tx_outputs: the number of tx_outputs being scanned
    243+ *                    scan_key: pointer to the recipient's scan key
    244+ *                 public_data: pointer to the input public key sum (optionaly, with the `input_hash`
    245+ *                              multiplied in, see `_recipient_compute_public_data`).
    


    theStack commented at 6:00 pm on May 28, 2024:
    0 *                              multiplied in, see `_recipient_public_data_create`).
    
  43. in include/secp256k1_silentpayments.h:289 in b58f89117e outdated
    284+ *  Returns: 1 if shared secret creation was successful. 0 if an error occured.
    285+ *  Args:                  ctx: pointer to a context object
    286+ *  Out:       shared_secret33: pointer to the resulting 33-byte shared secret
    287+ *  In:     recipient_scan_key: pointer to the recipient's scan key
    288+ *                 public_data: pointer to the input public key sum, tweaked with the input_hash
    289+ *                              (see `_recipient_compute_public_data`)
    


    theStack commented at 6:01 pm on May 28, 2024:
    0 *                              (see `_recipient_public_data_create`)
    
  44. in src/bench.c:190 in baafed004c outdated
    182@@ -179,7 +183,7 @@ int main(int argc, char** argv) {
    183     char* valid_args[] = {"ecdsa", "verify", "ecdsa_verify", "sign", "ecdsa_sign", "ecdh", "recover",
    184                          "ecdsa_recover", "schnorrsig", "schnorrsig_verify", "schnorrsig_sign", "ec",
    185                          "keygen", "ec_keygen", "ellswift", "encode", "ellswift_encode", "decode",
    186-                         "ellswift_decode", "ellswift_keygen", "ellswift_ecdh"};
    187+                         "ellswift_decode", "ellswift_keygen", "ellswift_ecdh", "silentpayments"};
    


    theStack commented at 6:04 pm on May 28, 2024:
    nit: should also add this on the help function on top of the file
  45. in src/modules/silentpayments/main_impl.h:374 in d43636b96d outdated
    357+        secp256k1_gej_add_ge_var(&A_sum_gej, &A_sum_gej, &addend, NULL);
    358+    }
    359+    if (secp256k1_gej_is_infinity(&A_sum_gej)) {
    360+        /* TODO: do we need a special error return code for this case? */
    361+        return 0;
    362+    }
    


    theStack commented at 6:17 pm on May 28, 2024:
    I’d say the answer to this open TODO question is “yes”. If a user scans a tx where the silent payments eligible input’s public keys cancel each other out, this probably deserves its own return code to differentiate between the cases “user misuses API / passed in wrong data” and “passed in data could be legit, but we can’t continue” (as discussed before in the earlier PR, e.g. #1471 (comment)). Would be interesting to hear maintainers opinions on API design here; to my knowledge, currently there is no public API functions that returns anything different than 0 or 1. If the successful return value is still 1, having two errors codes would be e.g. 0 and 2, which seems kind of strange.
  46. theStack commented at 6:19 pm on May 28, 2024: contributor

    Went through another round. To the best of my knowledge, this PR matches the BIP352 specification and I’m close to non-cryptographer-light-ACKing it :-)

    Found some nits an one open TODO that should probably be discussed though.

  47. josibake force-pushed on May 31, 2024
  48. josibake commented at 12:45 pm on May 31, 2024: member
    CI failure seems related to not being able to install valgrind via homebrew and unrelated to my change so ignoring for now (cc @real-or-random for confirmation?).
  49. josibake force-pushed on May 31, 2024
  50. josibake commented at 1:45 pm on May 31, 2024: member

    Thanks for the review @theStack ! Sorry for the slow response, I somehow missed the notification for your review :sweat_smile:


    Update https://github.com/bitcoin-core/secp256k1/commit/bd66eaa22acf434f0134d7f93c4fb694303708c3 -> https://github.com/bitcoin-core/secp256k1/commit/2dde8f1fa13687d2bd8328f85ac412a4052b040c (bip352-silentpayments-module-05-rebase -> bip352-silentpayments-module-06, compare)

    • spelling, grammar, and fixups per @theStack ’s review
    • Added ARG_CHECKs to check for the sum of the private keys / public keys being zero

    Per #1519 (review), I agree returning 0 is not the right thing to do, but having multiple error codes also seemed gross. I think an ARG_CHECK makes sense here because if the caller passed all valid seckeys / pubkeys and then they sum to zero, in principle its the caller passing incorrect arguments. The only thing the caller can do at this point is try again with different arguments. For the sender, this would mean repeating coin selection to get a different input set, and for the recipient this would mean skipping the transaction and moving on to the next one. Also happy to change if there is a better suggestion!

  51. real-or-random commented at 2:22 pm on May 31, 2024: contributor

    CI failure seems related to not being able to install valgrind via homebrew and unrelated to my change so ignoring for now (cc @real-or-random for confirmation?).

    Indeed, see #1536

  52. real-or-random commented at 4:23 pm on May 31, 2024: contributor

    Some general notes

    On error handling in general

    Error handling is hard, and the caller usually can’t really recover from an error anyway. This is in particular true on malicious inputs: there’s no reason to try to continue dealing with the attacker, and you simply want to abort. That’s why, as a general rule, we try to avoid error paths as much as possible. This usually boils down to merging all errors into a single one, i.e., a) have just a single error “code” for all possible errors, b) and in the case of a multi-stage thing involving multiple function calls, have just a single place where errors are returned.

    Signature verification is a good example. A (signature, message, pubkey) triple is either valid or not. The caller should not care why exactly a signature fails to verify, so we don’t even want to expose this to the caller.

    However, signature verification this is also a nice example of a case in which we stretch the rules a bit. Signature verification is implemented as two-stage process: 1. Parse the public key (which can fail). 2. Check the signature (which can fail). Purely from a “safe” API point of view, this is not great because we give the user two functions and two error paths instead of one. Ideally, there could just be one verification function which also takes care of parsing (this is how it’s defined BIP340). The primary reason why we want to have a separate parsing function in this case is performance: if you check several signatures under the same key, you don’t want to parse, which involves computing the y-coordinate, every time.

    ARG_CHECK

    ARG_CHECK will call the “illegal (argument) callback”, which, by default, crashes. See the docs here: https://github.com/bitcoin-core/secp256k1/blob/1791f6fce4d4856a4ce2b1982768a4ffa23fcc0a/include/secp256k1.h#L324 The callback/crash indicates to the caller that there’s a bug in the caller’s code.

    What does this mean for this discussion?

    • Added ARG_CHECKs to check for the sum of the private keys / public keys being zero

    Per #1519 (comment), I agree returning 0 is not the right thing to do, but having multiple error codes also seemed gross. I think an ARG_CHECK makes sense here because if the caller passed all valid seckeys / pubkeys and then they sum to zero, in principle its the caller passing incorrect arguments. The only thing the caller can do at this point is try again with different arguments. For the sender, this would mean repeating coin selection to get a different input set, and for the recipient this would mean skipping the transaction and moving on to the next one. Also happy to change if there is a better suggestion!

    So let’s take a look at the two sides:

    On the sender side: The secret keys sum up to zero (sum_i a_i = 0)

    This will happen only with negligible probability for honestly generated (=random) secret keys. That is, this will in practice only happen if the caller has a bug, or the caller has been tricked into using these secret keys, e.g., if someone else has crafted malicious secret keys for the caller. Since the latter is not a crazy scenario, we should not use ARG_CHECK here.

    We can just return 0 here to indicate to the caller that we can’t continue with these function inputs. And even if there are other error cases, I don’t see a reason why the caller code should care much about why the function fails. As long as you call the function with honestly generated inputs, everything will work out. (Devs will be interested in the exact failure case when debugging the caller’s code, but I think they can figure out during debugging then. “Normal” caller code should get just a single error code.)

    On the recipient side: The public keys sum up to infinity (sum_i A_i = 0) [1]

    Again, this can only happen if the sender is malicious. But since we’re not the sender, it’s entirely possible that the sender is malicious. And then these inputs are certainly legal, they’re just not valid. (In the same sense as it’s perfectly legal to use the signature verification algorithm on an invalid signature.) So an ARG_CHECK will not be appropriate at all: a malicious sender could trigger it and crash the scanning process.

    We should also simply return 0 to indicate that this transaction is not well-formed/not eligible for SP. And again, even if there are other error cases, I don’t see a reason why the caller should care why this transaction is not eligible.

    Alternatively, we could even return 1, store infinity in the public_data, and simply make sure that scanning won’t find any payments in that case. This would avoid the error path for this function entirely. But if the caller then calls secp256k1_silentpayments_recipient_create_shared_secret, I think we’d just postpone the error to this function, and for this function, I don’t see another way than returning an error. So I’m not convinced that this is better.

    [1] We should perhaps rename “infinity” to “zero”… ;)

  53. josibake commented at 6:06 pm on May 31, 2024: member

    @real-or-random thanks for the response, this is super helpful.

    Devs will be interested in the exact failure case when debugging the caller’s code, but I think they can figure out during debugging then

    In hindsight, I think my preference for ARG_CHECK was “better error messages as to what went wrong,” but I now realize it was because I was thinking as a dev ;). Also an oversight on my part: I didn’t realize/forgot that ARG_CHECK is actually crashing the program by default. I certainly agree that we don’t want this in either failure case.

    Alternatively, we could even return 1, store infinity in the public_data, and simply make sure that scanning won’t find any payments in that case. This would avoid the error path for this function entirely. But if the caller then calls secp256k1_silentpayments_recipient_create_shared_secret, I think we’d just postpone the error to this function, and for this function, I don’t see another way than returning an error. So I’m not convinced that this is better.

    If we imagine an index + light client scenario, the public_data would be created by the index and then sent to the light client, where the light client would call secp256k1_silentpayments_recipient_create_shared_secret (and then get the error). Given this, I think it would be better to have the error path so that the index ends up not storing any data at all for the malicious crafted transaction, which saves space for the index and bandwidth for the light client.


    Thinking about this a bit more:

    That’s why, as a general rule, we try to avoid error paths as much as possible. This usually boils down to merging all errors into a single one, i.e., a) have just a single error “code” for all possible errors, b) and in the case of a multi-stage thing involving multiple function calls, have just a single place where errors are returned.

    Most of the high-level functions in our API are calling multiple lower-level functions and so far the approach has been something like:

    0if (!secp256k1_func_that_returns_0_on_error(args)) {
    1    return 0;
    2}
    3...
    4if (!secp256k1_another_func_that_returns_0_on_error(args)) {
    5    return 0;
    6}
    

    Perhaps its worth looking to consolidate and try and only return an error at the end of a multi-stage process? This would mean ignoring the return values for a lot of the lower level function calls, which initially made me feel a bit weird. But in light of your recent comment, feels like this might be the preferred approach?

    EDIT: reading your comment again, I realize “error paths” is not really talking about branches in the code and more error paths for the user.

  54. theStack commented at 0:23 am on June 1, 2024: contributor

    We should also simply return 0 to indicate that this transaction is not well-formed/not eligible for SP. And again, even if there are other error cases, I don’t see a reason why the caller should care why this transaction is not eligible.

    Makes sense. My worry was that without an explicit error-code for this corner case, some users wouldn’t even be aware of an indirect “not eligible” case and more likely interpret a return value of 0 as “only possible if there’s a logic error on our side, so let’s assert for success” (given the passed in data is public and already verified for consensus-validity). But in the end that’s more a matter of good API documentation I guess.

    An example for the “input public keys sum up to point of infinity” case ($\sum_i A_i = 0$) is now available on the Signet chain via tx d73f4a19f3973e90af6df62e735bb7b31f3d5ab8e7e26e7950651b436d093313 [1], mined in block 198023. It consists of two inputs spending P2WPKH prevouts with negated pubkeys $(x,y)$ and $(x,-y)$ (easy to verify by looking at the second item of the witness stack each, where only the first byte for encoding the sign bit differs), and one dummy P2TR output. It hopefully helps SP implementations to identify potential problems with this corner case early. As first example and proof that it triggers the discussed code path, it makes the Silent Payment Index PR #28241 crash, which asserts on a return value of 1 for _recipient_public_data_create.

    I think it would be also a good idea to add this scenario to the BIP352 test vectors, or at least a unit test in this PR?

    [1] created with the following Python script: https://github.com/theStack/bitcoin/blob/202405-contrib-bip352_input_pubkeys_cancelled/contrib/silentpayments/submit_input_pubkeys_infinity_tx.py

  55. real-or-random commented at 10:24 am on June 3, 2024: contributor

    Perhaps its worth looking to consolidate and try and only return an error at the end of a multi-stage process? This would mean ignoring the return values for a lot of the lower level function calls, which initially made me feel a bit weird. But in light of your recent comment, feels like this might be the preferred approach?

    EDIT: reading your comment again, I realize “error paths” is not really talking about branches in the code and more error paths for the user.

    Right, it’s about avoiding errors that the user would need to deal with, e.g., by more branches on the user side. So the idea is that to try to avoid complexity in the user code, perhaps at the cost of adding complexity to our code. But let me also add that this and most all of what I above is more a rule of thumb than a strict policy.

    We should also simply return 0 to indicate that this transaction is not well-formed/not eligible for SP. And again, even if there are other error cases, I don’t see a reason why the caller should care why this transaction is not eligible.

    Makes sense. My worry was that without an explicit error-code for this corner case, some users wouldn’t even be aware of an indirect “not eligible” case and more likely interpret a return value of 0 as “only possible if there’s a logic error on our side, so let’s assert for success” (given the passed in data is public and already verified for consensus-validity). But in the end that’s more a matter of good API documentation I guess.

    Okay, I see the concern. And I agree, this should probably be a matter of documentation. I think, assuming our code is correct, there won’t be a reason to return 0 unless there’s something wrong with the transaction (invalid public key, keys sum up to 0), and we should just list these reasons in the docs.

    I think it would be also a good idea to add this scenario to the BIP352 test vectors, or at least a unit test in this PR?

    Indeed, this should be in the BIP, ideally even in the pseudocode. If our code starts to reject “payments”, then better be authorized by the standard. And I believe it’s the right approach: There’s no need to add complexity to implementations to deal with these malicious cases.1


    1. One thing to keep in mind are multi-sender scenarios where blaming the malicious participant could be hard. Say honest A and B have pkA and pkB, and malicious C then claims to have pkC = -(pkA + pkC) s.t. pkA + pkB + pkC = 0. And then A and B can’t conclude that C is malicious. We run into such a thing in the MuSig BIP, see the first paragraph of https://github.com/bitcoin/bips/blob/master/bip-0327.mediawiki#dealing-with-infinity-in-nonce-aggregation. But IIUC this cannot happen here since C needs to show a signature under pkC. ↩︎

  56. theStack referenced this in commit 1ceb0b6986 on Jun 18, 2024
  57. theStack referenced this in commit bbc706cf4f on Jun 19, 2024
  58. theStack referenced this in commit a96df5abcb on Jun 19, 2024
  59. theStack referenced this in commit 3623b60784 on Jun 19, 2024
  60. theStack referenced this in commit fe0f83531e on Jun 19, 2024
  61. theStack referenced this in commit 6304666575 on Jun 19, 2024
  62. theStack referenced this in commit 59cc43d727 on Jun 21, 2024
  63. josibake force-pushed on Jun 29, 2024
  64. josibake commented at 12:01 pm on June 29, 2024: member

    Update https://github.com/bitcoin-core/secp256k1/commit/2dde8f1fa13687d2bd8328f85ac412a4052b040c -> https://github.com/bitcoin-core/secp256k1/commit/ac591050262d9d00b629943d598f62b47e1ca7ae (bip352-silentpayments-module-06 -> bip352-silentpayments-module-07, compare)

    • Removes ARG_CHECKS for the sum to zero / infinity case
    • Adds the new test vector for sum to zero / infinity
    • Minor fix ups, e.g., s/receiver/recipient
  65. josibake force-pushed on Jul 3, 2024
  66. josibake commented at 9:57 am on July 3, 2024: member

    Update https://github.com/bitcoin-core/secp256k1/commit/ac591050262d9d00b629943d598f62b47e1ca7ae -> https://github.com/bitcoin-core/secp256k1/commit/5dd552ccbb5243047e0ad967561796f15a42bfbb (bip352-silentpayments-module-07 -> bip352-silentpayments-module-08, compare)

    (note: I also included a rebase inadvertently, so I rebased the -07-rebase branch as well to make comparing the diff easier with this compare link)

    The main change in this push is adding full test coverage for the module API. To make this easier, I ended up re-organizing the commits so that everything for sending is one commit, everything for labelled address is one commit, and everything for receiving is one commit. While this does make each commit larger, I think overall this makes things easier to review. I find it especially helpful to have the API tests in the same commit as it makes it easier to reason on how the functions will be used and makes it easy to modify the tests during testing to exercise the API.

    While adding the tests, I made the following implementation changes:

    • Only have ARG_CHECKS in functions exposed in the API. This meant removing ARG_CHECKS from some functions and adding them to others
    • Accumulate errors in a return value (i.e., use ret &= func instead of if (!func) { return 0 }
      • In many cases where we were returning early, I couldn’t think of a way to trigger an error in the function since the error would have already been caught with an ARG_CHECK or when initializing a struct. In these cases, it seemed better to remove the if branch and instead accumulate the error so that the function will still fail if an error is ever encountered
    • Some stylistic changes, e.g., comments and rewriting ARG_CHECK parsing for clarity

    Running gcov after these changes shows that the test coverage for the module is ~100% :smile: image

  67. josibake commented at 9:58 am on July 3, 2024: member
    Not quite sure what is happening with the CI failures, since it looks like the all of the tests and examples are passing. Will investigate more and push a fix.
  68. josibake force-pushed on Jul 3, 2024
  69. josibake commented at 12:47 pm on July 3, 2024: member

    Update https://github.com/bitcoin-core/secp256k1/commit/5dd552ccbb5243047e0ad967561796f15a42bfbb -> https://github.com/bitcoin-core/secp256k1/commit/1136e0c6aa6884d67d984d62f480986b9824db99

    CI was failing due to the benchmark executable. In the previous push, I added an ARG_CHECK that if a label_lookup callback is passed, label_cache cannot be NULL. However, I didn’t run the benchmarks locally and missed that the benchmark was calling _scan_outputs with a label lookup callback but NULL labels cache. Fixed by passing a noop labels cache pointer. This is fine since the purpose of using the label_lookup in the benchmark isn’t to actually scan for labels but to make sure that the label lookup branch of the code gets triggered during the benchmark.

  70. theStack commented at 5:19 pm on July 5, 2024: contributor

    Looks generally good to me, I think the new structuring of the commits makes sense and reviewing is easier. One thing that should be taken care of (which I also only became fully aware of recently while reviewing the MuSig2 PR) is cleaning out secrets, as stated in CONTRIBUTING.md:

    https://github.com/bitcoin-core/secp256k1/blob/ca06e58b2ce18d170546ad13429fb9319451f48a/CONTRIBUTING.md?plain=1#L50

    I believe this should be applied at least to the following local variables (probably also to the shared secret ones):

    • tweaked_secret_component in _create_shared_secret
    • a_sum_scalar, addend, a_sum in _sender_create_outputs
  71. jlest01 commented at 5:07 am on July 6, 2024: none

    Looks good to me overall.

    Why does secp256k1_silentpayments_sender_create_outputs have const secp256k1_silentpayments_recipient **recipients as parameter instead of const secp256k1_silentpayments_recipient *recipients ? Is there a specific reason for this ? const secp256k1_silentpayments_recipient **recipients can be somewhat confusing. Why not have a pointer to the first element of an array of secp256k1_silentpayments_recipient structures ?

  72. josibake force-pushed on Jul 9, 2024
  73. josibake commented at 9:36 am on July 9, 2024: member

    Update https://github.com/bitcoin-core/secp256k1/commit/1136e0c6aa6884d67d984d62f480986b9824db99 -> https://github.com/bitcoin-core/secp256k1/commit/8ce68efb9511124877d50e40bcea1563e1384ef8 (bip352-silentpayments-module-08 -> bip352-silentpayments-module-09, compare)

    Primarily:

    • Replace secp256k1_ecdh with ecmult_const
    • Refactor create_shared_secret
    • Clear secrets at each stage

    I noticed while clearing the secrets that we were creating the shared secrets in a really inefficient way:

    • deserialize unsigned char seckeys into secp256k1_scalars and add them
    • serialize the summed scalar back to unsigned char so it can be passed to secp256k1_ecdh
    • deserialize inside secp256k1_ecdh back to a scalar :man_facepalming:, call ecmult_const
    • serialize the resulting point coordinates to unsigned char and pass them to the noop hash function
    • deserialize the unsigned char x,y pair back into a point :man_facepalming:
    • serialize the point into a compressed public key

    Instead, I opted to remove secp256k1_ecdh and just calculate the the shared secret directly with ecmult_const. This avoids a lot of back and forth with serializing and deserializing and allowed me to remove the rather complicated noop hash function. Since secp256k1_ecdh is calling ecmult_const under the hood, this should be the same. However, I did notice that _cmov is being called inside secp256k1_ecdh, which led me down a bit of a rabbit hole. I left some TODO comments in places where we deserialize the secret keys into scalars for future reviewers, because I’m not quite sure what the right approach here is (cc the friendly neighborhood cryptographers @real-or-random , @jonasnick ).

    I also refactored the create_shared_secret function to avoid needing to create an intermediate tweaked scalar, which also improves the readability of the code imo.

  74. josibake force-pushed on Jul 9, 2024
  75. josibake commented at 10:04 am on July 9, 2024: member

    Why does secp256k1_silentpayments_sender_create_outputs have const secp256k1_silentpayments_recipient **recipients as parameter instead of const secp256k1_silentpayments_recipient *recipients ? Is there a specific reason for this ?

    Thanks for the review, @jlest01 ! For the _recipient structs specifically, a pointer to an array of pointers is preferable for sorting the recipients in place. With an array of pointers each entry is 8 bytes vs with recipient structs each entry is ~136 bytes, making it more efficient to move the pointers around.

    In general, pointers to an array of pointers is preferred as it provides more flexibility for the caller. For example, if the caller is using n inputs that all have the same public key, they can initialize the secp256k1_pubkey object once and then create a pointer to it n times. This is more efficient than requiring them to initialize n structs for the same public key.

  76. jlest01 commented at 11:54 am on July 9, 2024: none
    Thanks for the clarification @josibake . Just out of curiosity. Is there a PR in bitcoin core that uses the interface being implemented here ? I would like to better understand how this interface is intended to interact with Bitcoin Core classes.
  77. in src/modules/silentpayments/tests_impl.h:458 in ef597945b2 outdated
    435+
    436+
    437+    /* prepare the inputs */
    438+    {
    439+        for (i = 0; i < test->num_plain_inputs; i++) {
    440+            CHECK(secp256k1_ec_pubkey_parse(CTX, &plain_pubkeys_objs[i], test->plain_pubkeys[i], 33));
    


    Sjors commented at 12:37 pm on July 9, 2024:
    ef597945b2632862158a9477a74c1a525e92ffe9: to sanity check the test vectors themselves, if a matching plain_seckeys[i] entry exists, derive the public key to see that it matches. I can currently change a byte in the first test vector sec and pub key without breaking a test.

    Sjors commented at 12:56 pm on July 9, 2024:
    Mmm, even if malform vectors.h by deleting the struct bip352_test_vector { line make check is fine. That take a make clean. Seems like a Makefile bug.

    Sjors commented at 1:02 pm on July 9, 2024:
    So when I modify the first sec key in the first test vector the test does fail, and test-suite.log shows it happens at line 412 - which makes sense. The suggested sanity check would help debugging though.

    josibake commented at 3:36 pm on July 9, 2024:

    Mmm, even if malform vectors.h by deleting the struct bip352_test_vector { line make check is fine.

    Sounds like make is not picking up that vectors.h changed? Will look into this.

    to sanity check the test vectors themselves, if a matching plain_seckeys[i] entry exists, derive the public key to see that it matches

    This would make the tests run slower, and I don’t really see the value. Since the values for the tests are static, seems sufficient to do a one-time manual check as part of testing vs make this run every time in the CI?


    Sjors commented at 8:22 am on July 11, 2024:

    a one-time manual check

    Maybe the Python script can do it?


    real-or-random commented at 8:35 am on July 11, 2024:

    You can add the file here: https://github.com/bitcoin-core/secp256k1/blob/0055b86780f2aa7272a1c307f6b9cd298584340f/Makefile.am#L237-L243

    Then CI will check that the files match: https://github.com/bitcoin-core/secp256k1/blob/0055b86780f2aa7272a1c307f6b9cd298584340f/ci/ci.sh#L138-L147

    Note in case you’re wondering: There’s no CMake equivalent for this. It’s our own internal stuff, so there was no need to port it to CMake.


    josibake commented at 8:24 am on July 12, 2024:
    Added.
  78. Sjors commented at 12:38 pm on July 9, 2024: member

    Tested that tests_silentpayments_generate.py indeed produces silentpayments/vectors.h with the latest vectors.

    06618bd9d3361f68ed0cdc139ec7c66365d81b6ce87c2119905225fab8ab833c9  send_and_receive_test_vectors.json
    

    Maybe move the changes in src/modules/silentpayments/tests_impl.h to their own commit? It would make it slightly easier to review changes (before merge) in the test vectors vs. changes to their usage.

    I’m having a hard time modifying any of the test vectors in a way that makes the test fail. See one suggestion inline for a sanity check you could add. See also #1568.

  79. in examples/silentpayments.c:224 in c71791c629 outdated
    196+            generated_output_ptrs[i] = &tx_outputs[i];
    197+        }
    198+        ret = secp256k1_silentpayments_sender_create_outputs(ctx,
    199+            generated_output_ptrs,
    200+            recipient_ptrs, N_TX_OUTPUTS,
    201+            smallest_outpoint,
    


    Sjors commented at 1:19 pm on July 9, 2024:

    c71791c629ec88e2e253aae4157d7113b60bee60: it might be safer to have this function take an array of all outpoints of the previous transaction (and N_TX_INPUTS) and pick the “smallest” one. An implementer who knows what they’re doing can pick it themselves and simply pass an array of size 1.

    If you keep the interface the same, then it would be good to document unsigned char smallest_outpoint above, because it’s the only non-obvious variable in this example.


    josibake commented at 9:38 am on July 11, 2024:

    I’d prefer to keep the interface the same, but agree “smallest outpoint” needs better documentation.

    If we change this to accept an array of outpoints, it doesn’t actually prevent the caller from making a mistake: they can pass an array of size 1 with the wrong outpoint, or pass a subset of outpoints that doesn’t include the smallest outpoint. It does, however, require the caller to pass extra data to the function that is then immediately discarded and makes the API a bit more complicated.

    In this case, seems better to prefer the simpler API along with good documentation on the importance of choosing this value correctly.


    Sjors commented at 7:16 am on July 12, 2024:
    Good documentation sounds fine too. IIRC the main gotcha is to not interpret the output position as an integer when doing the comparison, as most wallets probably store outpoints as {uint256, uint32} instead of 36 (?) serialized bytes.

    josibake commented at 7:43 am on July 12, 2024:
    Yep, this was an issue we ran into before. There is now a test for exactly this mistake in the test vectors.
  80. in examples/silentpayments.c:183 in c71791c629 outdated
    170+        /*** Create the recipient objects ***/
    171+
    172+        /* Alice is sending to Bob and Carol in this transaction:
    173+         *
    174+         *     1. One output to Bob's labelled address
    175+         *     2. Two outputs for Carol (1.0 and 3.0 bitcoin)
    


    Sjors commented at 1:25 pm on July 9, 2024:
    c71791c629ec88e2e253aae4157d7113b60bee60: is there a reason to make this more contrived than just a single change address? The “Note” below would be fine even if the second Carol address wasn’t used in the example.

    josibake commented at 11:19 am on July 11, 2024:

    The reason is to demonstrate how to create multiple outputs for the same user. This is mentioned in the documentation for the API but I felt this would be a good one to have an example for, e.g., to show how to set the indexes on the recipient object correctly and to demonstrate how the generated outputs are matched back up to the correct recipients.

    I think it’s better to have a more complicated example for demonstrating the API usage considering in this case, all that’s needed for a more simple use case is to just remove objects from the recipient array.


    Sjors commented at 7:23 am on July 12, 2024:

    The problem with complicated examples in general is that they distract the reader into trying to understand what the example is about, rather than focus on how to code it.

    Another approach could be:

    0sp_addresses[0] = &carol_address; /* : 1.0 BTC */
    1sp_addresses[1] = &bob_address;   /* : 2.0 BTC */
    2/*
    3 * To create multiple outputs for Carol, Alice simply passes Carol's silent payment
    4 * address again, e.g.:
    5 * 
    6 * sp_addresses[2] = &carol_address; /* : 3.0 BTC */
    7 * 
    8 */
    

    josibake commented at 7:50 am on July 12, 2024:

    The problem with complicated examples in general is that they distract the reader into trying to understand what the example is about, rather than focus on how to code it.

    I don’t think that is the case here. I’d prefer to keep the line in, considering it’s about as complicated as the comment.

  81. in examples/silentpayments.c:177 in c71791c629 outdated
    172+        /* Alice is sending to Bob and Carol in this transaction:
    173+         *
    174+         *     1. One output to Bob's labelled address
    175+         *     2. Two outputs for Carol (1.0 and 3.0 bitcoin)
    176+         *
    177+         * Alice creates the recipient objects and adds the index of the original ordering (i.e. the ordering
    


    Sjors commented at 1:28 pm on July 9, 2024:
    c71791c629ec88e2e253aae4157d7113b60bee60: maybe move “adds the index of the …” below to where you set index.
  82. in examples/silentpayments.c:192 in c71791c629 outdated
    183+         */
    184+        sp_addresses[0] = &carol_address; /* : 1.0 BTC */
    185+        sp_addresses[1] = &bob_address;   /* : 2.0 BTC */
    186+        sp_addresses[2] = &carol_address; /* : 3.0 BTC */
    187+        for (i = 0; i < N_TX_OUTPUTS; i++) {
    188+            ret = secp256k1_ec_pubkey_parse(ctx, &recipients[i].scan_pubkey, (*(sp_addresses[i]))[0], 33);
    


    Sjors commented at 1:30 pm on July 9, 2024:

    c71791c629ec88e2e253aae4157d7113b60bee60

    0/** Read the scan public key, which is the first part of a silent payment address */
    
  83. in examples/silentpayments.c:196 in c71791c629 outdated
    191+            assert(ret);
    192+            recipients[i].index = i;
    193+            recipient_ptrs[i] = &recipients[i];
    194+        }
    195+        for (i = 0; i < N_TX_OUTPUTS; i++) {
    196+            generated_output_ptrs[i] = &tx_outputs[i];
    


    Sjors commented at 1:35 pm on July 9, 2024:

    c71791c629ec88e2e253aae4157d7113b60bee60: I find myself confused about the difference between tx_outputs and generated_output_ptrs. Can you make tx_outputs an array of pointers and just use that? I might have missed something.

    A better name might be:

    0/** The x-only pubkey for each output taproot scriptPubKey. */
    1secp256k1_xonly_pubkey *tx_output_pubkey_ptrs[N_TX_OUTPUTS];
    

    josibake commented at 11:25 am on July 11, 2024:

    Good catch, I think this would be less confusing if more consistent naming were used, e.g., tx_outputs; tx_output_ptrs. Regarding the API: arrays are passed as a pointer to an array of pointers to objects. This means we need to first create the array of objects (tx_outputs), and then create the array of pointers to those objects (tx_output_ptrs).

    In the case of tx_outputs, it’s less obvious why the we want the pointer to array of pointers approach, but I think its better to keep the API consistent, i.e., if passing an array to the function, it must be a pointer to an array of pointers.

    I think this might warrant a doc/silentpayments.md explaining the usage of the API, similar to the musig2 PR.


    Sjors commented at 7:28 am on July 12, 2024:

    Agree it’s better to keep the API consistent.

    There’s no magical C way to create an array of pointers to objects without first going through an array of objects? In any case the naming change makes it clear that tx_outputs and tx_output_ptrs refer to the same “stuff”.

  84. in examples/silentpayments.c:223 in c71791c629 outdated
    195+        for (i = 0; i < N_TX_OUTPUTS; i++) {
    196+            generated_output_ptrs[i] = &tx_outputs[i];
    197+        }
    198+        ret = secp256k1_silentpayments_sender_create_outputs(ctx,
    199+            generated_output_ptrs,
    200+            recipient_ptrs, N_TX_OUTPUTS,
    


    Sjors commented at 1:48 pm on July 9, 2024:

    If you like the above rename, then this one would be:

    0/** The silent recipient for each output or null for a regular address. */
    1const secp256k1_silentpayments_recipient *tx_output_recipient_ptrs[N_TX_OUTPUTS];
    

    (the description here is more important than the name, just recipient_ptrs is probably fine)


    josibake commented at 12:41 pm on July 11, 2024:
    The array here would never contain nulls for “regular addresses,” so I don’t think the description here is accurate.

    Sjors commented at 7:29 am on July 12, 2024:
    Then what’s in the output positions that aren’t a silent payment?

    josibake commented at 11:41 am on July 14, 2024:
    You would never include non-silent payment outputs when calling this module. These functions are not for general transaction construction and only for the silent payments specific logic needed during transaction construction.

    josibake commented at 11:47 am on July 14, 2024:
    Perhaps the confusion is coming from the fact these variables are named N_TX_INPUTS and N_TX_OUTPUTS ? Considering this example is “hardcoded” to a set number of inputs and outputs, might as well just use numbers here instead of variables. I’ll update this if I end up touching the example again.
  85. in examples/silentpayments.c:137 in c71791c629 outdated
    122+    assert(ret);
    123+
    124+    /*** Sending ***/
    125+    {
    126+        secp256k1_keypair sender_seckeys[N_TX_INPUTS];
    127+        const secp256k1_keypair *sender_seckey_ptrs[N_TX_INPUTS];
    


    Sjors commented at 1:52 pm on July 9, 2024:
    0/** The private key for each silent payment eligible input, null otherwise. */
    

    josibake commented at 12:41 pm on July 11, 2024:
    Same comment as above, these arrays would never contain non-silent payment eligible data.
  86. in examples/silentpayments.c:274 in c71791c629 outdated
    231+         *     1. The transaction inputs, `tx_input_ptrs`
    232+         *     2. The transaction outputs, `tx_output_ptrs`
    233+         *
    234+         * These will be used to demonstrate scanning as a full node and scanning as a light client.
    235+         */
    236+        const secp256k1_xonly_pubkey *tx_input_ptrs[N_TX_INPUTS];
    


    Sjors commented at 2:02 pm on July 9, 2024:

    c71791c629ec88e2e253aae4157d7113b60bee60

    0/** For each Silent Payment eligible input we collect all pubkeys
    1  * found in the scriptSig or witness. For simplicity in this 
    2  * example inputs have only one pubkey.
    3  */
    4const secp256k1_xonly_pubkey *tx_input_pubkey_ptrs[N_TX_INPUTS];
    

    josibake commented at 11:28 am on July 11, 2024:
    Actually, the eligible inputs from the BIP were chosen so that there is only one public key per input. This is why OP_CHECKMULTISIG inputs are not eligible for silent payments, for example.

    Sjors commented at 7:33 am on July 12, 2024:

    Ah, nice! In that case:

    0/** For each Silent Payment eligible input we collect the pubkey
    1  * found in its scriptSig or witness. See Inputs For Shared Secret Derivation
    2  * in the BIP.
    3  */
    

    josibake commented at 8:10 am on July 12, 2024:
    This comment still feels like protocol exposition and doesn’t add any information on how to use the API, which is what the example is aimed at, so I’d prefer to keep it as is.
  87. in examples/silentpayments.c:256 in c71791c629 outdated
    251+         *     1. Collect the relevant data from the transaction inputs and call
    252+         *        `secp256k1_silentpayments_recipient_public_data_create`
    253+         *     2. Call `secp256k1_silentpayments_recipient_scan_outputs`
    254+         *
    255+         * Additionally, since Bob has access to the full transaction outputs when scanning its easy for him
    256+         * to scan with labels, as demonstrated below. For efficient scanning, Bob keeps a cache of
    


    Sjors commented at 2:08 pm on July 9, 2024:
    c71791c629ec88e2e253aae4157d7113b60bee60: the example would be easier to follow if you first demonstrate usage without labels, since they’re optional.

    josibake commented at 11:30 am on July 11, 2024:

    I considered this, but as mentioned in #1519 (review), the goal of the example is to provide guidance for the more complicated usage of the API. In this case, the goal is to give a concrete example of what the label_context is.

    Perhaps its worth adding a comment mentioning that labels are optional and using without labels is as simple as calling the function with NULL, NULL for label_lookup and label_context.


    Sjors commented at 7:38 am on July 12, 2024:

    Same reaction. I think you need to guide people through a simple usage before covering a more advanced one. Otherwise anyone just looking for the simple use case will give up confused. And most people looking for an advanced use case probably want to start with the simple one and then expand it.

    It’s already clear from the comment that labels are optional, but it’s not clear how to avoid using them (without staring for a long time). You could perhaps add comments along the lines of “// Skip this if you’re not using labels”.


    josibake commented at 8:00 am on July 12, 2024:
    I’d prefer to keep labels in the example since it shows usage of the secp256k1_silentpayments_recipient_create_label_tweak function. I added a comment near the top that makes it more clear that labels are optional and that the function can be called with NULL, NULL if not using labels. This comment is then reiterated in the scanning section with a comment at the call site of the scanning function.
  88. in examples/silentpayments.c:295 in c71791c629 outdated
    275+            /* In this contrived example, our label context needs the secp256k1 context because our lookup function
    276+             * is calling `secp256k1_ec_pubkey_cmp`. In practice, this context can be anything the lookup function needs.
    277+             */
    278+            labels_cache.ctx = ctx;
    279+
    280+            /* Load Bob's spend public key */
    


    Sjors commented at 2:10 pm on July 9, 2024:
    c71791c629ec88e2e253aae4157d7113b60bee60: maybe emphasise that we only need the scan private key

    josibake commented at 11:31 am on July 11, 2024:
    I’m not sure what you mean here? In order to scan, Bob needs his scan key and spend public key.

    Sjors commented at 7:43 am on July 12, 2024:

    Ah, it’s already loaded.

    0/* Load Bob's spend public key. We already loaded his scan (secret) key. */
    
  89. in examples/silentpayments.c:384 in c71791c629 outdated
    313+            ret = secp256k1_silentpayments_recipient_public_data_serialize(ctx, light_client_data33, &public_data);
    314+            assert(ret);
    315+
    316+            /* Scan the transaction */
    317+            n_found_outputs = 0;
    318+            ret = secp256k1_silentpayments_recipient_scan_outputs(ctx,
    


    Sjors commented at 2:14 pm on July 9, 2024:
    The documentation for secp256k1_silentpayments_recipient_scan_outputs needs to be moved below the typedef and struct. At least otherwise VSCode doesn’t find it.

    josibake commented at 11:31 am on July 11, 2024:
    Nice catch! Will fix and add documentation for the struct.

    josibake commented at 7:55 am on July 12, 2024:
    Fixed.
  90. in examples/silentpayments.c:297 in c71791c629 outdated
    260+         * labels (e.g. 0..100_000) and use that while scanning.
    261+         */
    262+        {
    263+            secp256k1_silentpayments_found_output found_outputs[N_TX_OUTPUTS];
    264+            secp256k1_silentpayments_found_output *found_output_ptrs[N_TX_OUTPUTS];
    265+            secp256k1_silentpayments_public_data public_data;
    


    Sjors commented at 2:24 pm on July 9, 2024:

    Can be demystified with something like this:

    0/** Public data is an x-only (?) public key that can be generated
    1  * for each transaction without knowledge of either the input
    2  * private keys or the recipient silent payment address. This
    3  * function performs ECDH on it and checks if any found_output_ptrs
    4  * contains the resulting x-only pubkey - indication it's a silent
    5  * payment to us.
    6  * See also light client scanning below.
    7  */
    

    Sjors commented at 2:31 pm on July 9, 2024:
    It might even make sense to move the light client example up, since by necessity it explains some of the things that happen automagically.

    real-or-random commented at 8:44 am on July 11, 2024:
    0/** Public data is an x-only (?) public key that can be generated
    1  * for each transaction without knowledge of either the input
    2  * private keys or the recipient silent payment address. This
    3  * function performs ECDH on it and checks if any found_output_ptrs
    4  * contains the resulting x-only pubkey - indication it's a silent
    5  * payment to us.
    6  * See also light client scanning below.
    7  */
    

    Demystifying is a good idea, but I think we shouldn’t mix API usage with protocol internals. (If people want to know how it works under the hood, they should just read the BIP.)

    • The public data is public data that can be generated for each transaction without knowledge of either the input private keys or the recipient silent payment address. Whether it’s a public key is not relevant for API users.
    • This checks if it’s (potential) silent payment to us. The fact that it does an ECDH key exchange is a detail of the protocol.

    edit: Of course, this is not a strict requirement but rather a rule of thumb. For example, the first few lines of the module docs in include/secp256k1_silentpayments.h mention ECDH, and I guess that’s fine as a tiny bit of background what this module does at all.


    Sjors commented at 9:26 am on July 11, 2024:

    I wrote most of this based on my own initial confusion with the example code, and I have read the BIP. Agree it shouldn’t duplicate the whole thing, just enough so people understand what the code is trying to do.

    E.g. the only reason I understood what “public data” referred to was from @josibake’s way of phrasing stuff in a podcast.


    real-or-random commented at 11:12 am on July 11, 2024:

    E.g. the only reason I understood what “public data” referred to was from @josibake’s way of phrasing stuff in a podcast.

    Perhaps we can find a better term? (Public) scanning data?

    By the way, I listened to that epsiode yesterday… :)


    josibake commented at 12:58 pm on July 11, 2024:

    I agree its better to refer people to the BIP if they are interested in the internals. For public_data specifically, I think the documentation for _public_data_create should be sufficient for explaining to the caller what a public_data object is.

    As @Sjors pointed out in a different comment, smallest_outpoint isn’t well documented, but aside from that I think referring to this function should be sufficient for the caller.


    Sjors commented at 7:49 am on July 12, 2024:
    The term “public data” doesn’t occur in the BIP. Even if you point to the section “Creating outputs” it’s not obvious. But it might be better to clarify this in the BIP instead of in the demo. It could more clearly separate the derivation that needs only public data from the tweaks with the private key.

    josibake commented at 7:56 am on July 12, 2024:
    Updated the wording and added documentation for smallest_outpoint.
  91. Sjors commented at 2:30 pm on July 9, 2024: member
    I find the send and receive example easy to follow (despite some feedback - and minus the label stuff), which suggests that this is a good interface.
  92. josibake commented at 3:58 pm on July 9, 2024: member

    Thanks for the review, @Sjors !

    Maybe move the changes in src/modules/silentpayments/tests_impl.h to their own commit?

    This is how it was structured before, but I found it much easier to work with having the tests in the same commit where functions are added to the API. This allows you to easily make changes to the API and the relevant tests in the same commit and also lets reviews step through each commit and verify that the commit compiles and passes the tests.

    Will digest your feedback on the examples/silentpayments.c and update!


    Just out of curiosity. Is there a PR in bitcoin core that uses the interface being implemented here ? @jlest01 all of the Bitcoin Core PRs can be found here: https://github.com/bitcoin/bitcoin/issues/28536

  93. josibake force-pushed on Jul 11, 2024
  94. josibake commented at 2:05 pm on July 11, 2024: member
    Rebased to fix merge conflict
  95. josibake force-pushed on Jul 11, 2024
  96. josibake commented at 2:21 pm on July 11, 2024: member

    Updated https://github.com/bitcoin-core/secp256k1/commit/4a8b707fa84819dda7a663a92c0e32f519f9bacf -> https://github.com/bitcoin-core/secp256k1/commit/0c63b8b1911ef1183f411a5e232165b543c668ea (bip352-silentpayments-module-10-rebase -> bip352-silentpayments-module-11, compare)

    • Add the json test vectors from the BIP and ensure the Makefile is aware of changes to the test vectors
    • Update the example based on feedback from @Sjors
    • Update the API documentation
    • No API/implementation changes

    Per @real-or-random ’s comment #1519 (review), I removed details about the protocol internals from a few spots in the API documentation. The caller already needs to be familiar with BIP352 before using this library considering this library does not do things like get the smallest outpoint, filter the transaction inputs, or extract the public keys from the inputs. Given that, I agree its better to keep the documentation here focused on API usage and not on explaining how BIP352 works under the hood.

  97. in src/modules/silentpayments/main_impl.h:22 in 605096d3f4 outdated
    19+        &(*(const secp256k1_silentpayments_recipient **)pk2)->scan_pubkey
    20+    );
    21+}
    22 
    23-/* TODO: implement functions for receiver side. */
    24+static int secp256k1_silentpayments_recipient_sort(const secp256k1_context* ctx, const secp256k1_silentpayments_recipient **recipients, size_t n_recipients) {
    


    theStack commented at 11:48 pm on July 11, 2024:
    as this function always succeeds, it could just be void with the return 1; removed

    josibake commented at 2:14 pm on July 12, 2024:
    Nice, fixed.
  98. in src/modules/silentpayments/main_impl.h:72 in 605096d3f4 outdated
    69+}
    70+
    71+static int secp256k1_silentpayments_create_shared_secret(const secp256k1_context *ctx, unsigned char *shared_secret33, const secp256k1_scalar *secret_component, const secp256k1_pubkey *public_component) {
    72+    secp256k1_gej ss_j;
    73+    secp256k1_ge ss, pk;
    74+    size_t len = 33;
    


    theStack commented at 11:49 pm on July 11, 2024:

    micro-nit: len is a pure out-parameter for _eckey_pubkey_serialize, so it doesn’t need to be initialized (in contrast to the public API function _ec_pubkey_serialize where it is an In/Out param)

    0    size_t len;
    

    josibake commented at 2:14 pm on July 12, 2024:
    Fixed.
  99. in src/modules/silentpayments/main_impl.h:80 in 605096d3f4 outdated
    77+    secp256k1_pubkey_load(ctx, &pk, public_component);
    78+
    79+    /* Compute shared_secret = tweaked_secret_component * Public_component */
    80+    secp256k1_ecmult_const(&ss_j, &pk, secret_component);
    81+    secp256k1_ge_set_gej(&ss, &ss_j);
    82+    ret &= secp256k1_eckey_pubkey_serialize(&ss, shared_secret33, &len, 1);
    


    theStack commented at 11:55 pm on July 11, 2024:
    I think this call should always succeed (same as in _silentpayments_calculate_input_hash above), given that the only reason to return 0 is if the passed group element is the point at inifinity, which should never happen? If that assumption is right, then we could make the whole function void, and a VERIFY_CHECK on ret and len would be sufficient

    josibake commented at 2:14 pm on July 12, 2024:
    Updated, good call!
  100. in src/modules/silentpayments/main_impl.h:222 in 605096d3f4 outdated
    210+    ret &= !overflow;
    211+    /* TODO: any concerns here with multiplying a "secret" by a scalar here, i.e., input_hash * a_sum
    212+     * Mostly concerned about cases where a_sum is a single key and leaking information about a_sum
    213+     * since input_hash can be calculated by anyone
    214+     */
    215+    secp256k1_scalar_mul(&a_sum_scalar, &a_sum_scalar, &input_hash_scalar);
    


    theStack commented at 0:14 am on July 12, 2024:
    Regarding the TODO, I strongly assume that this is fine. For example, in secp256k1_ecdsa_sig_sign we also have a _scalar_mul call with a scalar corresponding to a secret key.

    jlest01 commented at 4:00 am on July 12, 2024:
    But if a_sum is a single key, is it possible to calculate the secret key, given any derived public information (such as the generated output) ?

    josibake commented at 7:31 am on July 12, 2024:

    I strongly assume that this is fine

    Same. Left the comment more to draw eyes from the “Friendly Neighborhood Cryptographers.” I’m not aware of any timing attacks that work with multiplying scalars or information that could be leaked, but wanted to double check.

    is it possible to calculate the secret key, given any derived public information

    No. This is a question about whether or not an attacker can learn anything from the multiplication step.


    real-or-random commented at 12:23 pm on July 12, 2024:
    This is fine. The entire scalar module treats scalars as secrets (because they typically represent secret keys).

    josibake commented at 2:13 pm on July 12, 2024:
    Removed the comment. I left a few of the other TODO comments in regarding _cmov and loading unsigned char secret keys into scalars because that’s something I’d appreciate some eyes on / feedback.
  101. in src/modules/silentpayments/main_impl.h:108 in 605096d3f4 outdated
     98+    hash->bytes = 64;
     99+}
    100+
    101+static void secp256k1_silentpayments_create_t_k(secp256k1_scalar *t_k_scalar, const unsigned char *shared_secret33, unsigned int k) {
    102+    secp256k1_sha256 hash;
    103+    unsigned char hash_ser[32];
    


    theStack commented at 0:19 am on July 12, 2024:
    hash_ser should probably also be cleaned out, as it’s just another representation of t_k_scalar (which is in turn cleaned out in the calling function below)

    josibake commented at 2:12 pm on July 12, 2024:
    Fixed.
  102. in examples/silentpayments.c:230 in 816796300a outdated
    225+            tx_outputs[i] = generated_outputs[i];
    226+        }
    227+
    228+        /* It's best practice to try to clear secrets from memory after using them.
    229+         * This is done because some bugs can allow an attacker to leak memory, for
    230+         * example through "out of bounds" array access (see Heartbleed), Or the OS
    



    josibake commented at 2:08 pm on July 12, 2024:
    Fixed.
  103. in examples/silentpayments.c:25 in 816796300a outdated
    20+/* Static data for Bob and Carol's silent payment addresses */
    21+static unsigned char smallest_outpoint[36] = {
    22+    0x16,0x9e,0x1e,0x83,0xe9,0x30,0x85,0x33,0x91,
    23+    0xbc,0x6f,0x35,0xf6,0x05,0xc6,0x75,0x4c,0xfe,
    24+    0xad,0x57,0xcf,0x83,0x87,0x63,0x9d,0x3b,0x40,
    25+    0x96,0xc5,0x4f,0x18,0xf4,0x00,0x00,0x00,0x00
    


    real-or-random commented at 8:45 am on July 12, 2024:
    nit: We typically have spaces after the commas here.

    josibake commented at 2:08 pm on July 12, 2024:
    Fixed.
  104. in tools/tests_silentpayments_generate.py:103 in 0b6827182d outdated
     98+            out += to_c_array(keys[i])
     99+            out += "}"
    100+        else:
    101+            out += '""'
    102+        if i != MAX_INPUTS_PER_TEST_CASE - 1:
    103+            out += ','
    


    real-or-random commented at 8:55 am on July 12, 2024:
    Believe it or not, but C89 allows trailing commas here, so you don’t need to check.

    josibake commented at 2:12 pm on July 12, 2024:
    A rare instance where C89 is not evil :laughing: , will fix in the next round of review.

    josibake commented at 11:35 am on July 14, 2024:
    Fixed.
  105. in examples/silentpayments.c:147 in 816796300a outdated
    142+        /*** Generate private keys for the sender ***
    143+         *
    144+         * In this example, only taproot inputs are used but the function can be called with
    145+         * a mix of taproot seckeys and plain seckeys. Taproot seckeys are passed as keypairs
    146+         * to allow the sending function to check if the private keys need to be negated without needing
    147+         * to do an expensive pubkey generation. This is not needed for plain seckeys since there is no need
    


    real-or-random commented at 9:00 am on July 12, 2024:
    nit: I think it will be good to limit the lines to 80 chars if possible/reasonable. (We have a rule about this in CONTRIBUTING.md for headers. I guess it should apply to examples, too.)

    josibake commented at 2:09 pm on July 12, 2024:
    Fixed.
  106. in examples/silentpayments.c:224 in 816796300a outdated
    219+            print_hex(xonly_print, sizeof(xonly_print));
    220+        }
    221+        /* Store the generated outputs in the `tx_outputs` array. The `tx_outputs` array is used
    222+         * to represent the final transaction, which is what Bob and Carol would use for scanning.
    223+         */
    224+        for(i = 0; i < N_TX_OUTPUTS; i++) {
    


    real-or-random commented at 9:01 am on July 12, 2024:
    nit: spacing

    josibake commented at 2:09 pm on July 12, 2024:
    Fixed.
  107. in examples/silentpayments.c:182 in 816796300a outdated
    177+        }
    178+        /*** Create the recipient objects ***/
    179+
    180+        /* Alice is sending to Bob and Carol in this transaction:
    181+         *
    182+         *     1. One output to Bob's labelled address (2.0 bitcoin)
    


    real-or-random commented at 9:09 am on July 12, 2024:
    nit: s/bitcoin/BTC for consistency>

    josibake commented at 2:10 pm on July 12, 2024:
    Fixed.
  108. in examples/silentpayments.c:455 in 816796300a outdated
    385+                int found = 0;
    386+                size_t k = 0;
    387+                secp256k1_xonly_pubkey potential_output;
    388+
    389+                while(1) {
    390+
    


    real-or-random commented at 9:12 am on July 12, 2024:
    nit: add space after while and remove the empty line

    josibake commented at 2:11 pm on July 12, 2024:
    Missed this but will address in the next round of review

    josibake commented at 11:42 am on July 14, 2024:
    Fixed.
  109. in examples/silentpayments.c:293 in 816796300a outdated
    288+            }
    289+
    290+            /* In this contrived example, our label context needs the secp256k1 context because our lookup function
    291+             * is calling `secp256k1_ec_pubkey_cmp`. In practice, this context can be anything the lookup function needs.
    292+             */
    293+            labels_cache.ctx = ctx;
    


    real-or-random commented at 9:14 am on July 12, 2024:
    Do you think this could be simplified by working with the serializations of the pubkeys?

    josibake commented at 12:03 pm on July 12, 2024:

    Good question! Initially, I think I was worried going from ge -> secp256k1_pubkey -> bytes would add extra overhead during scanning, but since then I’ve discovered we can go from ge -> bytes. I also benchmarked using secp256k1_pubkey vs unsigned char for the label callback and there was no difference in scanning.

    I’d say this is preferable too because now the callback function doesn’t need to be aware of secp256k1_pubkey and it’s very likely the labels will be stored in the labels cache as bytes (which would mean in the previous version the callback function would need to do the serialization, anyways).


    josibake commented at 2:10 pm on July 12, 2024:
    Updated to use unsigned char, thanks for the suggestion!
  110. in examples/silentpayments.c:358 in 816796300a outdated
    353+             * means she will need to first generate an output, check if it exists in the UTXO set (e.g.
    354+             * BIP158 or some other means of querying) and only proceed to check the next output (by
    355+             * incrementing `k`) if the first output exists.
    356+             *
    357+             * Additionally, Carol likely does not have access to the transaction inputs and prevout information,
    358+             * so she uses the `public_data` object creatd by Bob's full node earlier. This serialized `public_data` object
    


    real-or-random commented at 9:16 am on July 12, 2024:
    nit: created

    josibake commented at 2:10 pm on July 12, 2024:
    Fixed.
  111. in examples/silentpayments.c:2 in 816796300a outdated
    0@@ -0,0 +1,432 @@
    1+/*************************************************************************
    2+ * Written in 2024 by josibake                                           *
    


    real-or-random commented at 9:21 am on July 12, 2024:
    Our current recommendation is to skip all years and author info in new files (also in the C files), see the ellswift module for example. If there’s no data, it cannot become wrong and outdated, and the git history anyway does a better job here. (And copyright notices are simply not required to obtain copyright.) But this is really just a recommendation, feel free to keep your name if you feel that attribution is a good idea.

    josibake commented at 2:10 pm on July 12, 2024:
    Fixed.
  112. in examples/silentpayments.c:312 in 816796300a outdated
    307+            assert(ret);
    308+            labels_cache.entries_used = 1;
    309+
    310+            /* Bob collects the public data from the transaction inputs and creates a `secp256k1_silentpayments_public_data` object.
    311+             * He uses this for his own scanning and also serializes the `public_data` object to send to light clients. We will
    312+             * use this later for Carol, who is scanning as a light client. Note, anyone can create and vend these `public_data`
    


    real-or-random commented at 9:24 am on July 12, 2024:
    nit: “vend” is a word that I had to look up, which is a bit uncommon for me, even as a non-native speaker. Maybe you can find a more common word. Perhaps just “provide”?

    josibake commented at 2:10 pm on July 12, 2024:
    Fixed.
  113. in examples/silentpayments.c:321 in 816796300a outdated
    316+             */
    317+            ret = secp256k1_silentpayments_recipient_public_data_create(ctx,
    318+                &public_data,
    319+                smallest_outpoint,
    320+                tx_input_ptrs, N_TX_INPUTS,
    321+                NULL, 0 /* null because no eligible plain pubkey inputs were found in the tx */
    


    real-or-random commented at 9:24 am on July 12, 2024:
    s/null/NULL

    josibake commented at 2:10 pm on July 12, 2024:
    Fixed.
  114. real-or-random commented at 9:25 am on July 12, 2024: contributor
    I think the example is really helpful, and I don’t really have ideas to improve conceptually. (That’s why I have mostly just nits.)
  115. in src/bench.c:67 in 0c63b8b191 outdated
    62@@ -63,6 +63,10 @@ static void help(int default_iters) {
    63     printf("    ellswift_ecdh     : ECDH on ElligatorSwift keys\n");
    64 #endif
    65 
    66+#ifdef ENABLE_MODULE_SILENTPAYMENTS
    67+    printf("    silentpayments    : Silent payments recipient scanning\n");
    


    real-or-random commented at 9:26 am on July 12, 2024:
    Not just scanning, right?

    josibake commented at 12:04 pm on July 12, 2024:
    Currently, I only have a benchmark for the scanning function (since that’s the only function were performance is absolutely critical). Perhaps I can update this to silentpayments_scanning ?

    real-or-random commented at 12:19 pm on July 12, 2024:

    I only have a benchmark for the scanning function

    Ah, nevermind, I got this wrong.

  116. real-or-random commented at 9:26 am on July 12, 2024: contributor
    The module should also be mentioned in the README
  117. josibake force-pushed on Jul 12, 2024
  118. josibake commented at 2:07 pm on July 12, 2024: member

    Updated https://github.com/bitcoin-core/secp256k1/commit/0c63b8b1911ef1183f411a5e232165b543c668ea -> https://github.com/bitcoin-core/secp256k1/commit/602e11d910083582b139f2c45045c27e26805b92 (bip352-silentpayments-module-11 -> bip352-silentpayments-module-12, compare)

    • Updated the label_lookup function to take a 33-byte pubkey serialization (as opposed to a secp256k1_pubkey) (h/t @real-or-random)
    • Made _sort and _create_shared_secret void functions and added a few tests to verify all cases are covered (h/t @theStack)
    • Updated comments in the example to make it more clear that labels is an optional feature (h/t @Sjors)
    • General spelling and formatting fix ups
    • Added a commit for the README

    The diff looks rather large, but its mostly formatting and spelling fixes. Thanks @Sjors , @real-or-random , @theStack for the thorough review!

  119. in include/secp256k1_silentpayments.h:192 in 0910032d92 outdated
    187+ *  payment.
    188+ *
    189+ *  The public keys have to be passed in via two different parameter pairs, one
    190+ *  for regular and one for x-only public keys, in order to avoid the need of
    191+ *  users converting to a common pubkey format before calling this function.
    192+ *  The resulting data is can be used for scanning on the recipient side, or
    


    theStack commented at 4:37 pm on July 12, 2024:
    0 *  The resulting data can be used for scanning on the recipient side, or
    

    josibake commented at 11:34 am on July 14, 2024:
    Fixed.
  120. in include/secp256k1_silentpayments.h:295 in 0910032d92 outdated
    290+} secp256k1_silentpayments_found_output;
    291+
    292+/** Scan for Silent Payment transaction outputs.
    293+ *
    294+ *  Given a input public sum, an input_hash, a recipient's spend public key
    295+ *  B_spend, and the relevant transaction outputs, scan for outputs belong to
    


    theStack commented at 4:40 pm on July 12, 2024:
    0 *  B_spend, and the relevant transaction outputs, scan for outputs belonging to
    

    josibake commented at 11:34 am on July 14, 2024:
    Fixed.
  121. in src/modules/silentpayments/main_impl.h:359 in 0910032d92 outdated
    354+    secp256k1_ge_set_gej(&A_sum_ge, &A_sum_gej);
    355+    secp256k1_silentpayments_calculate_input_hash(input_hash_local, outpoint_smallest36, &A_sum_ge);
    356+    secp256k1_pubkey_save(&A_sum, &A_sum_ge);
    357+    /* serialize the public_data struct */
    358+    public_data->data[0] = 0;
    359+    secp256k1_ec_pubkey_serialize(ctx, &public_data->data[1], &pubkeylen, &A_sum, SECP256K1_EC_UNCOMPRESSED);
    


    theStack commented at 5:05 pm on July 12, 2024:

    could use internal function _eckey_pubkey_serialize instead, which accepts a group element, so the pubkey object A_sum is not needed anymore (verified that the tests still pass):

     0diff --git a/src/modules/silentpayments/main_impl.h b/src/modules/silentpayments/main_impl.h
     1index a1d4e15..840c9b0 100644
     2--- a/src/modules/silentpayments/main_impl.h
     3+++ b/src/modules/silentpayments/main_impl.h
     4@@ -311,7 +311,6 @@ int secp256k1_silentpayments_recipient_public_data_create(
     5 ) {
     6     size_t i;
     7     size_t pubkeylen = 65;
     8-    secp256k1_pubkey A_sum;
     9     secp256k1_ge A_sum_ge, addend;
    10     secp256k1_gej A_sum_gej;
    11     unsigned char input_hash_local[32];
    12@@ -353,10 +352,9 @@ int secp256k1_silentpayments_recipient_public_data_create(
    13     /* Compute input_hash = hash(outpoint_L || A_sum) */
    14     secp256k1_ge_set_gej(&A_sum_ge, &A_sum_gej);
    15     secp256k1_silentpayments_calculate_input_hash(input_hash_local, outpoint_smallest36, &A_sum_ge);
    16-    secp256k1_pubkey_save(&A_sum, &A_sum_ge);
    17     /* serialize the public_data struct */
    18     public_data->data[0] = 0;
    19-    secp256k1_ec_pubkey_serialize(ctx, &public_data->data[1], &pubkeylen, &A_sum, SECP256K1_EC_UNCOMPRESSED);
    20+    secp256k1_eckey_pubkey_serialize(&A_sum_ge, &public_data->data[1], &pubkeylen, 0);
    21     memcpy(&public_data->data[1 + pubkeylen], input_hash_local, 32);
    22     return ret;
    23 }
    

    Maybe there are other places as well where internal functions can be used to need less variables/code.


    josibake commented at 11:34 am on July 14, 2024:
    Good catch! Fixed.
  122. in src/modules/silentpayments/main_impl.h:490 in 0910032d92 outdated
    460+        int overflow = 0;
    461+
    462+        secp256k1_silentpayments_recipient_public_data_load_input_hash(input_hash, public_data);
    463+        secp256k1_scalar_set_b32(&input_hash_scalar, input_hash, &overflow);
    464+        /* TODO: any concerns with leaking the scan key when multiplying by a publicly known scalar? */
    465+        secp256k1_scalar_mul(&rsk_scalar, &rsk_scalar, &input_hash_scalar);
    


    theStack commented at 5:08 pm on July 12, 2024:
    seems like this TODO can be removed as well (see #1519 (review)).

    josibake commented at 11:33 am on July 14, 2024:
    Done.
  123. josibake force-pushed on Jul 14, 2024
  124. josibake commented at 11:33 am on July 14, 2024: member

    Updated https://github.com/bitcoin-core/secp256k1/commit/602e11d910083582b139f2c45045c27e26805b92 -> https://github.com/bitcoin-core/secp256k1/commit/00b0cb19a97718dfaab70aa7505ff157f22a31bd (bip352-silentpayments-module-12 -> bip352-silentpayments-module-13, compare)

    • Formatting fix ups for examples/silentpayments.c (introduced in the previous commit :doh:)
    • Use eckey_pubkey_serialize (h/t @theStack)
    • Spelling fixups in include/secp256k1_silentpayments.h (h/t @theStack)
    • Allow trailing comma in test vectors (h/t @real-or-random)
    • Fix up memset when clear unsigned char arrays
    • Fix CI by replacing memcmp with secp256k1_memcmp_var
  125. in examples/silentpayments.c:260 in 00b0cb19a9 outdated
    255+        secure_erase(seckey, sizeof(seckey));
    256+        for (i = 0; i < N_TX_INPUTS; i++) {
    257+            secure_erase(&sender_seckeys[i], sizeof(sender_seckeys[i]));
    258+        }
    259+    }
    260+
    


    jlest01 commented at 1:24 am on July 16, 2024:

    The example file can also show how to use the secp256k1_silentpayments_recipient_create_labelled_spend_pubkey function. Here is a suggestion.

     0{
     1        /*** Label ***/
     2
     3        unsigned char label_tweak[32];
     4        secp256k1_pubkey label;
     5        secp256k1_pubkey labelled_spend_pubkey;
     6        secp256k1_pubkey bob_spend_ec_pubkey;
     7        unsigned char labelled_spend_pubkey_print[33];
     8        size_t outputlen = 33;
     9
    10        ret = secp256k1_ec_pubkey_parse(ctx, &bob_spend_ec_pubkey, bob_spend_pubkey, sizeof(bob_spend_pubkey));
    11        assert(ret);
    12
    13        ret = secp256k1_silentpayments_recipient_create_label_tweak(ctx, &label, label_tweak, bob_scan_key, 1);
    14        assert(ret);
    15        ret = secp256k1_silentpayments_recipient_create_labelled_spend_pubkey(ctx, &labelled_spend_pubkey, &bob_spend_ec_pubkey, &label);
    16        assert(ret);
    17
    18        ret = secp256k1_ec_pubkey_serialize(ctx, labelled_spend_pubkey_print, &outputlen, &labelled_spend_pubkey, SECP256K1_EC_COMPRESSED);
    19        assert(ret);
    20        printf("Labelled Bob Spend Pubkey: ");
    21        print_hex(labelled_spend_pubkey_print, sizeof(labelled_spend_pubkey_print));
    22        assert(memcmp(labelled_spend_pubkey_print, bob_address[1], 33) == 0);
    23        printf("It is the same as bob_address[1]. This is how we derive labeled addresses.\n");
    24    }
    25    
    

    josibake commented at 11:22 am on August 6, 2024:
    Thanks for the suggestion! I tried a few ways to work this into the example but was never satisfied with the result. It always ended up making the example more complicated (first create bob’s address, then do the sending, then do the receiving), and I don’t think it adds much since this function is fairly self explanatory.

    jonasnick commented at 1:53 pm on October 11, 2024:
    IMHO demonstrating key generation in the example would make sense. I don’t really know why it would be particularly complicated, but maybe I’m missing something. If key generation is not added, then maybe we could add a few comments to the beginning of the file, where the keys are defined.
  126. jlest01 commented at 1:26 am on July 16, 2024: none
    nit: a suggestion to make the example file cover all functions.
  127. benma referenced this in commit 1d49769ad4 on Jul 22, 2024
  128. benma referenced this in commit ad4bd73d4f on Jul 22, 2024
  129. benma referenced this in commit e881028893 on Jul 23, 2024
  130. benma referenced this in commit a361bdc22d on Jul 23, 2024
  131. in include/secp256k1_silentpayments.h:56 in 79562d0cd1 outdated
    63+ *
    64+ *  `outpoint_smallest36` refers to the smallest outpoint lexicographically
    65+ *  from the transaction inputs (both silent payments eligible and non-eligible
    66+ *  inputs). This value MUST be the smallest outpoint out of all of the
    67+ *  transaction inputs, otherwise the recipient will be unable to find the the
    68+ *  payment.
    


    theStack commented at 2:54 pm on July 31, 2024:
    0 *  transaction inputs, otherwise the recipient will be unable to find the
    1 *  payment.
    

    josibake commented at 1:04 pm on August 6, 2024:
    Fixed.
  132. in include/secp256k1_silentpayments.h:115 in 23c7aead63 outdated
    106@@ -107,6 +107,56 @@ SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_sender_c
    107     size_t n_plain_seckeys
    108 ) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(5);
    109 
    110+/** Create Silent Payment label tweak and label.
    111+ *
    112+ *  Given a recipient's scan key b_scan and a label integer m, calculate the
    113+ *  corresponding label tweak and label:
    114+ *
    115+ *  label_tweak = hash(b_scan || m) label = label_tweak * G
    


    theStack commented at 2:56 pm on July 31, 2024:

    seems like a newline got lost here:

    0 *  label_tweak = hash(b_scan || m)
    1 *  label = label_tweak * G
    

    josibake commented at 1:04 pm on August 6, 2024:
    Fixed.
  133. in src/modules/silentpayments/main_impl.h:128 in 79562d0cd1 outdated
    125+    secp256k1_scalar t_k_scalar;
    126+    int ret;
    127+
    128+    /* Calculate and return P_output_xonly = B_spend + t_k * G */
    129+    secp256k1_silentpayments_create_t_k(&t_k_scalar, shared_secret33, k);
    130+    secp256k1_pubkey_load(ctx, &P_output_ge, recipient_spend_pubkey);
    


    theStack commented at 3:28 pm on July 31, 2024:

    Should we check the return value of secp256k1_pubkey_load, here and at other places? Note sure if there’s a general guideline in this library whether passed in secp256k1_pubkey instance parameters can be just treated as valid due to its type (after all, they must have been created before with libsecp as well) or a check is preferred as belts-and-suspenders.

    For example, the ellswift module does a verification check: https://github.com/bitcoin-core/secp256k1/blob/fded437c4cec1a29921c90be325d62ba866f44ce/src/modules/ellswift/main_impl.h#L406 while the ECDH module doesn’t: https://github.com/bitcoin-core/secp256k1/blob/fded437c4cec1a29921c90be325d62ba866f44ce/src/modules/ecdh/main_impl.h#L47

    (Probably it also depends on the code following, haven’t gone deeper there yet…)


    josibake commented at 9:53 am on August 6, 2024:
    secp256k1_pubkey_load fails if ge->x is zero, so in our case I think we should check the return value since the spend public key is a user supplied value and its not inconceivable someone would try to pass an illegal public key as a argument.

    josibake commented at 1:04 pm on August 6, 2024:
    Added error handling for secp256k1_pubkey_load, since in the case of the recipient_spend_pubkey this is a value the caller is responsible for setting.
  134. in include/secp256k1_silentpayments.h:101 in 00b0cb19a9 outdated
     96+ *                            keys
     97+ */
     98+SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_sender_create_outputs(
     99+    const secp256k1_context *ctx,
    100+    secp256k1_xonly_pubkey **generated_outputs,
    101+    const secp256k1_silentpayments_recipient **recipients,
    


    jlest01 commented at 8:16 pm on August 2, 2024:

    Is there any reason not to make the outer pointer constant since it is not modified in this function?

    0    const secp256k1_silentpayments_recipient * const *recipients,
    

    josibake commented at 10:58 am on August 6, 2024:
    The recipients array is passed to secp256k1_hsort, where the array is sorted in place.
  135. in include/secp256k1_silentpayments.h:130 in 00b0cb19a9 outdated
    124+ */
    125+SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipient_create_label_tweak(
    126+    const secp256k1_context *ctx,
    127+    secp256k1_pubkey *label,
    128+    unsigned char *label_tweak32,
    129+    const unsigned char *recipient_scan_key,
    


    jlest01 commented at 10:49 pm on August 4, 2024:

    nit: this change makes it clear that what is expected is the secret key

    0    const unsigned char *recipient_scan_seckey,
    

    josibake commented at 11:04 am on August 6, 2024:
    The scan key isn’t really a secret key, so I’ve been trying to avoid using seckey in the variable names. Conceptually, a scan key is more similar to an xpub in that whoever has the scan key can find the transactions, but cannot spend them. The scan key is also safe to give to a 3rd party without loss of funds. In hindsight, we probably should have chosen a better name in the BIP e.g. scan entropy or scan code (like chain code).
  136. josibake force-pushed on Aug 6, 2024
  137. josibake force-pushed on Aug 6, 2024
  138. josibake commented at 12:59 pm on August 6, 2024: member

    Update https://github.com/bitcoin-core/secp256k1/commit/a804ae7439f0c88d12dfbeb2e2706ea8d4ca52b7) -> https://github.com/bitcoin-core/secp256k1/commit/b2b904b30a62cd5f90432b659b7b955243bcb5df (bip352-silentpayments-module-13-rebase -> bip352-silentpayments-module-14, compare)

    • Update the example variable names to avoid confusing with general transaction input and outputs
    • Add benchmark for scanning a single output (light client scanning)
    • Typo fixes (h/t @theStack)
    • Handle secp256k1_pubkey_load errors
  139. josibake force-pushed on Aug 6, 2024
  140. josibake commented at 3:32 pm on August 6, 2024: member

    Update https://github.com/bitcoin-core/secp256k1/commit/b2b904b30a62cd5f90432b659b7b955243bcb5df -> https://github.com/bitcoin-core/secp256k1/commit/3165b6b091a30a4ace948d67d55142c61a12929d (bip352-silentpayments-module-14 -> bip352-silentpayments-module-15, compare)

    • Add test for malformed secp256k1_pubkey_load
    • Handle _pubkey_load error missed in the last push @theStack I spent some time trying to create a test where we create a secp256k1_pubkey object that is in an invalid state, but it doesn’t seem possible, since errors are caught when calling _ec_pubkey_parse. This is why I decided not to make the internal create_shared_secret function handle an error when loading a public key because by that point the public key object has either been a) created by us, or b) validated by us.

    In all other cases, I think it makes sense to check the error because the pubkey object would have been created by the caller and we have no way of knowing whether or not they created it correctly (e.g., using the ec_pubkey_parse function).

  141. jlest01 commented at 1:03 am on August 7, 2024: none

    Given the following secp256k1_silentpayments_public_data :

    0x00043e715c3499e40448bf23b53113f7218c5b9805b454a1003eb1009339a48ef69385272624f954ce1e87000eb8dc162e7952ede7d0d0d1d0e38dd7f0588101fe9506a48141afe37530269d77c297681e7efe6cb1d516f7ed50634570d3e0e0c54f

    If serialized using secp256k1_silentpayments_recipient_public_data_serialize, the result will be:

    0x039fefa1c745f46da57dc11376e80eb6c11cdd0e28733c50eae777cf5c4b0f95d7.

    So, if this same light_client_data33 is parsed back to secp256k1_silentpayments_public_data using secp256k1_silentpayments_recipient_public_data_parse, the result will be:

    0x01049fefa1c745f46da57dc11376e80eb6c11cdd0e28733c50eae777cf5c4b0f95d76df2de74016f234aef7957b66fa7a5eb8be025ded9122e32c50556f088efbbff0000000000000000000000000000000000000000000000000000000000000000

    This is not clear to me. Shouldn’t parsing the serialized data result in the same secp256k1_silentpayments_public_data?

  142. josibake commented at 7:53 am on August 7, 2024: member

    Thanks for testing @jlest01 . The public data object contains a public key and 32 byte input_hash. This can be used directly in _scan_outputs or it can be serialised for later use (in an index or for sending to light clients). When the data is serialised , the public key is tweaked with the input_hash scalar, i.e., $A_{tweaked} = inputhash \cdot A_{sum}$.

    $A_{tweaked}$ is what gets serialised, so when $A_{tweaked}$ is deserialised the public data object now only contains a public key. In your example, you can see that the compressed serialisation of the first public data object has the x coordinate 9fefa1c745f46da57dc11376e80eb6c11cdd0e28733c50eae777cf5c4b0f95d7, which is the same x-coordinate in the de-serialised public data at the end. This isn’t really explained in the API docs, by design, since this is supposed to be an opaque data type and these details aren’t really relevant for the caller.

  143. in src/modules/silentpayments/main_impl.h:72 in 31a6da2cf7 outdated
    69+static void secp256k1_silentpayments_create_shared_secret(const secp256k1_context *ctx, unsigned char *shared_secret33, const secp256k1_scalar *secret_component, const secp256k1_pubkey *public_component) {
    70+    secp256k1_gej ss_j;
    71+    secp256k1_ge ss, pk;
    72+    size_t len;
    73+    int ret;
    74+    memset(shared_secret33, 0, 33);
    


    theStack commented at 2:00 pm on August 8, 2024:
    nit: I think this memset can be removed, as there is no early return in this function and it’s guaranteed that shared_secret33 will be filled with the _pubkey_serialize call below
  144. in src/modules/silentpayments/main_impl.h:89 in 31a6da2cf7 outdated
    85+     * the secret key being used
    86+     */
    87+    ret = secp256k1_eckey_pubkey_serialize(&ss, shared_secret33, &len, 1);
    88+    VERIFY_CHECK(ret && len == 33);
    89+    (void)ret;
    90+}
    


    theStack commented at 2:04 pm on August 8, 2024:

    could clear out the ge/gej shared secret representations at the end (with the same “while not technically ‘secret’ data…” argumentation that’s already used in other functions)

    0    (void)ret;
    1    secp256k1_ge_clear(&ss);
    2    secp256k1_gej_clear(&ss_j);
    3}
    
  145. theStack commented at 3:17 pm on August 8, 2024: contributor

    @theStack I spent some time trying to create a test where we create a secp256k1_pubkey object that is in an invalid state, but it doesn’t seem possible, since errors are caught when calling _ec_pubkey_parse. This is why I decided not to make the internal create_shared_secret function handle an error when loading a public key because by that point the public key object has either been a) created by us, or b) validated by us.

    Thanks for following up! It seems that for the sender API function, the public component (i.e. a recipient’s scan public key) is never validated though? With the following change in the send API test:

     0diff --git a/src/modules/silentpayments/tests_impl.h b/src/modules/silentpayments/tests_impl.h
     1index fdca422..bb4e4b9 100644
     2--- a/src/modules/silentpayments/tests_impl.h
     3+++ b/src/modules/silentpayments/tests_impl.h
     4@@ -254,7 +254,7 @@ static void test_send_api(void) {
     5      * trying to parse the public key with _ec_pubkey_parse
     6      */
     7     p[0] = ALICE_SECKEY;
     8-    memset(&r[0].spend_pubkey.data, 0, sizeof(secp256k1_pubkey));
     9+    memset(&r[0].scan_pubkey.data, 0, sizeof(secp256k1_pubkey));
    10     CHECK_ILLEGAL(CTX, secp256k1_silentpayments_sender_create_outputs(CTX, op, rp, 2, SMALLEST_OUTPOINT, NULL, 0, p, 1));
    11 }
    

    the test fails on that pubkey’s serialization in the shared secret creation helper:

    0$ ./tests
    1test count = 64
    2random seed = de0af255fa70eafa2f1bb5b09fdfd112
    3src/modules/silentpayments/main_impl.h:86: test condition failed: ret && len == 33
    4Aborted (core dumped)
    
  146. in src/modules/silentpayments/main_impl.h:313 in bc9ac6237f outdated
    300+    ret &= secp256k1_pubkey_load(ctx, &label_addend, label);
    301+    if (!ret) {
    302+        return ret;
    303+    }
    304+    secp256k1_gej_set_ge(&result_gej, &B_m);
    305+    secp256k1_gej_add_ge_var(&result_gej, &result_gej, &label_addend, NULL);
    


    theStack commented at 3:53 pm on August 8, 2024:
    should we add a check here if the result is the point at infinity and return 0?
  147. in include/secp256k1_silentpayments.h:321 in 6b1f86b3e5 outdated
    316+ *           recipient_scan_key: pointer to the recipient's scan key
    317+ *                  public_data: pointer to the input public key sum
    318+ *                               (optionally, with the `input_hash` multiplied
    319+ *                               in, see `_recipient_public_data_create`).
    320+ *       recipient_spend_pubkey: pointer to the recipient's spend pubkey
    321+  *                label_lookup: pointer to a callback function for looking up
    


    theStack commented at 4:43 pm on August 8, 2024:

    whitespace nit

    0 *       recipient_spend_pubkey: pointer to the recipient's spend pubkey
    1 *                 label_lookup: pointer to a callback function for looking up
    
  148. in src/modules/silentpayments/main_impl.h:380 in 6b1f86b3e5 outdated
    363+    /* Compute input_hash = hash(outpoint_L || A_sum) */
    364+    secp256k1_ge_set_gej(&A_sum_ge, &A_sum_gej);
    365+    secp256k1_silentpayments_calculate_input_hash(input_hash_local, outpoint_smallest36, &A_sum_ge);
    366+    /* serialize the public_data struct */
    367+    public_data->data[0] = 0;
    368+    secp256k1_eckey_pubkey_serialize(&A_sum_ge, &public_data->data[1], &pubkeylen, 0);
    


    theStack commented at 4:47 pm on August 8, 2024:

    could check the return value and pubkeylen here if VERIFY is set (like also done in e.g. _silentpayments_calculate_input_hash), e.g.

    0    ser_ret = secp256k1_eckey_pubkey_serialize(&A_sum_ge, &public_data->data[1], &pubkeylen, 0);
    1    VERIFY_CHECK(ser_ret && pubkeylen == 65);
    2    (void)ser_ret;
    
  149. in examples/silentpayments.c:129 in 2f39991873 outdated
    124+    if (!fill_random(randomize, sizeof(randomize))) {
    125+        printf("Failed to generate randomness\n");
    126+        return 1;
    127+    }
    128+    /* Randomizing the context is recommended to protect against side-channel
    129+     * leakage See `secp256k1_context_randomize` in secp256k1.h for more
    


    theStack commented at 5:05 pm on August 8, 2024:

    missing dot nit

    0    /* Randomizing the context is recommended to protect against side-channel
    1     * leakage. See `secp256k1_context_randomize` in secp256k1.h for more
    
  150. in include/secp256k1_silentpayments.h:18 in 31a6da2cf7 outdated
    21+ * summing up private or public keys and the derivation of a shared secret
    22+ * using Elliptic Curve Diffie-Hellman. Combined are either: - spender's
    23+ * private keys and recipient's public key (a * B, sender side) - spender's
    24+ * public keys and recipient's private key (A * b, recipient side) With this
    25+ * result, the necessary key material for ultimately creating/scanning or
    26+ * spending Silent Payment outputs can be determined.
    


    theStack commented at 5:11 pm on August 8, 2024:
    here and below: seems like these format changes should be already part of the first commit
  151. theStack commented at 5:23 pm on August 8, 2024: contributor
  152. build: add skeleton for new silentpayments (BIP352) module 1c749416cc
  153. silentpayments: sending
    Add a routine for the entire sending flow which takes a set of private keys,
    the smallest outpoint, and list of recipients and returns a list of
    x-only public keys by performing the following steps:
    
    1. Sum up the private keys
    2. Calculate the input_hash
    3. For each recipient group:
        3a. Calculate a shared secret
        3b. Create the requested number of outputs
    
    This function assumes a single sender context in that it requires the
    sender to have access to all of the private keys. In the future, this
    API may be expanded to allow for a multiple senders or for a single
    sender who does not have access to all private keys at any given time,
    but for now these modes are considered out of scope / unsafe.
    
    Internal to the library, add:
    
    1. A function for creating shared secrets (i.e., a*B or b*A)
    2. A function for generating the "SharedSecret" tagged hash
    3. A function for creating a single output public key
    
    Finally, add tests for the sender API.
    9d6769f4ae
  154. silentpayments: recipient label support
    Add function for creating a label tweak. This requires a tagged hash
    function for labels. This function is used by the receiver for creating
    labels to be used for a) creating labelled addresses and b) to populate
    a labels cache when scanning.
    
    Add function for creating a labelled spend pubkey. This involves taking
    a label tweak, turning it into a public key and adding it to the spend
    public key. This function is used by the receiver to create a labelled
    silent payment address.
    
    Add tests for the label API.
    7229d49d1b
  155. silentpayments: receiving
    Add routine for scanning a transaction and returning the necessary
    spending data for any found outputs. This function works with labels via
    a lookup callback and requires access to the transaction outputs.
    Requiring access to the transaction outputs is not suitable for light
    clients, but light client support is enabled by exposing the
    `_create_shared_secret` and `_create_output_pubkey` functions in the
    API. This means the light client will need to manage their own scanning
    state, so wherever possible it is preferrable to use the
    `_recipient_scan_ouputs` function.
    
    Add an opaque data type for passing around the summed input public key (A_sum)
    and the input hash tweak (input_hash). This data is passed to the scanner
    before the ECDH step as two separate elements so that the scanner can
    multiply b_scan * input_hash before doing ECDH.
    
    Add functions for deserializing / serializing a public_data object to
    and from a public key. When serializing a public_data object, the
    input_hash is multplied into A_sum. This is so the object can be stored
    as public key for wallet rescanning later, or to vend to light clients.
    For the light client, a `_parse` function is added which parses the
    compressed public key serialization into a `public_data` object.
    
    Finally, add test coverage for the recieiving API.
    94c6e1f542
  156. silentpayments: add examples/silentpayments.c
    Demonstrate sending, scanning, and light client scanning.
    5c546e2874
  157. silentpayments: add benchmarks for scanning
    Add a benchmark for a full transaction scan and for scanning a single
    output. Only benchmarks for scanning are added as this is the most
    performance critical portion of the protocol.
    566b5b8db5
  158. tests: add BIP-352 test vectors
    Add the BIP-352 test vectors. The vectors are generated with a Python script
    that converts the .json file from the BIP to C code:
    
    $ ./tools/tests_silentpayments_generate.py test_vectors.json > ./src/modules/silentpayments/vectors.h
    5ce0db1c12
  159. ci: enable silentpayments module 5b9714fce2
  160. docs: update README f42e0dde59
  161. josibake force-pushed on Aug 15, 2024
  162. josibake commented at 10:57 am on August 15, 2024: member

    Update https://github.com/bitcoin-core/secp256k1/commit/3165b6b091a30a4ace948d67d55142c61a12929d -> https://github.com/bitcoin-core/secp256k1/commit/f42e0dde59943ad0c9247c61f7af521b70838e8f (bip352-silentpayments-module-15 -> bip352-silentpayments-module-16, compare)

    • Added test for malformed scan_pubkey and moved _pubkey_load out of _create_shared_secret. Overall, I think this makes the error handling more clear to validate the public keys before calling create_shared_secret
    • Added VERIFY and infinity checks (h/t @theStack)
    • Spelling and formatting fix-ups (h/t @theStack)

    Thanks again for the thorough review, @theStack !

  163. in src/modules/silentpayments/main_impl.h:229 in 9d6769f4ae outdated
    226+    last_recipient = *recipients[0];
    227+    k = 0;
    228+    for (i = 0; i < n_recipients; i++) {
    229+        if ((i == 0) || (secp256k1_ec_pubkey_cmp(ctx, &last_recipient.scan_pubkey, &recipients[i]->scan_pubkey) != 0)) {
    230+            /* If we are on a different scan pubkey, its time to recreate the the shared secret and reset k to 0.
    231+             * It's very unlikely tha the scan public key is invalid by this point, since this means the caller would
    


    theStack commented at 9:52 pm on August 19, 2024:
    0             * It's very unlikely that the scan public key is invalid by this point, since this means the caller would
    
  164. in include/secp256k1_silentpayments.h:137 in 7229d49d1b outdated
    132+) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
    133+
    134+/** Create Silent Payment labelled spend public key.
    135+ *
    136+ *  Given a recipient's spend public key B_spend and a label, calculate the
    137+ *  corresponding serialized labelled spend public key:
    


    theStack commented at 10:22 pm on August 19, 2024:
    0 *  corresponding labelled spend public key:
    

    (I assume that’s a leftover from an earlier version of the API where the resulting label was indeed serialized)

  165. in src/modules/silentpayments/tests_impl.h:240 in 7229d49d1b outdated
    235+        0x46,0x38,0x60,0x28,0xa8,0x1a,0x77,0xd4,0x91
    236+    };
    237+
    238+    /* Create a label and labelled spend public key, verify we get the expected result */
    239+    CHECK(secp256k1_ec_pubkey_parse(CTX, &s, BOB_ADDRESS[1], 33));
    240+    CHECK(secp256k1_silentpayments_recipient_create_label_tweak(CTX, &l, lt, ALICE_SECKEY, 1));
    


    theStack commented at 10:33 pm on August 19, 2024:
    nit: could also check the label and label tweak results here

    josibake commented at 12:39 pm on November 7, 2024:
    yes, although the values are used in the next call, _create_labelled_spend_pubkey and then the final result of that is checked against an exact expected value, so the extra checks don’t seem worth the added verbosity?
  166. in include/secp256k1_silentpayments.h:347 in 94c6e1f542 outdated
    342+    SECP256K1_ARG_NONNULL(6) SECP256K1_ARG_NONNULL(7) SECP256K1_ARG_NONNULL(8);
    343+
    344+/** Create Silent Payment shared secret.
    345+ *
    346+ *  Given the public input data (secp256k1_silentpayments_public_data),
    347+ *  calculate the shared secret.
    


    theStack commented at 10:57 pm on August 19, 2024:
    0 *  Given the public input data (secp256k1_silentpayments_public_data)
    1 *  and a recipient's scan key, calculate the shared secret.
    
  167. in src/modules/silentpayments/main_impl.h:504 in 94c6e1f542 outdated
    499+        secp256k1_ge P_output_ge = recipient_spend_pubkey_ge;
    500+        /* Calculate t_k = hash(shared_secret || ser_32(k)) */
    501+        secp256k1_silentpayments_create_t_k(&t_k_scalar, shared_secret, k);
    502+
    503+        /* Calculate P_output = B_spend + t_k * G
    504+         * This can fail if t_k overflows the curver order, but this is statistically improbable
    


    theStack commented at 11:18 pm on August 19, 2024:
    0         * This can fail if t_k overflows the curve order, but this is statistically improbable
    

    stratospher commented at 4:57 am on December 12, 2024:
    94c6e1f: it don’t think it can fail because of overflow at this point (the overflow is ignored and t_k has already been converted to a scalar in secp256k1_silentpayments_create_t_k). we could move this comment up into secp256k1_silentpayments_create_t_k and handle failure inside that function.
  168. in src/modules/silentpayments/tests_impl.h:318 in 94c6e1f542 outdated
    313+    CHECK(secp256k1_silentpayments_recipient_public_data_parse(CTX, &pd, BOB_ADDRESS[0]));
    314+    CHECK_ILLEGAL(CTX, secp256k1_silentpayments_recipient_public_data_serialize(CTX, o, &pd));
    315+    /* Try to create a shared secret with a malformed recipient scan key (all zeros) */
    316+    CHECK(secp256k1_silentpayments_recipient_create_shared_secret(CTX, o, MALFORMED_SECKEY, &pd) == 0);
    317+    /* Try to create a shared secret with a malformed public key (all zeros) */
    318+    memset(&pd.data[1], 0, sizeof(&pd.data - 1));
    


    theStack commented at 4:27 pm on August 20, 2024:

    I think this was meant to be

    0    memset(&pd.data[1], 0, sizeof(pd.data) - 1));
    

    (resulting in 97, rather than 8 due to being the sizeof a pointer)

  169. theStack commented at 4:35 pm on August 20, 2024: contributor

    Light ACK modulo some more smaller doc and test nits I found

    As an additional testing idea, could add some checks that the initialized “BIP0352/…” tagged hashes have the expected state (inspired by the MuSig2 PR), but that seems fine to also do in a follow-up.

  170. jlest01 commented at 3:04 pm on August 24, 2024: none

    The function secp256k1_silentpayments_recipient_public_data_create fails for the public keys:

    . 02aba9d21ae5bb3832e845eeca8c0b58e07b27b501debc759cdccb9d86beef3d48 . 03aba9d21ae5bb3832e845eeca8c0b58e07b27b501debc759cdccb9d86beef3d48

    And the smallest outpoint:

    . 66bd2917be75420a789924737f3e47500bf134cea57a10ce67bf33d8a256e7b000000000

    These values represent this signet P2TR transaction: https://mempool.space/signet/tx/d73f4a19f3973e90af6df62e735bb7b31f3d5ab8e7e26e7950651b436d093313

    Code below for reproduction:

     0static unsigned char smallest_outpoint[36] = {
     1    0x66, 0xbd, 0x29, 0x17, 0xbe, 0x75, 0x42, 0x0a, 0x78, 
     2    0x99, 0x24, 0x73, 0x7f, 0x3e, 0x47, 0x50, 0x0b, 0xf1, 
     3    0x34, 0xce, 0xa5, 0x7a, 0x10, 0xce, 0x67, 0xbf, 0x33, 
     4    0xd8, 0xa2, 0x56, 0xe7, 0xb0, 0x00, 0x00, 0x00, 0x00
     5};
     6
     7static unsigned char first_pubkey_bytes[33] = {
     8    0x02, 0xab, 0xa9, 0xd2, 0x1a, 0xe5, 0xbb, 0x38, 
     9    0x32, 0xe8, 0x45, 0xee, 0xca, 0x8c, 0x0b, 0x58,
    10    0xe0, 0x7b, 0x27, 0xb5, 0x01, 0xde, 0xbc, 0x75, 
    11    0x9c, 0xdc, 0xcb, 0x9d, 0x86, 0xbe, 0xef, 0x3d, 0x48
    12};
    13
    14static unsigned char second_pubkey_bytes[33] = {
    15    0x03, 0xab, 0xa9, 0xd2, 0x1a, 0xe5, 0xbb, 0x38, 
    16    0x32, 0xe8, 0x45, 0xee, 0xca, 0x8c, 0x0b, 0x58,
    17    0xe0, 0x7b, 0x27, 0xb5, 0x01, 0xde, 0xbc, 0x75, 
    18    0x9c, 0xdc, 0xcb, 0x9d, 0x86, 0xbe, 0xef, 0x3d, 0x48
    19};
    20
    21int main(void) {
    22
    23    enum { N_INPUTS = 2 };
    24
    25    unsigned char randomize[32];
    26
    27    secp256k1_pubkey first_pubkey;
    28    secp256k1_pubkey second_pubkey;
    29    const secp256k1_pubkey *tx_input_ptrs[N_INPUTS];
    30    secp256k1_silentpayments_public_data public_data;
    31
    32    int ret;
    33
    34    secp256k1_context* ctx = secp256k1_context_create(SECP256K1_CONTEXT_NONE);
    35    if (!fill_random(randomize, sizeof(randomize))) {
    36        printf("Failed to generate randomness\n");
    37        return 1;
    38    }
    39
    40    ret = secp256k1_context_randomize(ctx, randomize);
    41    assert(ret);
    42
    43    ret = secp256k1_ec_pubkey_parse(ctx, &first_pubkey, first_pubkey_bytes, 33);
    44    ret = secp256k1_ec_pubkey_parse(ctx, &second_pubkey, second_pubkey_bytes, 33);
    45
    46    tx_input_ptrs[0] = &first_pubkey;
    47    tx_input_ptrs[1] = &second_pubkey;
    48
    49    ret = secp256k1_silentpayments_recipient_public_data_create(ctx,
    50                &public_data,
    51                smallest_outpoint,
    52                NULL, 0, 
    53                tx_input_ptrs, N_INPUTS
    54            );
    55
    56    assert(ret); // main: Assertion `ret' failed.
    57
    58    return 0;
    59
    60}
    
  171. jlest01 commented at 3:48 pm on August 24, 2024: none

    Testing the same values ​​using rust silentpayments crate, you get an error description: Secp256k1Error(InvalidPublicKeySum) (code below).

    I assume the problem is that adding these two public keys results in a point at infinity (since they have opposite parity).

    Would it be the caller’s responsibility to do this check before calling secp256k1_silentpayments_recipient_public_data_create ?

     0use silentpayments::{secp256k1::PublicKey, utils::receiving::calculate_tweak_data};
     1
     2fn main() {
     3
     4    let mut pubkeys: Vec<PublicKey> = Vec::with_capacity(2);
     5
     6    let first_pubkey_bytes = hex::decode("02aba9d21ae5bb3832e845eeca8c0b58e07b27b501debc759cdccb9d86beef3d48").unwrap();
     7    let first_pubkey = PublicKey::from_slice(first_pubkey_bytes.as_slice()).unwrap();
     8    pubkeys.push(first_pubkey);
     9
    10    let second_pubkey_bytes = hex::decode("03aba9d21ae5bb3832e845eeca8c0b58e07b27b501debc759cdccb9d86beef3d48").unwrap();
    11    let second_pubkey = PublicKey::from_slice(second_pubkey_bytes.as_slice()).unwrap();
    12    pubkeys.push(second_pubkey);
    13
    14    let input_pub_keys: Vec<&PublicKey> = pubkeys.iter().collect();
    15    
    16    let mut outpoints_data: Vec<(String, u32)> = Vec::with_capacity(2);
    17
    18    outpoints_data.push(("b0e756a2d833bf67ce107aa5ce34f10b50473e7f732499780a4275be1729bd66".to_string(), 0));
    19    outpoints_data.push(("b0e756a2d833bf67ce107aa5ce34f10b50473e7f732499780a4275be1729bd66".to_string(), 1));
    20
    21    let tweak_data = calculate_tweak_data(&input_pub_keys, &outpoints_data).unwrap();
    22    // panicked! called `Result::unwrap()` on an `Err` value: Secp256k1Error(InvalidPublicKeySum)
    23}
    
  172. josibake commented at 1:00 pm on August 26, 2024: member

    Hey @jlest01 , thanks for the thorough testing! The error you’re encountering is from here:

    https://github.com/bitcoin-core/secp256k1/blob/f42e0dde59943ad0c9247c61f7af521b70838e8f/src/modules/silentpayments/main_impl.h#L369-L374

    As you mention, this is because the public keys sum to the point at infinity, i.e., the second pubkey is a negation of the first one. It is expected that callers should check the return value from _public_data_create and proceed to the next transaction if they get an error. Earlier in this PR we did discuss having more specific error codes but ultimately decided it’s best to try and make it such that if a user encounters an error, there’s likely nothing they can do but skip the transaction. From a user’s perspective, if they can’t do anything to fix the error, they don’t really gain anything by knowing why it failed.

    I think we have a case for this in the test vectors, but if not I’ll definitely add this to the BIP test vectors since scanning implementations should be checking for this.

  173. in src/modules/silentpayments/main_impl.h:444 in f42e0dde59
    439+    const secp256k1_xonly_pubkey * const *tx_outputs, size_t n_tx_outputs,
    440+    const unsigned char *recipient_scan_key,
    441+    const secp256k1_silentpayments_public_data *public_data,
    442+    const secp256k1_pubkey *recipient_spend_pubkey,
    443+    const secp256k1_silentpayments_label_lookup label_lookup,
    444+    const void *label_context
    


    antonilol commented at 2:04 pm on September 7, 2024:
    Could label_context (here and in the callback type) be made mutable? Is there a particular reason it is const?

    antonilol commented at 4:01 pm on September 7, 2024:
    Does this actually disallow (make undefined behavior) mutating the context from the callback?

    josibake commented at 12:47 pm on November 7, 2024:
    What would be the reason for making the pointer to label context mutable here? Making it const is meant to clearly communicate to the caller that this is meant to be a reference to some external data store. For your second question, I don’t think making the pointer const prevents the lookup function itself from mutating the context, but I don’t think there is a way we can prevent that.

    antonilol commented at 3:03 pm on November 7, 2024:

    If I understand correctly (after searching online about this), the C compiler will assume that no mutation happens to the pointed to data using the pointer label_context, because it is declared const. In Rust however, you can’t prevent users from mutating captured variables of a closure. (Rust’s shared references can be mutated through, still they are called immutable references.)

    A fix on the Rust side would be to add a level of indirection, instead of passing a pointer to the context directly, a pointer to a pointer is passed. The pointed to data of the const void * is never mutated (that would of type context_t * then, in C terms, where context_t is a struct defined by the user). A fix on the C side would be to remove the const keyword.

    I am fine with both, but the extra level of indirection will be slightly less performant, but modern CPU caches can make this (almost) unobservable.


    sipa commented at 3:08 pm on November 7, 2024:

    the C compiler will assume that no mutation happens to the pointed to data using the pointer label_context, because it is declared const.

    This is, as far as I understand, not correct. A const pointer does not imply that the data pointed to cannot be mutated. It only means it cannot be mutated through that pointer.

    What is true however, is that if all references to an object are const, and the compiler can prove this, then no mutation to the object can happen to it at all. This in practice only applies to objects that are themselves declared const.

    If the object itself is non-const, but you hold a const pointer to it, then it is perfectly legal even to cast the constness of the pointer away and mutate it through that casted pointer.


    antonilol commented at 3:26 pm on November 7, 2024:

    Thanks for the clarification!

    If the object itself is non-const, but you hold a const pointer to it, then it is perfectly legal even to cast the constness of the pointer away and mutate it through that casted pointer.

    Especially this, this is new for me, thanks!

  174. in src/modules/silentpayments/main_impl.h:469 in f42e0dde59
    464+    ARG_CHECK(recipient_spend_pubkey != NULL);
    465+    if (label_lookup != NULL) {
    466+        ARG_CHECK(label_context != NULL);
    467+    } else {
    468+        ARG_CHECK(label_context == NULL);
    469+    }
    


    antonilol commented at 4:23 pm on September 7, 2024:
    Why is the label_context ARG_CHECK’ed, shouldn’t this be treated as opaque data for the callback alone?

    Sjors commented at 1:13 pm on September 10, 2024:
    That means the (wallet) implementer has to make sure to perform these checks in their implementation of secp256k1_silentpayments_label_lookup. Seems better to do it here.

    antonilol commented at 8:54 am on September 11, 2024:
    That seems unnecessary, as both the callback and the context pointer are passed by the caller. If the caller wants to use global state or some other way that does not require label_context it should be allowed to set it to NULL.

    josibake commented at 12:53 pm on November 7, 2024:
    Since the label_lookup accepts a void* pointer for the context, I think it would be fine to make passing the context optional. Perhaps the better arg check would be to do this in reverse: complain if the caller passes a context and not a lookup function. Will update.
  175. in tools/tests_silentpayments_generate.py:1 in 5ce0db1c12 outdated
    0@@ -0,0 +1,297 @@
    1+#!/usr/bin/env python3
    


    Sjors commented at 8:18 am on September 9, 2024:

    5ce0db1c124aa98f62e13274eef22765da0ee335

    0+'''
    1+A script to convert BIP352 test vectors from JSON to a C header.
    2+
    3+Usage:
    4+
    5+    ./tools/tests_silentpayments_generate.py src/modules/silentpayments/bip352_send_and_receive_test_vectors.json  > ./src/modules/silentpayments/vectors.h
    6+'''
    
  176. in tools/tests_silentpayments_generate.py:164 in 5ce0db1c12 outdated
    159+        out += ",\n"
    160+    out += spacing*" " + "}"
    161+    if not last:
    162+        out += ","
    163+    out += "\n"
    164+
    


    Sjors commented at 8:19 am on September 9, 2024:

    5ce0db1c124aa98f62e13274eef22765da0ee335

    0if len(sys.argv) != 2:
    1    print("Usage: tests_silentpayments_generate.py vectors.json > vectors.h")
    2    sys.exit(1)
    
  177. in src/modules/silentpayments/tests_impl.h:10 in 9d6769f4ae outdated
     5+
     6+#ifndef SECP256K1_MODULE_SILENTPAYMENTS_TESTS_H
     7+#define SECP256K1_MODULE_SILENTPAYMENTS_TESTS_H
     8+
     9+#include "../../../include/secp256k1_silentpayments.h"
    10+#include "include/secp256k1.h"
    


    Sjors commented at 8:35 am on September 9, 2024:

    9d6769f4aed4691463f9c591e147df6a0ccfbca3: You don’t need this include and it breaks the cmake build.

    0cd build
    1cmake .. -DSECP256K1_ENABLE_MODULE_SILENT_PAYMENTS=O
    2cmake --build .
    3...
    4.../secp256k1/src/modules/silentpayments/tests_impl.h:11:10: fatal error: 'include/secp256k1.h' file not found
    5#include "include/secp256k1.h"
    
  178. in configure.ac:192 in 1c749416cc outdated
    187@@ -188,6 +188,10 @@ AC_ARG_ENABLE(module_ellswift,
    188     AS_HELP_STRING([--enable-module-ellswift],[enable ElligatorSwift module [default=yes]]), [],
    189     [SECP_SET_DEFAULT([enable_module_ellswift], [yes], [yes])])
    190 
    191+AC_ARG_ENABLE(module_silentpayments,
    192+    AS_HELP_STRING([--enable-module-silentpayments],[enable Silent Payments module [default=no]]), [],
    


    Sjors commented at 8:39 am on September 9, 2024:
    1c749416ccf4878fff5d103db81dadca367c10c0: can we just break the convention and make it --enable-module-silent-payments? :-)

    josibake commented at 1:00 pm on November 7, 2024:
    Haha no :)
  179. in include/secp256k1_silentpayments.h:86 in 9d6769f4ae outdated
    81+ *              n_recipients: the number of recipients. This is equal to the
    82+ *                            total number of outputs to be generated as each
    83+ *                            recipient may passed multiple times to generate
    84+ *                            multiple outputs for the same recipient
    85+ *       outpoint_smallest36: serialized smallest outpoint (lexicographically)
    86+ *                            from the transaction inputs
    


    Sjors commented at 9:52 am on September 9, 2024:

    9d6769f4aed4691463f9c591e147df6a0ccfbca3: the 36 suffix is weird. outpoint36_smallest would be slightly more clear.

    Suggest adding to the doc (instead): (36 bytes)

    Given how many times this has gone wrong, the following warning seems justified:

    0Lexicographical sorting applies to the concatenated outpoint hash and
    1index (vout). The latter is little-endian encoded, so must not be
    2sorted in its integer representation.
    

    The test vector in this PR doesn’t prevent this, because sorting is left to the implementer.

    Test vectors in the BIP don’t necessary prevent this either, because implementers will assume it’s all covered in libsecp.


    josibake commented at 1:03 pm on November 7, 2024:

    There was a separate discussion on the Musig2 PR that I need to follow up on regarding this, where there might be a more clever way to enforce the array length without these suffix hints. For now, I think I will remove the 36 and add it to the docs, as you suggest.

    Agree that we can’t enforce they are passing the correct outpoint out of a list of outpoints here, but I do think we can make it clear in the header file that this is not handled by libsecp and implementations MUST ensure they are following the test vectors from the BIP. I’ll add something as such to the header documentation.

  180. in include/secp256k1_silentpayments.h:68 in 9d6769f4ae outdated
    63+ *  Returns: 1 if creation of outputs was successful. 0 if an error occured.
    64+ *  Args:                ctx: pointer to a context object
    65+ *  Out:   generated_outputs: pointer to an array of pointers to xonly pubkeys,
    66+ *                            one per recipient.
    67+ *                            The order of outputs here matches the original
    68+ *                            ordering of the recipients array.
    


    Sjors commented at 10:42 am on September 9, 2024:

    9d6769f4aed4691463f9c591e147df6a0ccfbca3: the words “matches the original ordering” had me thoroughly confused, because I thought it referred to the array order. Try instead:

    0The outputs here are sorted by the index value provided in recipients. 
    
  181. in include/secp256k1_silentpayments.h:73 in 9d6769f4ae outdated
    68+ *                            ordering of the recipients array.
    69+ *  In:           recipients: pointer to an array of pointers to silent payment
    70+ *                            recipients, where each recipient is a scan public
    71+ *                            key, a spend public key, and an index indicating
    72+ *                            its position in the original ordering. The
    73+ *                            recipient array will be sorted in place, but
    


    Sjors commented at 10:47 am on September 9, 2024:
    9d6769f4aed4691463f9c591e147df6a0ccfbca3: grouped by scan and spend key,

    josibake commented at 1:10 pm on November 7, 2024:
    I think this is an implementation detail the caller does not need to be aware of, since we handle the grouping internally. From a callers perspective, all they need to do is pass an array of recipients, in any order.
  182. in include/secp256k1_silentpayments.h:91 in 9d6769f4ae outdated
    86+ *                            from the transaction inputs
    87+ *           taproot_seckeys: pointer to an array of pointers to 32-byte
    88+ *                            private keys of taproot inputs (can be NULL if no
    89+ *                            private keys of taproot inputs are used)
    90+ *         n_taproot_seckeys: the number of sender's taproot input private keys
    91+ *             plain_seckeys: pointer to an array of pointers to 32-byte
    


    Sjors commented at 10:55 am on September 9, 2024:

    9d6769f4aed4691463f9c591e147df6a0ccfbca3: why can’t this be secp256k1_keypair as well? If not documented in the API, at least a clarifying code comment would be useful.

    The PR description implies the logic is the other way around:

    taproot_seckeys are passed as keypairs to avoid the function needing to compute the public key to determine parity.

    So you want to save the caller from having to use secp256k1_keypair? But only for legacy inputs, so that doesn’t seem that useful towards the future.


    josibake commented at 1:12 pm on November 7, 2024:
    Good catch, this is outdated documentation. The function does expect taproot_seckeys to be secp256k1_keypairs.
  183. in CMakeLists.txt:55 in 1c749416cc outdated
    51@@ -52,9 +52,14 @@ option(SECP256K1_ENABLE_MODULE_RECOVERY "Enable ECDSA pubkey recovery module." O
    52 option(SECP256K1_ENABLE_MODULE_EXTRAKEYS "Enable extrakeys module." ON)
    53 option(SECP256K1_ENABLE_MODULE_SCHNORRSIG "Enable schnorrsig module." ON)
    54 option(SECP256K1_ENABLE_MODULE_ELLSWIFT "Enable ElligatorSwift module." ON)
    55+option(SECP256K1_ENABLE_MODULE_SILENTPAYMENTS "Enable Silent Payments module." OFF)
    


    Sjors commented at 11:52 am on September 9, 2024:

    1c749416ccf4878fff5d103db81dadca367c10c0: since the next commit adds a dependency on extrakeys, and silent payments obviously depends on schnorr:

     0diff --git a/CMakeLists.txt b/CMakeLists.txt
     1index 8b450bf..f036a3f 100644
     2--- a/CMakeLists.txt
     3+++ b/CMakeLists.txt
     4@@ -55,10 +55,17 @@ option(SECP256K1_ENABLE_MODULE_ELLSWIFT "Enable ElligatorSwift module." ON)
     5 option(SECP256K1_ENABLE_MODULE_SILENTPAYMENTS "Enable Silent Payments module." OFF)
     6 
     7 # Processing must be done in a topological sorting of the dependency graph
     8 # (dependent module first).
     9 if(SECP256K1_ENABLE_MODULE_SILENTPAYMENTS)
    10+  if(DEFINED SECP256K1_ENABLE_MODULE_SCHNORRSIG AND NOT SECP256K1_ENABLE_MODULE_SCHNORRSIG)
    11+    message(FATAL_ERROR "Module dependency error: You have disabled the schnorrsig module explicitly, but it is required by the silentpayments module.")
    12+  endif()
    13+  if(DEFINED SECP256K1_ENABLE_MODULE_EXTRAKEYS AND NOT SECP256K1_ENABLE_MODULE_EXTRAKEYS)
    14+    message(FATAL_ERROR "Module dependency error: You have disabled the extrakeys module explicitly, but it is required by the silentpayments module.")
    15+  endif()
    16+  set(SECP256K1_ENABLE_MODULE_EXTRAKEYS ON)
    17   add_compile_definitions(ENABLE_MODULE_SILENTPAYMENTS=1)
    18 endif()
    19 
    20 if(SECP256K1_ENABLE_MODULE_ELLSWIFT)
    21   add_compile_definitions(ENABLE_MODULE_ELLSWIFT=1)
    22diff --git a/configure.ac b/configure.ac
    23index fe778b1..da6f707 100644
    24--- a/configure.ac
    25+++ b/configure.ac
    26@@ -397,10 +397,20 @@ SECP_CFLAGS="$SECP_CFLAGS $WERROR_CFLAGS"
    27 ###
    28 
    29 # Processing must be done in a reverse topological sorting of the dependency graph
    30 # (dependent module first).
    31 if test x"$enable_module_silentpayments" = x"yes"; then
    32+  if test x"$enable_module_schnorrsig" = x"no"; then
    33+    AC_MSG_ERROR([Module dependency error: You have disabled the schnorrsig module explicitly, but it is required by the silentpayments module.])
    34+  fi
    35+  enable_module_schnorrsig=yes
    36+
    37+  if test x"$enable_module_extrakeys" = x"no"; then
    38+    AC_MSG_ERROR([Module dependency error: You have disabled the extrakeys module explicitly, but it is required by the silentpayments module.])
    39+  fi
    40+  enable_module_extrakeys=yes
    41+
    42   SECP_CONFIG_DEFINES="$SECP_CONFIG_DEFINES -DENABLE_MODULE_SILENTPAYMENTS=1"
    43 fi
    44 
    45 if test x"$enable_module_ellswift" = x"yes"; then
    46   SECP_CONFIG_DEFINES="$SECP_CONFIG_DEFINES -DENABLE_MODULE_ELLSWIFT=1"
    

    extrakeys already checks for schnorr, but I added it an explicit check for schnorr because otherwise the error is confusing.

    Try with:

    • ./configure --enable-module-silentpayments --disable-module-extrakeys
    • cmake .. -DSECP256K1_ENABLE_MODULE_SILENTPAYMENTS=ON -DSECP256K1_ENABLE_MODULE_SCHNORRSIG=OFF
    • etc
  184. in src/modules/silentpayments/main_impl.h:15 in 9d6769f4ae outdated
    11 #include "../../../include/secp256k1_silentpayments.h"
    12 
    13-/* TODO: implement functions for sender side. */
    14+/** Sort an array of silent payment recipients. This is used to group recipients by scan pubkey to
    15+ *  ensure the correct values of k are used when creating multiple outputs for a recipient. */
    16+static int secp256k1_silentpayments_recipient_sort_cmp(const void* pk1, const void* pk2, void *ctx) {
    


    Sjors commented at 12:15 pm on September 9, 2024:
    9d6769f4aed4691463f9c591e147df6a0ccfbca3: shouldn’t this also sort by spend key, and use index as a tie breaker? It seems BIP352 is ambiguous here. IIUC it doesn’t matter in the end, because the recipient is expected to reconsider all remaining outputs for each increment of k.

    josibake commented at 1:21 pm on November 7, 2024:

    When we have multiple spend keys for the same scan key, the order of the spend keys doesn’t matter. We purposely did not specify a sorting order for the spend keys in BIP352 in order to avoid making clients do more work than necessary.

    As you mention, the algorithm for scanning ensures it does not rely on any order dependence of the outputs to be able to find all outputs.

  185. in src/modules/silentpayments/main_impl.h:186 in 9d6769f4ae outdated
    183+        ARG_CHECK(n_plain_seckeys > 0);
    184+    } else {
    185+        ARG_CHECK(n_plain_seckeys == 0);
    186+    }
    187+    ARG_CHECK(outpoint_smallest36 != NULL);
    188+    /* ensure the index field is set correctly */
    


    Sjors commented at 12:54 pm on September 9, 2024:
    9d6769f4aed4691463f9c591e147df6a0ccfbca3: a wallet might want to use the output index here. So maybe just enforce that each number is strictly higher than the previous? Or even just uniqueness. The documentation should clarify what is expected, because libsecp errors are no fun to debug.

    josibake commented at 1:23 pm on November 7, 2024:
    To be clear: this is not the output index from the transaction. This is an index going from 0 - r-1 where r represents the total number of silent payment outputs in the final transaction. There may be other non silent payment outputs in the transaction, hence this has no relation to vout index.
  186. in src/modules/silentpayments/main_impl.h:203 in 9d6769f4ae outdated
    200+        secp256k1_scalar_add(&a_sum_scalar, &a_sum_scalar, &addend);
    201+    }
    202+    /* private keys used for taproot outputs have to be negated if they resulted in an odd point */
    203+    for (i = 0; i < n_taproot_seckeys; i++) {
    204+        secp256k1_ge addend_point;
    205+        /* TODO: why don't we need _cmov here after calling keypair_load? Because the ret is declassified? */
    


    Sjors commented at 1:08 pm on September 9, 2024:
    9d6769f4aed4691463f9c591e147df6a0ccfbca3: My impression is that _cmov is used to clear addend in case _load fails. But since we don’t have an early return inside the loop, might as well do it at the end of the loop.

    Sjors commented at 9:39 am on October 16, 2024:
    Offline discussion: key pair load should never fail, because key pair object has already been validated. So constant time is not an issue here.
  187. in src/modules/silentpayments/main_impl.h:206 in 9d6769f4ae outdated
    203+    for (i = 0; i < n_taproot_seckeys; i++) {
    204+        secp256k1_ge addend_point;
    205+        /* TODO: why don't we need _cmov here after calling keypair_load? Because the ret is declassified? */
    206+        ret &= secp256k1_keypair_load(ctx, &addend, &addend_point, taproot_seckeys[i]);
    207+        if (secp256k1_fe_is_odd(&addend_point.y)) {
    208+            secp256k1_scalar_negate(&addend, &addend);
    


    Sjors commented at 1:15 pm on September 9, 2024:

    Could simplify the early return below by:

    9d6769f4aed4691463f9c591e147df6a0ccfbca3: ret &=

  188. in src/modules/silentpayments/main_impl.h:210 in 9d6769f4ae outdated
    207+        if (secp256k1_fe_is_odd(&addend_point.y)) {
    208+            secp256k1_scalar_negate(&addend, &addend);
    209+        }
    210+        secp256k1_scalar_add(&a_sum_scalar, &a_sum_scalar, &addend);
    211+    }
    212+    /* If there are any failures in loading/summing up the secret keys, fail early */
    


    Sjors commented at 1:15 pm on September 9, 2024:

    9d6769f4aed4691463f9c591e147df6a0ccfbca3

    0const int zero = 0;
    1secp256k1_int_cmov(addend, &zero, !ret);
    2secp256k1_int_cmov(a_sum_scalar, &zero, !ret);
    3
    4/* If there are any failures ... */
    5if (!ret) return 0;
    

    Sjors commented at 9:35 am on October 16, 2024:

    @josibake says to clear the a_sum_scalar in early returns. Probably addend too.

    cmove vs memset: the former is constant time, but that’s not always necessary.

    In general the library is not always consistent when it comes to early returns. Those are often not constant time (invalid secret behaves different from valid secret).

    cc @jonasnick please brainstorm a bit more about this code


    real-or-random commented at 5:32 pm on October 21, 2024:

    Fwiw, we have this: https://github.com/bitcoin-core/secp256k1/blob/f0868a9b3d809565d5a6784cd16fc22c76bba63c/src/util.h#L209-L210

    But AFAIU, this is “just” about clearing secrets from memory, and then memset is the right answer. (The compiler will optimize those memsets, but this PR will address this: #1579) No need to be constant-time in an error branch.

  189. in include/secp256k1_silentpayments.h:167 in 94c6e1f542 outdated
    162+ *
    163+ *  This structure does not contain secret data. Guaranteed to be 98 bytes in
    164+ *  size. It can be safely copied/moved. Created with
    165+ *  `secp256k1_silentpayments_public_data_create`. Can be serialized as a
    166+ *  compressed public key using
    167+ *  `secp256k1_silentpayments_public_data_serialize`. The serialization is
    


    Sjors commented at 2:16 pm on September 9, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: there is no secp256k1_silentpayments_public_data_serialize, did you mean secp256k1_silentpayments_recipient_public_data_serialize? Did you also mean to drop “as a compressed public key” (since it contains more than that)?

    I do think it’s useful to briefly document what is in the opaque structure: the summed public key and input_hash.

    Also, is there a light client use case that requires more than just tweak data? If not, then this struct could be reduced to 33 bytes. That’s what the index uses.


    Sjors commented at 9:26 am on September 10, 2024:

    Alright, this is thoroughly confusing, but I think I figured it out.

    I rewrote the documentation to fix the method names and make it more clear that:

    1. this struct can be created in two ways
    2. the struct itself is not intended to be sent to light clients
    3. use terminology from the BIP (“public tweak data”)
     0diff --git a/include/secp256k1_silentpayments.h b/include/secp256k1_silentpayments.h
     1index 05cabb8..11164e1 100644
     2--- a/include/secp256k1_silentpayments.h
     3+++ b/include/secp256k1_silentpayments.h
     4@@ -159,17 +159,20 @@ SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipien
     5 ) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4);
     6 
     7 /** Opaque data structure that holds silent payments public input data.
     8  *
     9  *  This structure does not contain secret data. Guaranteed to be 98 bytes in
    10- *  size. It can be safely copied/moved. Created with
    11- *  `secp256k1_silentpayments_public_data_create`. Can be serialized as a
    12- *  compressed public key using
    13- *  `secp256k1_silentpayments_public_data_serialize`. The serialization is
    14- *  intended for sending the public input data to light clients. Light clients
    15- *  can use this serialization with
    16- *  `secp256k1_silentpayments_public_data_parse`.
    17+ *  size. It can be safely copied/moved.
    18+ *
    19+ *  When the summed public key and input_hash are available, it's created using
    20+ *  `secp256k1_silentpayments_recipient_public_data_create`. Otherwise it can be
    21+ *  constructed from a compressed public key with the `input_hash` multiplied in
    22+ *  (public tweak data) using secp256k1_silentpayments_recipient_public_data_parse.
    23+ *
    24+ *  To serve light clients, the public tweak data can generated and serialized to
    25+ *  33 bytes using `secp256k1_silentpayments_recipient_public_data_serialize`,
    26+ *  which they can parse with `secp256k1_silentpayments_recipient_public_data_parse`.
    27  */
    28 typedef struct {
    29     unsigned char data[98];
    

    josibake commented at 12:48 pm on November 14, 2024:

    I updated the name of the struct to be silentpayments_recipient_public_data and updated the documentation to have consistent naming.

    On the doc rewrite, I’m not convinced this is better. It mentions input_hash explicitly, which is an implementation detail internal to the module, and also mentions some of the low-level operations going on inside the functions, which I don’t think we should be exposing/explaining to the caller here. To me, it seems sufficient to tell the caller: “hey, create a public data object using this function. If you’re a light client and receive a previously created, serialised public data object, parse it with this function.”

    Going to leave the documentation as is for now, but would be interested to hear others thoughts on this.

  190. in include/secp256k1_silentpayments.h:170 in 94c6e1f542 outdated
    165+ *  `secp256k1_silentpayments_public_data_create`. Can be serialized as a
    166+ *  compressed public key using
    167+ *  `secp256k1_silentpayments_public_data_serialize`. The serialization is
    168+ *  intended for sending the public input data to light clients. Light clients
    169+ *  can use this serialization with
    170+ *  `secp256k1_silentpayments_public_data_parse`.
    


    Sjors commented at 2:17 pm on September 9, 2024:
    94c6e1f54299dfc83aa34bc92f9f861808449a23: secp256k1_silentpayments_recipient_public_data_parse
  191. Sjors commented at 2:30 pm on September 9, 2024: member

    Reviewed the scaffold, send, labels and part of the receive commit (94c6e1f54299dfc83aa34bc92f9f861808449a23). Did not study the tests.

    You could split the light client / indexer functions out of the receive commit 94c6e1f54299dfc83aa34bc92f9f861808449a23.

  192. in src/modules/silentpayments/main_impl.h:381 in 94c6e1f542 outdated
    376+    secp256k1_ge_set_gej(&A_sum_ge, &A_sum_gej);
    377+    secp256k1_silentpayments_calculate_input_hash(input_hash_local, outpoint_smallest36, &A_sum_ge);
    378+    /* serialize the public_data struct */
    379+    public_data->data[0] = 0;
    380+    secp256k1_eckey_pubkey_serialize(&A_sum_ge, &public_data->data[1], &pubkeylen, 0);
    381+    ser_ret = secp256k1_eckey_pubkey_serialize(&A_sum_ge, &public_data->data[1], &pubkeylen, 0);
    


    Sjors commented at 10:05 am on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: why are you doing this twice?

    Maybe use a compressed key? I know the struct doesn’t need to be particularly small, but it seems better to use compressed keys by default unless there’s a reason not to.


    josibake commented at 4:00 pm on November 17, 2024:
    good catch, because I am a dum dum! I think the first line is left over from when I added the ser_ret check.
  193. in include/secp256k1_silentpayments.h:236 in 94c6e1f542 outdated
    231+ *  Out:     output33: pointer to a 33-byte array to place the serialized
    232+ *                     `silentpayments_public_data` in
    233+ *  In:   public_data: pointer to an initialized silentpayments_public_data
    234+ *                     object
    235+ */
    236+SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipient_public_data_serialize(
    


    Sjors commented at 11:24 am on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: the name of this function implies:

    1. That it serializes everything in the struct, which it doesn’t
    2. That it merely serializes data, which it doesn’t - it performs a hash

    Maybe call it secp256k1_silentpayments_get_serialized_public_tweak_data?

    And to make it more clear that this thing can go in the function below, you could make a:

    0/** Can be sent to light clients */
    1typedef struct {
    2     unsigned char public_tweak_data[33];
    3}
    

    josibake commented at 1:09 pm on November 14, 2024:
    I would argue it does serialise everything in the struct: it takes a public_data object and returns the serialised format, which is 33 bytes. Also, this function does not do a hash. The input_hash part of the object is created when the public data object is created.
  194. in include/secp256k1_silentpayments.h:267 in 94c6e1f542 outdated
    262+ *  the recipients label cache during scanning.
    263+ *
    264+ *  Returns: pointer to the 32-byte label tweak if there is a match.
    265+ *           NULL pointer if there is no match.
    266+ *
    267+ *  In:         label: pointer to the label value to check (computed during
    


    Sjors commented at 11:37 am on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: BIP352 does not define the terms “label tweak” and “label value”. I’m guessing “tweak” refers to hash(bscan || m)·G?

    Maybe just add “See secp256k1_silentpayments_recipient_create_label_tweak”. From the formula there it seems that “value” is B1 - Bspend. Could call that “tweak point”.


    josibake commented at 1:20 pm on November 14, 2024:
    I changed the language to “label pubkey,” as this is more consistent with where it is used elsewhere. Also added a reference to the secp256k1_silentpayments_recipient_create_label_tweak function per your suggestion.
  195. in include/secp256k1_silentpayments.h:295 in 94c6e1f542 outdated
    290+    secp256k1_pubkey label;
    291+} secp256k1_silentpayments_found_output;
    292+
    293+/** Scan for Silent Payment transaction outputs.
    294+ *
    295+ *  Given a input public sum, an input_hash, a recipient's spend public key
    


    Sjors commented at 11:53 am on September 10, 2024:
    94c6e1f54299dfc83aa34bc92f9f861808449a23: , a recipient's scan key b_scan and spend public key
  196. in include/secp256k1_silentpayments.h:364 in 94c6e1f542 outdated
    359+ *  In:     recipient_scan_key: pointer to the recipient's scan key
    360+ *                 public_data: pointer to the input public key sum, tweaked
    361+ *                              with the input_hash (see
    362+ *                              `_recipient_public_data_create`)
    363+ */
    364+SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipient_create_shared_secret(
    


    Sjors commented at 12:05 pm on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: derive_ instead of create_? (from the POV of the universe, the sender created it)

    Ditto for recipient_create_output_pubkey below.


    josibake commented at 1:25 pm on November 14, 2024:
    Slightly prefer create, as I want to avoid any confusion with bip32 style derivation. From the recipients perspective, they are also creating a shared secret per the spec. Its only after scanning with the shared secret that they can confirm this transaction was indeed a silent payment (i.e., the sender created a shared secret and outputs from that shared secret).
  197. in include/secp256k1_silentpayments.h:349 in 94c6e1f542 outdated
    344+/** Create Silent Payment shared secret.
    345+ *
    346+ *  Given the public input data (secp256k1_silentpayments_public_data),
    347+ *  calculate the shared secret.
    348+ *
    349+ *  The resulting shared secret is needed as input for creating silent payments
    


    Sjors commented at 12:09 pm on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: here the word “deriving” would reduce confusion imo. Otherwise it’s sounds like this is a sender creating an output (of a transaction).

    Maybe make it:

    0* .. deriving silent payment
    1* pubkeys for this recipient scan public key, which can be
    2* matched against transaction outputs.
    

    josibake commented at 1:26 pm on November 14, 2024:
    Same comment as above, derived is heavily associated with BIP32 style public key/private key derivation, so I want to avoid using it here as much as possible.
  198. in include/secp256k1_silentpayments.h:260 in 94c6e1f542 outdated
    255+    secp256k1_silentpayments_public_data *public_data,
    256+    const unsigned char *input33
    257+) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
    258+
    259+/** Callback function for label lookups
    260+ *
    


    Sjors commented at 12:46 pm on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: Lookup the tweak corresponding to a tweak point.

    (or something to that effect, to make it more clear why you need a lookup table)

  199. in include/secp256k1_silentpayments.h:262 in 94c6e1f542 outdated
    257+) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
    258+
    259+/** Callback function for label lookups
    260+ *
    261+ *  This function is implemented by the recipient to check if a value exists in
    262+ *  the recipients label cache during scanning.
    


    Sjors commented at 12:51 pm on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: the term “cache” is confusing. It implies data that you can derive yourself, but is quicker to lookup. But the function that uses this callback can’t do that derivation.

    (from a higher level perspective it is a cache, since a wallet will do the necessarily derivations just once)

    Better to say “recipients label tweak lookup table” (as the function name does)


    josibake commented at 1:31 pm on November 14, 2024:
    I think cache is appropriate here? As you mention, the wallet could derive this itself for each scan, but better to derive once and cache the results. “Lookup table” implies a specific data structure to me, which is something I’d like to avoid.
  200. in src/modules/silentpayments/main_impl.h:470 in 94c6e1f542 outdated
    465+    if (label_lookup != NULL) {
    466+        ARG_CHECK(label_context != NULL);
    467+    } else {
    468+        ARG_CHECK(label_context == NULL);
    469+    }
    470+    /* TODO: do we need a _cmov call here to avoid leaking information about the scan key?
    


    Sjors commented at 1:12 pm on September 10, 2024:
    94c6e1f54299dfc83aa34bc92f9f861808449a23: leaking privacy still sucks, and afaik the _cmov is super cheap. So just do it?

    Sjors commented at 9:40 am on October 16, 2024:
    Offline discussion: just do a memset
  201. in src/modules/silentpayments/main_impl.h:482 in 94c6e1f542 outdated
    477+    ret &= secp256k1_pubkey_load(ctx, &recipient_spend_pubkey_ge, recipient_spend_pubkey);
    478+    /* If there is something wrong with the recipient scan key, recipient spend pubkey, or the public data, return early */
    479+    if (!ret) {
    480+        return 0;
    481+    }
    482+    combined = (int)public_data->data[0];
    


    Sjors commented at 1:26 pm on September 10, 2024:

    94c6e1f54299dfc83aa34bc92f9f861808449a23: so much for opaque datatype :-)

    Maybe add a helper function secp256k1_silentpayments_recipient_public_data_get_tweak_data_scalar(&rsk_scalar, recipient_scan_key, public_data)


    josibake commented at 1:32 pm on November 14, 2024:
    It is opaque to users of the library :) Not sure I see the value of creating a separate function that is not exposed in the public API and only used internally?
  202. in src/modules/silentpayments/main_impl.h:509 in 94c6e1f542 outdated
    504+         * This can fail if t_k overflows the curver order, but this is statistically improbable
    505+         */
    506+        ret &= secp256k1_eckey_pubkey_tweak_add(&P_output_ge, &t_k_scalar);
    507+        found = 0;
    508+        secp256k1_xonly_pubkey_save(&P_output_xonly, &P_output_ge);
    509+        for (i = 0; i < n_tx_outputs; i++) {
    


    Sjors commented at 1:45 pm on September 10, 2024:
    94c6e1f54299dfc83aa34bc92f9f861808449a23: could be slightly optimised by skipping over outputs where you already found something. But this loop seems reasonably cheap.

    josibake commented at 1:33 pm on November 14, 2024:
    Had considered this! However, this is only relevant for large transactions with many outputs, where multiple outputs are created for the same recipient. For now, I think I’ll leave this for a follow up.
  203. in examples/silentpayments.c:8 in 5c546e2874 outdated
    0@@ -0,0 +1,503 @@
    1+/*************************************************************************
    2+ * To the extent possible under law, the author(s) have dedicated all    *
    3+ * copyright and related and neighboring rights to the software in this  *
    4+ * file to the public domain worldwide. This software is distributed     *
    5+ * without any warranty. For the CC0 Public Domain Dedication, see       *
    6+ * EXAMPLES_COPYING or https://creativecommons.org/publicdomain/zero/1.0 *
    7+ *************************************************************************/
    8+
    


    Sjors commented at 2:32 pm on September 10, 2024:

    5c546e2874012dccc40a3e91ed2502c538a00356 for cmake:

     0diff --git a/examples/CMakeLists.txt b/examples/CMakeLists.txt
     1index fd1ebce..66e989e 100644
     2--- a/examples/CMakeLists.txt
     3+++ b/examples/CMakeLists.txt
     4@@ -3,6 +3,7 @@ function(add_example name)
     5   add_executable(${target_name} ${name}.c)
     6   target_include_directories(${target_name} PRIVATE
     7     ${PROJECT_SOURCE_DIR}/include
     8+    ${PROJECT_SOURCE_DIR}
     9   )
    10   target_link_libraries(${target_name}
    11     secp256k1
    12@@ -32,3 +33,7 @@ endif()
    13 if(SECP256K1_ENABLE_MODULE_ELLSWIFT)
    14   add_example(ellswift)
    15 endif()
    16+
    17+if(SECP256K1_ENABLE_MODULE_SILENTPAYMENTS)
    18+  add_example(silentpayments)
    19+endif()
    
    0cmake .. -DSECP256K1_BUILD_EXAMPLES=1 -DSECP256K1_ENABLE_MODULE_SILENTPAYMENTS=1
    
  204. Sjors commented at 2:37 pm on September 10, 2024: member

    Reviewed the rest. This PR @ f42e0dde59943ad0c9247c61f7af521b70838e8f seems to be in pretty good shape, but I’m not a low level C guru.

    I didn’t check the changes since my last review of the example, but its usage of labels is now clear to me.

    I didn’t review the tests and benchmarks.

  205. in examples/silentpayments.c:330 in f42e0dde59
    325+                size_t len = 33;
    326+                secp256k1_pubkey label;
    327+                unsigned int m = 1;
    328+
    329+                /* Load Bob's spend public key */
    330+                ret = secp256k1_ec_pubkey_parse(ctx,
    


    jonasnick commented at 1:30 pm on October 11, 2024:
    This ret is unused.
  206. in examples/silentpayments.c:342 in f42e0dde59
    337+                 * called
    338+                 * `secp256k1_silentpayments_recipient_create_labelled_spend_pubkey`
    339+                 * and is using the resulting labelled spend pubkey to encode a
    340+                 * labelled silent payments address.
    341+                 */
    342+                ret = secp256k1_silentpayments_recipient_create_label_tweak(ctx,
    


    jonasnick commented at 1:32 pm on October 11, 2024:
    ret is only checked after ec_pubkey_serialize.
  207. in examples/silentpayments.c:384 in f42e0dde59
    379+            );
    380+            assert(ret);
    381+
    382+            /* Scan the transaction */
    383+            n_found_outputs = 0;
    384+            ret = secp256k1_silentpayments_recipient_scan_outputs(ctx,
    


    jonasnick commented at 1:32 pm on October 11, 2024:
    ret is unchecked.
  208. in examples/silentpayments.c:443 in f42e0dde59
    438+             */
    439+            ret = secp256k1_silentpayments_recipient_public_data_parse(ctx,
    440+                &public_data,
    441+                light_client_data33
    442+            );
    443+            assert(ret);
    


    jonasnick commented at 1:38 pm on October 11, 2024:
    IIUC public_data typically comes from an untrusted source, so I don’t think callers want to assert this. I think there are other places in the code where an assert may mislead readers of the example. For example, when parsing the recipient address. It may also make sense to add a comment to assert(n_found_outputs == 1); explaining that this is only true in this particular test. And btw, I noticed that the number of found outputs is not asserted for Carol.
  209. in examples/silentpayments.c:137 in f42e0dde59
    132+    assert(ret);
    133+
    134+    /*** Sending ***/
    135+    {
    136+        secp256k1_keypair sender_seckeys[N_INPUTS];
    137+        const secp256k1_keypair *sender_seckey_ptrs[N_INPUTS];
    


    jonasnick commented at 1:39 pm on October 11, 2024:
    Why is this called sender_seckeys instead of sender_keypair? Same in the silentpayments.h.
  210. in examples/silentpayments.c:141 in f42e0dde59
    136+        secp256k1_keypair sender_seckeys[N_INPUTS];
    137+        const secp256k1_keypair *sender_seckey_ptrs[N_INPUTS];
    138+        secp256k1_silentpayments_recipient recipients[N_OUTPUTS];
    139+        const secp256k1_silentpayments_recipient *recipient_ptrs[N_OUTPUTS];
    140+        secp256k1_xonly_pubkey generated_outputs[N_OUTPUTS];
    141+        secp256k1_xonly_pubkey *generated_output_ptrs[N_OUTPUTS];
    


    jonasnick commented at 1:41 pm on October 11, 2024:

    nit: Why have a separate generated_outputs when it’s identical to tx_outputs? If

    0        for (i = 0; i < N_OUTPUTS; i++) {
    1            tx_output_ptrs[i] = &tx_outputs[i];
    2        }
    

    from below is moved to the top, then generated_output_ptrs could be replaced by tx_output_ptrs.


    josibake commented at 5:07 pm on November 17, 2024:
    Good point, will simplify to use only tx_outputs.
  211. in examples/silentpayments.c:317 in f42e0dde59
    312+                 *
    313+                 *      _silentpayments_recipient_scan_outputs(..., NULL, NULL);
    314+                 *
    315+                 *  In this case, since Bob has access to the full transaction
    316+                 *  outputs when scanning its easy for him to scan with labels,
    317+                 *  as demonstrated below. For efficient scanning, Bob keeps a
    


    jonasnick commented at 1:57 pm on October 11, 2024:

    nit

    0                 *  In this case, since Bob has access to the full transaction
    1                 *  outputs when scanning, it's easy for him to scan with labels,
    2                 *  as demonstrated below. For efficient scanning, Bob keeps a
    
  212. in examples/silentpayments.c:368 in f42e0dde59
    363+             * and provide these `public_data` objecs, i.e. you don't need to be
    364+             * a silent payments wallet, just someone interested in vending this
    365+             * data to light clients, e.g. a wallet service provider. In our
    366+             * example, Bob is scanning for himself but also sharing this data
    367+             * with light clients.
    368+             */
    


    jonasnick commented at 2:00 pm on October 11, 2024:
     0            /* Bob collects the public data from the transaction inputs and
     1             * creates a `secp256k1_silentpayments_public_data` object. He uses
     2             * this for his own scanning and also serializes the `public_data`
     3             * object to send to light clients. We will use this later for
     4             * Carol, who is scanning as a light client. Note, anyone can create
     5             * and provide these `public_data` objects, i.e. you don't need to be
     6             * a silent payments wallet, just someone interested in vending this
     7             * data to light clients, e.g. a wallet service provider. In our
     8             * example, Bob is scanning for himself but also sharing this data
     9             * with light clients.
    10             */
    

    nit: Also, “vending” sounds a bit strange.

  213. in include/secp256k1_silentpayments.h:13 in f42e0dde59
     8+extern "C" {
     9+#endif
    10+
    11+/* This module provides an implementation for Silent Payments, as specified in
    12+ * BIP352. This particularly involves the creation of input tweak data by
    13+ * summing up private or public keys and the derivation of a shared secret using
    


    jonasnick commented at 2:37 pm on October 11, 2024:
    s/private/secret key for consistency with the rest of the lib
  214. in include/secp256k1_silentpayments.h:126 in f42e0dde59
    121+ *  Out:               label: pointer to the resulting label public key
    122+ *             label_tweak32: pointer to the 32 byte label tweak
    123+ *  In:   recipient_scan_key: pointer to the recipient's scan key
    124+ *                         m: label integer (0 is used for change outputs)
    125+ */
    126+SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_silentpayments_recipient_create_label_tweak(
    


    jonasnick commented at 2:40 pm on October 11, 2024:
    micronit: could be just called secp256k1_silentpayments_recipient_create_label to save typing because it computes both label and tweak.
  215. in include/secp256k1_silentpayments.h:177 in f42e0dde59
    172+typedef struct {
    173+    unsigned char data[98];
    174+} secp256k1_silentpayments_public_data;
    175+
    176+/** Compute Silent Payment public data from input public keys and transaction
    177+ * inputs.
    


    jonasnick commented at 2:41 pm on October 11, 2024:
    0 *  inputs.
    
  216. in include/secp256k1_silentpayments.h:295 in f42e0dde59
    290+    secp256k1_pubkey label;
    291+} secp256k1_silentpayments_found_output;
    292+
    293+/** Scan for Silent Payment transaction outputs.
    294+ *
    295+ *  Given a input public sum, an input_hash, a recipient's spend public key
    


    jonasnick commented at 2:42 pm on October 11, 2024:
    maybe just “public data” would be clearer than “input public sum and input_hash”.
  217. in include/secp256k1_silentpayments.h:319 in f42e0dde59
    314+ *  In:              tx_outputs: pointer to the tx's x-only public key outputs
    315+ *                 n_tx_outputs: the number of tx_outputs being scanned
    316+ *           recipient_scan_key: pointer to the recipient's scan key
    317+ *                  public_data: pointer to the input public key sum
    318+ *                               (optionally, with the `input_hash` multiplied
    319+ *                               in, see `_recipient_public_data_create`).
    


    jonasnick commented at 2:44 pm on October 11, 2024:
    I didn’t find anything in the _recipient_public_data_create doc that would explain this optional multiplication.
  218. in include/secp256k1_silentpayments.h:244 in f42e0dde59
    239+    const secp256k1_silentpayments_public_data *public_data
    240+) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3);
    241+
    242+/** Parse a 33-byte sequence into a silent_payments_public_data object.
    243+ *
    244+ *  Returns: 1 if the data was able to be parsed. 
    


    jonasnick commented at 2:45 pm on October 11, 2024:
    trailing whitespace
  219. in include/secp256k1_silentpayments.h:325 in f42e0dde59
    320+ *       recipient_spend_pubkey: pointer to the recipient's spend pubkey
    321+ *                 label_lookup: pointer to a callback function for looking up
    322+ *                               a label value. This function takes a label
    323+ *                               pubkey as an argument and returns a pointer to
    324+ *                               the label tweak if the label exists, otherwise
    325+ *                               returns a nullptr (NULL if labels are not
    


    jonasnick commented at 2:45 pm on October 11, 2024:
    s/nullptr/NULL pointer would be more consistent.
  220. in include/secp256k1_silentpayments.h:29 in f42e0dde59
    24+ * any wallet software already using libsecp256k1, this API should provide all
    25+ * the functions needed for a Silent Payments implementation without requiring
    26+ * any further elliptic-curve operations from the wallet.
    27+ */
    28+
    29+/* This struct serves as an In param for passing the silent payment address
    


    jonasnick commented at 7:03 am on October 13, 2024:
    0/* This struct serves as an input argument for passing the silent payment address
    

    would be more consistent with secp256k1.h

  221. in include/secp256k1_silentpayments.h:37 in f42e0dde59
    32+ * and used to return the generated outputs matching the original ordering.
    33+ * When more than one recipient is used the recipient array will be sorted in
    34+ * place as part of generating the outputs, but the generated outputs will be
    35+ * returned in the original ordering specified by the index to ensure the
    36+ * caller is able to match up the generated outputs to the correct silent
    37+ * payment address (e.g. to be able to assign the correct amounts to the
    


    jonasnick commented at 7:04 am on October 13, 2024:
    0 * When more than one recipient is used, the recipient array will be sorted in
    1 * place as part of generating the outputs, but the generated outputs will be
    2 * returned in the original ordering specified by the index to ensure the
    3 * caller is able to match up the generated outputs to the correct silent
    4 * payment address (e.g., to be able to assign the correct amounts to the
    
  222. in src/modules/silentpayments/main_impl.h:227 in f42e0dde59
    222+    secp256k1_scalar_mul(&a_sum_scalar, &a_sum_scalar, &input_hash_scalar);
    223+    secp256k1_silentpayments_recipient_sort(ctx, recipients, n_recipients);
    224+    last_recipient = *recipients[0];
    225+    k = 0;
    226+    for (i = 0; i < n_recipients; i++) {
    227+        if ((i == 0) || (secp256k1_ec_pubkey_cmp(ctx, &last_recipient.scan_pubkey, &recipients[i]->scan_pubkey) != 0)) {
    


    Sosthene00 commented at 8:58 pm on October 25, 2024:
    I was a bit worried about that part, because it seems that if we have 3 recipients with 2 scan_pubkeys that are not contiguous, i.e. ABA instead of AAB, we would reset k to 0 when hitting A for the second time instead of correctly incrementing k to 1, resulting in generating the same pubkey twice (assuming identical spend_key of course). I must be missing something though because I can’t make the tests fail with this pattern, it seems that the same (correct) outputs are being generated regardless of the order.

    theStack commented at 9:22 pm on October 25, 2024:
    Note that a few lines above before the loop, the recipients are sorted by scan pubkey (see secp256k1_silentpayments_recipient_sort) in order to avoid the problem you described. The outputs are still returned in the original order, by taking use of the index field of the recipient structure that a user has to set in ascending order (0, 1, 2, …).

    Sosthene00 commented at 8:28 am on October 26, 2024:
    You’re right, nothing to see here then
  223. in examples/silentpayments.c:411 in f42e0dde59
    406+             *
    407+             * Being a light client, Carol likely does not have access to the
    408+             * transaction outputs. This means she will need to first generate
    409+             * an output, check if it exists in the UTXO set (e.g. BIP158 or
    410+             * some other means of querying) and only proceed to check the next
    411+             * output (by incrementing `k`) if the first output exists.
    


    Sosthene00 commented at 9:52 am on October 26, 2024:

    It seems that this description of the light client workflow would only work for users that don’t use labels at all. While we could assume that indeed most light client users wouldn’t use labels, I think we’d rather make a slighter more general example here assuming that Carol also use labels, or if we don’t want to make the example more complicated at the very least explain it here in comments. Here’s a quick rewriting of this paragraph based on how we made it work for Dana:

     0/** Being a light client, Carol likely does not have access to the
     1* transaction outputs. This means she will need to first generate
     2* the first outputs she would get in a transaction, meaning the output 
     3* from her plain spend_pubkey + one for each label she monitors 
     4* with _k_ == 0 , and check if it exists in the UTXO set (e.g. BIP158 or 
     5* some other means of querying).
     6*  
     7* As soon as she finds an output she can request the whole transaction
     8* (or rather the whole block to not reveal which transactions 
     9* she's interested in) and use the same function than Bob from there 
    10* to find any other output that belongs to her.
    11*  
    12* She would then repeat this operation for each `public_data` she received
    13*/
    

    Alternatively I guess she could also increment k when she has a match and keep looking for outputs this way, but she needs to generate the k+1 output for each label also. Maybe that would indeed make more sense if we go for UTXO set scanning instead of scanning each block sequentially.


    josibake commented at 5:34 pm on November 17, 2024:
    This is a good call out. I added a sentence to indicate that, while its not recommended, Carol can still uses labels as a light client using the addition method.

    josibake commented at 5:35 pm on November 17, 2024:
    I think another reason I didn’t include light client labels is because this would require a function for adding arbitrary public keys, which at this point we are trying to avoid.

    Sosthene00 commented at 8:02 am on November 28, 2024:
    Yes I agree it’s kind of an edge case and we want to be careful not to compromise too much because of it, but on the other hand we should acknowledge it because it will definitely happen I think. I’ll try to complete the tests on that part and see how bad it makes things
  224. in include/secp256k1_silentpayments.h:63 in 9d6769f4ae outdated
    58+ *  If necessary, the private keys are negated to enforce the right y-parity.
    59+ *  For that reason, the private keys have to be passed in via two different
    60+ *  parameter pairs, depending on whether the seckeys correspond to x-only
    61+ *  outputs or not.
    62+ *
    63+ *  Returns: 1 if creation of outputs was successful. 0 if an error occured.
    


    stratospher commented at 5:51 am on November 1, 2024:
    9d6769f: typo nit: s/occured/occurred here and in few more places in the file
  225. josibake commented at 1:27 pm on November 7, 2024: member
    Big thanks to all the review over the last few months and apologies for my slow response. I’m in the process of rebasing and incorporating review feedback, should have the PR updated shortly.
  226. in src/modules/silentpayments/main_impl.h:122 in 9d6769f4ae outdated
    119+    secp256k1_scalar_set_b32(t_k_scalar, hash_ser, NULL);
    120+    /* While not technically "secret" data, explicitly clear hash_ser since leaking this would allow an attacker
    121+     * to identify the resulting transaction as a silent payments transaction and potentially link the transaction
    122+     * back to the silent payment address
    123+     */
    124+    memset(hash_ser, 0, sizeof(hash_ser));
    


    theuni commented at 6:36 pm on November 12, 2024:

    Note that this will probably be a noop. See “notes” here: https://en.cppreference.com/w/c/string/byte/memset

    secp has secure_erase in examples. Maybe that could be brought out? Surely there are other places where a cleanse is necessary?


    theStack commented at 6:48 pm on November 12, 2024:
    A memory cleanse function secp256k1_memclear is available since the recently merged #1579 (using the same memory-barrier-approach as in Bitcoin Core) and should be used for that purpose.

    theuni commented at 8:09 pm on November 12, 2024:
    Aha, perfect, thanks! I was grepping on an old branch and missed this.
  227. in src/modules/silentpayments/main_impl.h:106 in 9d6769f4ae outdated
    103+    hash->s[7] = 0xf25e5e0aul;
    104+
    105+    hash->bytes = 64;
    106+}
    107+
    108+static void secp256k1_silentpayments_create_t_k(secp256k1_scalar *t_k_scalar, const unsigned char *shared_secret33, unsigned int k) {
    


    stratospher commented at 12:57 pm on December 2, 2024:
    9d6769f: micro nit: might make sense to add const qualifier to the int too. also for the int in other functions - secp256k1_silentpayments_recipient_create_output_pubkey, secp256k1_silentpayments_recipient_create_label_tweak, secp256k1_silentpayments_create_output_pubkey
  228. in src/modules/silentpayments/main_impl.h:221 in 9d6769f4ae outdated
    218+    secp256k1_ge_set_gej(&A_sum_ge, &A_sum_gej);
    219+
    220+    /* Calculate the input hash and tweak a_sum, i.e., a_sum_tweaked = a_sum * input_hash */
    221+    secp256k1_silentpayments_calculate_input_hash(input_hash, outpoint_smallest36, &A_sum_ge);
    222+    secp256k1_scalar_set_b32(&input_hash_scalar, input_hash, &overflow);
    223+    ret &= !overflow;
    


    stratospher commented at 8:59 am on December 3, 2024:
    9d6769f: BIP sounds like it ignores overflow in https://github.com/bitcoin/bips/blob/master/bip-0352.mediawiki#creating-outputs. though everywhere in the code input_hash overflow would lead to failure. It’s a very unlikely scenario and not sure which option is preferable, but would be nice to keep BIP and code consistent.
  229. in src/modules/silentpayments/main_impl.h:117 in 9d6769f4ae outdated
    114+    secp256k1_silentpayments_sha256_init_sharedsecret(&hash);
    115+    secp256k1_sha256_write(&hash, shared_secret33, 33);
    116+    secp256k1_write_be32(k_serialized, k);
    117+    secp256k1_sha256_write(&hash, k_serialized, sizeof(k_serialized));
    118+    secp256k1_sha256_finalize(&hash, hash_ser);
    119+    secp256k1_scalar_set_b32(t_k_scalar, hash_ser, NULL);
    


    stratospher commented at 9:00 am on December 3, 2024:

    9d6769f: BIP says to fail If t_k is not valid tweak - https://github.com/bitcoin/bips/blob/master/bip-0352.mediawiki#creating-outputs here, we’re ignoring the overflow. we could keep it consistent with the BIP?

    (have a similar comment in call site of secp256k1_silentpayments_create_t_k)

  230. in src/modules/silentpayments/main_impl.h:408 in 94c6e1f542 outdated
    403+
    404+    VERIFY_CHECK(ctx != NULL);
    405+    ARG_CHECK(output33 != NULL);
    406+    ARG_CHECK(public_data != NULL);
    407+    /* Only allow public_data to be serialized if it has the hash and the summed public key
    408+     * This helps protect against accidentally serialiazing just a the summed public key A
    


    stratospher commented at 8:52 am on December 5, 2024:

    94c6e1f: typo nit:

    0     * This helps protect against accidentally serializing just the summed public key A
    
  231. in src/modules/silentpayments/main_impl.h:414 in 94c6e1f542 outdated
    409+     */
    410+    ARG_CHECK(public_data->data[0] == 0);
    411+    ret &= secp256k1_silentpayments_recipient_public_data_load_pubkey(ctx, &pubkey, public_data);
    412+    secp256k1_silentpayments_recipient_public_data_load_input_hash(input_hash, public_data);
    413+    ret &= secp256k1_ec_pubkey_tweak_mul(ctx, &pubkey, input_hash);
    414+    secp256k1_ec_pubkey_serialize(ctx, output33, &pubkeylen, &pubkey, SECP256K1_EC_COMPRESSED);
    


    stratospher commented at 9:01 am on December 5, 2024:
    94c6e1f: micro nit: could add VERIFY_CHECK(pubkeylen == 33)
  232. in src/modules/silentpayments/tests_impl.h:308 in 94c6e1f542 outdated
    303+
    304+    /* Check that malformed serializations are rejected */
    305+    CHECK(secp256k1_silentpayments_recipient_public_data_parse(CTX, &pd, malformed) == 0);
    306+
    307+    /* This public_data object was created with combined = 0, i.e., it has both the input hash and summed public keypair.
    308+     * In instances where the caller has access the the full transaction, they should use `_scan_outputs` instead, so
    


    stratospher commented at 12:41 pm on December 5, 2024:

    94c6e1f:

    0     * In instances where the caller has access to the full transaction, they should use `_scan_outputs` instead, so
    
  233. in src/modules/silentpayments/main_impl.h:483 in 94c6e1f542 outdated
    478+    /* If there is something wrong with the recipient scan key, recipient spend pubkey, or the public data, return early */
    479+    if (!ret) {
    480+        return 0;
    481+    }
    482+    combined = (int)public_data->data[0];
    483+    if (!combined) {
    


    stratospher commented at 1:47 pm on December 8, 2024:

    94c6e1f: I think comments here would be useful! (thanks @theStack for explaining the intuition behind this!)

    maybe something like:

    0/**
    1    * If combined is set, pubkey in `public_data` contains input_hash * pubkey. Save an extra point multiplication
    2    * by only having to compute shared_secret = recipient_scan_key * (input_hash * pubkey).
    3    * 
    4    * If combined is not set, update recipient_scan_key to contain recipient_scan_key * input_hash and then compute
    5    * shared_secret = (recipient_scan_key * input_hash) * pubkey.
    6*/
    

    EDIT: part I got confused with was I thought everyone could do (recipient_scan_key * input_hash) but light clients wouldn’t be able to since they don’t have access to summed pubkey, input_hash(computed from summed pubkey). would have liked to see that part in the comments but also think it doesn’t fit here since it’s bitcoin specific.

  234. in src/modules/silentpayments/main_impl.h:228 in 9d6769f4ae outdated
    225+    secp256k1_silentpayments_recipient_sort(ctx, recipients, n_recipients);
    226+    last_recipient = *recipients[0];
    227+    k = 0;
    228+    for (i = 0; i < n_recipients; i++) {
    229+        if ((i == 0) || (secp256k1_ec_pubkey_cmp(ctx, &last_recipient.scan_pubkey, &recipients[i]->scan_pubkey) != 0)) {
    230+            /* If we are on a different scan pubkey, its time to recreate the the shared secret and reset k to 0.
    


    stratospher commented at 1:21 pm on December 9, 2024:

    9d6769f:

    0            /* If we are on a different scan pubkey, its time to recreate the shared secret and reset k to 0.
    
  235. in src/modules/silentpayments/main_impl.h:193 in 9d6769f4ae outdated
    190+        ARG_CHECK(recipients[i]->index == i);
    191+    }
    192+
    193+    /* Compute input private keys sum: a_sum = a_1 + a_2 + ... + a_n */
    194+    a_sum_scalar = secp256k1_scalar_zero;
    195+    for (i = 0; i < n_plain_seckeys; i++) {
    


    Eunovo commented at 9:56 am on December 16, 2024:
    How should we handle secret keys that add up to SECP256k1_N? I tested this locally with ALICE_SECKKEY = 0xeadc78165ff1f8ea94ad7cfdc54990738a4c53f6e0507b42154201b8e5dff3b1 and another secret key I generated with SECP256K1_N - ALICE_SECKEY = 0x152387e9a00e07156b5283023ab66f8b306288efcef824f9aa905cd3ea564d90. Passing these two plain secret keys causes secp256k1_silentpayments_sender_create_outputs to fail

    Eunovo commented at 4:48 am on December 20, 2024:
    I had an offline conversation with @josibake. The conclusion is that there is nothing to do. The sender has to try again with a different set of pubkeys

    Eunovo commented at 4:31 pm on December 20, 2024:

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin-core/secp256k1. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-21 13:15 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me