test: add BIP352 scanning to the functional test framework #36835

pull rustaceanrob wants to merge 2 commits into bitcoin:master from rustaceanrob:sp-recv-functional changing 4 files +135 −0
  1. rustaceanrob commented at 1:45 PM on November 11, 2026: member

    Picked from #35302 as an isolated change that can be evaluated independently. This should make #35302 a bit easier to review. This adds a Python implementation of BIP352 scanning, without labels, so the functional tests can check silent payments outputs independently of bitcoind support. It is tested against the BIP352 receiving test vectors.

  2. DrahtBot added the label Tests on Nov 11, 2026
  3. DrahtBot commented at 1:46 PM on November 11, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK theStack, ismaelsadeeq, Eunovo

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #35302 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/35302.svg"></sub> (Silent Payments: Sending (take 2) by Eunovo)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. in test/functional/test_framework/bip352.py:53 in c24ff83c38
      48 | +            if hash160(pubkey) == prevout_spk[3:23]:
      49 | +                return GE.from_bytes(pubkey)
      50 | +    return None
      51 | +
      52 | +
      53 | +def is_witness_v2_or_higher(spk: bytes) -> bool:
    


    ismaelsadeeq commented at 1:25 PM on November 12, 2026:

    nit: could add a small comment here, explaining the function?


    rustaceanrob commented at 8:37 PM on November 12, 2026:

    Moved to fdfd7467cf77bdbea4d2c8bfdff137172006834a with a description

  5. in test/functional/test_framework/bip352.py:54 in c24ff83c38
      49 | +                return GE.from_bytes(pubkey)
      50 | +    return None
      51 | +
      52 | +
      53 | +def is_witness_v2_or_higher(spk: bytes) -> bool:
      54 | +    return 4 <= len(spk) <= 42 and 0x52 <= spk[0] <= 0x60 and spk[1] == len(spk) - 2
    


    ismaelsadeeq commented at 1:25 PM on November 12, 2026:

    too many magic nums here and other places


    rustaceanrob commented at 2:22 PM on November 12, 2026:

    re #36835 (review)

    I think this suggestion makes sense, so the magic numbers and such can move to script_util.py

  6. in test/functional/test_framework/bip352.py:90 in c24ff83c38 outdated
      85 | +    """Scan a transaction, looking up its prevouts in the UTXO set of node, or with getrawtransaction."""
      86 | +    tx = from_hex(CTransaction(), tx_hex)
      87 | +    inputs = []
      88 | +    for i, txin in enumerate(tx.vin):
      89 | +        txid, n = f"{txin.prevout.hash:064x}", txin.prevout.n
      90 | +        prevout = node.gettxout(txid, n, include_mempool=False) or node.getrawtransaction(txid, True)["vout"][n]
    


    ismaelsadeeq commented at 1:42 PM on November 12, 2026:

    Isn't this an issue when the output is in a transaction that is confirmed? getrawtransaction will return confirmed tx too when tx index is enabled.

    I can include an old payment, and you may think it's a new payment during this scan, so you want to check the utxo set and getmempool entry or gate and only return the output when getmempool entry also returns the tx?


    rustaceanrob commented at 2:26 PM on November 12, 2026:

    getrawtransaction will return confirmed tx too

    This is by design. We check for confirmed payments here as well, whose inputs will inherently not be in the UTXO set. See this commit for the use cases. FWIW, a "real" scanner would use the undo data and block rather than -txindex and getrawtransaction.

    Side note related to thsee calls, either we assume -txindex is enabled as is done here, or the user has to pass a block hash they expect the payment to be in, which would be cumbersome for writing tests IMO.

  7. in test/functional/test_framework/bip352.py:114 in c24ff83c38
     109 | +                    ) for vin in given["vin"]]
     110 | +                    output_spks = [bytes.fromhex("5120" + output) for output in given["outputs"]]
     111 | +                    spend_privkey = int(given["key_material"]["spend_priv_key"], 16)
     112 | +                    found = scan(inputs, output_spks, bytes.fromhex(given["key_material"]["scan_priv_key"]),
     113 | +                                 (spend_privkey * G).to_bytes_compressed())
     114 | +                    expected = [{"output": output["pub_key"], "tweak": output["priv_key_tweak"]} for output in receiving["expected"]["outputs"]]
    


    ismaelsadeeq commented at 1:43 PM on November 12, 2026:

    nit: too many nested loops; can u simplify? It would make it easy to follow.


    rustaceanrob commented at 2:06 PM on November 12, 2026:

    I don't think extracting into a separate function would help much for readability


    rustaceanrob commented at 8:16 PM on November 12, 2026:

    It did a bit. I added a function that creates an iterator for the receiving test vectors after parsing the json.


    ismaelsadeeq commented at 2:52 PM on November 17, 2026:

    Thanks, this reads much better now.

  8. ismaelsadeeq commented at 1:43 PM on November 12, 2026: member

    Concept ACK, well scoped change.

    I did a brief review, see comments below

  9. in test/functional/test_framework/bip352.py:30 in c24ff83c38
      25 | +ANNEX_TAG = b"\x50"
      26 | +
      27 | +
      28 | +def get_input_pubkey(script_sig: bytes, witness: list[bytes], prevout_spk: bytes) -> GE | None:
      29 | +    """Return the public key of an input as a group element, or None if the input is not eligible."""
      30 | +    if len(prevout_spk) == 34 and prevout_spk[:2] == P2TR_PREFIX:
    


    theStack commented at 1:44 PM on November 12, 2026:

    non-blocking suggestion: I wonder if introducing is_{p2tr,p2pkh...}_script helpers (or a Solver equivalent) in script_util.py would make sense to keep the output script matching logic out of this file, kind of surprised that we don't have that already in the functional test framework


    rustaceanrob commented at 8:17 PM on November 12, 2026:

    Added fdfd7467cf77bdbea4d2c8bfdff137172006834a with some is_p2* utilities and a witness version function

  10. theStack commented at 1:53 PM on November 12, 2026: contributor

    Concept ACK

  11. rustaceanrob force-pushed on Nov 12, 2026
  12. rustaceanrob force-pushed on Nov 12, 2026
  13. DrahtBot added the label CI failed on Nov 12, 2026
  14. DrahtBot commented at 8:21 PM on November 12, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task lint: https://github.com/bitcoin/bitcoin/actions/runs/43298469093/job/131462961033</sub> <sub>LLM reason (✨ experimental): CI failed in Python lint: bip352.py:35 passes an int to is_p2wpkh, which expects bytes.</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  15. test: Add `is_p2*` and witness version `script_util`
    The following commit inspects byte arrays as script, but these could be
    broadly useful elsewhere.
    fdfd7467cf
  16. rustaceanrob force-pushed on Nov 12, 2026
  17. DrahtBot removed the label CI failed on Nov 12, 2026
  18. in test/functional/test_framework/bip352.py:63 in 6af1669814 outdated
      58 | +    input_hash = int.from_bytes(TaggedHash("BIP0352/Inputs", smallest_outpoint + A.to_bytes_compressed()), "big")
      59 | +    ecdh_shared_secret = (input_hash * int.from_bytes(scan_privkey, "big") % ORDER) * A
      60 | +    B_spend = GE.from_bytes(spend_pubkey)
      61 | +    outputs = {spk[2:] for spk in output_spks if is_p2tr_script(spk)}
      62 | +    found = []
      63 | +    for k in range(K_MAX):
    


    bartoli commented at 9:51 PM on November 12, 2026:

    Shouldn't we loop on outputs and see if they match an output for the current (increasing) k instead? Here, there is a very minimal, but non null, chance that there is a k that matches, but is not actually a sp output?

    OR, if we keep the loop in that way because it is 'safe enough', then maybe we should at least stop the range at min(K_MAX, len(outputs)) to save on scan time?


    rustaceanrob commented at 7:32 AM on November 13, 2026:

    Shouldn't we loop on outputs and see if they match an output for the current (increasing) k instead?

    This matches the BIP.

    then maybe we should at least stop the range at min(K_MAX, len(outputs)) to save on scan time?

    k = 0 and no found outputs exits the loop.


    bartoli commented at 7:56 AM on November 13, 2026:

    Oh you're right. I read the loop assuming the function would have to scan for potentially multiple scan keys of multiple sp receive addresses of the wallet. Now i understand.

  19. rustaceanrob force-pushed on Nov 16, 2026
  20. rustaceanrob commented at 12:32 PM on November 16, 2026: member

    6ad35e8686075cdc51ebb8ff3857ea50e13325fb added the make_sp_addr function from #35302 along with tests. Also included a short note on -txindex in the scan_tx function.

  21. in test/functional/test_framework/bip352.py:68 in 6ad35e8686
      63 | +        return []
      64 | +    smallest_outpoint = min(outpoint.serialize() for outpoint, *_ in inputs)
      65 | +    input_hash = int.from_bytes(TaggedHash("BIP0352/Inputs", smallest_outpoint + A.to_bytes_compressed()), "big")
      66 | +    ecdh_shared_secret = (input_hash * int.from_bytes(scan_privkey, "big") % ORDER) * A
      67 | +    B_spend = GE.from_bytes(spend_pubkey)
      68 | +    outputs = {spk[2:] for spk in output_spks if is_p2tr_script(spk)}
    


    Eunovo commented at 10:49 AM on November 17, 2026:
  22. in test/functional/test_framework/bip352.py:88 in 6ad35e8686 outdated
      83 | +    """
      84 | +    tx = from_hex(CTransaction(), tx_hex)
      85 | +    inputs = []
      86 | +    for i, txin in enumerate(tx.vin):
      87 | +        txid, n = f"{txin.prevout.hash:064x}", txin.prevout.n
      88 | +        prevout = node.gettxout(txid, n, include_mempool=False) or node.getrawtransaction(txid, True)["vout"][n]
    


    Eunovo commented at 11:29 AM on November 17, 2026:

    https://github.com/bitcoin/bitcoin/pull/36835/commits/6ad35e8686075cdc51ebb8ff3857ea50e13325fb:

    When a Coinbase TX is supplied, txid will be 000...00, and getrawtransaction will fail with test_framework.util.JSONRPCException: No such mempool or blockchain transaction. Use gettransaction for wallet transactions. (-5).

    Not sure what approach is most appropriate here; on the one hand, a user shouldn't try to scan a coinbase tx, and maybe the code should crash and force the user to adjust the test, but I also see this happening when a user tries to scan a block, and an argument could be made that the function should simply skip a coinbase tx the way the production scanning function would.

  23. Eunovo approved
  24. Eunovo commented at 11:29 AM on November 17, 2026: contributor
  25. DrahtBot requested review from ismaelsadeeq on Nov 17, 2026
  26. DrahtBot requested review from theStack on Nov 17, 2026
  27. rustaceanrob force-pushed on Nov 17, 2026
  28. rustaceanrob commented at 1:09 PM on November 17, 2026: member

    c4580eae5817a22d3630f996ff5455b9f7b23581 skips coinbases, per the BIP, and also adds the early return in the case there are no PT2R outputs in the transaction

  29. in test/functional/test_framework/bip352.py:81 in c4580eae58
      76 | +            break
      77 | +        found.append({"output": P_k.hex(), "tweak": tweak.hex()})
      78 | +    return found
      79 | +
      80 | +
      81 | +def scan_tx(node: TestNode, tx_hex: str, scan_privkey: bytes, spend_pubkey: bytes) -> list[dict[str, str]]:
    


    ismaelsadeeq commented at 2:50 PM on November 17, 2026:

    scan_tx is dead for now; we are not sure it compiles in C.I, and it does not seem like a utility because it needs state and a -txindex requirement. Probably fits better as a helper in the actual test?


    rustaceanrob commented at 2:56 PM on November 17, 2026:

    Sure, we can have this module be self tested entirely. I don't think #35302 uses this in multiple files anyway.

  30. DrahtBot requested review from ismaelsadeeq on Nov 17, 2026
  31. rustaceanrob force-pushed on Nov 17, 2026
  32. test: add BIP352 scanning to the functional test framework
    Add a Python implementation of BIP352 scanning, without labels, so the
    functional tests can check silent payments outputs independently of the
    node. It is tested against the BIP352 receiving test vectors.
    
    Co-authored-by: rustaceanrob <rob.netzke@gmail.com>
    cd6d7e240b
  33. rustaceanrob force-pushed on Nov 17, 2026
  34. DrahtBot added the label CI failed on Nov 17, 2026
  35. DrahtBot commented at 3:09 PM on November 17, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task lint: https://github.com/bitcoin/bitcoin/actions/runs/44109510291/job/134130936328</sub> <sub>LLM reason (✨ experimental): CI failed because Python linting found three unused imports in test/functional/test_framework/bip352.py (ruff F401 errors).</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  36. in test/functional/test_framework/bip352.py:25 in cd6d7e240b
      20 | +
      21 | +
      22 | +def make_sp_addr(scan_key: ECKey, spend_key: ECKey, hrp: str = "sprt") -> str:
      23 | +    """Encode an SP address from two ECKey objects (scan and spend)."""
      24 | +    payload = scan_key.get_pubkey().get_bytes() + spend_key.get_pubkey().get_bytes()
      25 | +    return bech32_encode(Encoding.BECH32M, hrp, [0] + convertbits(list(payload), 8, 5))
    


    theStack commented at 3:28 PM on November 17, 2026:

    nit: could consider moving this function (and the decoding counterpart) to address.py in the future

  37. theStack approved
  38. theStack commented at 3:30 PM on November 17, 2026: contributor

    ACK cd6d7e240bb22e5d9515bd8cf8dbc081ff471dc8

    I agree that deferring the introduction of scan_tx to another PR (likely #35302) makes sense, given that it it's currently unused/untested.

  39. DrahtBot requested review from Eunovo on Nov 17, 2026
  40. in test/functional/test_framework/bip352.py:19 in cd6d7e240b
      14 | +from .messages import COutPoint, deser_string_vector
      15 | +from .script import ANNEX_TAG, CScript, hash160
      16 | +from .script_util import is_p2pkh_script, is_p2sh_script, is_p2tr_script, is_p2wpkh_script, script_witness_version
      17 | +from .segwit_addr import bech32_encode, convertbits, Encoding
      18 | +
      19 | +K_MAX = 2323
    


    ismaelsadeeq commented at 3:34 PM on November 17, 2026:

    nit: reference the bip here.

  41. DrahtBot requested review from ismaelsadeeq on Nov 17, 2026
  42. ismaelsadeeq approved
  43. ismaelsadeeq commented at 3:35 PM on November 17, 2026: member

    ACK cd6d7e240bb22e5d9515bd8cf8dbc081ff471dc8

  44. DrahtBot removed the label CI failed on Nov 17, 2026

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-18 16:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me