Picked from #35302 as an isolated change that can be evaluated independently. This should make #35302 a bit easier to review. This adds a Python implementation of BIP352 scanning, without labels, so the functional tests can check silent payments outputs independently of bitcoind support. It is tested against the BIP352 receiving test vectors.
test: add BIP352 scanning to the functional test framework #36835
pull rustaceanrob wants to merge 2 commits into bitcoin:master from rustaceanrob:sp-recv-functional changing 4 files +135 −0-
rustaceanrob commented at 1:45 PM on November 11, 2026: member
- DrahtBot added the label Tests on Nov 11, 2026
-
DrahtBot commented at 1:46 PM on November 11, 2026: contributor
<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--006a51241073e994b41acfe9ec718e94-->
External sites
- Code coverage & benchmarks: https://corecheck.dev/bitcoin/bitcoin/pulls/36835
- Ralph review <img src=https://review.fish.foo/assets/bitcoin/bitcoin/36835.png alt="Ralph review status"> https://git.fish.foo/bitcoin/bitcoin/pulls/36835#author-ralph
<!--021abf342d371248e50ceaed478a90ca-->
Reviews
See the guideline and AI policy for information on the review process.
Type Reviewers ACK theStack, ismaelsadeeq, Eunovo If your review is incorrectly listed, please copy-paste <code><!--meta-tag:bot-skip--></code> into the comment that the bot should ignore.
<!--174a7506f384e20aa4161008e828411d-->
Conflicts
Reviewers, this pull request conflicts with the following ones:
- #35302 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/35302.svg"></sub> (Silent Payments: Sending (take 2) by Eunovo)
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
<!--5faf32d7da4f0f540f40219e4f7537a3-->
-
in test/functional/test_framework/bip352.py:53 in c24ff83c38
48 | + if hash160(pubkey) == prevout_spk[3:23]: 49 | + return GE.from_bytes(pubkey) 50 | + return None 51 | + 52 | + 53 | +def is_witness_v2_or_higher(spk: bytes) -> bool:
ismaelsadeeq commented at 1:25 PM on November 12, 2026:nit: could add a small comment here, explaining the function?
rustaceanrob commented at 8:37 PM on November 12, 2026:Moved to fdfd7467cf77bdbea4d2c8bfdff137172006834a with a description
in test/functional/test_framework/bip352.py:54 in c24ff83c38
49 | + return GE.from_bytes(pubkey) 50 | + return None 51 | + 52 | + 53 | +def is_witness_v2_or_higher(spk: bytes) -> bool: 54 | + return 4 <= len(spk) <= 42 and 0x52 <= spk[0] <= 0x60 and spk[1] == len(spk) - 2
ismaelsadeeq commented at 1:25 PM on November 12, 2026:too many magic nums here and other places
rustaceanrob commented at 2:22 PM on November 12, 2026:I think this suggestion makes sense, so the magic numbers and such can move to
script_util.pyin test/functional/test_framework/bip352.py:90 in c24ff83c38 outdated
85 | + """Scan a transaction, looking up its prevouts in the UTXO set of node, or with getrawtransaction.""" 86 | + tx = from_hex(CTransaction(), tx_hex) 87 | + inputs = [] 88 | + for i, txin in enumerate(tx.vin): 89 | + txid, n = f"{txin.prevout.hash:064x}", txin.prevout.n 90 | + prevout = node.gettxout(txid, n, include_mempool=False) or node.getrawtransaction(txid, True)["vout"][n]
ismaelsadeeq commented at 1:42 PM on November 12, 2026:Isn't this an issue when the output is in a transaction that is confirmed? getrawtransaction will return confirmed tx too when tx index is enabled.
I can include an old payment, and you may think it's a new payment during this scan, so you want to check the utxo set and getmempool entry or gate and only return the output when getmempool entry also returns the tx?
rustaceanrob commented at 2:26 PM on November 12, 2026:getrawtransaction will return confirmed tx too
This is by design. We check for confirmed payments here as well, whose inputs will inherently not be in the UTXO set. See this commit for the use cases. FWIW, a "real" scanner would use the undo data and block rather than
-txindexandgetrawtransaction.Side note related to thsee calls, either we assume
-txindexis enabled as is done here, or the user has to pass a block hash they expect the payment to be in, which would be cumbersome for writing tests IMO.in test/functional/test_framework/bip352.py:114 in c24ff83c38
109 | + ) for vin in given["vin"]] 110 | + output_spks = [bytes.fromhex("5120" + output) for output in given["outputs"]] 111 | + spend_privkey = int(given["key_material"]["spend_priv_key"], 16) 112 | + found = scan(inputs, output_spks, bytes.fromhex(given["key_material"]["scan_priv_key"]), 113 | + (spend_privkey * G).to_bytes_compressed()) 114 | + expected = [{"output": output["pub_key"], "tweak": output["priv_key_tweak"]} for output in receiving["expected"]["outputs"]]
ismaelsadeeq commented at 1:43 PM on November 12, 2026:nit: too many nested loops; can u simplify? It would make it easy to follow.
rustaceanrob commented at 2:06 PM on November 12, 2026:I don't think extracting into a separate function would help much for readability
rustaceanrob commented at 8:16 PM on November 12, 2026:It did a bit. I added a function that creates an iterator for the receiving test vectors after parsing the json.
ismaelsadeeq commented at 2:52 PM on November 17, 2026:Thanks, this reads much better now.
ismaelsadeeq commented at 1:43 PM on November 12, 2026: memberConcept ACK, well scoped change.
I did a brief review, see comments below
in test/functional/test_framework/bip352.py:30 in c24ff83c38
25 | +ANNEX_TAG = b"\x50" 26 | + 27 | + 28 | +def get_input_pubkey(script_sig: bytes, witness: list[bytes], prevout_spk: bytes) -> GE | None: 29 | + """Return the public key of an input as a group element, or None if the input is not eligible.""" 30 | + if len(prevout_spk) == 34 and prevout_spk[:2] == P2TR_PREFIX:
theStack commented at 1:44 PM on November 12, 2026:non-blocking suggestion: I wonder if introducing
is_{p2tr,p2pkh...}_scripthelpers (or aSolverequivalent) inscript_util.pywould make sense to keep the output script matching logic out of this file, kind of surprised that we don't have that already in the functional test framework
rustaceanrob commented at 8:17 PM on November 12, 2026:Added fdfd7467cf77bdbea4d2c8bfdff137172006834a with some
is_p2*utilities and a witness version functiontheStack commented at 1:53 PM on November 12, 2026: contributorConcept ACK
rustaceanrob force-pushed on Nov 12, 2026rustaceanrob force-pushed on Nov 12, 2026DrahtBot added the label CI failed on Nov 12, 2026DrahtBot commented at 8:21 PM on November 12, 2026: contributor<!--85328a0da195eb286784d51f73fa0af9-->
🚧 At least one of the CI tasks failed. <sub>Task
lint: https://github.com/bitcoin/bitcoin/actions/runs/43298469093/job/131462961033</sub> <sub>LLM reason (✨ experimental): CI failed in Python lint:bip352.py:35passes aninttois_p2wpkh, which expectsbytes.</sub><details><summary>Hints</summary>
Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:
Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.
Leave a comment here, if you need help tracking down a confusing failure.
</details>
fdfd7467cftest: Add `is_p2*` and witness version `script_util`
The following commit inspects byte arrays as script, but these could be broadly useful elsewhere.
rustaceanrob force-pushed on Nov 12, 2026DrahtBot removed the label CI failed on Nov 12, 2026in test/functional/test_framework/bip352.py:63 in 6af1669814 outdated
58 | + input_hash = int.from_bytes(TaggedHash("BIP0352/Inputs", smallest_outpoint + A.to_bytes_compressed()), "big") 59 | + ecdh_shared_secret = (input_hash * int.from_bytes(scan_privkey, "big") % ORDER) * A 60 | + B_spend = GE.from_bytes(spend_pubkey) 61 | + outputs = {spk[2:] for spk in output_spks if is_p2tr_script(spk)} 62 | + found = [] 63 | + for k in range(K_MAX):
bartoli commented at 9:51 PM on November 12, 2026:Shouldn't we loop on outputs and see if they match an output for the current (increasing) k instead? Here, there is a very minimal, but non null, chance that there is a k that matches, but is not actually a sp output?
OR, if we keep the loop in that way because it is 'safe enough', then maybe we should at least stop the range at min(K_MAX, len(outputs)) to save on scan time?
rustaceanrob commented at 7:32 AM on November 13, 2026:Shouldn't we loop on outputs and see if they match an output for the current (increasing) k instead?
This matches the BIP.
then maybe we should at least stop the range at min(K_MAX, len(outputs)) to save on scan time?
k = 0and no found outputs exits the loop.
bartoli commented at 7:56 AM on November 13, 2026:Oh you're right. I read the loop assuming the function would have to scan for potentially multiple scan keys of multiple sp receive addresses of the wallet. Now i understand.
rustaceanrob force-pushed on Nov 16, 2026rustaceanrob commented at 12:32 PM on November 16, 2026: member6ad35e8686075cdc51ebb8ff3857ea50e13325fb added the
make_sp_addrfunction from #35302 along with tests. Also included a short note on-txindexin thescan_txfunction.in test/functional/test_framework/bip352.py:68 in 6ad35e8686
63 | + return [] 64 | + smallest_outpoint = min(outpoint.serialize() for outpoint, *_ in inputs) 65 | + input_hash = int.from_bytes(TaggedHash("BIP0352/Inputs", smallest_outpoint + A.to_bytes_compressed()), "big") 66 | + ecdh_shared_secret = (input_hash * int.from_bytes(scan_privkey, "big") % ORDER) * A 67 | + B_spend = GE.from_bytes(spend_pubkey) 68 | + outputs = {spk[2:] for spk in output_spks if is_p2tr_script(spk)}
Eunovo commented at 10:49 AM on November 17, 2026:https://github.com/bitcoin/bitcoin/pull/36835/commits/6ad35e8686075cdc51ebb8ff3857ea50e13325fb:
nit: Can return early if
outputsis empty.in test/functional/test_framework/bip352.py:88 in 6ad35e8686 outdated
83 | + """ 84 | + tx = from_hex(CTransaction(), tx_hex) 85 | + inputs = [] 86 | + for i, txin in enumerate(tx.vin): 87 | + txid, n = f"{txin.prevout.hash:064x}", txin.prevout.n 88 | + prevout = node.gettxout(txid, n, include_mempool=False) or node.getrawtransaction(txid, True)["vout"][n]
Eunovo commented at 11:29 AM on November 17, 2026:https://github.com/bitcoin/bitcoin/pull/36835/commits/6ad35e8686075cdc51ebb8ff3857ea50e13325fb:
When a Coinbase TX is supplied, txid will be
000...00, andgetrawtransactionwill fail withtest_framework.util.JSONRPCException: No such mempool or blockchain transaction. Use gettransaction for wallet transactions. (-5).Not sure what approach is most appropriate here; on the one hand, a user shouldn't try to scan a coinbase tx, and maybe the code should crash and force the user to adjust the test, but I also see this happening when a user tries to scan a block, and an argument could be made that the function should simply skip a coinbase tx the way the production scanning function would.
Eunovo approvedEunovo commented at 11:29 AM on November 17, 2026: contributorDrahtBot requested review from ismaelsadeeq on Nov 17, 2026DrahtBot requested review from theStack on Nov 17, 2026rustaceanrob force-pushed on Nov 17, 2026rustaceanrob commented at 1:09 PM on November 17, 2026: memberc4580eae5817a22d3630f996ff5455b9f7b23581 skips coinbases, per the BIP, and also adds the early return in the case there are no PT2R outputs in the transaction
Eunovo commented at 1:37 PM on November 17, 2026: contributorin test/functional/test_framework/bip352.py:81 in c4580eae58
76 | + break 77 | + found.append({"output": P_k.hex(), "tweak": tweak.hex()}) 78 | + return found 79 | + 80 | + 81 | +def scan_tx(node: TestNode, tx_hex: str, scan_privkey: bytes, spend_pubkey: bytes) -> list[dict[str, str]]:
ismaelsadeeq commented at 2:50 PM on November 17, 2026:scan_txis dead for now; we are not sure it compiles in C.I, and it does not seem like a utility because it needs state and a-txindexrequirement. Probably fits better as a helper in the actual test?
rustaceanrob commented at 2:56 PM on November 17, 2026:Sure, we can have this module be self tested entirely. I don't think #35302 uses this in multiple files anyway.
DrahtBot requested review from ismaelsadeeq on Nov 17, 2026rustaceanrob force-pushed on Nov 17, 2026cd6d7e240btest: add BIP352 scanning to the functional test framework
Add a Python implementation of BIP352 scanning, without labels, so the functional tests can check silent payments outputs independently of the node. It is tested against the BIP352 receiving test vectors. Co-authored-by: rustaceanrob <rob.netzke@gmail.com>
rustaceanrob force-pushed on Nov 17, 2026DrahtBot added the label CI failed on Nov 17, 2026DrahtBot commented at 3:09 PM on November 17, 2026: contributor<!--85328a0da195eb286784d51f73fa0af9-->
🚧 At least one of the CI tasks failed. <sub>Task
lint: https://github.com/bitcoin/bitcoin/actions/runs/44109510291/job/134130936328</sub> <sub>LLM reason (✨ experimental): CI failed because Python linting found three unused imports intest/functional/test_framework/bip352.py(ruff F401 errors).</sub><details><summary>Hints</summary>
Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:
Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.
Leave a comment here, if you need help tracking down a confusing failure.
</details>
in test/functional/test_framework/bip352.py:25 in cd6d7e240b
20 | + 21 | + 22 | +def make_sp_addr(scan_key: ECKey, spend_key: ECKey, hrp: str = "sprt") -> str: 23 | + """Encode an SP address from two ECKey objects (scan and spend).""" 24 | + payload = scan_key.get_pubkey().get_bytes() + spend_key.get_pubkey().get_bytes() 25 | + return bech32_encode(Encoding.BECH32M, hrp, [0] + convertbits(list(payload), 8, 5))
theStack commented at 3:28 PM on November 17, 2026:nit: could consider moving this function (and the decoding counterpart) to
address.pyin the futuretheStack approvedtheStack commented at 3:30 PM on November 17, 2026: contributorACK cd6d7e240bb22e5d9515bd8cf8dbc081ff471dc8
I agree that deferring the introduction of
scan_txto another PR (likely #35302) makes sense, given that it it's currently unused/untested.DrahtBot requested review from Eunovo on Nov 17, 2026in test/functional/test_framework/bip352.py:19 in cd6d7e240b
14 | +from .messages import COutPoint, deser_string_vector 15 | +from .script import ANNEX_TAG, CScript, hash160 16 | +from .script_util import is_p2pkh_script, is_p2sh_script, is_p2tr_script, is_p2wpkh_script, script_witness_version 17 | +from .segwit_addr import bech32_encode, convertbits, Encoding 18 | + 19 | +K_MAX = 2323
ismaelsadeeq commented at 3:34 PM on November 17, 2026:nit: reference the bip here.
DrahtBot requested review from ismaelsadeeq on Nov 17, 2026ismaelsadeeq approvedismaelsadeeq commented at 3:35 PM on November 17, 2026: memberACK cd6d7e240bb22e5d9515bd8cf8dbc081ff471dc8
DrahtBot removed the label CI failed on Nov 17, 2026Eunovo commented at 12:07 PM on November 18, 2026: contributorLabels
github-metadata-mirror
This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-18 16:51 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me