test: fix TaprootSignatureMsg default codeseparator_pos #36184

TaprootSignatureMsg defaults codeseparator_pos to -1 and serializes it with .to_bytes(4, "little", signed=False), so any scriptpath=True call that relies on the default raises OverflowError.

The unsigned encoding was added so callers could pass 0xfffffffe/0xffffffff, and the call sites in feature_taproot.py were updated to default to 0xffffffff; this default was left at -1. The only in-tree scriptpath caller passes codeseparator_pos explicitly, so CI does not hit it.

Set the default to 0xFFFFFFFF: the value feature_taproot.py uses for "no codeseparator", and the bytes the previous signed encoding produced for -1.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/36184.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline and AI policy for information on the review process. A summary of reviews will appear here.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

LLM Linter (✨ experimental)

Possible places where comparison-specific test macros should replace generic comparisons:

• [test/functional/test_framework/script.py] assert input_index < len(txTo.vin) -> use assert_greater_than(len(txTo.vin), input_index)

^{2026-09-16 06:51:11}

The concrete case you asked about: writing a new tapscript test with no OP_CODESEPARATOR, the ordinary case. The natural call,

TaprootSignatureHash(tx, utxos, hashtype, idx, scriptpath=True, leaf_script=script)

raises OverflowError: can't convert negative int to unsigned today, from inside the framework, for a value BIP342 treats as normal ("...or 0xffffffff if none executed", BIP342, codesep_pos). That's the gap a 0xFFFFFFFF default closes — test-code ergonomics, not a consensus question. No in-tree test exercises this default either way: feature_taproot.py:252 is the only scriptpath caller, and it always passes an explicit codeseppos.

A few more points, briefly:

• This isn't a new convention, it's a leftover. 81e5c8385b made the serialisation unsigned and converted the -1 literals it owned — feature_taproot.py:436 and :1296 — to 0xffffffff. It touched two files, and the one -1 left behind is the default in script.py's signature. That one literal is the whole PR.
• Reverting that commit isn't available either: feature_taproot.py now relies on 0xffffffff and 0xfffffffe (lines 436, 787), which overflow under signed=True.
• The parameter is real, not decoration: script.py:856 serialises it into every scriptpath sighash.
• I'd rather this not fail loud, because 0xffffffff isn't an out-of-band sentinel marking a caller mistake — it's BIP342's own value for "no codeseparator executed". interpreter.cpp:444 sets it exactly that way, unconditionally, before the interpreter's opcode loop starts. Failing on it would mean failing on the normal case.
• fadf621825, which made leaf_script mandatory, set a standard I'd apply the same way here: no default without a real use case for it. codeseparator_pos clears that bar, because the fallback isn't invented for this PR — it's the same 0xffffffff already used in interpreter.cpp:444 and feature_taproot.py:436/:1296.
• No sighash changes either way: (-1).to_bytes(4, "little", signed=True) and (0xFFFFFFFF).to_bytes(4, "little", signed=False) are both ffffffff.

If a bare 0xFFFFFFFF literal is the actual objection, I can name it instead — a CODESEPARATOR_POS_NONE = 0xFFFFFFFF constant in script.py, used in the signature and, if useful, in place of the two feature_taproot.py literals. And if you'd still rather this be closed, say so and I will.