BIP-346: fix two-byte index parsing in the reference implementation #2314

pull Dev-next-gen wants to merge 3 commits into bitcoin:master from Dev-next-gen:bip346-two-byte-index changing 2 files +39 −6
  1. Dev-next-gen commented at 8:56 AM on September 29, 2026: none

    While reading bip-0346/ref-impl/src/main.rs I found that the two-byte form of an individual in/output index can never work.

    In parse_inout_selection, individual selection mode, an index byte whose highest bit is set means the index spans two bytes. The composition is:

    (((first & (1 << 7)) as usize) << 8) + next_byte as usize
    

    Inside that branch first & 0x80 is always 0x80 — that is exactly the condition that got us there. So the mask keeps the flag bit and discards the seven value bits, and every two-byte index evaluates to 0x8000 + next_byte, i.e. 32768 or more.

    Concretely, on the existing test transaction (4 inputs), selecting absolute input index 1:

    • single-byte encoding, index byte 0x01 → Ok(0x9973075dd63b8af73e73c9881eff129aa2051e1ce9d342d052f874f9bdd75b16)
    • two-byte encoding of the same index, bytes 0x80 0x01 → Err("selected index out of bounds")

    Both spell the same selection, so they must produce the same TxHash. In absolute mode the bug makes every two-byte index out of bounds, so valid selectors are rejected and the whole encoding is dead. In relative mode it is quieter and worse: read_i15 masks 0x7fff, which erases the 0x8000 that was just added, so an offset of 256 or more collapses onto its low 8 bits and commits to a different input than the one the selector names, with no error.

    The BIP text settles what the code should do — "If that byte's highest bit is set to 1, the remaining 7 bits, together with the next byte's 8 bits represent the absolute index to select" (and the same wording for the relative index) — and the Rationale states the encoding allows "absolute indices up to ~32k" and relative indices "from -16382 to +16383". Those ranges need 15 value bits; as written, no index below 32768 is reachable through the two-byte form. The leading mode, twenty lines above, already composes its two-byte count the intended way, with selection & TXFS_INOUT_SELECTION_MASK.

    So the fix is to mask the flag bit off instead of keeping only it.

    I added a test asserting that both encodings of the same absolute index yield the same TxHash; it fails before the change and passes after. Nothing else moves: generate_vectors only exercises single-byte indices (as its own TODO notes, testing index size needs more than 32 in/outputs), so the patched branch was unreachable, and running the binary regenerates txhash_vectors.json byte for byte identical (md5 56b8499b66d20582ceba6a18544a0145 before and after).

    Found by a defect-hunting pipeline I build and run (Dev-next-gen), using Claude Code with Anthropic's Claude Opus 5.

    Nothing this pipeline produces is submitted without human approval. The entire pull request — the code change, the tests and this description — was reviewed by a human before the pipeline was allowed to submit it.

  2. BIP-346: fix two-byte individual index parsing in reference impl
    In individual selection mode the BIP says that when an index byte has its
    highest bit set, "the remaining 7 bits, together with the next byte's 8
    bits represent the index". parse_inout_selection masked that byte with
    (1 << 7) instead of with its complement, so it kept only the flag bit and
    threw away the seven value bits: every two-byte index evaluated to
    0x8000 + next_byte.
    
    In absolute mode that is always out of bounds, so a perfectly valid
    selector using the two-byte encoding is rejected. In relative mode
    read_i15 masks 0x8000 away again, so offsets of 256 or more collapse to
    their low 8 bits and silently select a different input than the one the
    selector names.
    
    The committed test vectors are unaffected: generate_vectors only
    exercises single-byte indices (see its TODO about needing more than 32
    in/outputs), and txhash_vectors.json regenerates byte for byte. Added a
    test asserting that both encodings of the same absolute index produce the
    same TxHash.
    58de753cb4
  3. fametrano commented at 9:31 AM on October 9, 2026: contributor

    Tested ACK 58de753. I added a relative-mode case next to yours: index -1 written in one byte (0x7f) and in two bytes (0xff 0xff). Both now give the same hash. On master the two-byte form fails with "selected index out of bounds".

    While testing I noticed three places where the BIP text disagrees with the code. Relative indices run from -16384 to +16383, not -16382. The absolute and relative mode bullets say "second highest bit", but the flag is the third highest bit, as the line above them says. A 5-bit count allows at most 31 individually selected items, not 32.

  4. BIP-346: test two-byte relative index in reference impl
    Index -1 relative to input 1 written as one byte (0x7f) and as two
    bytes (0xff 0xff) must select the same input and give the same TxHash.
    Before the previous commit the two-byte form fails with "selected index
    out of bounds". Case suggested by fametrano in review.
    219b7c42a1
  5. BIP-346: align individual-mode text with the reference impl
    - absolute/relative mode is chosen by the third highest bit
      (TXFS_INOUT_INDIVIDUAL_MODE), not the second, as the line introducing
      individual mode already says
    - a two-byte relative index is a 15-bit two's complement value, so it
      ranges from -16384 to +16383, not -16382 (read_i15)
    - the count is the low 5 bits (TXFS_INOUT_SELECTION_MASK), so at most 31
      individual in/outputs can be selected, not 32
    
    Reported by fametrano in review.
    3ff55feca0
  6. Dev-next-gen commented at 11:31 PM on October 13, 2026: none

    Thanks for the tested ACK, and for taking the relative-mode path, which my test left alone. Index -1 as 0x7f against 0xff 0xff is a good probe because on master the second form goes through read_i15 and turns into +255, so it only errors here because the test transaction is small.

    I added your case as two_byte_relative_index_matches_single_byte in 219b7c4 and fixed the text in 3ff55fe. All three points hold against the code: the absolute and relative bullets now say third highest bit, matching TXFS_INOUT_INDIVIDUAL_MODE = 1 << 5; read_i15(0x4000) gives -16384, so the summary range is now -16384 to +16383; and the count is selection & TXFS_INOUT_SELECTION_MASK, five bits, so 32 became 31 both in the summary and in the malleability section, which repeated the same number.

    One caveat: there was no Rust toolchain where I pushed from, so the new test is the absolute one with only the selector bytes and input index changed, and I checked its expectation against a small port of parse_inout_selection rather than with cargo test. If your checkout is still around, a run would confirm it.

    Found by a defect-hunting pipeline I build and run (Dev-next-gen), using Claude Code with Anthropic's Claude Opus 5.5.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bips. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-10-14 03:10 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me