test: add tests in script test suites covering script.cpp live mutants #36360

pull Yudis-bit wants to merge 4 commits into bitcoin:master from Yudis-bit:test-script-coverage-mutants changing 4 files +169 −0
  1. Yudis-bit commented at 6:12 PM on September 27, 2026: none

    Adds unit tests for boundary checks and live mutants in src/script/script.cpp.

    <details> <summary><a href="https://mutanthub.space/mutants/533">script.cpp:173</a>: <code>lastOpcode >= OP_1</code> -> <code>> OP_1</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -173,1 +173,1 @@ unsigned int CScript::GetSigOpCount(bool fAccurate) const
    -            if (fAccurate && lastOpcode >= OP_1 && lastOpcode <= OP_16)
    +            if (fAccurate && lastOpcode > OP_1 && lastOpcode <= OP_16)
    

    Fails BOOST_CHECK_EQUAL(s_1of1.GetSigOpCount(true), 1U) in sigopcount_tests.cpp.

    </details>

    <details> <summary><a href="https://mutanthub.space/mutants/534">script.cpp:186</a>: <code>GetSigOpCount(true)</code> -> <code>GetSigOpCount(false)</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -186,1 +186,1 @@ unsigned int CScript::GetSigOpCount(const CScript& scriptSig) const
    -        return GetSigOpCount(true);
    +        return GetSigOpCount(false);
    

    Fails BOOST_CHECK_EQUAL(s1.GetSigOpCount(scriptSig), 3U) in sigopcount_tests.cpp.

    </details>

    <details> <summary><a href="https://mutanthub.space/mutants/536">script.cpp:198</a>: <code>opcode > OP_16</code> -> <code>>= OP_16</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -198,1 +198,1 @@ unsigned int CScript::GetSigOpCount(const CScript& scriptSig) const
    -        if (opcode > OP_16)
    +        if (opcode >= OP_16)
    

    Fails BOOST_CHECK_EQUAL(p2sh.GetSigOpCount(script_sig_op16), 3U) in sigopcount_tests.cpp.

    </details>

    <details> <summary><a href="https://mutanthub.space/mutants/545">script.cpp:220</a>: <code>program[0] == 0x4e &&</code> -> <code>||</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -220,1 +220,1 @@ bool CScript::IsPayToAnchor(int version, const std::vector<unsigned char>& progr
    -        program[0] == 0x4e &&
    +        program[0] == 0x4e ||
    

    Fails BOOST_CHECK(!CScript::IsPayToAnchor(1, {0x4e, 0x00})) in script_standard_tests.cpp.

    </details>

    <details> <summary>script.cpp:236: <code>size() == 34</code> -> <code>>= 34</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -236,1 +236,1 @@ bool CScript::IsPayToWitnessScriptHash() const
    -    return (this->size() == 34 &&
    +    return (this->size() >= 34 &&
    

    Fails BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToWitnessScriptHash()) for oversized scripts in script_segwit_tests.cpp.

    </details>

    <details> <summary>script.cpp:243: <code>size() == 34</code> -> <code>>= 34</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -243,1 +243,1 @@ bool CScript::IsPayToTaproot() const
    -    return (this->size() == 34 &&
    +    return (this->size() >= 34 &&
    

    Fails BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToTaproot()) for oversized scripts in script_segwit_tests.cpp.

    </details>

    <details> <summary><a href="https://mutanthub.space/mutants/560">script.cpp:292</a>: <code>if (i)</code> -> <code>if (1==1)</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -292,1 +292,1 @@ std::string CScriptWitness::ToString() const
    -        if (i) {
    +        if (1==1) {
    

    Fails BOOST_CHECK_EQUAL(wit_single.ToString(), "CScriptWitness(0102)") in script_tests.cpp.

    </details>

    <details> <summary><a href="https://mutanthub.space/mutants/563">script.cpp:306</a>: <code>opcode > MAX_OPCODE</code> -> <code>>= MAX_OPCODE</code></summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -306,1 +306,1 @@ bool CScript::HasValidOps() const
    -        if (!GetOp(it, opcode, item) || opcode > MAX_OPCODE || item.size() > MAX_SCRIPT_ELEMENT_SIZE) {
    +        if (!GetOp(it, opcode, item) || opcode >= MAX_OPCODE || item.size() > MAX_SCRIPT_ELEMENT_SIZE) {
    

    Fails BOOST_CHECK(script.HasValidOps()) on OP_NOP10 in script_tests.cpp.

    </details>

    <details> <summary>script.cpp:337, 343, 350: <code>return false</code> -> <code>return true</code> on truncated pushdata lengths</summary>

    --- a/src/script/script.cpp
    +++ b/src/script/script.cpp
    @@ -337,1 +337,1 @@ bool GetScriptOp(CScriptBase::const_iterator& pc, CScriptBase::const_iterator en
    -                return false;
    +                return true;
    @@ -343,1 +343,1 @@ bool GetScriptOp(CScriptBase::const_iterator& pc, CScriptBase::const_iterator en
    -                return false;
    +                return true;
    @@ -350,1 +350,1 @@ bool GetScriptOp(CScriptBase::const_iterator& pc, CScriptBase::const_iterator en
    -                return false;
    +                return true;
    

    Fails BOOST_CHECK(!s.GetOp(it, opcode, data)) on truncated OP_PUSHDATA1/2/4 scripts in script_tests.cpp.

    </details>

    Review per commit recommended.

  2. Yudis-bit requested review from Copilot on Sep 27, 2026
  3. DrahtBot added the label Tests on Sep 27, 2026
  4. Copilot commented at 6:12 PM on September 27, 2026: none

    Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

  5. DrahtBot commented at 6:12 PM on September 27, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK fametrano

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #36776 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36776.svg"></sub> (bip352: match P2SH-P2WPKH instead of evaluating it by Eunovo)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  6. fanquake commented at 7:02 PM on September 27, 2026: member
  7. brunoerg commented at 1:23 PM on September 28, 2026: contributor

    Is this fully LLM generated? Did you really check what you're claiming? e.g. the 16-of-16 P2SH test doesn't test the OP_16 boundary (b9ad09fe3), your script_GetOp_truncated test case does not kill any of the four GetOp mutants it claims.

  8. Yudis-bit force-pushed on Sep 29, 2026
  9. Yudis-bit commented at 3:42 AM on September 29, 2026: none

    You're right about the original OP_16 test. It put OP_16 inside the serialized redeem script, so it did not exercise the outer scriptSig boundary in mutant 2328. The replacement puts OP_16 directly before the redeem-script push. With 2328 applied, it fails with 0 != 3.

    For GetOp, three of the four IDs are detected by the original tests. On ec4d7b3bd, applying the exact dataset patches separately makes script_tests/script_GetOp_truncated fail at:

    2377: script_tests.cpp:1646: !s1.GetOp(it1, opcode, data)
    2380: script_tests.cpp:1651: !s2.GetOp(it2, opcode, data)
    2383: script_tests.cpp:1656: !s4.GetOp(it4, opcode, data)
    

    All three builds succeed; each test exits 201. Those patches change return false to return true at the incomplete length-field checks. The diff I put in the description was wrong.

    2372 was also an incorrect claim. Its return is unreachable after the preceding pc >= end guard for valid iterators. I've removed it from the description and commit message.

    The revised tests detect all 12 retained mutants when applied individually, and all 50 cases in the four affected suites pass without mutations. The follow-up audit was AI-assisted.

  10. sedited commented at 12:31 PM on October 8, 2026: contributor

    Re #36360 (comment)

    Per the AI policy, we require contributors to write their own comments and PR descriptions. We do this, because the bottleneck in this project is human review. Reading through wordy and often vacuous LLM-generated comments takes time and is frustrating. Can you re-write your comments here in your own words?

  11. Yudis-bit commented at 12:47 PM on October 8, 2026: none

    Fair point. I used an LLM earlier to draft the environment summary and format the text, which resulted in nonsensical toolchain versions and sloppy claims about the mutation run. That was a bad call, apologies for the noise.

    On the code itself, Bruno's critique on b9ad09fe3 was accurate: placing OP_16 inside the serialized redeem script left only a pushdata in scriptSig, never touching the opcode > OP_16 guard at line 198 of src/script/script.cpp. Commit 6c82c93763 fixes that by placing OP_16 directly in scriptSig ahead of the redeem script push, so a >= mutation drops the sigop count from 3 to 0.

    On the mutation run 21 dataset, my claim that all 12 were live was wrong. Checking the data directly, only four are actual surviving mutants on master: 2347 (P2TR 34-byte check), 2362 (CScriptWitness::ToString delimiter on single elements), 2365 (HasValidOps ceiling at MAX_OPCODE), and 2383 (GetScriptOp truncated OP_PUSHDATA4). Mutants 2377 and 2380 were already caught elsewhere on master, while 2372 at line 322 (end - pc < 1) is unreachable dead code in script.cpp due to the preceding pc >= end guard at line 318.

    The branch adds targeted unit test coverage for these boundaries across script_segwit_tests (Taproot push opcodes and size), sigopcount_tests (outer scriptSig OP_16), script_standard_tests (2-byte anchor mismatches on {0x4e, 0x73}), and script_tests (truncated pushdata headers and witness stringification). I have updated the PR description to remove the generated text.

  12. sedited requested review from brunoerg on Oct 21, 2026
  13. brunoerg commented at 6:39 PM on October 21, 2026: contributor

    Please, instead of saying "killing surviving mutant 2383", could you put the diff in the description, it would facilitate the reproduction? Also, you can check or suggest mutants at https://mutanthub.space/mutants?q=&project=bitcoin%2Fbitcoin.

    edit: the description doesn't seem accurate on what's claiming and doing.

  14. in src/test/sigopcount_tests.cpp:73 in 9c912324c1 outdated
      68 | +    BOOST_CHECK_EQUAL(s1.GetSigOpCount(scriptSig), 3U);
      69 | +
      70 | +    // 1-of-1 multisig to verify accurate counting when lastOpcode is OP_1 (boundary of OP_1..OP_16)
      71 | +    CScript s_1of1;
      72 | +    s_1of1 << OP_1 << ToByteVector(dummy) << OP_1 << OP_CHECKMULTISIG;
      73 | +    BOOST_CHECK_EQUAL(s_1of1.GetSigOpCount(true), 1U);
    


    brunoerg commented at 6:40 PM on October 21, 2026:

    #36130 already covers it.


    Yudis-bit commented at 4:15 AM on October 22, 2026:

    Dropped the redundant GetSigOpCount(false) check in 20d01e52a. Kept the GetSigOpCount(true) check on OP_1, which exercises the accurate counting lower boundary (killing mutant 533; master only tests OP_2 and OP_3). Clean range-diff against previous push.

  15. Yudis-bit commented at 4:03 AM on October 22, 2026: none

    Updated description with reproduction diffs and MutantHub links. Replaced the generic claims with the exact script.cpp boundary checks tested in each commit. All 27 CI builds green on 9c912324c.

  16. Yudis-bit force-pushed on Oct 22, 2026
  17. fametrano commented at 4:41 AM on October 23, 2026: none

    Nine of the ten mutants in the description fail the new tests, but the IsPayToTaproot one (size() == 34 to >= 34) does not.

    I applied each mutant on its own to 448176c7 and ran the four touched suites, with and without the new tests. Each of the nine fails the new checks and passes the old ones in those suites. I also recomputed the four new sigop counts (3, 1, 3, 0) with another implementation's counter, and they agree. I can share the scripts and the logs.

    The >= 34 mutant passes the new tests and the whole test_bitcoin. No new case starts with OP_1 0x20 and is longer than 34 bytes: in every longer one, the byte after OP_1 is not 0x20, so the unmutated checks already reject it. A case that catches the mutant is suggested inline.

  18. in src/test/script_segwit_tests.cpp:113 in 448176c749 outdated
     108 | +    BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToTaproot());
     109 | +
     110 | +    // 33 bytes
     111 | +    bytes = {OP_1, 33};
     112 | +    bytes.insert(bytes.end(), 33, 0);
     113 | +    BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToTaproot());
    


    fametrano commented at 4:41 AM on October 23, 2026:

    This case catches the >= 34 mutant: it passes unmutated and fails with the mutant.

        BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToTaproot());
    
        // 35 bytes starting with OP_1 0x20
        bytes = {OP_1, 0x20};
        bytes.insert(bytes.end(), 33, 0);
        BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToTaproot());
    

    Yudis-bit commented at 6:49 AM on October 26, 2026:

    Added the 35-byte test case in 279d5cab3. The script prefix matches OP_1 0x20 so the >= 34 mutant reaches the size check and fails. Passes the 50 cases across the four touched suites with clean range-diff.

  19. Yudis-bit force-pushed on Oct 26, 2026
  20. DrahtBot added the label CI failed on Oct 27, 2026
  21. DrahtBot removed the label CI failed on Oct 27, 2026
  22. in src/test/script_segwit_tests.cpp:123 in 76b4532b3f outdated
     118 | +    BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToTaproot());
     119 | +
     120 | +    // 35 bytes starting with OP_1 0x20
     121 | +    bytes = {OP_1, 0x20};
     122 | +    bytes.insert(bytes.end(), 33, 0);
     123 | +    BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToTaproot());
    


    fametrano commented at 6:23 PM on November 1, 2026:

    The P2WSH twin of this case is missing: size() == 34 to >= 34 in IsPayToWitnessScriptHash (script.cpp:236) survives the whole test_bitcoin.

    On 76b4532b3f, I changed line 236 of src/script/script.cpp to >= 34 and rebuilt. ctest --test-dir build/src/test then passed all 155 suites, script_assets_tests included with the qa-assets data. The case below, at the end of IsPayToWitnessScriptHash_Invalid_Size, passes unmutated and fails with the mutant. Could you add it there?

        // 35 bytes starting with OP_0 0x20
        std::vector<unsigned char> bytes = {OP_0, 0x20};
        bytes.insert(bytes.end(), 33, 0);
        BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToWitnessScriptHash());
    

    Yudis-bit commented at 4:31 AM on November 2, 2026:

    The case below, at the end of IsPayToWitnessScriptHash_Invalid_Size, passes unmutated and fails with the mutant. Could you add it there?

    Good catch, thanks. Added in 46e3efb53. Passes locally on unmutated code and fails against the >= 34 mutant.

  23. Yudis-bit force-pushed on Nov 2, 2026
  24. Yudis-bit commented at 4:39 AM on November 2, 2026: none

    Rebased and added the 35-byte P2WSH check. Range-diff against 76b4532b3:

    1:  279d5cab3 ! 1:  46e3efb53 test: cover CScript::IsPayToTaproot edge cases and malformed encodings
        @@ src/test/script_segwit_tests.cpp: BOOST_AUTO_TEST_CASE(IsPayToWitnessScriptHash_Invalid_Size)
             CScript notp2wsh;
             notp2wsh << OP_0 << ToByteVector(dummy);
             BOOST_CHECK(!notp2wsh.IsPayToWitnessScriptHash());
        +
        +    // 35 bytes starting with OP_0 0x20
        +    std::vector<unsigned char> bytes = {OP_0, 0x20};
        +    bytes.insert(bytes.end(), 33, 0);
        +    BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToWitnessScriptHash());
         }
    2:  f1955ccd2 = 2:  9be36c223 test: cover sigop counting boundaries
    3:  555e6d74d = 3:  06657b524 test: cover anchor payload byte mismatches
    4:  76b4532b3 = 4:  413a33e71 test: cover script parsing and formatting edges
    
  25. fametrano commented at 10:27 AM on November 2, 2026: none

    tACK 413a33e71629d95abb6d16806d1ef34358c70652

    Ralph traced the mutants without running them and suggested a per-mutant run on the base; this is that run. I applied each of the eleven mutants in the description on its own to src/script/script.cpp. On master, 6c01bd0b2d, and on the pull request's parent, ed7dd7cf4e, each passes the whole test_bitcoin, so no existing unit test catches any of them. On the head, each fails the touched suites at the check the description names, and unmutated the whole test_bitcoin passes. Release build without wallet, with the qa-assets data; commands, results and logs: https://gist.github.com/fametrano/e7a8cc0a1a900a7c8f5e27f47bd31082/25f148b77851ea322c50cce59fb73097b41b0f9f

    Not blocking: the P2WSH twin of Ralph's second-byte point also survives. With (*this)[1] == 0x20 removed from IsPayToWitnessScriptHash (line 238), the whole test_bitcoin still passes on the head. This case, at the end of IsPayToWitnessScriptHash_Invalid_Size, passes unmutated and fails with that mutant:

        // 34 bytes starting with OP_0 0x1f
        bytes = {OP_0, 0x1f};
        bytes.insert(bytes.end(), 32, 0);
        BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToWitnessScriptHash());
    
  26. test: cover CScript::IsPayToTaproot edge cases and malformed encodings
    Add unit test coverage for CScript::IsPayToTaproot() in script_segwit_tests:
    - Valid 34-byte SegWit v1 witness program (OP_1 0x20 <32-byte-key>).
    - Rejection of non-v1 witness programs (OP_0, OP_2, OP_16).
    - Rejection of mismatched program sizes (31, 33, and 34 bytes without OP_1 0x20).
    - Rejection of superfluous opcodes (OP_NOP) and empty scripts.
    - Rejection of pushdata-encoded 32-byte payloads (OP_PUSHDATA1/2/4).
    
    Kills live mutants 2346, 2347, and 2348 in src/script/script.cpp.
    a93f1cb375
  27. test: cover sigop counting boundaries
    Check accurate counting for non-P2SH outputs and multisig with OP_1.
    For P2SH outputs, put OP_16 directly in scriptSig before the serialized
    redeem script, so an early return of zero differs from its three sigops.
    Also check rejection of OP_NOP, the next opcode above OP_16.
    
    Detects script.cpp mutations 2325, 2326 and 2328 from the mutation dataset.
    Each mutation was applied separately and produced an assertion failure;
    the unmodified baseline passes.
    24416e0e2e
  28. test: cover anchor payload byte mismatches
    Check rejection when only one byte of the two-byte anchor program matches,
    both for the encoded script and for the static IsPayToAnchor overload.
    
    The static checks detect script.cpp mutation 2337, which replaces the
    conjunction between the payload byte comparisons with ||. Both inputs
    contain two bytes, so this does not depend on an out-of-bounds access in
    the mutated code.
    790c4ac3b0
  29. test: cover script parsing and formatting edges
    Check acceptance of MAX_OPCODE and rejection of the next opcode. Check
    that GetOp rejects incomplete PUSHDATA1, PUSHDATA2 and PUSHDATA4 length
    fields. Check witness formatting for empty, single-item and multi-item
    stacks.
    
    Detects script.cpp mutations 2362, 2365, 2377, 2380 and 2383 from the
    mutation dataset. Each mutation was applied separately and produced an
    assertion failure; the unmodified baseline passes.
    036bb5771c
  30. Yudis-bit force-pushed on Nov 2, 2026
  31. Yudis-bit commented at 2:18 PM on November 2, 2026: none

    Rebased and added the 34-byte P2WSH check to IsPayToWitnessScriptHash_Invalid_Size. Range-diff against 413a33e71:

    1:  46e3efb53 ! 1:  a93f1cb37 test: cover CScript::IsPayToTaproot edge cases and malformed encodings
        @@ src/test/script_segwit_tests.cpp: BOOST_AUTO_TEST_CASE(IsPayToWitnessScriptHash_
         +    // 35 bytes starting with OP_0 0x20
         +    std::vector<unsigned char> bytes = {OP_0, 0x20};
         +    bytes.insert(bytes.end(), 33, 0);
        ++    BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToWitnessScriptHash());
        ++
        ++    // 34 bytes starting with OP_0 0x1f
        ++    bytes = {OP_0, 0x1f};
        ++    bytes.insert(bytes.end(), 32, 0);
         +    BOOST_CHECK(!CScript(bytes.begin(), bytes.end()).IsPayToWitnessScriptHash());
          }
          
    2:  9be36c223 = 2:  24416e0e2 test: cover sigop counting boundaries
    3:  06657b524 = 3:  790c4ac3b test: cover anchor payload byte mismatches
    4:  413a33e71 = 4:  036bb5771 test: cover script parsing and formatting edges
    
  32. fametrano commented at 8:00 AM on November 4, 2026: none

    tACK 036bb5771c

    Since 413a33e7 the only change is the 34-byte OP_0 0x1f case in IsPayToWitnessScriptHash_Invalid_Size. With (*this)[1] == 0x20 removed from IsPayToWitnessScriptHash, that case fails at script_segwit_tests.cpp:47, the only failure in a test_bitcoin built without the wallet (137 of 138 suites pass). Each of the eleven mutants in the description still makes one of the touched suites fail, and without a mutant all 138 suites of that build pass.

    Ralph notes the new case without running it; this run shows it catches the mutant. Commands and logs: https://gist.github.com/fametrano/e7a8cc0a1a900a7c8f5e27f47bd31082/e19e349f3f028ede3330e34e38633dc178cfeb62

  33. DrahtBot added the label CI failed on Nov 14, 2026
  34. maflcko removed the label CI failed on Nov 16, 2026

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-18 16:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me