Remove template matching and pseudo opcodes #13194

pull sipa wants to merge 1 commits into bitcoin:master from sipa:201805_noscriptpattern changing 4 files +66 −103
  1. sipa commented at 3:27 AM on May 9, 2018: member

    The current code contains a rather complex script template matching engine, which is only used for 3 particular script types (P2PK, P2PKH, multisig). The first two of these are trivial to match for otherwise, and a specialized matcher for multisig is both more compact and more efficient than a generic one.

    The goal is being more flexible, so that for example larger standard multisigs inside SegWit outputs are easier to implement.

    As a side-effect, it also gets rid of the pseudo opcodes hack.

  2. sipa cross-referenced this on May 9, 2018 from issue [wallet] Support CMV signing of 17~20 key template by instagibbs
  3. laanwj added the label Wallet on May 9, 2018
  4. in src/script/script.h:190 in 688d79698f outdated
     185 | -    // template matching params
     186 | -    OP_SMALLINTEGER = 0xfa,
     187 | -    OP_PUBKEYS = 0xfb,
     188 | -    OP_PUBKEYHASH = 0xfd,
     189 | -    OP_PUBKEY = 0xfe,
     190 | -
    


    laanwj commented at 11:18 AM on May 9, 2018:

    Nice to get rid of these fake opcodes here.

  5. in src/script/standard.cpp:67 in 688d79698f outdated
      73 | +    opcodetype opcode;
      74 | +    valtype data;
      75 | +    CScript::const_iterator it = script.begin();
      76 | +    if (script.size() < 1 || script.back() != OP_CHECKMULTISIG) return false;
      77 | +
      78 | +    if (!script.GetOp(it, opcode, data) || (opcode != OP_0 && (opcode < OP_1 || opcode > OP_16))) return false;
    


    jtimon commented at 2:22 PM on May 9, 2018:

    Souldn't this return false if it's op_0 too? or are 0 required sigs multisigs valid?


    instagibbs commented at 3:17 PM on May 9, 2018:

    key count can be 0, I think. The only checks in the interpreter I see are for negative numbers.


    sipa commented at 3:36 PM on May 9, 2018:

    I'm trying to stick to the original logic as close as possible; optimizations / other tweaks can happen later.


    laanwj commented at 11:25 AM on May 10, 2018:

    Agree with @sipa here. One step at a time is easier to review.

  6. in src/script/standard.cpp:72 in 688d79698f outdated
      78 | +    if (!script.GetOp(it, opcode, data) || (opcode != OP_0 && (opcode < OP_1 || opcode > OP_16))) return false;
      79 | +    required = CScript::DecodeOP_N(opcode);
      80 | +    while (script.GetOp(it, opcode, data) && CPubKey::ValidSize(data)) {
      81 | +        pubkeys.emplace_back(std::move(data));
      82 |      }
      83 | +    if (opcode != OP_0 && (opcode < OP_1 || opcode > OP_16)) return false;
    


    jtimon commented at 2:26 PM on May 9, 2018:

    Same here, shouldn't OP_0 be invalid here too? If 0 is allowed, why not just opcode < OP_0 || opcode > OP_16?


    sipa commented at 3:31 PM on May 9, 2018:

    OP_0 is 0, OP_1 is 81. They aren't continuous.


    laanwj commented at 12:48 PM on May 10, 2018:

    An isOP_N(opcode) helper function would make this (and the expression above) slightly better readable.


    sipa commented at 8:32 PM on May 12, 2018:

    I added an IsSmallInteger function for this (which would be distinct from IsOP_N by not accepting OP_1NEGATE).

  7. jtimon approved
  8. jtimon commented at 2:54 PM on May 9, 2018: contributor

    utACK modulo nits and modulo this kind of unrelated thing I don't understand. Perhaps a stupid question but, in https://github.com/bitcoin/bitcoin/blob/master/src/pubkey.h#L59 why are there 3 ways to indicate non-compressed and 2 ways to indicate compressed? shouldn't one for each be enough? also, should using an enum or macros or constants be more clear?

  9. in src/script/standard.cpp:42 in 688d79698f outdated
      44 | -    {
      45 | -        // Standard tx, sender provides pubkey, receiver adds signature
      46 | -        mTemplates.insert(std::make_pair(TX_PUBKEY, CScript() << OP_PUBKEY << OP_CHECKSIG));
      47 | +    if (script.size() == 35 && script[0] == 33 && script[34] == OP_CHECKSIG) {
      48 | +        pubkey = valtype(script.begin() + 1, script.begin() + 34);
      49 | +        return CPubKey::ValidSize(pubkey);
    


    instagibbs commented at 3:18 PM on May 9, 2018:

    Redundant check here? We know the size.


    sipa commented at 3:32 PM on May 9, 2018:

    CpubKey::ValidSize checks the data-dependent size (depending on what the first byte is).

  10. in src/script/standard.cpp:46 in 688d79698f outdated
      48 | +        pubkey = valtype(script.begin() + 1, script.begin() + 34);
      49 | +        return CPubKey::ValidSize(pubkey);
      50 | +    }
      51 | +    if (script.size() == 67 && script[0] == 65 && script[66] == OP_CHECKSIG) {
      52 | +        pubkey = valtype(script.begin() + 1, script.begin() + 66);
      53 | +        return CPubKey::ValidSize(pubkey);
    


    instagibbs commented at 3:18 PM on May 9, 2018:

    Redundant check here? We know the size.


    instagibbs commented at 3:19 PM on May 9, 2018:

    We could just use proper constants in the lines above for more belt and suspenders.

  11. sipa commented at 3:33 PM on May 9, 2018: member

    @jtimon Hybrid public keys (the 0x06 and 0x07 ones) are stupid, but were supported by OpenSSL, and because of that they're supported in Bitcoin. Nothing we can do about that.

  12. instagibbs commented at 3:46 PM on May 9, 2018: member

    light tACK

  13. in src/script/standard.cpp:75 in 688d79698f outdated
      75 | +    CScript::const_iterator it = script.begin();
      76 | +    if (script.size() < 1 || script.back() != OP_CHECKMULTISIG) return false;
      77 | +
      78 | +    if (!script.GetOp(it, opcode, data) || (opcode != OP_0 && (opcode < OP_1 || opcode > OP_16))) return false;
      79 | +    required = CScript::DecodeOP_N(opcode);
      80 | +    while (script.GetOp(it, opcode, data) && CPubKey::ValidSize(data)) {
    


    laanwj commented at 12:42 PM on May 10, 2018:

    Shouldn't it check the opcodes here? Oh I guess not, all the push opcodes valid here will return non-empty data, and CPubKey::ValidSize rejects empty data.


    sipa commented at 8:31 PM on May 12, 2018:

    Indeed (that's also what the old code does).

  14. sipa force-pushed on May 12, 2018
  15. in src/script/standard.cpp:44 in 2fc4fdd094 outdated
      46 | -        mTemplates.insert(std::make_pair(TX_PUBKEY, CScript() << OP_PUBKEY << OP_CHECKSIG));
      47 | +    if (script.size() == 35 && script[0] == 33 && script[34] == OP_CHECKSIG) {
      48 | +        pubkey = valtype(script.begin() + 1, script.begin() + 34);
      49 | +        return CPubKey::ValidSize(pubkey);
      50 | +    }
      51 | +    if (script.size() == 67 && script[0] == 65 && script[66] == OP_CHECKSIG) {
    


    Empact commented at 2:06 AM on May 16, 2018:

    CPubKey::PUBLIC_KEY_SIZE == 65, would add clarity


    sipa commented at 12:15 AM on May 17, 2018:

    Done.

  16. in src/script/standard.cpp:40 in 2fc4fdd094 outdated
      42 | -    static std::multimap<txnouttype, CScript> mTemplates;
      43 | -    if (mTemplates.empty())
      44 | -    {
      45 | -        // Standard tx, sender provides pubkey, receiver adds signature
      46 | -        mTemplates.insert(std::make_pair(TX_PUBKEY, CScript() << OP_PUBKEY << OP_CHECKSIG));
      47 | +    if (script.size() == 35 && script[0] == 33 && script[34] == OP_CHECKSIG) {
    


    Empact commented at 2:06 AM on May 16, 2018:

    CPubKey::COMPRESSED_PUBLIC_KEY_SIZE == 33, would add clarity


    Empact commented at 2:10 AM on May 16, 2018:

    Incidentally, I have #12461 to make these references more succinct.


    sipa commented at 12:15 AM on May 17, 2018:

    Done.

  17. in src/script/standard.cpp:51 in 2fc4fdd094 outdated
      53 | +        return CPubKey::ValidSize(pubkey);
      54 | +    }
      55 | +    return false;
      56 | +}
      57 | +
      58 | +static bool MatchPayToPubkeyHash(const CScript& script, valtype& pubkeyhash)
    


    Empact commented at 2:36 AM on May 16, 2018:

    See also IsToKeyID in compressor.cpp, which looks equivalent.

    Could split this up as CScript::IsPayToPubkeyHash, and do the unpacking in Solver, ala IsPayToScriptHash. Would make the case handling more consistent in Solver.


    sipa commented at 12:21 AM on May 17, 2018:

    I'd rather not merging with the versions in compressor - they may end up living separate lives as the standard templates change.

  18. in src/script/standard.cpp:38 in 2fc4fdd094 outdated
      34 | @@ -35,22 +35,55 @@ const char* GetTxnOutputType(txnouttype t)
      35 |      return nullptr;
      36 |  }
      37 |  
      38 | -bool Solver(const CScript& scriptPubKey, txnouttype& typeRet, std::vector<std::vector<unsigned char> >& vSolutionsRet)
      39 | +static bool MatchPayToPubkey(const CScript& script, valtype& pubkey)
    


    Empact commented at 2:37 AM on May 16, 2018:

    See also IsToPubKey in compressor, which looks equivalent.

  19. Empact commented at 2:42 AM on May 16, 2018: member

    Concept ACK! Much simpler way to handle these 3 cases.

  20. jonasschnelli commented at 6:20 AM on May 16, 2018: contributor

    Nice! utACK 2fc4fdd0945a3800af8213676182ebb2c63689d8

  21. sipa force-pushed on May 17, 2018
  22. gmaxwell commented at 9:35 PM on May 25, 2018: contributor

    ACK.

  23. in src/script/standard.cpp:80 in fa1f642772 outdated
      86 | +    while (script.GetOp(it, opcode, data) && CPubKey::ValidSize(data)) {
      87 | +        pubkeys.emplace_back(std::move(data));
      88 |      }
      89 | +    if (!IsSmallInteger(opcode)) return false;
      90 | +    unsigned int keys = CScript::DecodeOP_N(opcode);
      91 | +    if (pubkeys.size() != keys || required == 0 || keys < required) return false;
    


    Empact commented at 12:29 AM on May 26, 2018:

    nit: this required == 0 check could be an earlier return


    sipa commented at 12:39 AM on May 26, 2018:

    Fixed.

  24. in src/script/standard.cpp:150 in fa1f642772 outdated
     221 | +    std::vector<std::vector<unsigned char>> keys;
     222 | +    if (MatchMultisig(scriptPubKey, required, keys)) {
     223 | +        typeRet = TX_MULTISIG;
     224 | +        vSolutionsRet.push_back({static_cast<unsigned char>(required)});
     225 | +        vSolutionsRet.insert(vSolutionsRet.end(), keys.begin(), keys.end());
     226 | +        vSolutionsRet.push_back({static_cast<unsigned char>(keys.size())});
    


    Empact commented at 12:30 AM on May 26, 2018:

    Maybe worth noting these casts are safe due to small integer limits?


    sipa commented at 12:39 AM on May 26, 2018:

    Done.

  25. Empact commented at 12:31 AM on May 26, 2018: member

    utACK fa1f642

  26. sipa force-pushed on May 26, 2018
  27. Empact commented at 12:43 AM on May 26, 2018: member

    re-utACK 89736a9 just addresses comments above

  28. in src/script/standard.cpp:82 in 89736a9fbe outdated
      88 | +        pubkeys.emplace_back(std::move(data));
      89 |      }
      90 | +    if (!IsSmallInteger(opcode)) return false;
      91 | +    unsigned int keys = CScript::DecodeOP_N(opcode);
      92 | +    if (pubkeys.size() != keys || keys < required) return false;
      93 | +    if (it + 1 != script.end()) return false;
    


    promag commented at 8:46 AM on May 26, 2018:

    nit

    return it + 1 == script.end();
    

    sipa commented at 6:24 PM on May 26, 2018:

    Done.

  29. promag commented at 8:47 AM on May 26, 2018: member

    utACK 89736a9.

  30. sipa force-pushed on May 26, 2018
  31. in src/script/standard.cpp:40 in c639f5ac20 outdated
      42 | -    static std::multimap<txnouttype, CScript> mTemplates;
      43 | -    if (mTemplates.empty())
      44 | -    {
      45 | -        // Standard tx, sender provides pubkey, receiver adds signature
      46 | -        mTemplates.insert(std::make_pair(TX_PUBKEY, CScript() << OP_PUBKEY << OP_CHECKSIG));
      47 | +    if (script.size() == CPubKey::PUBLIC_KEY_SIZE + 2 && script[0] == CPubKey::PUBLIC_KEY_SIZE && script[CPubKey::PUBLIC_KEY_SIZE + 1] == OP_CHECKSIG) {
    


    jl2012 commented at 4:53 PM on May 29, 2018:

    what about script.back() == OP_CHECKSIG?


    sipa commented at 9:42 PM on May 29, 2018:

    Done.

  32. Remove template matching and pseudo opcodes
    The current code contains a rather complex script template matching engine,
    which is only used for 3 particular script types (P2PK, P2PKH, multisig).
    The first two of these are trivial to match for otherwise, and a specialized
    matcher for multisig is both more compact and more efficient than a generic
    one.
    
    The goal is being more flexible, so that for example larger standard multisigs
    inside SegWit outputs are more easy to implement.
    
    As a side-effect, it also gets rid of the pseudo opcodes hack.
    c814e2e7e8
  33. in src/script/standard.cpp:63 in c639f5ac20 outdated
      67 | +}
      68 | +
      69 | +/** Test for "small integer" script opcodes - OP_0 through OP_16. */
      70 | +static constexpr bool IsSmallInteger(opcodetype opcode)
      71 | +{
      72 | +    return opcode == OP_0 || (opcode >= OP_1 && opcode <= OP_16);
    


    jl2012 commented at 5:02 PM on May 29, 2018:

    if you remove opcode == OP_0 here, you can remove if (required == 0) return false; below


    sipa commented at 9:42 PM on May 29, 2018:

    Fixed.

  34. sipa force-pushed on May 29, 2018
  35. jl2012 commented at 9:07 AM on May 30, 2018: contributor

    utACK c814e2e

  36. laanwj commented at 2:50 PM on May 30, 2018: member

    utACK c814e2e7e81fd01fcb07f4a28435741bdc463801

  37. laanwj merged this on May 30, 2018
  38. laanwj closed this on May 30, 2018

  39. laanwj referenced this in commit fd96d54f39 on May 30, 2018
  40. jl2012 cross-referenced this on May 30, 2018 from issue Standard template for CHECKMULTISIG with 17~20 keys by jl2012
  41. Bushstar cross-referenced this on Jun 1, 2018 from issue commits from bitcoin/master by Bushstar
  42. Empact cross-referenced this on Jun 1, 2018 from issue scripted-diff: Avoid temporary copies when looping over std::map by Empact
  43. Empact cross-referenced this on Jun 1, 2018 from issue Avoid unused locals for CScript::IsWitnessProgram by Empact
  44. Empact cross-referenced this on Jun 1, 2018 from issue scripted-diff: Rename key size consts to be relative to their class by Empact
  45. jonasschnelli cross-referenced this on Aug 30, 2018 from issue 0.17.2rc issue (standardness change for bare multisig) by preminem
  46. jonasschnelli commented at 9:09 AM on August 30, 2018: contributor

    This change has the side effect that invalid pubkeys (with correct size of 33 or 65) in bare multisig are now non-standard (see #14104)

  47. jonasschnelli added the label Needs release notes on Aug 30, 2018
  48. laanwj cross-referenced this on Aug 30, 2018 from issue TODO for release notes 0.17.0 by laanwj
  49. fanquake removed the label Needs release note on Mar 20, 2019
  50. nkostoulas cross-referenced this on Apr 11, 2019 from issue Guardnode: CLTV multisig transaction Solver/Signing and Request changes by nkostoulas
  51. sickpig referenced this in commit c2912c15e2 on Oct 10, 2019
  52. sickpig cross-referenced this on Oct 10, 2019 from issue [port] Remove template matching and pseudo opcodes by sickpig
  53. sickpig referenced this in commit 028f48415e on Oct 17, 2019
  54. sickpig referenced this in commit c62ad82032 on Oct 17, 2019
  55. sickpig referenced this in commit 79cbe72a3a on Oct 17, 2019
  56. sickpig referenced this in commit 3146c738e0 on Oct 17, 2019
  57. sickpig referenced this in commit 60c818236f on Oct 30, 2019
  58. str4d cross-referenced this on Nov 14, 2019 from issue Bitcoin script PRs 1 by str4d
  59. sickpig referenced this in commit b3e587dce2 on Dec 6, 2019
  60. zkbot referenced this in commit 63c3d1ec94 on Dec 17, 2019
  61. sickpig referenced this in commit 79c1ff95be on Mar 11, 2020
  62. DeckerSU referenced this in commit 337442edc1 on Mar 17, 2020
  63. furszy cross-referenced this on Jun 18, 2020 from issue Upstream scripts back ports [Step 1] by furszy
  64. furszy referenced this in commit a5265a4db4 on Jun 27, 2020
  65. UdjinM6 referenced this in commit 6900d720b9 on Jun 19, 2021
  66. UdjinM6 referenced this in commit 23e5ca0964 on Jun 24, 2021
  67. UdjinM6 referenced this in commit b6364db126 on Jun 26, 2021
  68. UdjinM6 referenced this in commit e883e76989 on Jun 26, 2021
  69. UdjinM6 referenced this in commit e8da18d43b on Jun 28, 2021
  70. bitcoin locked this on Dec 16, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-19 14:54 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me