CAT in Tapscript (BIP-347) #29247

pull 0xBEEFCAF3 wants to merge 3 commits into bitcoin:master from 0xBEEFCAF3:arm/re-enable-opcat changing 10 files +813 −8
  1. 0xBEEFCAF3 commented at 10:40 pm on January 14, 2024: none

    CAT in Tapscript

    This PR provides the necessary code to enable the opcode OP_CAT in Tapscript as specified in BIP-347: OP_CAT in Tapscript and BIN-2024-0001,

    Important: This PR does not include miner activation functionality. This means that merging this PR into bitcoin-core will not make OP_CAT functional in Bitcoin. If this PR is merged it is not a signal of community consensus around activating CAT nor be read as a portent about the activation process or timeline. The PR is not a stand-in for consensus around such decisions.

    This PR includes:

    • An implementation of the Tapscript opcode OP_CAT along with the flags SCRIPT_VERIFY_OP_CAT and SCRIPT_VERIFY_DISCOURAGE_OP_CAT.
    • Integration and unit tests which ensure that our CAT implementation works as expected. This includes ensuring that CAT never uses more that 520 bytes of stack memory. A full description of what these tests cover is given below.
    • An improvement to src/test/script_tests.cpp JSON script format enabling the rapid creation of new Tapscript unit tests.

    Activation on Bitcoin-inquisition (signet)

    Bitcoin-inquisition PR 37 which contained our implementation of OP_CAT was merged into bitcoin-inquisition (signet) on Apr 25 2024, released as bitcoin-inquisition release 25.2 on Apr 26 2024. and activated in bitcoin-inquisition Apr 30 2024. The bitcoin-inquisition PR was reviewed in the PR review club (transcript of discussion). Since Apr 30 2024 there have been many OP_CAT transactions created and spent on signet.

    The code merged into bitcoin-inquisition differs from this PR, as we have removed the consensus logic which was used to activate it on signet.

    OP_CAT Tapscript Implementation

    We implement OP_CAT as a new Tapscript op code by redefining the opcode OP_SUCCESS126 (126 in decimal and 0x7e in hexadecimal). This is the same opcode value used by the original OP_CAT.

    When evaluated, the OP_CAT instruction:

    1. Pops the top two values off the stack,
    2. concatenates the popped values together in stack order,
    3. and then pushes the concatenated value on the top of the stack.

    OP_CAT fails if there are fewer than two values on the stack or if a concatenated value would have a combined size greater than the maximum script element size of 520 bytes.

    See BIP 347 for a deeper description.

    Errors thrown

    If evaluated OP_CAT can throw the following errors:

    • If at time of evaluation the stack has fewer than two elements we throw the error: SCRIPT_ERR_INVALID_STACK_OPERATION
    • If at time of evaluation the top two stack elements have a combined size greater than MAX_SCRIPT_ELEMENT_SIZE (520 bytes) we throw the error: SCRIPT_ERR_PUSH_SIZE.

    Script verification flags

    While this PR does not contain any miner signaling and activation logic and can not activate OP_CAT, it does contain two flags which future activation logic could set to control activation of OP_CAT.

    • SCRIPT_VERIFY_OP_CAT IF a bitcoin node has this set to true, then it treat OP_CAT enabled for Tapscript. That is, OP_SUCCESS126 will be redefining to OP_CAT in Tapscript. If this was set to true at the consensus level this would cause a soft fork.

    SCRIPT_VERIFY_DISCOURAGE_OP_CAT When set to true, a node receiving any Tapscript transaction containing the opcode OP_CAT or OP_SUCCESS126 will reject the transaction throwing the error SCRIPT_ERR_DISCOURAGE_OP_CAT but not banning the node which relayed the transaction. This prevents nodes from relaying transactions with OP_CAT. This is equivalent to the behavior of SCRIPT_ERR_DISCOURAGE_OP_CAT = true when SCRIPT_VERIFY_OP_CAT = false as OP_CAT is an OP_SUCCESS (OP_SUCCESS126).

    This is how these two flags are intended to be used to ensure a smooth soft fork.

    Stage SCRIPT_VERIFY OP_CAT SCRIPT_VERIFY DISCOURAGE_OP_CAT Status
    1.Default False True This does not represent any change in behavior of the bitcoin-core node.
    2.Upgrading True True OP_CAT is activating.
    3.Upgraded True False It is clear that OP_CAT has been activated and the network has been upgraded.

    The flag SCRIPT_ERR_DISCOURAGE_OP_CAT provides a window of time for the network to fully activate, before nodes will relay or accept transactions containing OP_CAT in their mem pools.

    Tests

    This PR contains a suite of script tests to ensure that OP_CAT functions as expected. In the JSON script tests (script_tests.json), we test:

    • That if OP_CAT is not activated that there are no changes to bitcoin consensus.
    • That regardless of the activation or non-activation of OP_CAT in Tapscript, pre-Tapscript scripts, i.e. Bitcoin scripts, have no changes in behavior.
    • That OP_CAT when evaluated throws the expected error when there are less than two elements on the stack.
    • That OP_CAT when evaluated in a variety of circumstances and edge cases, successfully concatenates elements of the stack. This includes:
      • multiple calls to OP_CAT in a row,
      • evaluating OP_CAT inside of a IF conditional,
      • evaluating OP_CAT on zero size stack elements, random and large stack elements,
      • evaluating OP_CAT on values being moved to and from the alt stack,
      • and checking that when evaluated OP_CAT concatenates the elements in the expected order.

    All of these tests are designed to cover the happy path of OP_CAT, the various errors which OP_CAT can throw and all the corner cases between those two outcomes.

    Additionally we include three tests outside of the JSON script tests.

    • cat_simple and cat_empty_stack are designed to test OP_CAT outside of the JSON serialization regime. Ensure that we catch bugs that we might miss in the JSON script tests due to a bug introduced at the JSON serialization layer.
    • cat_dup_test enumerates all stack element sizes from 1 to 522 bytes and then enumerates up to 10 repetitions of OP\_DUP OP\_CAT. It then tests if the stack element would exceed 520 bytes and if so did OP_CAT throw the error SCRIPT_ERR_PUSH_SIZE. This allows us to be certain that OP_CAT will not introduce any OP\_DUP OP\_CAT memory exhaustion attacks.

    Better Tapscript tests in JSON script tests

    While writing these JSON script tests (script_tests.json) we ran into the following problem. The JSON script tests are simple and easy to write for pre-Tapscript scripts, but adding or changing a Tapscript test requires substantial work per test. Consider the following pre-tapscript test:

    0["'aa' 'bb'", "CAT 0x4c 0x02 0xaabb EQUAL", "P2SH,STRICTENC", "DISABLED_OPCODE", "CAT disabled"]
    

    whereas a Tapscript test for the same script (annotated with comments for better readability) would look like:

     0[
     1    [
     2        "aa",
     3        "bb",
     4        "7e4c02aabb87", // output script
     5        "c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d", // control block
     6        0.00000001
     7    ],
     8    "",
     9    "0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99", // Tapscript output
    10    "P2SH,WITNESS,TAPROOT",
    11    "OK",
    12    "TAPSCRIPT CATs aa and bb together and checks if EQUAL to aabb"
    13]
    

    Computing the Tapscript output, such as 0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99, requires writing custom code and running it for each test. The same is true for the Tapscript control block, such as c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d. If a test is changed or updated new outputs and control blocks must be computed. The complexity of doing this is likely the reason that no one has added any Tapscript tests to JSON script tests until this PR.

    In this PR we address this issue by adding the following improvements to JSON script tests:

    • Adding simple macros ("#SCRIPT# and #CONTROLBLOCK#) that allow the script test parser to automatically generate and inject a valid Tapscript output and control block to be computed automatically from the JSON script.
    • Allowing Tapscript scripts to use the human readable strings like pre-script scripts by marking the location of the script in the witness stack using #SCRIPT#. This transforms the unreadable script 7e4c02aabb87 into #SCRIPT# CAT 0x4c 0x02 0xaabb EQUAL.

    This results in the following JSON script test which is far easier to write and easier to read.

     0[
     1    [
     2        "aa",
     3        "bb",
     4        "#SCRIPT# CAT",
     5        "#CONTROLBLOCK#",
     6        0.00000001
     7    ],
     8    "",
     9    "0x51 0x20 #TAPROOTOUTPUT#",
    10    "P2SH,WITNESS,TAPROOT,OP_CAT",
    11    "OK",
    12    "TAPSCRIPT Test of OP_CAT flag by calling CAT on two elements. TAPSCRIPT_OP_CAT flag is set so CAT is executed."
    13],
    
  2. DrahtBot commented at 10:40 pm on January 14, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29247.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #29270 (Implement OP_CHECKSIGFROMSTACK(VERIFY) by reardencode)
    • #29269 (Add OP_INTERNALKEY for Tapscript by reardencode)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot renamed this:
    Reenable OP_CAT
    Reenable OP_CAT
    on Jan 14, 2024
  4. DrahtBot added the label CI failed on Jan 14, 2024
  5. DrahtBot commented at 11:27 pm on January 14, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/20473822722

  6. ajtowns added the label Consensus on Jan 15, 2024
  7. 0xBEEFCAF3 marked this as a draft on Feb 1, 2024
  8. 0xBEEFCAF3 commented at 10:05 pm on February 1, 2024: none
    Drafting until I get the inquisition PR approved and I can get the builds passing.
  9. moonsettler commented at 10:39 am on February 15, 2024: none

    Some of us have been playing with the idea of neutered CAT: instead of the MAX_SCRIPT_ELEMENT_SIZE (520 bytes) the maximum output size would be 80 bytes.

    Rationale: It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

    Combined with CSFS it can be used as a signed datacarrier, for which the ‘standard’ limit is 80 bytes, it also has to be smaller than 84 bytes which is required for building CTV templates on the stack, and larger than 64/65/72 bytes which are respectively needed for:

    • Merkle inclusion: 2x 20/32 byte hashes
    • LN-symmetry (LNhance): 2x 32 byte hashes
    • Separate sig: 64/72 bytes (0-conf bonds, staking contracts)

    Could be a livable compromise between the conservatives that want to preserve certain characteristics of bitcoin and the prometheans who want to give the developers more practical and useful tools to build with.

  10. bigspider commented at 9:16 am on February 16, 2024: none

    Rationale: It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

    Script is already expressive enough to (awkwardly, expensively) compute arbitrary SHA256 hashes on the stack, except that the result would be broken in smaller pieces of at most 4 bytes. With CAT, those can be concatenated to get a single 32-byte result.

    Therefore, neutering CATs does not achieve the desired result of preventing the CHECKSIG tricks, unless you limit the length of the result to less than 32 bytes - which would also neuter most of the utility of the opcode.

  11. 0xBEEFCAF3 force-pushed on May 26, 2024
  12. 0xBEEFCAF3 force-pushed on May 26, 2024
  13. 0xBEEFCAF3 force-pushed on May 27, 2024
  14. 0xBEEFCAF3 force-pushed on May 27, 2024
  15. DrahtBot removed the label CI failed on May 27, 2024
  16. 0xBEEFCAF3 marked this as ready for review on May 27, 2024
  17. 0xBEEFCAF3 force-pushed on Aug 27, 2024
  18. DrahtBot added the label CI failed on Aug 27, 2024
  19. DrahtBot commented at 3:16 pm on August 27, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/29308305455

    Make sure to run all tests locally, according to the documentation.

    The failure may happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  20. 0xBEEFCAF3 force-pushed on Aug 29, 2024
  21. 0xBEEFCAF3 renamed this:
    Reenable OP_CAT
    CAT in Tapscript (BIP-347)
    on Aug 29, 2024
  22. 0xBEEFCAF3 force-pushed on Aug 29, 2024
  23. 0xBEEFCAF3 force-pushed on Aug 31, 2024
  24. 0xBEEFCAF3 force-pushed on Aug 31, 2024
  25. awin28522 approved
  26. 0xBEEFCAF3 force-pushed on Aug 31, 2024
  27. DrahtBot removed the label CI failed on Aug 31, 2024
  28. DrahtBot added the label CI failed on Sep 11, 2024
  29. in src/script/interpreter.cpp:1823 in f5807a52a3 outdated
    1818@@ -1800,10 +1819,19 @@ static bool ExecuteWitnessScript(const Span<const valtype>& stack_span, const CS
    1819             }
    1820             // New opcodes will be listed here. May use a different sigversion to modify existing opcodes.
    1821             if (IsOpSuccess(opcode)) {
    1822-                if (flags & SCRIPT_VERIFY_DISCOURAGE_OP_SUCCESS) {
    1823-                    return set_error(serror, SCRIPT_ERR_DISCOURAGE_OP_SUCCESS);
    1824+                if (opcode == OP_CAT) {
    1825+                    if (flags & SCRIPT_VERIFY_DISCOURAGE_OP_CAT) {
    


    rot13maxi commented at 7:56 pm on September 12, 2024:
    SCRIPT_VERIFY_DISCOURAGE_OP_CAT is acting as a CAT-specific SCRIPT_VERIFY_DISCOURAGE_OP_SUCCESS that can be turned off while still not relaying transasctions with other OP_SUCCESS opcodes. Since this is the first time we’re carving out an OP_SUCCESS and doing this, it would be great to have a test that exercises the combinations of SCRIPT_VERIFY_DISCOURAGE_OP_CAT and SCRIPT_VERIFY_OP_CAT

    0xBEEFCAF3 commented at 3:25 pm on September 17, 2024:
    Great suggestion! Added some additional unit tests to script_tests.json testing different combinations of scripts with and without (SCRIPT_VERIFY_DISCOURAGE_OP_CAT, SCRIPT_VERIFY_OP_CAT and SCRIPT_VERIFY_DISCOURAGE_OP_SUCCESS)

    ajtowns commented at 2:19 pm on October 9, 2024:
  30. DrahtBot removed the label CI failed on Sep 15, 2024
  31. 0xBEEFCAF3 force-pushed on Sep 17, 2024
  32. DrahtBot added the label CI failed on Sep 17, 2024
  33. DrahtBot commented at 2:33 am on September 17, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/30231832503

    Make sure to run all tests locally, according to the documentation.

    The failure may happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  34. 0xBEEFCAF3 force-pushed on Sep 17, 2024
  35. DrahtBot removed the label CI failed on Sep 17, 2024
  36. achow101 commented at 4:04 pm on October 15, 2024: member
    Converting to draft while waiting for consensus on this.
  37. achow101 marked this as a draft on Oct 15, 2024
  38. EthanHeilman commented at 2:55 pm on October 19, 2024: contributor

    @achow101 What does draft vs non-draft mean in the bitcoin-core github? My understanding was moving from draft to non-draft means code is ready for review. Similar to moving out of WIP.

    Is draft/non-draft is being used to signal community consensus or bitcoin-core consensus? If so, I agree this should be a draft.

  39. achow101 commented at 8:40 pm on October 19, 2024: member

    In general, draft means that a PR is not in a state where it could be merged. It indicates to reviewers that they may not want to do any in depth code review of the PR yet.

    In the context of this PR, there does not appear to be consensus for the concept of OP_CAT in TapScript yet, so this PR cannot be merged even if the code is okay. Hence marking it as a draft.

    Note that this is a result of a Kill/Shill/Merge session that occurred at the most recent CoreDev.

  40. EthanHeilman commented at 2:28 pm on October 20, 2024: contributor
    @achow101 Thanks, that’s helpful for understanding the context. If consensus forms around OP_CAT should I or @0xBEEFCAF3 mark this as no longer as draft? I can’t speak for @0xBEEFCAF3 but I would feel slightly uncomfortable making that decision as it would open me up to criticism that I was attempting to manufacture consensus, even if there was consensus. Then again if that is the expectation I suppose we’d have to do that and take the licks. What is the process here?
  41. achow101 commented at 3:10 pm on October 20, 2024: member

    If consensus forms around OP_CAT should I or @0xBEEFCAF3 mark this as no longer as draft?

    Yes. A maintainer may also do so as well.

  42. 0xBEEFCAF3 force-pushed on Oct 29, 2024
  43. DrahtBot commented at 1:11 am on October 30, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/32250870897

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  44. DrahtBot added the label CI failed on Oct 30, 2024
  45. 0xBEEFCAF3 force-pushed on Nov 9, 2024
  46. DrahtBot added the label Needs rebase on Nov 13, 2024
  47. OP_CAT in tapscript
    Implement OP_CAT as a new Tapscript op code by redefining the opcode OP_SUCCESS126 (126 in decimal and 0x7e in hexadecimal). This is the same opcode value used by the original OP_CAT.
    
    When evaluated, the OP_CAT instruction:
    
    Pops the top two values off the stack,
    concatenates the popped values together in stack order,
    and then pushes the concatenated value on the top of the stack.
    OP_CAT fails if there are fewer than two values on the stack or if a concatenated value would have a combined size greater than the maximum script element size of 520 bytes.
    
    See BIP 347 for a deeper description.
    ffeabdc599
  48. tests: add unit tests for OP_CAT
    Co-authored-by: Ethan Heilman <ethan.r.heilman@gmail.com>
    
    This commit introduces unit tests for opcat in script_tests.json.
    Additionally, it creates new test utilities for future Taproot script
    tests within script_tests.json. The key features of this commit are the
    addition of three new tags: `#SCRIPT#`, `#CONTROLBLOCK#`, and
    `#TAPROOTOUTPUT#`. These tags streamline the test creation process by
    eliminating the need to manually generate these components outside the
    test suite.
    
    * `#SCRIPT#`: Parses Tapscript and outputs a byte string of opcodes.
    * `#CONTROLBLOCK#`: Automatically generates the control block for a given
    Taproot output.
    * `#TAPROOTOUTPUT#`: Generates the final Taproot scriptPubKey.
    33565e7975
  49. test: add functional test for OP_CAT spends
    The goal of this functional test is to ensure OP_CAT spends are still
    disabled by default in segwitv0 and legacy spends. Spending such inputs
    should result in `mandatory-script-verify-flag-failed (Attempted to use a disabled opcode)`.
    While spending OP_CAT inputs in tapscript should be discouraged under
    the default `STANDARD_SCRIPT_VERIFY_FLAGS`.
    4e337f7245
  50. 0xBEEFCAF3 force-pushed on Nov 15, 2024
  51. DrahtBot removed the label CI failed on Nov 15, 2024
  52. DrahtBot removed the label Needs rebase on Nov 15, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-21 09:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me