tests: Add witness commitment if we have a witness transaction in FullBlockTest.update_block() #31823

pull Christewart wants to merge 1 commits into bitcoin:master from Christewart:2025-02-07-featureblockpy-witnesscommitment changing 2 files +20 −0
  1. Christewart commented at 6:05 pm on February 7, 2025: contributor

    This is useful for test cases where we want to test logic invalid blocks that contain witness transactions. If we don’t add the witness commitment as per BIP141, blocks will be rejected with the error Block mutated.

    This change was needed in https://github.com/ajtowns/bitcoin/pull/13 which is a soft fork proposal to disallow 64 byte transactions. We want to test that 64 byte transactions serialized without the witness are invalid. If we do not have this change, we cannot directly test the logic that rejects 64 byte transactions.

    I decided to PR this upstream as many soft fork proposals may not see the light of day, but this functionality seems strictly additive to the test framework.

  2. DrahtBot commented at 6:05 pm on February 7, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31823.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK theStack, sedited, glozow
    Stale ACK kevkevinpal

    If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

  3. DrahtBot added the label Tests on Feb 7, 2025
  4. stark-3k commented at 11:13 pm on February 8, 2025: none

    Hi, looking at feature_block.py — it doesn’t seem to contain any SegWit-specific validation tests. They remain in feature_segwit.py, p2p_segwit.py, etc; although I’m not completely confident about the scope of this file. I have a couple of questions(avoid if irrelevant):

    1. Is there any test case that can be covered in the current implementation that needs this change to be implemented?
    2. Wouldn’t it be better to create a new test file like feature_<soft_fork>.py to test the consensus changes made in a new soft fork, following the convention(feature_segwit,feature_taproot,etc)? Also, why disallow 64 byte transactions? Thanks
  5. Christewart commented at 6:15 pm on February 9, 2025: contributor
    1. Is there any test case that can be covered in the current implementation that needs this change to be implemented?

    There is not currently. This short coming in the test framework was discovered while working on test cases for disallowing 64 byte transactions in the bitcoin blockchain. This python test case would not fail correctly if this change wasn’t added to the test framework: https://github.com/ajtowns/bitcoin/pull/13/files#diff-d390e9e12bfa34a462fdff9f1894d7dc3edb45c26cf55df486d92864a3052fafR139

    2. Wouldn't it be better to create a new test file like `feature_<soft_fork>.py` to test the consensus changes made in a new soft fork, following the convention(`feature_segwit`,`feature_taproot`,etc)?
    

    This PR is not proposing a soft fork. This PR makes the update_block function BIP141 compliant.

    Also, why disallow 64 byte transactions?

    Here are some resources on why 64 byte transactions are problematic.

  6. in test/functional/feature_block.py:1422 in d39e7287e8 outdated
    1400@@ -1400,6 +1401,9 @@ def update_block(self, block_number, new_transactions):
    1401         self.add_transactions_to_block(block, new_transactions)
    1402         old_sha256 = block.sha256
    1403         block.hashMerkleRoot = block.calc_merkle_root()
    1404+        has_witness_tx = any(not tx.wit.is_null() for tx in block.vtx)
    1405+        if has_witness_tx:
    1406+            add_witness_commitment(block)
    


    kevkevinpal commented at 2:44 am on February 16, 2025:

    Right now this code would not be executed at all, do you think it makes sense to add a new invalid tx with witness data to invalid_txs.py?

    Similar to what you had here this should execute add_witness_commitment


    Christewart commented at 3:03 pm on February 27, 2025:

    Hi @kevkevinpal !

    I took your suggestion and added the test in 6109bc5

  7. Christewart force-pushed on Feb 27, 2025
  8. Christewart requested review from kevkevinpal on Feb 27, 2025
  9. kevkevinpal commented at 5:36 pm on February 27, 2025: contributor
    ACK 6109bc529e234b464d406495bb9644254ca7fc9e
  10. kevkevinpal commented at 5:44 pm on February 27, 2025: contributor
    The CI failure looks unrelated to your change
  11. Christewart commented at 6:08 pm on February 27, 2025: contributor

    The CI failure looks unrelated to your change

    Maybe the gods (AKA bitcoin core maintainers 👼 📿 ) will restart the build for me :pray: 😄

  12. in test/functional/data/invalid_txs.py:134 in 6109bc529e outdated
    135+# but doesnt spend a segwit output
    136+class ExtraWitness(BadTxTemplate):
    137+    expect_disconnect = False
    138+    valid_in_block = False
    139+    block_reject_reason = "mandatory-script-verify-flag-failed (Witness provided for non-witness script)"
    140+
    


    maflcko commented at 8:44 am on February 28, 2025:
    Missing reject_reason field?

    Christewart commented at 5:19 pm on February 28, 2025:

    It is my understanding that the feature_block.py test suites turns on -acceptnonstdtxn=1 which turns off this policy check. Thus we don’t receive any error message when submitting the transaction to the mempool.

    https://github.com/bitcoin/bitcoin/blob/3c1f72a36700271c7c1293383549c3be29f28edb/src/validation.cpp#L886


    theStack commented at 3:24 pm on November 12, 2025:

    It is my understanding that the feature_block.py test suites turns on -acceptnonstdtxn=1 which turns off this policy check. Thus we don’t receive any error message when submitting the transaction to the mempool.

    I assume you are referring to the functional test p2p_invalid_tx.py rather than feature_block.py, as the latter only sends blocks to the node rather than individual txs? Specifying the error message as reject_reason (meaning the invalid tx case is used to test both via block and individual tx submission) seems to work after increasing the tx size, in order to avoid a “tx-size-small” rejection:

     0diff --git a/test/functional/data/invalid_txs.py b/test/functional/data/invalid_txs.py
     1index db8871e001..d128ac1682 100644
     2--- a/test/functional/data/invalid_txs.py
     3+++ b/test/functional/data/invalid_txs.py
     4@@ -135,14 +135,13 @@ class SizeTooSmall(BadTxTemplate):
     5 # reject a transaction that contains a witness
     6 # but doesnt spend a segwit output
     7 class ExtraWitness(BadTxTemplate):
     8-    expect_disconnect = False
     9-    valid_in_block = False
    10-    block_reject_reason = "mandatory-script-verify-flag-failed (Witness provided for non-witness script)"
    11+    expect_disconnect = True
    12+    reject_reason = "mandatory-script-verify-flag-failed (Witness provided for non-witness script)"
    13
    14     def get_tx(self):
    15         tx = CTransaction()
    16         tx.vin.append(self.valid_txin)
    17-        tx.vout.append(CTxOut(0, CScript()))
    18+        tx.vout.append(CTxOut(0, basic_p2sh))
    19         tx.wit.vtxinwit = [CTxInWitness()]
    20         tx.wit.vtxinwit[0].scriptWitness.stack = [CScript([OP_TRUE])]
    21         return tx
    22diff --git a/test/functional/p2p_invalid_tx.py b/test/functional/p2p_invalid_tx.py
    23index ae9771e7cb..b1979cc76d 100755
    24--- a/test/functional/p2p_invalid_tx.py
    25+++ b/test/functional/p2p_invalid_tx.py
    26@@ -160,7 +160,7 @@ class InvalidTxRequestTest(BitcoinTestFramework):
    27             node.p2ps[0].send_txs_and_test([rejected_parent], node, success=False)
    28
    29         self.log.info('Test that a peer disconnection causes erase its transactions from the orphan pool')
    30-        with node.assert_debug_log(['Erased 100 orphan transaction(s) from peer=26']):
    31+        with node.assert_debug_log(['Erased 100 orphan transaction(s) from peer=27']):
    32             self.reconnect_p2p(num_connections=1)
    33
    34         self.log.info('Test that a transaction in the orphan pool is included in a new tip block causes erase this transaction from the orphan pool')
    

    In general, I think only specifying a block_reject_reason rarely makes sense, as a consensus-invalid tx is usually also rejected by mempool. The only exception I could think of is txs that land in the orphanage due to missing inputs. Currently that’s the case for the invalid tx classes BadInputOutpointIndex and NonexistentInput.


    Christewart commented at 4:42 pm on November 12, 2025:

    Hi @theStack , thanks for taking a look at this.

    I assume you are referring to the functional test p2p_invalid_tx.py rather than feature_block.py, as the latter only sends blocks to the node rather than individual txs?

    No, my intention was relaying invalid blocks. This is useful for testing new consensus proposals that are currently policy - which 64 byte transactions are currently disallowed by policy.


    theStack commented at 2:59 pm on November 13, 2025:

    I see. Note that my comment was referring to your reply about why the reject_reason field is missing in the ExtraWitness invalid tx test case class, particularly about this sentence: “Thus we don’t receive any error message when submitting the transaction to the mempool.”. All txs specified in data/invalid_txs.py are submitted to mempool in p2p_invalid_tx.py and are expected to be rejected (unless reject_reason is set explicitly to None), otherwise this functional test would fail. I think it makes sense to specify for what exact reason it fails. With the test case as-is (if you want to keep it that small), this would be “tx-size-small”:

     0diff --git a/test/functional/data/invalid_txs.py b/test/functional/data/invalid_txs.py
     1index db8871e001..fca10ae416 100644
     2--- a/test/functional/data/invalid_txs.py
     3+++ b/test/functional/data/invalid_txs.py
     4@@ -137,6 +137,7 @@ class SizeTooSmall(BadTxTemplate):
     5 class ExtraWitness(BadTxTemplate):
     6     expect_disconnect = False
     7     valid_in_block = False
     8+    reject_reason = "tx-size-small"
     9     block_reject_reason = "mandatory-script-verify-flag-failed (Witness provided for non-witness script)"
    10
    11     def get_tx(self):
    

    (note that the second part of my previous posted patch is not relevant anymore after rebase)


    Christewart commented at 5:20 pm on November 14, 2025:

    Ok i think i understand what you are saying conceptually - but perhaps I’m being a bit dense - but toggling the reject_reason comment that I have in https://github.com/bitcoin/bitcoin/pull/31823/commits/94a1bbb0d7f70513b68b28d4111cac56efce864a doesn’t seem to affect whether the p2p_invalid_tx.py test suite passes or fails. I.e. the reject_reason seems irrelevant?

    I’m a bit short on time at the moment and will take a look further in the next day or 2, but figured I would throw that out here in case you know why that is the case.


    theStack commented at 6:09 pm on November 14, 2025:

    No worries, I think it’s indeed a bit confusing. The testing of a reject reason is done by asserting that the corresponding string appears in the bitcoind log file. If no explicit reject_reason is specified in an invalid tx test case, the default value of "" (empty string) is used, i.e. a submission error is still expected, but the exact reason for why it fails doesn’t matter.

    You would see a difference in behaviour if you specified reject_reason = None, meaning that the expected outcome is that the tx submission succeeds. As your introduced invalid tx doesn’t is rejected by the mempool, the functional test p2p_invalid_tx.py would then fail. // EDIT: that was wrong, see below


    Christewart commented at 8:23 pm on November 14, 2025:

    I tried reject_reason = None locally and the p2p_invalid_tx.py tests seem to still pass 🤔.

    If I understand this line of code correctly, it seems that passing in None or "" as the reject_reason results in the same behavior? I’m not super familiar with python so its a bit unclear to me if None and "" are evaluated as false in this if statement.

    https://github.com/bitcoin/bitcoin/blob/e221b2524659d22cc5a2b0d0023254115a7a5622/test/functional/test_framework/p2p.py#L915


    Christewart commented at 8:46 pm on November 14, 2025:

    Ok, hopefully bringing this home. You are right that the reject_reason = "tx-size-small" makes sense. I’ve added it in a626caec847c1c542c663be6664229857d2bf220.

    The other stuff I think is incorrect as None and """ seem to result in the same behavior.

    Long winded way of saying you were right and I’ve incorporated your review.

    Please re-review when you have time.


    theStack commented at 8:51 pm on November 14, 2025:

    Oops yes, you’re right, sorry for creating even more confusion. It seems that all tx test cases in invalid_txs.py are expected to be rejected from mempool (which makes sense I guess, as they are… invalid), and omitting a reject_reason or setting it explicitly to None leads indeed to the same behaviour.

    I’m not super familiar with python so its a bit unclear to me if None and "" are evaluated as false in this if statement.

    Yeah, an empty string is falsy:

    0>>> "true" if None else "false"
    1'false'
    2>>> "true" if "" else "false"
    3'false'
    
  13. Christewart requested review from maflcko on Mar 3, 2025
  14. Christewart force-pushed on Apr 8, 2025
  15. kevkevinpal commented at 9:49 pm on April 11, 2025: contributor
    reACK 9ab4e3f
  16. DrahtBot added the label CI failed on Jun 15, 2025
  17. in test/functional/data/invalid_txs.py:147 in 9ab4e3fe7e outdated
    142+        tx = CTransaction()
    143+        tx.vin.append(self.valid_txin)
    144+        tx.vout.append(CTxOut(0, CScript()))
    145+        tx.wit.vtxinwit = [CTxInWitness()]
    146+        tx.wit.vtxinwit[0].scriptWitness.stack = [CScript([OP_TRUE])]
    147+        tx.calc_sha256()
    


    DrahtBot commented at 8:11 am on June 16, 2025:
    [01:04:52.315]  AttributeError: ‘CTransaction’ object has no attribute ‘calc_sha256’

    kevkevinpal commented at 1:39 pm on June 19, 2025:

    So I see calc_sha256 on your branch but on the master bitcoin/bitcoin branch it is missing

    looks to be removed in this commit https://github.com/bitcoin/bitcoin/commit/9b3dce24a3336a02563412b541a3fb2003e92506

    so you’ll need to add it back in


    Christewart commented at 5:51 pm on June 27, 2025:
    The line wasn’t necessary, I removed it in 43c2613305697255d294d9dd02f1cd19246913c9. Thanks for pointing this out though as the DrahtBot message wasn’t clear to me!
  18. Christewart force-pushed on Jun 27, 2025
  19. Christewart requested review from stark-3k on Jun 27, 2025
  20. DrahtBot removed the label CI failed on Jun 27, 2025
  21. kevkevinpal commented at 6:13 pm on September 29, 2025: contributor
    reACK 43c2613
  22. in test/functional/data/invalid_txs.py:130 in 43c2613305 outdated
    131@@ -130,6 +132,21 @@ def get_tx(self):
    132         assert MIN_STANDARD_TX_NONWITNESS_SIZE - 1 == 64
    133         return tx
    134 
    135+# reject a transaction that contains a witness
    136+# but doesnt spend a segwit output
    


    maflcko commented at 9:25 am on November 13, 2025:
    doesnt -> doesn’t [missing apostrophe in contraction “doesn't”]
    

    Christewart commented at 8:46 pm on November 14, 2025:
    Taken in a626caec847c1c542c663be6664229857d2bf220
  23. maflcko added the label Needs rebase on Nov 13, 2025
  24. maflcko closed this on Nov 13, 2025

  25. maflcko reopened this on Nov 13, 2025

  26. DrahtBot added the label CI failed on Nov 13, 2025
  27. DrahtBot removed the label Needs rebase on Nov 13, 2025
  28. theStack commented at 3:00 pm on November 13, 2025: contributor
    Concept ACK, happy to re-review after rebase
  29. Christewart force-pushed on Nov 14, 2025
  30. Christewart force-pushed on Nov 14, 2025
  31. Christewart force-pushed on Nov 14, 2025
  32. Christewart force-pushed on Nov 14, 2025
  33. DrahtBot removed the label CI failed on Nov 14, 2025
  34. tests: Add witness commitment if we have a witness transaction in FullBlockTest.update_block() a7c96f874d
  35. in test/functional/data/invalid_txs.py:133 in a626caec84
    125@@ -124,6 +126,22 @@ def get_tx(self):
    126         assert MIN_STANDARD_TX_NONWITNESS_SIZE - 1 == 64
    127         return tx
    128 
    129+# reject a transaction that contains a witness
    130+# but doesn't spend a segwit output
    131+class ExtraWitness(BadTxTemplate):
    132+    expect_disconnect = False
    133+    valid_in_block = False
    


    theStack commented at 10:00 pm on November 14, 2025:

    these two lines can be dropped; expect_disconnect was removed in #33050 (commit 876dbdfb4702410dfd4037614dc9298a0c09c63e) and valid_in_block is set to False by default anyways

    other than that, lgtm


    Christewart commented at 0:44 am on November 15, 2025:
    Done in a7c96f8
  36. Christewart force-pushed on Nov 15, 2025
  37. DrahtBot added the label CI failed on Nov 15, 2025
  38. theStack approved
  39. theStack commented at 1:09 am on November 15, 2025: contributor
    ACK a7c96f874de13ace9814da92fd6160a126a97ebf
  40. DrahtBot removed the label CI failed on Nov 15, 2025
  41. sedited approved
  42. sedited commented at 9:59 am on November 17, 2025: contributor
    ACK a7c96f874de13ace9814da92fd6160a126a97ebf
  43. Christewart commented at 9:07 pm on November 19, 2025: contributor
    I think this is good for merge?
  44. glozow commented at 4:09 pm on December 9, 2025: member
    ACK a7c96f874de13ace9814da92fd6160a126a97ebf
  45. glozow merged this on Dec 9, 2025
  46. glozow closed this on Dec 9, 2025


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-11 03:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me