Add txids with non-standard inputs to reject filter #19620

pull sdaftuar wants to merge 2 commits into bitcoin:master from sdaftuar:2020-07-reject-unknown-wit changing 5 files +55 −8
  1. sdaftuar commented at 3:37 pm on July 29, 2020: member

    Our policy checks for non-standard inputs depend only on the non-witness portion of a transaction: we look up the scriptPubKey of the input being spent from our UTXO set (which is covered by the input txid), and the p2sh checks only rely on the scriptSig portion of the input.

    Consequently it’s safe to add txids of transactions that fail these checks to the reject filter, as the witness is irrelevant to the failure. This is helpful for any situation where we might request the transaction again via txid (either from txid-relay peers, or if we might fetch the transaction via txid due to parent-fetching of orphans).

    Further, in preparation for future witness versions being deployed on the network, ensure that WITNESS_UNKNOWN transactions are rejected in AreInputsStandard(), so that transactions spending v1 (or greater) witness outputs will fall into this category of having their txid added to the reject filter.

  2. sdaftuar commented at 3:46 pm on July 29, 2020: member

    I need to test this still myself, but I wanted to open this as a potential alternative to feeling the need to backport #18044 (as @jnewbery has proposed in #19606) in order to protect older software from wasting bandwidth in the event of taproot activation in the future.

    I have a branch where I backported this patch to 0.20 here: https://github.com/sdaftuar/bitcoin/tree/2020-07-reject-unknown-wit-0.20.

    Note that this PR should be independently useful to mitigate an issue @ariard brought up here: #18044 (review)

  3. in src/validation.cpp:693 in be23c2af9f outdated
    689@@ -690,7 +690,7 @@ bool MemPoolAccept::PreChecks(ATMPArgs& args, Workspace& ws)
    690 
    691     // Check for non-standard pay-to-script-hash in inputs
    692     if (fRequireStandard && !AreInputsStandard(tx, m_view))
    693-        return state.Invalid(TxValidationResult::TX_NOT_STANDARD, "bad-txns-nonstandard-inputs");
    694+        return state.Invalid(TxValidationResult::TX_INPUTS_NOT_STANDARD, "bad-txns-nonstandard-inputs");
    


    jnewbery commented at 4:33 pm on July 29, 2020:
    Strictly speaking, do we want to move this check above any other check that could return TX_NOT_STANDARD, so transactions with non-standard inputs are guaranteed to be put in the recentrejects filter? I don’t think it matters, since upgraded peers won’t relay transactions that fail other standardness checks, unless we loosen those checks for some other reason.

    sdaftuar commented at 4:50 pm on July 29, 2020:
    Yeah I agree. It doesn’t seem worth the effort to figure out how to make it so that a tx that would fail in AreInputsStandard() will always have its txid go into the rejects filter – there are a lot of checks that happen before that (including checks that don’t depend on looking up the inputs first), which we’d have to move around somehow and complicate the logic. But, if this proves to be insufficient for some reason we can always try to revisit.

    instagibbs commented at 8:31 pm on July 31, 2020:
    nit: since we’re touching the line could stand to put brackets on the if statement block
  4. jnewbery commented at 4:33 pm on July 29, 2020: member

    Concept ACK. Looks like a good change.

    I think we might independently want to backport wtxid relay. It’s really not that difficult of a backport, and we eventually want wtxid to be the standard way to relay txs and getting it into v0.20 would help with that.

  5. DrahtBot added the label P2P on Jul 29, 2020
  6. DrahtBot added the label TX fees and policy on Jul 29, 2020
  7. DrahtBot added the label Validation on Jul 29, 2020
  8. JeremyRubin commented at 5:44 pm on July 29, 2020: contributor

    Concept ACK and lite code review ACK.

    It seems to me that this change won’t help that much in the future upgrade case because even though this fixes witness versions things like leaf versions in taproot (which may be upgraded more often) will require a witness dependent check, and can’t be txid reject filtered, unless we made it a policy that transactions spending taproots that mix and match leaf versions are non-standard (but only detectable in the case where an previous attempted witness is nonstandard).

  9. sdaftuar commented at 7:48 pm on July 29, 2020: member

    It seems to me that this change won’t help that much in the future upgrade case […] @JeremyRubin I agree, this is a bandaid to help this particular scenario. Going forward, wtxid-relay + a change to the p2p protocol for announcing/requesting unconfirmed ancestors of a transaction by wtxid seems like the right way to solve this type of problem more generally.

  10. sipa commented at 7:55 pm on July 29, 2020: member
    @sdaftuar @naumenkogs Random idea about this: post-Erlay it may make sense to have a message “i do not have have dependencies of txid txid”, at which point both sides add all known direct and indirect unconfirmed parents of txid they know of to their to-be-broadcast set; if erlay is in effect, the overlap will get cancelled out automatically.
  11. sdaftuar force-pushed on Jul 30, 2020
  12. sdaftuar commented at 9:33 pm on July 30, 2020: member
    Updated some comments and also used this same logic when processing orphans.
  13. in src/consensus/validation.h:29 in ba645670e6 outdated
    25@@ -26,7 +26,8 @@ enum class TxValidationResult {
    26      * is uninteresting.
    27      */
    28     TX_RECENT_CONSENSUS_CHANGE,
    29-    TX_NOT_STANDARD,          //!< didn't meet our local policy rules
    30+    TX_INPUTS_NOT_STANDARD,   //!< inputs (covered by txid) failed policy rules
    


    sipa commented at 9:38 pm on July 30, 2020:
    Would it make sense to call this TX_NOWIT_NON_STANDARD instead or so, so that perhaps later it could be extended to cover other forms of nonstandardness that we know are not due to witness data?

    sdaftuar commented at 9:43 pm on July 30, 2020:

    Well I didn’t expect that we’d ever go to the trouble of even doing this step (wtxid-relay was always the right solution, rather than trying to scatter detection of txid-is-definitely-bad failures through transaction validation), but if you think it’s plausible that we would repeat this type of thing in the future, then I agree this would make sense.

    We could also just as easily change this enum at that point though…


    sipa commented at 5:44 pm on July 31, 2020:
    Right, of course. The name can be changed later if it becomes more expansive.
  14. DrahtBot commented at 4:06 am on July 31, 2020: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #19498 (Tidy up ProcessOrphanTx by jnewbery)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  15. in src/net_processing.cpp:203 in ba645670e6 outdated
    195@@ -196,6 +196,12 @@ namespace {
    196      * the reject filter store wtxids is exactly what we want to avoid
    197      * redownload of a rejected transaction.
    198      *
    199+     * In cases where we can tell that a segwit transaction will fail
    200+     * validation no matter the witness, we may add the txid of such
    201+     * transaction to the filter as well. This can be helpful when
    202+     * communicating with txid-relay peers or if we were to fetch a transaction
    203+     * via wtxid (eg in our orphan handling).
    


    jnewbery commented at 9:39 am on July 31, 2020:
    Should be ‘via txid’

    sdaftuar commented at 6:31 pm on July 31, 2020:
    Thanks, fixed.
  16. jnewbery commented at 10:33 am on July 31, 2020: member

    Code changes look good to me. One suggested change to a comment inline.

    I think a functional test would be useful here. This change depends on the ordering of checks in ATMP, so there could potentially be subtle interactions there. Proving this out with a test for txid-relay peers and orphan fetching would be great.

  17. sdaftuar force-pushed on Jul 31, 2020
  18. sdaftuar commented at 6:37 pm on July 31, 2020: member
    Happy to include a functional test if anyone is able to put one together for this.
  19. JeremyRubin commented at 6:47 pm on July 31, 2020: contributor
    @instagibbs might?
  20. instagibbs commented at 6:51 pm on July 31, 2020: member
    I think this deserves a test and shouldn’t be too bad. I’ll take a crack at it this weekend.
  21. luke-jr approved
  22. luke-jr commented at 7:23 pm on July 31, 2020: member
    utACK
  23. sipa commented at 8:30 pm on July 31, 2020: member
    Code review ACK acdd7f3dda6d6c89e6f91c781bf941bfc66d8ba3. A test would be nice.
  24. in src/net_processing.cpp:2071 in acdd7f3dda outdated
    2066+                // (the scriptPubKey being spent is covered by the txid).
    2067+                // Add the txid to the reject filter to prevent repeated
    2068+                // processing of this transaction in the event that child
    2069+                // transactions are later received (resulting in
    2070+                // parent-fetching by txid via the orphan-handling logic).
    2071+                if (orphan_state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && orphanTx.GetWitnessHash() != orphanTx.GetHash()) {
    


    instagibbs commented at 8:35 pm on July 31, 2020:
    why does orphanTx.GetWitnessHash() != orphanTx.GetHash() matter?

    instagibbs commented at 8:42 pm on July 31, 2020:
    Ok, because the wtxid is inserted above already, it would be a redundant entry for a non-witness-having tx?

    luke-jr commented at 9:16 pm on July 31, 2020:
    Yes, inserting the same hash twice degrades the filter IIRC

    jnewbery commented at 9:48 pm on July 31, 2020:
    Not quite. Adding the same entry is a no-op on the underlying bloom filter (since the same entry hashes to the same values). It does however increment the internal nEntriesThisGeneration counter, which means we’ll be able to add fewer entries before wiping a previous generation. It effectively lowers the actual capacity of the rolling bloom filter.

    sdaftuar commented at 5:43 pm on August 3, 2020:
    Added a comment to explain this.
  25. instagibbs commented at 9:17 pm on July 31, 2020: member

    Yeah might be worth noting that’s what it’s for. Took me a bit of work.

    On Fri, Jul 31, 2020, 5:16 PM Luke Dashjr notifications@github.com wrote:

    @luke-jr commented on this pull request.

    In src/net_processing.cpp https://github.com/bitcoin/bitcoin/pull/19620#discussion_r463848628:

    @@ -2053,6 +2060,17 @@ void static ProcessOrphanTx(CConnman& connman, CTxMemPool& mempool, std::set<uin // if we start doing this too early. assert(recentRejects); recentRejects->insert(orphanTx.GetWitnessHash());

    •            // If the transaction failed for TX_INPUTS_NOT_STANDARD,
      
    •            // then we know that the witness was irrelevant to the policy
      
    •            // failure, since this check depends only on the txid
      
    •            // (the scriptPubKey being spent is covered by the txid).
      
    •            // Add the txid to the reject filter to prevent repeated
      
    •            // processing of this transaction in the event that child
      
    •            // transactions are later received (resulting in
      
    •            // parent-fetching by txid via the orphan-handling logic).
      
    •            if (orphan_state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && orphanTx.GetWitnessHash() != orphanTx.GetHash()) {
      

    Yes, inserting the same hash twice degrades the filter IIRC

    — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bitcoin/pull/19620#discussion_r463848628, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMAFUZZCX3AQ6GZCIXYOX3R6MYCNANCNFSM4PLXHWJQ .

  26. in src/policy/policy.cpp:156 in acdd7f3dda outdated
    151@@ -152,6 +152,8 @@ bool IsStandardTx(const CTransaction& tx, bool permit_bare_multisig, const CFeeR
    152  * script can be anything; an attacker could use a very
    153  * expensive-to-check-upon-redemption script like:
    154  *   DUP CHECKSIG DROP ... repeated 100 times... OP_1
    155+ *
    156+ * Note that only the non-witness portion of the transaction is checked here.
    


    ariard commented at 0:19 am on August 1, 2020:
    I think new comment add a bit of ambiguity with regards to name WITNESS_UNKNOWN. Solver() only check the witnessversion to qualify a spent output type as such. Maybe WITNESSVERSION_UNKNOWN ?

    sdaftuar commented at 5:46 pm on August 3, 2020:
    Not taking this change – I think this is clear enough, and not worth changing the 15 places in the codebase that enum appears because of its usage in this context.
  27. in src/policy/policy.cpp:169 in acdd7f3dda outdated
    165@@ -164,7 +166,7 @@ bool AreInputsStandard(const CTransaction& tx, const CCoinsViewCache& mapInputs)
    166 
    167         std::vector<std::vector<unsigned char> > vSolutions;
    168         TxoutType whichType = Solver(prev.scriptPubKey, vSolutions);
    169-        if (whichType == TxoutType::NONSTANDARD) {
    170+        if (whichType == TxoutType::NONSTANDARD || whichType == TxoutType::WITNESS_UNKNOWN) {
    


    ariard commented at 0:29 am on August 1, 2020:

    This new check seems to overrule the SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_WITNESS_PROGRAM one in VerifyWitnessProgram() ? AFAICT, we don’t have test coverage for this standard script flag.

    0                                   AssertionError: [node 1] Expected messages "['non-mandatory-script-verify-flag (Witness version reserved for soft-fork upgrades)']" does not partially match log:
    1                                    - 2020-08-01T00:14:31.620579Z [msghand] received: tx (87 bytes) peer=1
    2                                    - 2020-08-01T00:14:31.620845Z [msghand] 52ba45c4ff1da3119475d9ac8201c4ea7018be75444de05aeda9dba22cde28d6 from peer=1 was not accepted: bad-txns-nonstandard-inputs
    3                                    - 2020-08-01T00:14:31.661127Z [msghand] received: ping (8 bytes) peer=1
    

    See new test on both master/diff: https://github.com/ariard/bitcoin/commit/a97e4635ae592e582d186079911d943c58df0e54


    sdaftuar commented at 5:47 pm on August 3, 2020:
    I’m not sure I understand whether you’re asking for a change, or pointing out a bug, or something else? Could you explain if anything needs to be done here?

    ariard commented at 9:38 pm on August 3, 2020:

    Ah sorry, previously a segwit v1+ spend would have been rejected by this check : https://github.com/bitcoin/bitcoin/blob/34eb2362581d4d8f0bfd3baa12ba750afaf85c62/src/script/interpreter.cpp#L1552 in VerifyWitnessProgram

    Now, a segwit v+1 spend is going to be rejected by this new check in AreInputsStandards. As linked test confirms if you run it on both master/diff, it’s duplicating a policy rule enforcement.

    Is this the first case where we duplicate policy rule checks ? I don’t know, but we should avoid issues in case of future changes making them inconsistent (like verifying first with bitcoinconsensus_verify_script then broadcasting through ATMP and obtaining divergent results).

    If you think it’s worthy to fix, we can a) remove L1552 in script/interpreter.cpp or b) add a check against a common `MAX_STANDARD_SEGWIT_VERSION const in L1552 and L133 of script/standard.cpp or c) just add a comment around both about duplication ?


    sdaftuar commented at 5:38 pm on August 4, 2020:

    Is this the first case where we duplicate policy rule checks ?

    First, it’s not an identical check, because the AreInputsStandard() check is only enabled on mainnet and not testnet, via the fRequireStandard flag also needing to be set. Whereas the script flag is enabled on all networks.

    Second, I think I’d be pretty surprised if anyone has a mental model of validation where they were certain that policy rule enforcement is not potentially duplicated: the validation rules are so complex and the checks so verbose that we generally tend to take a belt-and-suspenders approach, rather than trying to strictly ensure that any given check only happens once. If anyone has an understanding that any given transaction policy failure can only happen once in our code, I’d be curious to learn more about how we can be certain of this!

    So this doesn’t strike me as problematic; moreover I think trying to change validation code in order to ensure there is no duplication is risky and has no benefit. So I’ve instead added a comment pointing out that the check might be duplicative.

  28. sdaftuar force-pushed on Aug 3, 2020
  29. instagibbs commented at 6:40 pm on August 3, 2020: member
    Turns out there’s already a nice natural spot for the test, feel free to snipe: https://github.com/instagibbs/bitcoin/commit/57f782d29e1a2d2f315bf99762e0aa6ce4e0f1c0
  30. sdaftuar commented at 7:07 pm on August 3, 2020: member
    Thanks @instagibbs! I added your test change here.
  31. in test/functional/p2p_segwit.py:1435 in 4644948aa3 outdated
    1432         tx3.rehash()
    1433+
    1434+        # First we test this transaction against fRequireStandard=true node
    1435+        # making sure the txid is added to the reject filter
    1436+        self.std_node.announce_tx_and_wait_for_getdata(tx3)
    1437+        test_transaction_acceptance(self.nodes[1], self.std_node, tx3, with_witness=True, accepted=False, reason="bad-txns-nonstandard-inputs")
    


    jnewbery commented at 9:24 am on August 4, 2020:
    If I remove this reason parameter, then this test passes on master (i.e. I don’t think the line below actually tests any functional change).

    instagibbs commented at 11:52 am on August 4, 2020:
    Good catch. Increasing the wait to 5 seconds seems to fix it, probably because of INV delay logic? 2 seconds isn’t enough time for the INV to get sent out by the node.

    instagibbs commented at 11:55 am on August 4, 2020:
    I wonder if having a negative log line is better for each negative test case so we don’t have to guess what the time required is.

    jnewbery commented at 12:08 pm on August 4, 2020:
    You could use setmocktime to move time forward and force the node to send out getdata requests.

    instagibbs commented at 2:25 pm on August 4, 2020:
    Ok, I’d suggest just doing 5 seconds for now for both instances of announce_tx_and_wait_for_getdata success=False in this test.

    sdaftuar commented at 5:34 pm on August 4, 2020:
    Changed this to 5 seconds.
  32. Add txids with non-standard inputs to reject filter
    Our policy checks for non-standard inputs depend only on the non-witness
    portion of a transaction: we look up the scriptPubKey of the input being
    spent from our UTXO set (which is covered by the input txid), and the p2sh
    checks only rely on the scriptSig portion of the input.
    
    Consequently it's safe to add txids of transactions that fail these checks to
    the reject filter, as the witness is irrelevant to the failure. This is helpful
    for any situation where we might request the transaction again via txid (either
    from txid-relay peers, or if we might fetch the transaction via txid due to
    parent-fetching of orphans).
    
    Further, in preparation for future witness versions being deployed on the
    network, ensure that WITNESS_UNKNOWN transactions are rejected in
    AreInputsStandard(), so that transactions spending v1 (or greater) witness
    outputs will fall into this category of having their txid added to the reject
    filter.
    7989901c7e
  33. test addition of unknown segwit spends to txid reject filter 9f88ded82b
  34. sdaftuar force-pushed on Aug 4, 2020
  35. sdaftuar commented at 5:39 pm on August 4, 2020: member
    Updated with a comment to address @ariard’s feedback and tried to fix the test by bumping the timeout to 5 seconds.
  36. naumenkogs commented at 7:00 am on August 5, 2020: member
    utACK 9f88ded82b2898ca63d44c08072f1ba52f0e18d7
  37. in src/net_processing.cpp:2074 in 9f88ded82b
    2069+                // transactions are later received (resulting in
    2070+                // parent-fetching by txid via the orphan-handling logic).
    2071+                if (orphan_state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && orphanTx.GetWitnessHash() != orphanTx.GetHash()) {
    2072+                    // We only add the txid if it differs from the wtxid, to
    2073+                    // avoid wasting entries in the rolling bloom filter.
    2074+                    recentRejects->insert(orphanTx.GetHash());
    


    jonatack commented at 9:13 am on August 5, 2020:

    For my understanding, is there any reason to not use the existing const orphanHash local variable, e.g. (without the assert)

    0+                assert(orphanHash == orphanTx.GetHash());
    1-                if (orphan_state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && orphanTx.GetWitnessHash() != orphanTx.GetHash()) {
    2+                if (orphan_state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && orphanTx.GetWitnessHash() != orphanHash) {
    3                     // We only add the txid if it differs from the wtxid, to
    4                     // avoid wasting entries in the rolling bloom filter.
    5-                    recentRejects->insert(orphanTx.GetHash());
    6+                    recentRejects->insert(orphanHash);
    
  38. in src/net_processing.cpp:3066 in 9f88ded82b
    3061+                // Add the txid to the reject filter to prevent repeated
    3062+                // processing of this transaction in the event that child
    3063+                // transactions are later received (resulting in
    3064+                // parent-fetching by txid via the orphan-handling logic).
    3065+                if (state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && tx.GetWitnessHash() != tx.GetHash()) {
    3066+                    recentRejects->insert(tx.GetHash());
    


    jonatack commented at 9:17 am on August 5, 2020:

    Similarly, any reason to not use the existing local variables txid and wtxid that are set next to tx; all three are const. Tested running this with asserts:

    0-                if (state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && tx.GetWitnessHash() != tx.GetHash()) {
    1-                    recentRejects->insert(tx.GetHash());
    2+                if (state.GetResult() == TxValidationResult::TX_INPUTS_NOT_STANDARD && wtxid != txid) {
    3+                    recentRejects->insert(txid);
    
  39. in test/functional/p2p_segwit.py:1437 in 9f88ded82b
    1434+        # First we test this transaction against fRequireStandard=true node
    1435+        # making sure the txid is added to the reject filter
    1436+        self.std_node.announce_tx_and_wait_for_getdata(tx3)
    1437+        test_transaction_acceptance(self.nodes[1], self.std_node, tx3, with_witness=True, accepted=False, reason="bad-txns-nonstandard-inputs")
    1438+        # Now the node will no longer ask for getdata of this transaction when advertised by same txid
    1439+        self.std_node.announce_tx_and_wait_for_getdata(tx3, timeout=5, success=False)
    


    jnewbery commented at 9:33 am on August 5, 2020:

    I don’t like that this is testing transaction acceptance by relying on our transaction download logic. It means that changes to transaction download (eg #19184) may cause unrelated tests to fail, or (perhaps worse) make it such that tests are no longer actually testing what we want them to test.

    Unfortunately, we have no way to directly query a node’s recent rejects filter, so we do need to test this indirectly somehow. Here’s an alternative that uses the debug log to determine why the transaction has been rejected. It’s not ideal, but I think it’s better than testing via a separate subcomponent (tx download):

     0diff --git a/test/functional/p2p_segwit.py b/test/functional/p2p_segwit.py
     1index 519fb0438f..e0c671f55d 100755
     2--- a/test/functional/p2p_segwit.py
     3+++ b/test/functional/p2p_segwit.py
     4@@ -123,13 +123,14 @@ def get_virtual_size(witness_block):
     5     vsize = int((3 * base_size + total_size + 3) / 4)
     6     return vsize
     7 
     8-def test_transaction_acceptance(node, p2p, tx, with_witness, accepted, reason=None):
     9+def test_transaction_acceptance(node, p2p, tx, with_witness, accepted, reason=None, non_reason=None):
    10     """Send a transaction to the node and check that it's accepted to the mempool
    11 
    12     - Submit the transaction over the p2p interface
    13     - use the getrawmempool rpc to check for acceptance."""
    14     reason = [reason] if reason else []
    15-    with node.assert_debug_log(expected_msgs=reason):
    16+    non_reason = [non_reason] if non_reason else []
    17+    with node.assert_debug_log(expected_msgs=reason, unexpected_msgs=non_reason):
    18         p2p.send_and_ping(msg_tx(tx) if with_witness else msg_no_witness_tx(tx))
    19         assert_equal(tx.hash in node.getrawmempool(), accepted)
    20 
    21@@ -1418,7 +1419,7 @@ class SegWitTest(BitcoinTestFramework):
    22         temp_utxo.pop()  # last entry in temp_utxo was the output we just spent
    23         temp_utxo.append(UTXO(tx2.sha256, 0, tx2.vout[0].nValue))
    24 
    25-        # Spend everything in temp_utxo into an segwit v1 output.
    26+        # Spend everything in temp_utxo into a segwit v1 output.
    27         tx3 = CTransaction()
    28         total_value = 0
    29         for i in temp_utxo:
    30@@ -1433,8 +1434,9 @@ class SegWitTest(BitcoinTestFramework):
    31         # making sure the txid is added to the reject filter
    32         self.std_node.announce_tx_and_wait_for_getdata(tx3)
    33         test_transaction_acceptance(self.nodes[1], self.std_node, tx3, with_witness=True, accepted=False, reason="bad-txns-nonstandard-inputs")
    34-        # Now the node will no longer ask for getdata of this transaction when advertised by same txid
    35-        self.std_node.announce_tx_and_wait_for_getdata(tx3, timeout=5, success=False)
    36+        # If we send the transaction again, it'll be rejected by the recent
    37+        # rejects filter before we try to accept it to mempool
    38+        test_transaction_acceptance(self.nodes[1], self.std_node, tx3, with_witness=True, accepted=False, non_reason="bad-txns-nonstandard-inputs")
    39 
    40         # Spending a higher version witness output is not allowed by policy,
    41         # even with fRequireStandard=false.
    

    sdaftuar commented at 1:39 pm on August 5, 2020:

    Unfortunately, we have no way to directly query a node’s recent rejects filter

    Would it be better to have a ~unit~ test that calls ATMP and then checks that the txid made it into the reject filter?


    jnewbery commented at 1:50 pm on August 5, 2020:
    I’m not sure if that counts as a unit test? ATMP and the reject filter are in different components.

    instagibbs commented at 2:53 pm on August 5, 2020:
    Not sure it’s “better”, I still think the best thing would be to add log lines in case of rejection due to recentRejects and scan for that line in logs.

    jnewbery commented at 3:24 pm on August 5, 2020:

    I still think the best thing would be to add log lines in case of rejection due to recentRejects and scan for that line in logs.

    This seems reasonable, and very easy to implement.


    instagibbs commented at 4:58 pm on August 5, 2020:

    After some noodling/test writing I actually think a simple timeout average_wait_for_typical_getdata<<n<<GETDATA_TX_INTERVAL is best, or a “unit” test.

    I think 5 seconds is ok for now, someone can replace later with something better.


    jnewbery commented at 10:58 am on August 6, 2020:

    I actually think a simple timeout

    Can you explain why you think that? (I’m not disagreeing, just trying to understand why you think relying on tx download is a better test for this tx acceptance change).

    There seems to be some confusion around the concept of unit tests here. This is higher-level functionality spread across two different components (validation and net_processing). The only testing framework that we have that encompasses those two components is the functional test framework. As far as I’m aware, we have cpp tests for net_processing (eg denialofservice_tests.cpp) and validation (eg validation_block_tests.cpp), but not tests that span both.

    Generally, the concept of unit tests is to test units of code in isolation (eg here we could test that validation returns TX_INPUTS_NOT_STANDARD when the inputs are non-standard). Functional test test higher-level system functionality (eg we don’t redownload or revalidate transactions which have non-standard inputs).


    instagibbs commented at 1:22 pm on August 6, 2020:
    I was mostly annoyed that the various failure cases have to be handled on a case by case basis. One case is AlreadyHave, the other is not but the getdata is not sent because another peer has already had it requested. I’d rather write something that covers both properly.
  40. jonatack commented at 9:52 am on August 5, 2020: member

    ACK 9f88ded82b2

    Thanks for working on test coverage. It would be good if the test could isolate the changes more with less dependance on other logic, but the changed test does fail repeatedly on master with Expected messages "['bad-txns-nonstandard-inputs']" does not partially match log.

    Feel free to ignore the nits below.

  41. sdaftuar commented at 8:01 pm on August 5, 2020: member
    @jonatack Thanks for taking a look – if I end up re-pushing I’ll address the nits.
  42. ariard commented at 9:04 am on August 6, 2020: member

    Code Review/Tested ACK 9f88ded

    Thanks for the comment on duplicative checks under some flags combination, I agree that alternatives aren’t worthy the burden review.

  43. jnewbery commented at 11:04 am on August 6, 2020: member
    Code review ACK 9f88ded82b2898ca63d44c08072f1ba52f0e18d7
  44. ajtowns commented at 6:31 pm on August 6, 2020: member
    ACK 9f88ded82b2898ca63d44c08072f1ba52f0e18d7 - code review
  45. laanwj added this to the "Blockers" column in a project

  46. fanquake merged this on Aug 6, 2020
  47. fanquake closed this on Aug 6, 2020

  48. fanquake removed this from the "Blockers" column in a project

  49. sidhujag referenced this in commit 51084b8ed6 on Aug 7, 2020
  50. sdaftuar referenced this in commit 9985edbb4b on Aug 7, 2020
  51. sdaftuar referenced this in commit a77c784e52 on Aug 7, 2020
  52. sdaftuar referenced this in commit 06f9c5c3be on Aug 7, 2020
  53. sdaftuar referenced this in commit 107cf1515e on Aug 7, 2020
  54. sdaftuar referenced this in commit e39cd0f823 on Aug 7, 2020
  55. sdaftuar referenced this in commit 127331e168 on Aug 7, 2020
  56. sdaftuar referenced this in commit e835a84531 on Aug 7, 2020
  57. sdaftuar referenced this in commit d67058014a on Aug 7, 2020
  58. sdaftuar referenced this in commit 0411236ccc on Aug 10, 2020
  59. sdaftuar referenced this in commit 568007cffe on Aug 10, 2020
  60. sdaftuar referenced this in commit 2ea826cfc4 on Aug 11, 2020
  61. sdaftuar referenced this in commit 52c3bec1ba on Aug 11, 2020
  62. jonasschnelli referenced this in commit aee9d2306a on Aug 28, 2020
  63. fanquake referenced this in commit 30926997fa on Sep 4, 2020
  64. backpacker69 referenced this in commit a80e7da4e3 on Sep 8, 2020
  65. backpacker69 referenced this in commit cfc53bc645 on Sep 8, 2020
  66. Bushstar referenced this in commit 211f854d85 on Oct 21, 2020
  67. Bushstar referenced this in commit 08e36013e2 on Oct 21, 2020
  68. Fabcien referenced this in commit 74b9af263e on Sep 9, 2021
  69. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 13:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me