package validation: relax the package-not-child-with-unconfirmed-parents rule #31385

pull glozow wants to merge 4 commits into bitcoin:master from glozow:2024-11-relax-unconf changing 8 files +180 −81
  1. glozow commented at 1:33 am on November 28, 2024: member

    Broadens the package validation interface, see #27463 for wider context.

    On master, package rules include that (1) the package topology must be child-wth-parents (2) all of the child’s unconfirmed parents must be present. This PR relaxes the second rule, leaving the first rule untouched (there are plans to change that as well, but not here).

    Original motivation for this rule was based on the idea that we would have a child-with-unconfirmed-parents package relay protocol, and this would verify that the peer provided the “correct” package. For various reasons, we’re not planning on doing this. We could potentially do this for ancestor packages (with a similar definition that all UTXOs to make the tx valid are available in this package), but it’s also questionable whether it’s useful to enforce this.

    This rule gets in the way of certain usage of 1p1c package relay currently. If a transaction has multiple parents, of which only 1 requires a package CPFP, this rule blocks the package from relaying. Even if all the non-low-feerate parents are already in mempool, when the p2p logic submits the 1p1c package, it gets rejected for not meeting this rule.

  2. glozow added the label TX fees and policy on Nov 28, 2024
  3. DrahtBot commented at 1:33 am on November 28, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31385.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK ishaanam, instagibbs

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #33106 ([WIP] policy: lower the default blockmintxfee, incrementalrelayfee, minrelaytxfee by glozow)
    • #33094 (refactor: unify container presence checks by l0rinc)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  4. DrahtBot added the label CI failed on Nov 28, 2024
  5. DrahtBot commented at 2:08 am on November 28, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33632131707

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  6. in test/functional/p2p_opportunistic_1p1c.py:393 in dad28726c9 outdated
    375-        # Same error if submitted through submitpackage without parent_high
    376-        package_hex_missing_parent = [parent_low["hex"], child["hex"]]
    377-        result_missing_parent = node.submitpackage(package_hex_missing_parent)
    378-        assert_equal(result_missing_parent["package_msg"], "package-not-child-with-unconfirmed-parents")
    379+        assert parent_low["txid"] in node_mempool
    380+        assert child["txid"] in node_mempool
    


    instagibbs commented at 4:10 pm on November 29, 2024:

    can expand this test case to be a little more interesting with a grandparent as well:

     0diff --git a/test/functional/p2p_opportunistic_1p1c.py b/test/functional/p2p_opportunistic_1p1c.py
     1index 00b94d1a71..8aa5b43a53 100755
     2--- a/test/functional/p2p_opportunistic_1p1c.py
     3+++ b/test/functional/p2p_opportunistic_1p1c.py
     4@@ -62,18 +62,18 @@ class PackageRelayTest(BitcoinTestFramework):
     5             "-datacarriersize=100000",
     6             "-maxmempool=5",
     7         ]]
     8         self.supports_cli = False
     9 
    10-    def create_tx_below_mempoolminfee(self, wallet):
    11+    def create_tx_below_mempoolminfee(self, wallet, utxo_to_spend=None):
    12         """Create a 1-input 1sat/vB transaction using a confirmed UTXO. Decrement and use
    13         self.sequence so that subsequent calls to this function result in unique transactions."""
    14 
    15         self.sequence -= 1
    16         assert_greater_than(self.nodes[0].getmempoolinfo()["mempoolminfee"], FEERATE_1SAT_VB)
    17 
    18-        return wallet.create_self_transfer(fee_rate=FEERATE_1SAT_VB, sequence=self.sequence, confirmed_only=True)
    19+        return wallet.create_self_transfer(fee_rate=FEERATE_1SAT_VB, sequence=self.sequence, utxo_to_spend=utxo_to_spend, confirmed_only=True) [@cleanup](/bitcoin-bitcoin/contributor/cleanup/)
    20     def test_basic_child_then_parent(self):
    21         node = self.nodes[0]
    22         self.log.info("Check that opportunistic 1p1c logic works when child is received before parent")
    23@@ -341,34 +341,42 @@ class PackageRelayTest(BitcoinTestFramework): [@cleanup](/bitcoin-bitcoin/contributor/cleanup/)
    24     def test_other_parent_in_mempool(self):
    25         self.log.info("Check opportunistic 1p1c works even if child already has another parent in mempool")
    26         node = self.nodes[0]
    27 
    28+        # Grandparent will enter mempool by itself
    29+        grandparent_high = self.wallet.create_self_transfer(fee_rate=FEERATE_1SAT_VB*10, confirmed_only=True)
    30+
    31         # This parent needs CPFP
    32-        parent_low = self.create_tx_below_mempoolminfee(self.wallet)
    33+        parent_low = self.create_tx_below_mempoolminfee(self.wallet, utxo_to_spend=grandparent_high["new_utxo"])
    34         # This parent does not need CPFP and can be submitted alone ahead of time
    35         parent_high = self.wallet.create_self_transfer(fee_rate=FEERATE_1SAT_VB*10, confirmed_only=True)
    36         child = self.wallet.create_self_transfer_multi(
    37             utxos_to_spend=[parent_high["new_utxo"], parent_low["new_utxo"]],
    38             fee_per_output=999*parent_low["tx"].get_vsize(),
    39         )
    40 
    41         peer_sender = node.add_outbound_p2p_connection(P2PInterface(), p2p_idx=1, connection_type="outbound-full-relay")
    42 
    43-        # 1. Send first parent which will be accepted.
    44+        # 1. Send grandparent which will be accepted
    45+        peer_sender.send_and_ping(msg_tx(grandparent_high["tx"]))
    46+        assert grandparent_high["txid"] in node.getrawmempool()
    47+
    48+        # 2. Send first parent which will be accepted.
    49         peer_sender.send_and_ping(msg_tx(parent_high["tx"]))
    50         assert parent_high["txid"] in node.getrawmempool()
    51 
    52-        # 2. Send child.
    53+        # 3. Send child.
    54         peer_sender.send_and_ping(msg_tx(child["tx"]))
    55 
    56-        # 3. Node requests parent_low.
    57+        # 4. Node requests parent_low.
    58         parent_low_txid_int = int(parent_low["txid"], 16)
    59         peer_sender.wait_for_getdata([parent_low_txid_int])
    60         peer_sender.send_and_ping(msg_tx(parent_low["tx"]))
    61 
    62         node_mempool = node.getrawmempool()
    63+        assert grandparent_high["txid"] in node_mempool
    64         assert parent_high["txid"] in node_mempool
    65         assert parent_low["txid"] in node_mempool
    66         assert child["txid"] in node_mempool
    67 
    68     def run_test(self):
    

    glozow commented at 1:55 pm on December 4, 2024:
    added
  7. glozow force-pushed on Dec 4, 2024
  8. glozow commented at 2:12 pm on December 4, 2024: member
    Rebased after #31096 and took @instagibbs’ comment, thank you
  9. glozow marked this as ready for review on Dec 4, 2024
  10. instagibbs commented at 2:55 pm on December 4, 2024: member

    Original motivation for this rule was based on the idea that we would have a child-with-unconfirmed-parents package relay protocol, and this would verify that the peer provided the “correct” package

    Was the idea that the full ancestor package was to be “pulled” into the package validation logic, including in-mempool things?

  11. in src/test/txpackage_tests.cpp:444 in 0f0f979c06 outdated
    440@@ -441,20 +441,6 @@ BOOST_AUTO_TEST_CASE(package_submission_tests)
    441         BOOST_CHECK_EQUAL(result_quit_early.m_state.GetResult(), PackageValidationResult::PCKG_TX);
    442     }
    443 
    444-    // Child with missing parent.
    


    instagibbs commented at 3:27 pm on December 4, 2024:

    Might still be an interesting case to keep around, shoving it to the end after parent is already accepted:

     0diff --git a/src/test/txpackage_tests.cpp b/src/test/txpackage_tests.cpp
     1index 09e719ff4c..be0ebac834 100644
     2--- a/src/test/txpackage_tests.cpp
     3+++ b/src/test/txpackage_tests.cpp
     4@@ -476,10 +476,30 @@ BOOST_AUTO_TEST_CASE(package_submission_tests)
     5             BOOST_CHECK(it_child_deduped->second.m_result_type == MempoolAcceptResult::ResultType::MEMPOOL_ENTRY);
     6         }
     7 
     8         BOOST_CHECK_EQUAL(m_node.mempool->size(), expected_pool_size);
     9     }
    10+
    11+    // In-mempool parent and child with missing parent.
    12+    mtx_child.vin.emplace_back(COutPoint(package_unrelated[0]->GetHash(), 0));
    13+    CTransactionRef tx_child_missing_parent = MakeTransactionRef(mtx_child);
    14+    Package package_missing_parent;
    15+    package_missing_parent.push_back(tx_parent);
    16+    package_missing_parent.push_back(tx_child_missing_parent);
    17+    {
    18+        const auto result_missing_parent = ProcessNewPackage(m_node.chainman->ActiveChainstate(), *m_node.mempool,
    19+                                                             package_missing_parent, /*test_accept=*/false, /*client_maxfeerate=*/{});
    20+        auto it_parent = result_missing_parent.m_tx_results.find(tx_parent->GetWitnessHash());
    21+        auto it_child = result_missing_parent.m_tx_results.find(tx_child_missing_parent->GetWitnessHash());
    22+        BOOST_CHECK(result_missing_parent.m_state.IsInvalid());
    23+        BOOST_CHECK_EQUAL(result_missing_parent.m_state.GetResult(), PackageValidationResult::PCKG_TX);
    24+        BOOST_CHECK_EQUAL(result_missing_parent.m_state.GetRejectReason(), "transaction failed");
    25+        BOOST_CHECK(it_parent->second.m_state.IsValid());
    26+        BOOST_CHECK_EQUAL(it_child->second.m_state.GetResult(), TxValidationResult::TX_MISSING_INPUTS);
    27+        BOOST_CHECK_EQUAL(it_child->second.m_state.GetRejectReason(), "bad-txns-inputs-missingorspent");
    28+        BOOST_CHECK_EQUAL(m_node.mempool->size(), expected_pool_size);
    29+    }
    30 }
    31 
    32 // Tests for packages containing a single transaction
    33 BOOST_AUTO_TEST_CASE(package_single_tx)
    34 {
    

    glozow commented at 12:44 pm on December 10, 2024:

    I’ve added a different test, creating a new set of transactions instead of borrowing stuff from earlier tests. Testing

    1. Missing parent: gives you TX_RECONSIDERABLE and TX_MISSING_INPUTS
    2. Submit the missing parent
    3. Try again, should work just fine
  12. instagibbs commented at 3:55 pm on December 4, 2024: member

    approach ACK 0f0f979c062055c28383bd4b06f69be90380fe53

    changes seem correct, and I like less code

    I have to think a bit about the UX that we currently have:

    1. submitpackage requires child-with-parents-tree (no other ancestors, unless single tx)
    2. AcceptPackage requires child-with-parents (no other ancestors)
    3. 1P1C relay allows relay of child-with-parent (no other ancestors)

    and after this PR:

    1. submitpackage requires child-with-parents-tree (but can have arbitrary in-mempool ancestors)
    2. AcceptPackage requires child-with-parents (but can have arbitrary in-mempool ancestors)
    3. 1P1C relays a child-with-parent (but can have arbitrary in-mempool ancestors)
  13. DrahtBot removed the label CI failed on Dec 4, 2024
  14. glozow commented at 4:23 pm on December 4, 2024: member

    Was the idea that the full ancestor package was to be “pulled” into the package validation logic, including in-mempool things?

    Yes, I remember thinking “this is restrictive but easy to work around because you can just fetch the parents from mempool and add them to the package if you need to” (and we could code that up) but that’s a pain. And I’m no longer convinced it’s useful at all to require all parents be present.

  15. glozow force-pushed on Jun 10, 2025
  16. glozow commented at 8:03 pm on June 10, 2025: member
    Rebased since it’s been a while. Still worth doing imo! Getting this out of the way would be good before we try to do subpackage evaluation / package RBF stuff, fwiw.
  17. instagibbs commented at 3:55 pm on June 11, 2025: member

    reviewed through f61d8d2eab6f5f07d6c34723a92f521737a05623

    This removes some technically-unnecessary hurdles to propagating packages

    Added a couple tests here that I think are worthy of inclusion: https://github.com/instagibbs/bitcoin/tree/2025-06-childparents-test

    Ran fuzzing for a bit.

    Needs a release note

    I’m unsure how to communicate this stuff to users though.

  18. glozow commented at 4:22 pm on June 11, 2025: member

    I think for users the change is small but nice:

    • when you submit thruogh the submitpackage RPC, you no longer need to have all unconfirmed parents present.
    • if you have a n-parent-1-child package where only 1 parent needs fee-bumping, you can somewhat expect it to propagate
  19. glozow commented at 8:58 pm on June 12, 2025: member
    Thanks, cherry-picked the two working tests @instagibbs and made a stab at a release note.
  20. glozow force-pushed on Jun 12, 2025
  21. DrahtBot added the label CI failed on Jun 12, 2025
  22. glozow commented at 9:22 pm on June 12, 2025: member
    Oop, rebased for silent merge conflict with #32421
  23. DrahtBot removed the label CI failed on Jun 13, 2025
  24. in doc/policy/packages.md:15 in bc9538823f outdated
     7@@ -8,12 +8,6 @@ Graph (a directed edge exists between a transaction that spends the output of an
     8 For every transaction `t` in a **topologically sorted** package, if any of its parents are present
     9 in the package, they appear somewhere in the list before `t`.
    10 
    11-A **child-with-unconfirmed-parents** package is a topologically sorted package that consists of
    12-exactly one child and all of its unconfirmed parents (no other transactions may be present).
    13-The last transaction in the package is the child, and its package can be canonically defined based
    14-on the current state: each of its inputs must be available in the UTXO set as of the current chain
    15-tip or some preceding transaction in the package.
    


    ishaanam commented at 5:39 pm on July 22, 2025:

    In bc9538823f52c7274d11ae28fa2b2f7236daa69c “relax child-with-unconfirmed-parents rule "

    Maybe instead of just deleting the definition of child-with-unconfirmed-parents, a definition of child-with-parents could be added?


    glozow commented at 6:16 pm on July 23, 2025:
    Good point, added a definition
  25. in src/policy/packages.h:29 in bc9538823f outdated
    25@@ -26,7 +26,7 @@ static_assert(MAX_PACKAGE_WEIGHT >= MAX_STANDARD_TX_WEIGHT);
    26 
    27 // If a package is to be evaluated, it must be at least as large as the mempool's ancestor/descendant limits,
    28 // otherwise transactions that would be individually accepted may be rejected in a package erroneously.
    29-// Since a submitted package must be child-with-unconfirmed-parents (all of the transactions are an ancestor
    30+// Since a submitted package must be child-with-parents (all of the transactions are an ancestor
    


    ishaanam commented at 5:54 pm on July 22, 2025:

    In bc9538823f52c7274d11ae28fa2b2f7236daa69c " relax child-with-unconfirmed-parents rule " nit for precision:

    0// Since a submitted package must be child-with-parents (all of the transactions are a parent
    

    glozow commented at 6:16 pm on July 23, 2025:
    done
  26. in test/functional/p2p_opportunistic_1p1c.py:397 in c8d96e23f7 outdated
    392+        peer_sender = node.add_p2p_connection(P2PInterface())
    393+
    394+        # The 1p1c that spends the confirmed utxo must be received first. Afterwards, the "younger" 1p1c can be received.
    395+        for package in [[low_fee_great_grandparent, high_fee_grandparent], [low_fee_parent, high_fee_child]]:
    396+            # Aliases
    397+            low_fee_parent, high_fee_child = package
    


    ishaanam commented at 6:48 pm on July 22, 2025:

    In c8d96e23f734e275dfee840df49b1f4cc78cd4b9 “test: add chained 1p1c propagation test "

    nit: it is a bit confusing to reuse the low_fee_parent and high_fee_child variables here.


    glozow commented at 6:16 pm on July 23, 2025:
    changed them to parent_relative, child_relative
  27. in test/functional/rpc_packages.py:515 in c8d96e23f7 outdated
    510+        parent_tx = self.wallet.create_self_transfer()
    511+        child_tx = self.wallet.create_self_transfer(utxo_to_spend=parent_tx["new_utxo"])
    512+        grandchild_tx = self.wallet.create_self_transfer(utxo_to_spend=child_tx["new_utxo"])
    513+        ggrandchild_tx = self.wallet.create_self_transfer(utxo_to_spend=grandchild_tx["new_utxo"])
    514+
    515+        # Submitting them all together doesn't work, as the topology is not parents-with-child
    


    ishaanam commented at 6:56 pm on July 22, 2025:

    In c8d96e23f734e275dfee840df49b1f4cc78cd4b9 “test: add chained 1p1c propagation test "

    nit for consistency:

    0        # Submitting them all together doesn't work, as the topology is not child-with-parents
    

    glozow commented at 6:16 pm on July 23, 2025:
    done, thanks
  28. ishaanam commented at 7:09 pm on July 22, 2025: contributor

    utACK 613e66dfec67ca747a8c75cc70a60a3343346d83

    This looks like a good improvement as it allows for package relay to be used in more situations, without updating a lot of code. This PR also also adds good testing. I only have some minor nits, please feel free to ignore.

  29. DrahtBot requested review from instagibbs on Jul 22, 2025
  30. glozow force-pushed on Jul 23, 2025
  31. glozow commented at 6:17 pm on July 23, 2025: member
    Thanks @ishaanam! Took all suggestions
  32. glozow force-pushed on Jul 23, 2025
  33. glozow commented at 7:52 pm on July 23, 2025: member
    Rebased as well
  34. in doc/policy/packages.md:12 in d61cb2edc6 outdated
    12-exactly one child and all of its unconfirmed parents (no other transactions may be present).
    13-The last transaction in the package is the child, and its package can be canonically defined based
    14-on the current state: each of its inputs must be available in the UTXO set as of the current chain
    15-tip or some preceding transaction in the package.
    16+A **child-with-parents** package is a topologically sorted package that consists of exactly one child and at least one
    17+of its unconfiremd parents.  Not all unconfirmed parents need to be present. No other transactions may be present; the
    


    maflcko commented at 8:19 am on July 24, 2025:
    unconfiremd -> unconfirmed [spelling error in “unconfiremd parents”]
    

    glozow commented at 1:45 pm on July 24, 2025:
    🤦 thank you, fixed
  35. in test/functional/p2p_opportunistic_1p1c.py:43 in 313169b163 outdated
    39@@ -40,7 +40,6 @@
    40 )
    41 from test_framework.test_framework import BitcoinTestFramework
    42 from test_framework.util import (
    43-    assert_equal,
    


    maflcko commented at 8:21 am on July 24, 2025:

    in the first commit (after the rebase): This is now required. Otherwise:

    0                                        assert_equal(testres[0]["reject-reason"], "missing-inputs")
    1                                       ^^^^^^^^^^^^
    2                                   NameError: name 'assert_equal' is not defined
    

    glozow commented at 1:45 pm on July 24, 2025:
    ah doi, restored
  36. relax child-with-unconfirmed-parents rule
    This rule was originally introduced along with a very early proposal for
    package relay as a way to verify that the "correct"
    child-with-unconfirmed-parents package was provided for a transaction,
    where correctness was defined as all of the transactions unconfirmed
    parents. However, we are not planning to introduce a protocol where
    peers would be asked to send these packages.
    
    This rule has downsides: if a transaction has multiple parents but only
    1 that requires package CPFP to be accepted, the current rule prevents
    us from accepting that package. Even if the other parents are already in
    mempool, the p2p logic will only submit the 1p1c package, which fails
    this check. See the test in p2p_1p1c_network.py
    f24771af05
  37. [unit test] package submission 2p1c with 1 parent missing 525be56741
  38. test: add chained 1p1c propagation test 12f48d5ed3
  39. [doc] release note for relaxing requirement of all unconfirmed parents present ea17a9423f
  40. glozow force-pushed on Jul 24, 2025
  41. ishaanam commented at 4:48 pm on July 25, 2025: contributor

    re-utACK ea17a9423fb431a86d36927b02d3624f654fd867

    I used range_diff to compare to my previous review. The changes were just to address review comments and rebase.

  42. instagibbs commented at 7:16 am on July 31, 2025: member

    ACK ea17a9423fb431a86d36927b02d3624f654fd867

    I think the doc update gives a decent explanation how this can be used. Code cleanup is nice.

  43. fanquake added this to the milestone 30.0 on Jul 31, 2025
  44. fanquake merged this on Aug 1, 2025
  45. fanquake closed this on Aug 1, 2025


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-08-08 15:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me