policy: nVersion=3 and Package RBF #25038

pull glozow wants to merge 15 commits into bitcoin:master from glozow:package-rbf changing 24 files +1510 −134
  1. glozow commented at 0:24 am on April 30, 2022: member

    Note: this PR was superseded by #28948 (v3) and #28984 (package RBF). Also, v3 became TRUC and includes other changes, see BIP 431 for more up-to-date documentation.

    See #27463 for overall package relay tracking.

    This PR contains 2 projects: v3 policy and package RBF. Mailing list posts: package RBF 1 and V3 + package RBF 2. It stems from a long discussion about RBF pinning, across a mailing list thread and gist.

    V3 Policy: A set of policy rules applied to transactions with their nVersion field set to 3. Namely, it allows users to opt in to more restrictive descendant limits for shared transactions. If adopted by many nodes in the network, V3 mitigates various RBF pinning attacks. See doc/policy/version3_transactions.md for the exact rules and rationale, and these review club notes for more background and discussion.

    Package RBF: In addition to allowing a child to pay for its parents within the package, also allow the child to pay for replacing the parent’s conflicts. For example, this allows LN users to replace commitment transactions existing in the mempool, simply by broadcasting their respective commitment transactions with a high-fee child. The commitment transactions can be signed with 0 fees, which means no overpaying.

    FAQ: is v3 still helpful even with cluster mempool (#27677) ?

    • Rule 3 pinning: This is addressed with v3 but not really with cluster mempool (descendant allowance is still too permissive).
    • Package RBF and ACP pinning: This PR allows for package RBF with v3 packages. V3 has an effective “cluster limit” of 2 which makes it very cheap to calculate the mining score of a v3 transaction. With cluster mempool, which also makes it easier to calculate mining score, we could have package RBF for non-v3 transactions.
    • Allowing 0fee transactions: This PR allows v3 transactions to be below minimum relay feerate, provided they are CPFP’d. This is because the simplified topology allows us to avoid situations like the ones described in #26933. With cluster mempool, we can allow this for non-v3 transactions.
  2. glozow added the label TX fees and policy on Apr 30, 2022
  3. glozow added the label Needs Conceptual Review on Apr 30, 2022
  4. DrahtBot commented at 4:17 am on April 30, 2022: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK theStack

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #28972 (test: Add and use option for tx-version in MiniWallet methods by maflcko)
    • #28960 (kernel: Remove dependency on CScheduler by TheCharlatan)
    • #28950 (RPC: Add maxfeerate and maxburnamount args to submitpackage by instagibbs)
    • #28948 (v3 transaction policy for anti-pinning by glozow)
    • #28886 (refactor: Replace sets of txiter with CTxMemPoolEntryRefs by TheCharlatan)
    • #28863 (wallet, mempool: propagete checkChainLimits error message to wallet by ismaelsadeeq)
    • #28690 (build: Introduce internal kernel library by TheCharlatan)
    • #28368 (Fee Estimator updates from Validation Interface/CScheduler thread by ismaelsadeeq)
    • #28335 (RFC: Remove boost usage from kernel api / headers by TheCharlatan)
    • #27432 (contrib: add tool to convert compact-serialized UTXO set to SQLite database by theStack)
    • #26593 (tracing: Only prepare tracepoint arguments when actually tracing by 0xB10C)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  5. fanquake requested review from ariard on May 1, 2022
  6. fanquake requested review from darosior on May 1, 2022
  7. fanquake requested review from t-bast on May 1, 2022
  8. jonatack commented at 2:21 pm on May 3, 2022: contributor
    0$ ./test/lint/lint-python.py
    1test/functional/feature_package_rbf.py:18:1: F401 'test_framework.util.assert_equal' imported but unused
    
  9. bitcoin deleted a comment on May 9, 2022
  10. in src/validation.cpp:515 in abef97a694 outdated
    504@@ -502,7 +505,7 @@ class MemPoolAccept
    505                             /* m_bypass_limits */ false,
    506                             /* m_coins_to_uncache */ coins_to_uncache,
    507                             /* m_test_accept */ false,
    508-                            /* m_allow_bip125_replacement */ false,
    509+                            /* m_allow_bip125_replacement */ true,
    


    ariard commented at 8:34 pm on May 9, 2022:
    Reason to not turn PackageTestAccept as true too ?

    glozow commented at 2:16 pm on May 16, 2022:
    This RBF policy is essentially treating the package as “one big transaction,” which works because everything is in the ancestor set of the child. PackageTestAccept doesn’t impose any topology restrictions - it even allows unrelated transactions to be submitted as a group. So it wouldn’t make sense to use each other’s fees.
  11. in src/validation.cpp:1028 in abef97a694 outdated
    1017                                    m_limit_descendant_size, err_string)) {
    1018+        // TODO: When replacements exist, to avoid overestimating descendant counts, deduct the
    1019+        // to-be-replaced mempool entries when counting descendants. Note that this is not
    1020+        // necessarily as simple as subtracting the count/size from descendant limits, because
    1021+        // multiple transactions may conflict with the same entries, causing us to double-count them
    1022+        // and their descendants.
    


    ariard commented at 9:42 pm on May 9, 2022:
    IIUC given that PackageMempoolChecks happens after package dedup there should not be remaining already-in-mempool transactions at that stage in txns ? So no descendants that could be to-be-replaced mempool entries to subtract or are you thinking to something else ?

    glozow commented at 2:31 pm on May 16, 2022:

    I mean a situation like this:

    {A, B, C} are in the mempool, and A is already at the descendant limit (101KvB). {X, Y, Z} is a package where X conflicts with B and will replace {B, C}. However, when we run descendant limit checks, it will look like A’s descendants include {B, C, X, Z} and reject it. But actually A’s descendants will only include {X, Z} after the replacement.

  12. in src/validation.cpp:955 in abef97a694 outdated
    951@@ -928,6 +952,9 @@ bool MemPoolAccept::ReplacementChecks(Workspace& ws)
    952     const uint256& hash = ws.m_hash;
    953     TxValidationState& state = ws.m_state;
    954 
    955+    m_rbf = ws.m_conflicts.size() > 0;
    


    ariard commented at 9:50 pm on May 9, 2022:
    Not sure if this check isn’t already redundant given that calling ReplacementChecks in AcceptSingleTransaction is dependent on m_rbf=true or unless there is another code path where it makes sense ?

    glozow commented at 2:38 pm on May 16, 2022:
    Ah good point, I forgot I had moved this to the bottom of PreChecks.
  13. in doc/policy/packages.md:142 in abef97a694 outdated
    132+   conflicting transactions or is from another transaction in the package.
    133+
    134+4. The package fee after deduplication pays an absolute fee of at least the sum paid by the
    135+   original transactions.
    136+
    137+5. The additional fees (difference between absolute fee paid by the package after deduplication and the
    


    ariard commented at 10:20 pm on May 9, 2022:
    Depending on the design of p2p packages, we might relay the “full-non-dedup” version of the package as our peers might not have the same mempool composition. So the additional fees might have to be paid on the non-dedup package. I don’t think we have to settle the question now though we might have to rework package-RBF in function of p2p design ?
  14. in doc/policy/packages.md:143 in abef97a694 outdated
    138+   sum paid by the original transactions) pays for the package's bandwidth after deduplication at or
    139+   above the rate set by the node's incremental relay feerate. For example, if the incremental relay
    140+   feerate is 1 satoshi/vB and the package after deduplication contains 500 virtual bytes total, then the
    141+   package fees after deduplication is at least 500 satoshis higher than the sum of the original transactions.
    142+
    143+6. The number of original transactions does not exceed 100.
    


    ariard commented at 10:25 pm on May 9, 2022:

    I think it could be a limitation for second-layer package issuers, where a child-with-parents package aims to replace more than 5 LN commitment transactions at the same time. A counterparty could extend the 5 non-related commitment transactions descendants until reaching max in-mempool limits, thus interfering with their potential package-RBF.

    Am I correct ? That said, I remember we’ve already said that child-with-parents to replace multiple LN commitment transactions open the door to malicious interferences.


    t-bast commented at 12:12 pm on May 10, 2022:

    I believe this issue can unfortunately arise even when you’re replacing a single LN commitment transaction. Imagine that you’re trying to replace {commitTxAlice, anchorTxAlice} but Alice has created anchorTxAlice to have 100 outputs and broadcast one child tx per output, you’re already hitting the 100 transactions limit…

    I believe that the only thing that can save Bob here is to broadcast {commitTxAlice, anchorTxBob} by relying on the CPFP carve-out rule to get Alice’s commitment to confirm. But in order to do that, Bob needs to somehow get the signed commitTxAlice (which never appeared in his mempool since his mempool contains commitTxBob).

    So this rule is an issue for L2 protocols, but it is also a protection against high mempool churn, so it’s hard to draw the line.


    glozow commented at 2:47 pm on May 16, 2022:

    I think it could be a limitation for second-layer package issuers, where a child-with-parents package aims to replace more than 5 LN commitment transactions at the same time. A counterparty could extend the 5 non-related commitment transactions descendants until reaching max in-mempool limits, thus interfering with their potential package-RBF.

    Yeah, batching multiple commitment transaction replacements isn’t very safe. That’s why we have the “Warning: Batched fee-bumping may be unsafe for some use cases. Users and application developers should take caution if utilizing multi-parent packages.” in the doc.

    I believe this issue can unfortunately arise even when you’re replacing a single LN commitment transaction.

    I don’t think this scenario is possible with our current mempool policy. With our descendant limit at 25, anchorTxAlice wouldn’t be able to have 100 children.

  15. ariard commented at 10:25 pm on May 9, 2022: member
    Just a first parse
  16. in doc/policy/packages.md:126 in abef97a694 outdated
    117@@ -117,3 +118,30 @@ backward-compatible for users and applications that rely on p2p transaction rela
    118 package validation should not prevent the acceptance of a transaction that would otherwise be
    119 policy-valid on its own. By always accepting a transaction that passes individual validation before
    120 trying package validation, we prevent any unintentional restriction of policy.
    121+
    122+### Package Replace by Fee
    123+
    124+Child-with-unconfirmed-parents packages may replace mempool transactions under the following conditions:
    125+
    126+1. The child transaction does not conflict with any transactions in the mempool.
    


    t-bast commented at 11:46 am on May 10, 2022:

    This rule startled me a bit at first, it may be worth mentioning that if you want to update the package’s child, you should do a normal RBF of just that child.

    What you mean is that package-rbf applies for the case where we want for example to replace {commitTxAlice, anchorAlice} by {commitTxBob, anchorBob} where commitTxAlice and commitTxBob conflict, but we cannot use package-rbf to replace {commitTxAlice, anchorAlice1} by {commitTxAlice, anchorAlice2} where anchorAlice2 pays more fees than anchorAlice1. In that case we should submit anchorAlice2 alone and it will go through the usual RBF rules. Is that correct?

    Also you don’t want to mix up package-rbf and normal rbf, so a child tx cannot create conflicts “outside of the package” IIUC.


    glozow commented at 3:17 pm on May 16, 2022:
    Yes you’re right. Thanks for the feedback and clear explanation, adding some similar language to the doc!
  17. in doc/policy/packages.md:132 in abef97a694 outdated
    127+
    128+2. The directly conflicting transactions all signal replaceability explicitly.
    129+
    130+3. All transactions in the package, including those that do not directly conflict with any mempool
    131+   transactions, only include an unconfirmed input if that input was included in one of the directly
    132+   conflicting transactions or is from another transaction in the package.
    


    t-bast commented at 12:03 pm on May 10, 2022:

    My head hurts :laughing:

    I’m not 100% sure I’m understanding this rule correctly, can you let me know if the following example makes sense?

    Let’s imagine we have {commitTxAlice, anchorTxAlice} in the mempool that we want to replace. What we really want to do is to replace commitTxAlice by commitTxBob (the child is just a tool to achieve that). We ask bitcoind to fundrawtransaction to ensure our anchorTxBob adds enough funds. bitcoind adds an unconfirmed (but safe) wallet input:

    0+-------------+
    1| commitTxBob |---------+
    2+-------------+         |      +-------------+
    3                        |      |             |
    4+------------------+    +----->| anchorTxBob |
    5| previousWalletTx |---------->|             |
    6+------------------+           +-------------+
    

    If we include previousWalletTx in the package, this package-rbf would work, right? But if we don’t include previousWalletTx in the package, this would be rejected?

    But what if previousWalletTx itself has an unconfirmed parent? We cannot include this grand-parent in the package (since packages are only parents-with-single-child).

    This isn’t directly bringing an unconfirmed input, but it is indirectly bringing one. Is that ok? If that’s ok, why wouldn’t it be ok to just allow adding new unconfirmed inputs in the first place?


    t-bast commented at 7:41 am on May 18, 2022:
    Gotcha, thanks for the confirmation, we’ll deal with that limitation then, it’s already a good start. And hopefully we’ll eventually figure out better RBF rules for both single txs and packages!

    glozow commented at 7:00 pm on May 20, 2022:

    (Sorry for my earlier comment, I was confused)

    If we include previousWalletTx in the package, this package-rbf would work, right?

    Yes. It’s fine because it’s one of the direct parents of anchorTxBob.

    But if we don’t include previousWalletTx in the package, this would be rejected?

    If you mean anchorTxBob now only has 1 unconfirmed input, commitTxBob, actually this would be accepted.

    If you mean that previousWalletTx was submitted ahead of time, that should still be okay because the code will remember to exempt previousWalletTx from the “no new unconfirmed” rule.

    But what if previousWalletTx itself has an unconfirmed parent?

    Based on this rule, if previousWalletTx has another unconfirmed parent, this would be rejected.


    t-bast commented at 6:18 am on May 24, 2022:
    Thanks for the clarification! I’ll create extensive tests of this behavior and will report back.

    t-bast commented at 2:53 pm on June 21, 2022:
    It’s looking good with the current state of the PR, I am able to include new unconfirmed inputs that themselves have an unconfirmed parent as shown by this test :+1:
  18. glozow force-pushed on May 16, 2022
  19. bitcoin deleted a comment on May 20, 2022
  20. bitcoin deleted a comment on May 20, 2022
  21. bitcoin deleted a comment on May 20, 2022
  22. instagibbs commented at 7:58 pm on June 2, 2022: member

    @t-bast

    to have 100 outputs and broadcast one child tx per output, you’re already hitting the 100 transactions limit…

    They cannot create this chain to begin with as from any single unconfirmed transaction with confirmed parents, you can only make 25 transactions.

    So if you for example had >4 commitment transactions that you want to CPFP with a single bump transaction, this is where the limit can be hit.

  23. t-bast commented at 10:26 pm on June 2, 2022: contributor

    They cannot create this chain to begin with as from any single unconfirmed transaction with confirmed parents, you can only make 25 transactions.

    Right, I had forgotten that restriction! However, this 25 descendants chain limit is only a configuration parameter on bitcoind, not a BIP 125 requirement, right? So in theory we cannot fully rely on it, nothing prevents an attacker to inject a 100 descendants chain in miners mempools if they have configured their bitcoind differently (or even use a different implementation)?

  24. instagibbs commented at 11:21 pm on June 2, 2022: member
    @t-bast unlikely but true, seems like something to consider scaling with other policies to be more robust?
  25. ajtowns commented at 11:49 pm on June 2, 2022: contributor

    this 25 descendants chain limit is only a configuration parameter on bitcoind, not a BIP 125 requirement

    Note that BIP 125 enforcement could be considered a configuration parameter too – eg knots nodes will allow replacement of any tx, whether they signal or not.

    Perhaps something we could consider is defining a service bit to correspond with a relay policy, and have nodes preferential peer with other nodes who set that bit; that way you’re more likely for nodes that support a particular policy to form a fully connected graph, and thus have a path to any miners that also accept that relay policy. Knots defines an experimental service bit for full-RBF nodes https://github.com/bitcoinknots/bitcoin/blob/23.x-knots/src/init.cpp#L1103 .

  26. in doc/policy/packages.md:175 in da7cc60fb5 outdated
    143+   sum paid by the original transactions) pays for the package's bandwidth at or above the rate set
    144+   by the node's incremental relay feerate. For example, if the incremental relay feerate is 1
    145+   satoshi/vB and the package after deduplication contains 500 virtual bytes total, then the package
    146+   fees after deduplication is at least 500 satoshis higher than the sum of the original transactions.
    147+
    148+*Rationale*: These fee-related rules are identical to the replacement [policy for individual
    


    sdaftuar commented at 2:36 pm on June 3, 2022:

    Since we are introducing a new form of RBF that doesn’t currently exist, now might be the best time to at least improve the BIP 125 rules to be incentive compatible in all situations (though this comes at the cost of making replacements potentially more expensive).

    In particular what I have in mind is that we require the ancestor feerate of the child transaction to be higher than the actual feerate of all transactions that would be evicted. This is conservative but I think would ensure that we don’t have any situations where there are transactions being replaced that might appear in the next block, while the new transactions do not.

    Would this be too conservative for the use cases that we have in mind?


    instagibbs commented at 3:11 pm on June 3, 2022:
    Can you elaborate with an example? It’s possible I don’t have the same mental mapping to terms and do not understand the scenario.

    sdaftuar commented at 3:44 pm on June 3, 2022:

    Sure – consider the case where you have a transaction A(100vb, 1sat/vb) and child B (100vb, 10sat/vb) in the mempool. Total fees = 1100 sat.

    A package comes in with parent A’(100vb, 1sat/vb) and child B’(1000vb, 2.5sat/vb). Total package fees = 2600 sat.

    Under the rules described here, package (A’, B’) could replace (A, B) in the mempool, because total fees are paid for and there’s enough left over fee to pay the incremental relay fee of the new package (1500 sat/1100 vb). Also the package feerate is higher than that of the direct conflicts[1].

    The deficiency in the logic is that we’re not looking at the feerates of the transactions that would be evicted – even though their absolute fee is being paid, their feerate could be even higher than the transactions that would replace them. Calculating the effective feerate that such a transaction would have in our mining code is tricky (because a sibling could pay for an ancestor, it’s not quite right to just consider ancestor feerate), but using its feerate is a conservative check here – any descendant that is paying for its confirmation would be caught in this logic, and its (higher) feerate considered if necessary.

    For evaluating the mining suitability of the new transactions coming in, I think we can just look at the ancestor feerate of the package, which is a lower bound on the attractiveness of the new transactions. (I think this would also allow us to relax the rule against new unconfirmed inputs.)

    [1] In looking at this example, I noticed that BIP 125 as drafted seems to misstate an important check we perform in our current RBF policy, namely that we compare the feerate of a new transaction coming in to the feerate of the existing transactions that it directly conflicts with. (Without this check, a small high fee transaction could be replaced by a large low fee transaction!). From my first glance at the code in this PR, it looks like the code itself does a similar check for packages (comparing the package feerate to the feerate of the direct conflicts) so that is good at least, but the description of the logic in the documentation should point out that there is some feerate check happening as well.


    instagibbs commented at 3:58 pm on June 3, 2022:
    It’s ancient lore by now, but I’m not sure it’s a “mis-statement” or just incentive incompatible stuff no one implements(becase feerate check is cheap). I agree with your reading of current texts either way and agree pulling that into the text is the right thing to do.

    glozow commented at 4:38 pm on June 15, 2022:

    require the ancestor feerate of the child transaction to be higher than the actual feerate of all transactions that would be evicted. This is conservative but I think would ensure that we don’t have any situations where there are transactions being replaced that might appear in the next block, while the new transactions do not.

    I think this is a sensible addition. I cannot think of any easy attacks since it’s only the directly conflicting transactions and not their descendants (i.e. I don’t have the concerns stated in #23121 (comment)).

    Btw, don’t we want min(package feerate, child ancestor feerate)?

    In pseudocode for a child-with-unconfirmed-parents package:

    0for each tx in directly_conflicting_transactions:
    1    if min(package_feerate, child.ancestor_feerate) <= tx.individual_feerate
    2        fail
    

    With this, do you think it would be acceptable to remove the “no new unconfirmed inputs” rule in package RBF?

    we compare the feerate of a new transaction coming in to the feerate of the existing transactions that it directly conflicts with

    Right, this wasn’t in BIP 125. I also misinterpreted that part of the code to be part of the other fee-related rules and just a “fail fast” piece of logic, and didn’t include it in # #23711 either (gah!). Just opened #25382 to add test coverage for it in isolation + add it to doc/policy/replacements.md


    t-bast commented at 12:34 pm on June 20, 2022:

    it’s only the directly conflicting transactions and not their descendants

    Could you clarify that? I’m not sure whether @sdaftuar’s example contradicts this claim or not.

    Let’s consider a mempool containing transaction A (1000vb, 1sat/vb), child B (1000vb, 10sat/vb) and grand-child C (100vb, 100sat/vb).

    If we want to replace this with a package containing transaction A’ (1000vb, 1sat/vb) and child B’ (1000vb, 20sat/vb), would that work? Or would we be forced to match the 100 sat/vb of grand-child C (since @sdaftuar says the ancestor feerate of that package must be higher than the actual feerate of all transactions that would be evicted)? If so, that would be an issue, an attacker could always put a low-fee high-feerate descendant to make it very expensive to replace the first transaction in the chain, can’t they?


    sdaftuar commented at 3:20 pm on June 21, 2022:

    require the ancestor feerate of the child transaction to be higher than the actual feerate of all transactions that would be evicted. This is conservative but I think would ensure that we don’t have any situations where there are transactions being replaced that might appear in the next block, while the new transactions do not.

    I think this is a sensible addition. I cannot think of any easy attacks since it’s only the directly conflicting transactions and not their descendants (i.e. I don’t have the concerns stated in #23121 (comment)).

    Ah, in writing this I overlooked what you had written up in that comment and forgot about that concern. When I commented before I did indeed mean to suggest that we’d compare our estimated mining score of the new child transaction (min(package feerate, ancestor feerate) as you say) with the actual feerate of all the transactions that would be evicted, and not just the direct conflicts – if we only compare to the feerate of direct conflicts, then I believe it is still possible for a transaction that would be included in the next block to be replaced by a transaction that will confirm much later.

    However now you have reminded me of the other concern, which is that a high actual feerate (but low ancestor feerate) transaction could be used to pin a package and make it difficult to evict. I’m not sure what to do about this!

    Perhaps a simple, good enough approach might be to require that the child transaction’s score is better than the ancestor feerate of all the transactions that would be evicted, and the actual feerates of the direct conflicts? I just think we should do something to ensure that a new transaction package isn’t strictly worse than something that would be evicted. We’d still be missing the case where something being evicted has a lower ancestor score than its actual mining score because of an unrelated transaction that is paying for a low feerate parent, but maybe that is an edge case that we can overlook (especially since the total fees in the mempool are still going up and the first order approximation of mining desirability is also going up).

    I was trying to think about whether this approach might be attackable in some way to either muck with fee estimation or attack someone’s mempool with low quality transactions, but I need to give it more thought.

    An alternate approach to this problem might be to define some new (opt in) relay policy that limits the scope of descendant transactions in some way, and then only implement a package RBF policy that would interact with transactions that meet that limited criteria. That might be a good direction to go if our broader attempts at this problem result in behavior that’s not incentive compatible..?


    ariard commented at 12:00 pm on June 22, 2022:

    Perhaps a simple, good enough approach might be to require that the child transaction’s score is better than the ancestor feerate of all the transactions that would be evicted, and the actual feerates of the direct conflicts?

    To my understanding of this dual-criteria approach (better ancestor score than the set of evicted transactions and better feerate of the direct conflicts) I’m not sure it’s robust against pinning attackers.

    Let’s say you have a mempool containing parent A (1000vb, 1 sat/vb), parent B (1000vb, 2 sat/vb) and child C (100vb, 20 sat/vb). You would like to replace C with newer transaction C’ (100vb, 10 sat/vb) spending parent A and parent D (1000vb, 20 sat/vb). While the ancestor score of C’ is better than C, the feerate is far less compelling and it should be rejected by the mempool. However, the low-ancestor score child C should not be mined for a while and the pin succeeds, if this scenario is correct.

    An alternate approach to this problem might be to define some new (opt in) relay policy that limits the scope of descendant transactions in some way, and then only implement a package RBF policy that would interact with transactions that meet that limited criteria.

    I don’t know if any opt-in approach limiting the size of the package (both ancestors/descendants) is compatible with miners incentives. I think a pinning attacker would be always able to attach an ancestor beyond the limit downgrading the high-feerate replacement candidate ancestor score.

    I’m not sure if we have considered other scoring approach like best-ancestor-score of the latest generation of package transactions. I.e, if you have parent A, parent B and child C and grandchild D, and you would like to replace with new package parent A, parent E and child C’ you evaluate ancestor score of C’ against grandchild D. I think it could be a correct answer to the miner viewpoint “Which competing chain of transactions is the most interesting to include ?”. Though I don’t know if it’s fully incentive compatible.


    glozow commented at 4:30 pm on June 27, 2022:

    When I commented before I did indeed mean to suggest that we’d compare our estimated mining score of the new child … However now you have reminded me of the other concern, which is that a high actual feerate (but low ancestor feerate) transaction could be used to pin a package and make it difficult to evict. I’m not sure what to do about this!

    My bad about misunderstanding your original suggestion! Yes, this is my concern with it and I’m glad you agree.

    Perhaps a simple, good enough approach might be to require that the child transaction’s score is better than the ancestor feerate of all the transactions that would be evicted, and the actual feerates of the direct conflicts?

    I think this would be fine, definitely an improvement upon our current logic. I can’t really think of a strong attack, will think about it more.

    Let’s say you have a mempool containing parent A … While the ancestor score of C’ is better than C, the feerate is far less compelling and it should be rejected by the mempool.

    IIUC since we use the minimum of ancestor feerate and individual feerate, this example doesn’t break it, since we’d use C'’s individual feerate of 10sat/vB to compare to C’s individual feerate of 20sat/vB.

    The natural “attack” scenario might be: A counterparty broadcasts their tx A (which we’ll want to replace). They also broadcast an unrelated high-fee tx B (50sat/vB), and a child C which spends from A and B. Since C spends from B, it has a high ancestor feerate (even though they wouldn’t be mined together), and is thus harder to replace. But since B is high feerate, it will be mined soon, so we’re probably fine.

    An alternate approach to this problem might be to define some new (opt in) relay policy that limits the scope of descendant transactions in some way, and then only implement a package RBF policy that would interact with transactions that meet that limited criteria. That might be a good direction to go if our broader attempts at this problem result in behavior that’s not incentive compatible..?

    I’m now pretty convinced that opt-in shrinking ancestor/descendant limits is the way to go (ping @instagibbs who had ideas about this being necessary in the future). Maybe we start allowing nVersion=3 (or whatever signal we decide) and it doubles as a signal for package RBF and stricter limits (and BIP 125 signaling would not be used for package RBF).

    A few questions:

    • Is it possible to have single transaction limits as well (e.g. a single transaction cannot be more than 10KvB)? If so, that seems simpler than restricting descendants based on the size of the original transaction. Or would that be a bad idea, since it would also imply fewer possible in-flight HTLCs?
    • In this policy, I think we would want to require “any tx spending from an unconfirmed nVersion=3 tx must also be nVersion=3?” And if a nVersion=3 transaction has ancestors that aren’t nVersion=3, we’d use the stricter limits? Following that, I guess we would also need “any tx replacing an nVersion=3 tx must also be nVersion=3?”
    • Assuming we can figure out something sensible, it seems better to have both ancestor and descendant limits? Adding descendant limits resolves the “you must pay for all conflicts” type of pinning. But adding an ancestor limit could be beneficial in resolving the “create a replacement tx with a giant ancestor” type pinning.

    instagibbs commented at 5:29 pm on June 27, 2022:

    I’ve posted this privately in a few places, but here’s a likely incomplete discussion of the opt-in limit reduction discussion, aimed at LN/eltoo problems, but by no means limited to them: https://gist.github.com/instagibbs/b3095752d6289ab52166c04df55c1c19

    “complete package” limits seem to be inherent to any usable solution


    sdaftuar commented at 9:37 pm on June 28, 2022:

    Assuming we can figure out something sensible, it seems better to have both ancestor and descendant limits? Adding descendant limits resolves the “you must pay for all conflicts” type of pinning. But adding an ancestor limit could be beneficial in resolving the “create a replacement tx with a giant ancestor” type pinning.

    Yeah I was thinking in this direction as well. I wonder if we could find some super minimal use case, like a 2-transaction package replacing another 2-transaction package, and just implement that to start. I think the rough idea would be:

    • some opt-in flag (eg nVersion=3, or anything else) indicates that a transaction can’t have more than 1 in-mempool descendant with no more than X vbytes of total size. Any descendant is also not allowed to have more than 1 ancestor.
    • When a 1-child, 1-parent package comes in, if it conflicts with at most 1 opt-in package, allow for RBF semantics along the lines of what we’re discussing. In this case, I think we would require the package feerate to be greater than the ancestor feerates of both transactions that would be replaced (along with the total fee check, and the incremental relay fee check).

    Would there be value to any layer 2 projects if something this minimal were to be implemented?


    t-bast commented at 6:30 am on June 29, 2022:

    This would be very valuable for lightning. @instagibbs has explored that space recently.

    A single descendant is too limiting though: in the lightning case, each participant has its own anchor output, so we need to allow 2 descendants if we apply that rule to the commitment transaction.


    ajtowns commented at 11:18 am on June 29, 2022:

    [1] In looking at this example, I noticed that BIP 125 as drafted seems to misstate an important check we perform in our current RBF policy, namely that we compare the feerate of a new transaction coming in to the feerate of the existing transactions that it directly conflicts with.

    Perhaps we should aim for a new rbf bip that describes bitcoin core’s actual policy for both rbf of single txs and packages of txs, and document that we implement the new bip, and not 125?


    instagibbs commented at 1:34 pm on June 29, 2022:

    To recap my thoughts:

    1. I really think this direction is somewhere we need to go. The fact that in my sketches I can get rid of carveout, and naively support N-party contracts speaks to me that we’re on the right path. The fact it makes simple systems like batched payouts potentially more robust is also a win.

    2. I am 99% sure we want to limit entire package limits to make rule#3 pinning not a kill shot for replacement replacement

    3. @t-bast Ephemeral dust, or dust values that are immediately spent within the package, enable LN and other systems to switch over to a single anchor output, obviating the requirement for carveout rules, and would fit within the one-parent-one-child pattern.

    We could expand this to two direct children limit, and preserve the carveout to make sure currently deployed systems get benefits, assuming it’s not otherwise a DoS/incentive compatible issue

    1. Another nice-to-have would be forced signaling of replacability of the child tx, since the single-anchor spend is mostly freeform. We can work around this via “0 CSV OP_1ADD” scriptpubkey which forces RBF signaling, but something to consider. Full RBF solves this longer term as well.

    t-bast commented at 1:44 pm on June 29, 2022:

    @t-bast Ephemeral dust, or dust values that are immediately spent within the package, enable LN and other systems to switch over to a single anchor output, obviating the requirement for carveout rules, and would fit within the one-parent-one-child pattern.

    :100:

    We could expand this to two direct children limit, and preserve the carveout to make sure currently deployed systems get benefits, assuming it’s not otherwise a DoS/incentive compatible issue

    Don’t bother if it’s only for lightning, since we’ll need a (small) change on the commitment transaction format to benefit from it, we might as well directly switch to a single anchor output.

  27. fanquake referenced this in commit d6832217ef on Jun 16, 2022
  28. sidhujag referenced this in commit 457143b0d8 on Jun 16, 2022
  29. t-bast commented at 2:51 pm on June 21, 2022: contributor

    I have added a first set of tests to eclair that exercise the package-rbf logic: https://github.com/ACINQ/eclair/commit/4f583b5725cba6594388d54c0b31affc1bf8cddf

    I extracted the transactions as test vectors here: https://gist.github.com/t-bast/7c553e61ff2bee3720ff4f7db04cc1b3 I’m not sure how easy it will be to translate them to bitcoin core tests, you’ll need to re-generate some of the inputs and re-sign the commit transactions. The file should have all the required data, but it may be quite painful still…

    The third test vector triggers the following error: Internal bug detected: "it != package_result.m_tx_results.end()"

  30. DrahtBot added the label Needs rebase on Jun 30, 2022
  31. glozow commented at 9:50 am on July 8, 2022: member

    Apologies for the delay, I am rebasing + working on the following:

    • Some unit tests for RBF code
    • Adding an ancestor feerate rule #25038 (review)
    • Adding a set of rules for nVersion=3, i.e. stricter ancestor/descendant limits, inheritance, signaling for RBF, etc. #25038 (review) Won’t be a finalized proposal, I’m hoping that having code written will help us iterate on what those rules should be.

    Will update soon!

  32. glozow force-pushed on Jul 19, 2022
  33. glozow renamed this:
    BIP125-based Package RBF
    policy: nVersion=3 and Package RBF
    on Jul 19, 2022
  34. glozow force-pushed on Jul 19, 2022
  35. glozow force-pushed on Jul 19, 2022
  36. DrahtBot removed the label Needs rebase on Jul 19, 2022
  37. in doc/policy/packages.md:136 in 18630747af outdated
    131+However, allowing replacement for all ancestor packages also might not make sense due to the
    132+increased complexity.
    133+
    134+2. At least one of the following signaling conditions is met:
    135+
    136+    1a. All of the directly conflicting transactions signal BIP125 replaceability explicitly.
    


    instagibbs commented at 3:10 pm on July 19, 2022:
    All of the directly conflicting transactions already in mempool?

    glozow commented at 3:43 pm on July 19, 2022:

    Yes, same as here: https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md#current-replace-by-fee-policy

    A transaction conflicts with an in-mempool transaction (“directly conflicting transaction”) if they spend one or more of the same inputs.

  38. in doc/policy/packages.md:180 in 18630747af outdated
    175+*Rationale*: These fee-related rules are identical to the replacement [policy for individual
    176+transactions](./mempool-replacements.md), treating the package (after deduplication) as one
    177+composite transaction. This works because all of the transactions in a
    178+child-with-unconfirmed-parents package are in the ancestor set of the child transaction.
    179+
    180+6. The number of original transactions does not exceed 100.
    


    instagibbs commented at 3:15 pm on July 19, 2022:
    replaced

    glozow commented at 3:44 pm on July 19, 2022:

    same as here: https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md#current-replace-by-fee-policy

    Would it be more clear if I change the terms, or repeat them in this doc?


    instagibbs commented at 5:08 pm on July 27, 2022:
    being explicit is always better imo
  39. in doc/policy/version3_transactions.md:16 in 18630747af outdated
    11+   replaceability. Use the (`-mempoolfullrbf`) configuration option to allow transaction
    12+   replacement without enforcement of any opt-in signaling rule.
    13+
    14+3. A V3 transaction cannot have more than 1 descendant.
    15+
    16+4. A V3 transaction that has a V3 ancestor cannot be larger than 4000 virtual bytes.
    


    instagibbs commented at 3:24 pm on July 19, 2022:

    I believe this falls short in that the parent V3 transaction can be just south of 101kvB, even in ANYONECANPAY and SIGHASH_SINGLE like scenarios where additional inputs cannot be excluded by the presigning parties.

    In imprecise language, this is what I expect to be required to solve it: “When evaluating a V3 transaction, if its total mempool package size after entry into the mempool would exceed 4000 virtual bytes, it is rejected.”


    glozow commented at 3:53 pm on July 19, 2022:

    Happy to change this of course, but just want to clarify what you’re suggesting.

    In imprecise language, this is what I expect to be required to solve it: “When evaluating a V3 transaction, if its total mempool package size after entry into the mempool would exceed 4000 virtual bytes, it is rejected.”

    I interpret this to mean that we should change both ancestor and descendant limits to 4000vB. This also means an individual V3 tx is not allowed to be more than 4000vB by itself. So in the LN case, a commitment tx isn’t allowed to be more than 4000vB. I believe this would necessitate lowering the max in-flight HTLC limit. Is that accurate and if so, is it ok?


    instagibbs commented at 3:59 pm on July 19, 2022:

    Your comment seems correct. Will have to think about this some more with respect to the case today.

    edit: Looks like that works out to ~85 HTLC outputs or so naively in the layered commitment style setting


    ajtowns commented at 2:24 am on July 20, 2022:

    I believe this falls short in that the parent V3 transaction can be just south of 101kvB,

    Correct me if I’m wrong, but I think this case would be something like: you’re trying to confirm tx A which pays no fees itself; your counterparty publishes a package of tx A, B and C; C spends both A and B; A and C are both small, but B is 99kvB with an ephemeral output that is spent by C. You can’t replace C without also replacing B because of the ephemeral output.

    What you could do, if the ephemeral output is anyone can spend, is replace C in two steps: one that creates tx D that spends B’s ephemeral output (it’s anyone can spend), which costs you C’s fees plus epsilon; then you create tx E that spends A with your desired feerate. So you’re paying ~100kvB at the “backlog” rate here.

    With a 4kvB “related txs” limit, I think the worst case setup is just A is small (100vB) and is spent by B which is large (3900vB), so you have to pay at least 3.9kvB at the backlog rate plus epsilon to replace it with something smaller, but that’s clearly already 25x better. Even if the backlog rate is not much smaller than the next block rate, you’re only overpaying by at most 39x rather than perhaps 1000x here.

    I interpret this to mean that we should change both ancestor and descendant limits to 4000vB.

    Hmm, I think the worst case scenario here would still be bad:

    • A is spent by {B1,..,B20} (descendent count = 21 so less than 25)
    • B1 also spends C1, B2 also spends C2, etc; all via ephemeral outputs
    • A, Bn are 100vB each; Cn is 3800vB each

    A has a descendent size of 2100vB; Bn has an ancestor size of 4000vB and a descendent size of 100vB; Cn has a descendent size of 3900vB. So the 4000vB restriction is okay.

    But replacing A thus means Bn are invalid, which then means Cn are invalid due to the now unspent ephemeral outputs, which means you’re replacing 78kvB in the mempool, not just 4kvB.


    glozow commented at 8:37 am on July 20, 2022:

    @ajtowns Note that ephemeral outputs aren’t proposed here, just the anc/desc limits.

    Also note that V3 descendant count is limited to 2 (i.e. a V3 transaction can only have 1 descendant), so A can’t be spent by {B1…B20}. But of course B1 could spend a bunch of things, up to {C1…C23}.

    Good point that with ephemeral outputs replacing a child means we need to remove the parent as well. I’m experimenting with implementing it, and my idea was to require the parent (the one with the dust output) pay 0 fee so it can’t stay in the mempool by itself. We’d still want to prevent replacing a large volume of transactions (i.e. include it in the RBF 100 limit), but users/devs don’t have to worry about high costs of replacement since its fee is 0?

    I’m thinking about ephemeral outputs but hoping to minimize the scope of this proposal. Correct me if I’m wrong, but it seems ephemeral outputs are not needed to make package relay work for LN penalty without pinning attacks. We could do ephemeral outputs as a next step - IIUC ephemeral outputs would be a “loosening” of policy rules so we could add it on to V3 rules later?


    instagibbs commented at 1:26 pm on July 20, 2022:

    Ephemeral outputs for ln-penalty are nice but definitely a stretch goal. I’m assuming some form exists for eltoo.

    Keeping inline with what is proposed here:

    For eltoo purposes, there’s no reason we cannot restrict each presigned transaction to having a single input only, and only the one ephemeral anchor for spending. Whenever a proper replacement for rule#3 is figured out, the one-input restriction could be relaxed for eltoo construction, and more typical ANYONECANPAY constructions likely work after this as well.


    instagibbs commented at 1:55 pm on July 20, 2022:

    But replacing A thus means Bn are invalid, which then means Cn are invalid due to the now unspent ephemeral outputs, which means you’re replacing 78kvB in the mempool, not just 4kvB.

    To be clear, as long as we have sane DoS limits in place, to the RBF-ing party, they don’t care about these extra bytes as they wouldn’t be paying for those additional bytes(by package CPFP logic already in place, the child is already paying for parent). These further-restricted limits are not about stopping network DoS, but about pinning.

    edit: Ok, the marginal DoS is there unless you ensure the dust-having transactions are unattractive to mine on their own, which 0-fee is definitely sufficient to do.


    t-bast commented at 7:50 am on July 28, 2022:

    Maybe this is a naive question, but is there a way we can make that value configurable instead of using a protocol hard-coded value?

    Informally, any v3 transaction would include a descendant_max_weight somewhere (handwave, handwave), covered by signatures. This lets L2 contracts decide what values are acceptable for each tx type. For example, in the anchor outputs LN-penalty case, we know that commit txs only need an anchor child that can be constrained to be quite small, so commit txs could use a low value here, that can take into account the maximum weight of a commit tx.

    Or would this be adding too much complexity for only a marginal improvement? If so, we should probably bikeshed the protocol hard-coded value and use the lowest value we can to minimize the harm a malicious child tx can do.


    t-bast commented at 7:55 am on July 28, 2022:

    I believe this falls short in that the parent V3 transaction can be just south of 101kvB, even in ANYONECANPAY and SIGHASH_SINGLE like scenarios where additional inputs cannot be excluded by the presigning parties.

    Can’t we easily fix that by relaxing the 100kvB policy limit to be 104kvB for v3 transactions?


    instagibbs commented at 3:02 pm on July 28, 2022:

    It’s definitely doable under an annex field regime(input commits to total package size), but it’s a question of complexity.

    I do think this kind of system would allow ANYONECANPAY style contracts to be secure against pinning, so it’s definitely worth further exploration.


    instagibbs commented at 3:03 pm on July 28, 2022:

    Can’t we easily fix that by relaxing the 100kvB policy limit to be 104kvB for v3 transactions?

    You would still have to replace 100kvB of griefing txns in general, so that doesn’t sidestep the issue?


    t-bast commented at 7:02 am on July 29, 2022:
    That is true, it’s more important to focus on the general issue taking the parents into account, you’re right, setting a limit on the whole v3 package would be useful.
  40. glozow commented at 11:52 am on July 28, 2022: member

    we should probably bikeshed the protocol hard-coded value and use the lowest value we can to minimize the harm a malicious child tx can do.

    Here’s my reasoning for the current value (4000vB), but open to feedback and I will of course add the rationale to the version3_transactions.md doc:

    • Upper bound: the larger we make this limit, the more vbytes we may need to replace. With default limits, if the child is e.g. 100,000vB, that might be an additional 100,000sat or more, depending on the feerate.
    • Lower bound: the smaller we make this limit, the fewer UTXOs we can use to fund this fee-bump. If we require all fee-bumps to only use 1 UTXO, for example, we’re be requiring wallets to always maintain a pool of high-value UTXOs, one for each channel they might close.

    If you’re broadcasting commitment tx commit_a and 4000vB is the largest output your counterparty can attach to their commitment tx commit_b, you can easily meet (commit_b.fees + 4000 * minfeerate) + (incrementalfeerate * commit_a.size) fee to replace.

    4000vB should give you at least 50 inputs depending on the output type, which I’m guessing is a reasonable restriction. If you say, hey, 10 inputs is enough, then we could lower this to 1000vB.

  41. DrahtBot added the label Needs rebase on Jul 28, 2022
  42. t-bast commented at 7:01 am on July 29, 2022: contributor

    Here’s my reasoning for the current value (4000vB), but open to feedback and I will of course add the rationale to the version3_transactions.md doc

    Thanks for the explanation!

    4000vB should give you at least 50 inputs depending on the output type, which I’m guessing is a reasonable restriction. If you say, hey, 10 inputs is enough, then we could lower this to 1000vB.

    Since that child transaction is only bringing fees, it shouldn’t need a big input amount, so I’d say 50 inputs is clearly too much, 10 inputs should be more than enough, shouldn’t it?

  43. instagibbs commented at 2:24 pm on July 29, 2022: member

    @t-bast and I realized that under this proposal, even excluding ephemeral outputs, we can do all the anchor improvements up/to the “ephemeral” part for ln-penatly.

    i.e. Change to an OP_TRUE single anchor output for commitment transaction, but right at the relay output dust level.

    Wouldn’t be applicable to eltoo-like constructs, but these are more rare it seems. Most contract types can survive bleeding off a few sats once. e.g. batched payouts, coinjoins, ln-penalty

    Scope creep to OP_TRUE becoming a standard output(up to 1 a tx?) to relay might be worth it? We can figure out the ephemeral part as a nice cleanup of current protocols further in future, and fix for eltoo-like stuff.

  44. glozow commented at 10:51 am on August 1, 2022: member

    Since that child transaction is only bringing fees, it shouldn’t need a big input amount, so I’d say 50 inputs is clearly too much, 10 inputs should be more than enough, shouldn’t it?

    Oh great! Then I’ll limit the v3 child to 1000vB. Very simple :ok_hand: :grinning:

  45. glozow force-pushed on Aug 2, 2022
  46. DrahtBot removed the label Needs rebase on Aug 2, 2022
  47. DrahtBot added the label Needs rebase on Aug 3, 2022
  48. glozow force-pushed on Aug 8, 2022
  49. DrahtBot removed the label Needs rebase on Aug 8, 2022
  50. fanquake referenced this in commit c5f0cbefa3 on Aug 22, 2022
  51. DrahtBot added the label Needs rebase on Aug 22, 2022
  52. glozow force-pushed on Aug 31, 2022
  53. DrahtBot removed the label Needs rebase on Aug 31, 2022
  54. glozow force-pushed on Sep 1, 2022
  55. glozow force-pushed on Sep 2, 2022
  56. glozow marked this as ready for review on Sep 3, 2022
  57. glozow commented at 10:16 am on September 3, 2022: member
    Ready for review! Writing a mailing list post at the moment. You can also look at doc/policy/packages.md and doc/policy/version3_transactions.md.
  58. glozow force-pushed on Sep 23, 2022
  59. glozow commented at 3:38 pm on September 23, 2022: member
    Rebase + deleted some unused code + tweak to the signaling logic. Posted to the mailing list just now https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
  60. in doc/policy/packages.md:145 in f48315af9c outdated
    117@@ -117,3 +118,53 @@ backward-compatible for users and applications that rely on p2p transaction rela
    118 package validation should not prevent the acceptance of a transaction that would otherwise be
    119 policy-valid on its own. By always accepting a transaction that passes individual validation before
    120 trying package validation, we prevent any unintentional restriction of policy.
    121+
    122+### Package Replace by Fee
    123+
    124+A transaction conflicts with an in-mempool transaction ("directly conflicting transaction") if they
    125+spend one or more of the same inputs. A transaction may conflict with multiple in-mempool
    126+transactions. A directly conflicting transaction's and its descendants (together, "original
    


    t-bast commented at 9:34 am on September 29, 2022:

    nit:

    0transactions. A directly conflicting transaction and its descendants (together, "original
    
  61. in doc/policy/packages.md:190 in f48315af9c outdated
    165+*Rationale*: Prevent network-wide DoS where replacements using the same outpoints are relayed over
    166+and over again; require the replacement transactions to pay for relay using "new" fees. This
    167+rule is taken from the [replacement policy for individual transactions](./mempool-replacements.md),
    168+treating the package as one composite transaction.
    169+
    170+5. The number of original transactions does not exceed 100.
    


    t-bast commented at 9:51 am on September 29, 2022:

    I was wondering whether this could be a pinning vector when used with v3 transactions, but it’s not :tada:

    Packages are by default restricted to 25 elements, so in theory we’re safe, but since this value is configurable, we cannot completely rely on this protection. However, the size restriction on the v3 child ensures that it cannot be bumping 100 parent transactions, so this ensures that we’re really safe (unless I’m missing something)!


    glozow commented at 12:58 pm on September 29, 2022:
    Yep! It should basically be impossible to hit this limit in the commitment tx bumping case. Your commitment tx will only conflict with that 1 commitment tx, and it cannot have more than 1 descendant as a V3 transaction, even if the node has configured a larger -limitdescendantcount.

    instagibbs commented at 1:24 pm on September 29, 2022:
    snuck in a fix for rule#5 pinning, nice. Probably should be noted somewhere.
  62. in doc/policy/version3_transactions.md:3 in f48315af9c outdated
    0@@ -0,0 +1,62 @@
    1+# Transactions with nVersion 3
    2+
    3+A transaction with its `nVersion` field set to 3 ("V3 transactions") are allowed in mempool and
    


    t-bast commented at 11:32 am on September 29, 2022:

    nit:

    0A transaction with its `nVersion` field set to 3 ("V3 transactions") is allowed in mempool and
    

    Or start the sentence with “Transactions with their”.


    glozow commented at 3:12 pm on November 2, 2022:
    Thanks, done
  63. in doc/policy/version3_transactions.md:43 in f48315af9c outdated
    38+transaction. The contract protocol can create presigned transactions paying 0 fees and 1 output for
    39+attaching a CPFP at broadcast time ("anchor output"). Without package RBF, multiple anchor outputs
    40+would be required to allow each counterparty to fee-bump any presigned transaction. With package
    41+RBF, since the presigned transactions can replace each other, 1 anchor output is sufficient.
    42+
    43+4. A V3 transaction that has an unconfirmed V3 ancestor cannot be larger than 1000 virtual bytes.
    


    t-bast commented at 11:39 am on September 29, 2022:

    Maybe an unrelated question, but currently bitcoind doesn’t let us create a transaction that has no outputs. In the context of CPFP, transactions with no outputs can be very useful: if you have inputs that let you reach the desired package feerate without creating any change, that’s what you want to do. But currently we can’t, so we have to overshoot the input amount and add a change output.

    Is there a consensus rule that disallows transactions with no outputs in blocks? If not, it would be quite useful to allow v3 transactions that have no outputs, introducing that new version is a good opportunity to do that.


    glozow commented at 1:15 pm on September 29, 2022:
    Yeah good point, seems perfectly reasonable to me to have no payments in a CPFP transaction. I agree it seems like it would be fine to allow empty vout. But it looks like non-empty vout is indeed a consensus rule. See CheckTransaction: https://github.com/bitcoin/bitcoin/blob/291e363ce500e492475c4ccd189ea1d031c43613/src/consensus/tx_check.cpp#L16-L17 Which is called by CheckBlock: https://github.com/bitcoin/bitcoin/blob/291e363ce500e492475c4ccd189ea1d031c43613/src/validation.cpp#L3370-L3381

    glozow commented at 1:20 pm on September 29, 2022:

    Extra trivia: seems like this consensus rule has been here since the very beginning :smiling_face_with_tear: can’t believe Satoshi didn’t predict the need for CPFPing presigned transactions.

    https://github.com/bitcoin/bitcoin/blob/4405b78d6059e536c36974088a8ed4d9f0f29898/main.h#L443-L448


    instagibbs commented at 1:57 pm on September 29, 2022:

    Is there a consensus rule that disallows transactions with no outputs in blocks?

    Yes. It’s a hard rule.


    t-bast commented at 2:03 pm on September 29, 2022:
    That’s too bad! Damn you Satoshi, you could have thought about this…

    instagibbs commented at 2:03 pm on September 29, 2022:
    OP_RETURN outputs can be pretty small :)

    glozow commented at 3:05 pm on September 29, 2022:
    Mm, could have a 0-value OP_RETURN?

    instagibbs commented at 3:06 pm on September 29, 2022:
    yes, you’ll pay 8+1+1 bytes for the output
  64. in src/policy/rbf.cpp:205 in f48315af9c outdated
    201+        // A package/transaction's ancestor feerate is not equivalent to the miner score; it may
    202+        // overestimate. Some subset of the ancestors could be included by itself if it has other
    203+        // high-feerate descendants or are themselves higher feerate than this package/transaction.
    204+        // For now, as a conservative estimate, use the minimum between the transaction's individual
    205+        // feerate and ancestor feerate.
    206+        const CFeeRate replacement_miner_score = std::min(replacement_individual_feerate, replacement_ancestor_feerate);
    


    t-bast commented at 12:10 pm on September 29, 2022:

    Can we relax the replacement_ancestor_feerate for v3 transactions? What would be ideal would be to simply ignore unconfirmed v3 ancestors from this calculation (some kind of replacement_non_v3_ancestor_feerate), which I believe is ok from an incentives’ point of view given the other restrictions on v3 packages.

    Otherwise we will have the following issue:

    • a commitment transaction pays 0 fees and weighs 19,000 vbytes (it has many HTLCs)
    • an attacker publishes a CPFP v3 child that weighs 1,000 vbytes and has an individual feerate of e.g. 100 sat/byte
    • the package feerate ends up being 5 sat/byte, and the package doesn’t confirm
    • if a feerate of 20 sat/byte for the package was enough to get the package confirmed, the honest participant would target that and thus create a CPFP v3 child with a feerate of 400 sat/byte
    • but that would be rejected under the current rules, and the honest participant is forced to make the package feerate be 100 sat/byte, which is extremely expensive

    I’m realizing now that this is actually just another way of formulating @instagibbs’s proposal to evict v3 siblings: when the direct conflict is a v3 transaction with an unconfirmed v3 parent, it would be great to only compare their individual feerates.


    glozow commented at 1:04 pm on September 29, 2022:
    To clarify, is the second v3 child (400sat/vbyte) replacing the first v3 child (100sat/vbyte) only? Assuming the commitment tx is already in the mempool. In that case, it would just be single transaction replacement so this rule isn’t used. Just the individual feerates are compared, and the child is accepted. Maybe I’m misunderstanding the scenario though?

    t-bast commented at 1:51 pm on September 29, 2022:

    I think we cannot do only individual replacement here, if the attacker publishes their commit + their child, and the honest participants wants to publish their own commit + their child (because for some reason they never receive the attacker’s package in their mempool so cannot attach a CPFP to that commit).

    So this is actually slightly different from the v3 sibling eviction problem, which as you point out probably just works :tm: with individual replacement rules already.


    t-bast commented at 1:58 pm on September 29, 2022:

    I’m not yet 100% sure whether package RBF currently lets us replace [CommitTxA (19,000 vbytes, 0 sat/byte), AnchorTxA (1,000 vbytes, 20 sat/byte)] with [CommitTxB (19,000 vbytes, 0 sat/byte), AnchorTxB (1,000 vbytes, 80 sat/byte)], where all txs are v3 (which is the issue I’m trying to describe here).

    I probably just need to write a test for it and see for myself, but what do you think?


    glozow commented at 1:59 pm on September 29, 2022:

    I see okay so the situation is:

    original transactions = commitment_A + cpfp_A replacement transactions (package) = commitment_B + cpfp_B

    • commitment_A: 19,000vB, 0sat.
      • individual feerate = 0sat/vB
      • ancestor feerate = 0sat/vB
    • cpfp_A: 1000vbytes, 100,000sat.
      • individual feerate = 100sat/vB
      • ancestor feerate = 5sat/vB
    • commitment_B: 19,000vB, 0sat.
      • individual feerate = 0sat/vB
      • ancestor feerate = 0sat/vB
    • cpfp_B: 1000vB, 400,000sat.
      • individual feerate = 400sat/vB
      • ancestor feerate = 20sat/vB package feerate = 20sat/vB

    replacement_miner_score = minimum(package feerate, ancestor feerate) = 20sat/vB the rule is that replacement_miner_score must be higher than

    • the direct conflict’s (commitment_A) indivdual feerate = 0sat/vB
    • the original transactions’ (commitment_A, cpfp_A) ancestor feerates = 0sat/vB and 5sat/vB

    So this should be fine AFAICT


    t-bast commented at 2:05 pm on September 29, 2022:
    That’s great! I’ll write a test for this in eclair anyway, thanks for clearing that up :+1:

    instagibbs commented at 5:06 pm on November 1, 2022:

    This rule is still sometimes problematic if you are doing batch fee bumping.

    Let’s imagine the case where we have two separate channels, symmetrical commitment transactions(for clarity of argument).

    If commitment_A is in the miner’s mempool, and Bob wants to fee bump commitment_A as well as CPFP commitment_B(different channel!) into the miner’s mempool, Bob will have to pay the replaced cpfp_A’s feerate for the entire CPFP commitment_B + cpfp_B package.

    In other words, batch bumping is probably not a good idea unless you know package RBF will evict the sponsored transaction directly.

  65. in src/policy/contract_policy.h:19 in f48315af9c outdated
    14+#include <string>
    15+
    16+// This module enforces rules for transactions with nVersion=3 ("V3 transactions") which are
    17+// intended for use in contracting protocols.
    18+
    19+/** Maximum virtual size of a tx which spends from a V3 transaction, in vB. */
    


    t-bast commented at 12:11 pm on September 29, 2022:

    nit:

    0/** Maximum virtual size of a tx which spends from an unconfirmed V3 transaction, in vB. */
    

    glozow commented at 3:36 pm on November 2, 2022:
    Thanks, done
  66. in src/policy/contract_policy.h:32 in f48315af9c outdated
    27+/** Maximum total virtual size of transactions, in KvB, including a tx and all its mempool ancestors. */
    28+static constexpr unsigned int V3_ANCESTOR_SIZE_LIMIT_KVB{DEFAULT_ANCESTOR_SIZE_LIMIT_KVB};
    29+
    30+/** Every transaction that spends an unconfirmed V3 transaction must also have V3. Check this rule
    31+ * for ptx and a package that may contain its unconfirmed ancestors. It's fine if ptx is one of the
    32+ * transactions in the package, and it's fine if none of the transactions are ancestors of ptx.
    


    t-bast commented at 12:17 pm on September 29, 2022:
    It looks like part of this comment should be on the CheckV3Inheritance overload below, the one that takes a ptx as argument?

    glozow commented at 1:23 pm on September 29, 2022:
    Ah yes :+1: thanks!
  67. in src/policy/contract_policy.cpp:64 in f48315af9c outdated
    59+std::optional<std::string> ApplyV3Rules(const CTransactionRef& ptx,
    60+                                        const CTxMemPool::setEntries& ancestors,
    61+                                        const std::set<uint256>& direct_conflicts)
    62+{
    63+    // These rules only apply to transactions with nVersion=3.
    64+    if (ptx->nVersion != 3) return std::nullopt;
    


    t-bast commented at 12:20 pm on September 29, 2022:
    Shouldn’t we call CheckV3Inheritance here instead of letting the caller check that before? There are no cases where we want to call ApplyV3Rules without also checking v3 inheritance, are there?

    glozow commented at 3:24 pm on November 2, 2022:
    Added, thanks!
  68. DrahtBot added the label Needs rebase on Oct 9, 2022
  69. in doc/policy/packages.md:127 in add1250fda outdated
    122+### Package Replace by Fee
    123+
    124+A transaction conflicts with an in-mempool transaction ("directly conflicting transaction") if they
    125+spend one or more of the same inputs. A transaction may conflict with multiple in-mempool
    126+transactions. A directly conflicting transaction's and its descendants (together, "original
    127+transactions") must be evicted in the event of a replacement.
    


    instagibbs commented at 7:00 pm on October 25, 2022:

    “evicted in the event of a replacement” is a tautology, it’s being replaced so obviously it’s being evicted.

    suggested: “in the event of mempool entry of the new transaction”


    glozow commented at 3:17 pm on November 2, 2022:
    Fixed, hopefully clearer now
  70. in doc/policy/version3_transactions.md:20 in add1250fda outdated
    15+V3 transactions:
    16+
    17+1. A V3 transaction can be replaced, even if it does not signal BIP125 replaceability. Other
    18+   conditions rules apply, see [RBF rules](./mempool-replacements.md) and [Package RBF
    19+rules][./packages.md#Package-Replace-By-Fee]. Use the (`-mempoolfullrbf`) configuration option to
    20+allow transaction replacement without enforcement of any opt-in signaling rule.
    


    instagibbs commented at 7:04 pm on October 25, 2022:
    … for non-V3 transactions

    glozow commented at 3:12 pm on November 2, 2022:
    Deleted comment
  71. in doc/policy/version3_transactions.md:51 in add1250fda outdated
    17+1. A V3 transaction can be replaced, even if it does not signal BIP125 replaceability. Other
    18+   conditions rules apply, see [RBF rules](./mempool-replacements.md) and [Package RBF
    19+rules][./packages.md#Package-Replace-By-Fee]. Use the (`-mempoolfullrbf`) configuration option to
    20+allow transaction replacement without enforcement of any opt-in signaling rule.
    21+
    22+2. Any descendant of an unconfirmed V3 transaction must also be V3.
    


    instagibbs commented at 7:05 pm on October 25, 2022:
    based on discussion at Coredev: Any ancestor as well!

    glozow commented at 2:33 pm on October 28, 2022:
    Yes, working on this! Sorry for the delay!

    glozow commented at 3:12 pm on November 2, 2022:
    Done and documented
  72. in doc/policy/version3_transactions.md:29 in add1250fda outdated
    24+*Rationale*: Combined with Rule 1, this gives us the property of "inherited signaling" when
    25+descendants of unconfirmed transactions are created. Additionally, checking whether a transaction
    26+signals replaceability this way does not require mempool traversal, and does not change based on
    27+what transactions are mined.
    28+
    29+*Note*: The descendant of a *confirmed* V3 transaction does not need to be V3.
    


    instagibbs commented at 7:06 pm on October 25, 2022:
    “Just like any other policy measure, it does not effect validity of data in blocks.”

    glozow commented at 3:18 pm on November 2, 2022:
    Clarified
  73. in doc/policy/version3_transactions.md:92 in add1250fda outdated
    40+would be required to allow each counterparty to fee-bump any presigned transaction. With package
    41+RBF, since the presigned transactions can replace each other, 1 anchor output is sufficient.
    42+
    43+4. A V3 transaction that has an unconfirmed V3 ancestor cannot be larger than 1000 virtual bytes.
    44+
    45+*Rationale*: (upper bound) the larger the descendant size limit, the more vbytes may need to be
    


    instagibbs commented at 7:16 pm on October 25, 2022:

    The pinning “damage” is specifically bounded between the tx size of whatever an honest user would have needed to make to bump, and the total allowed limit. For example, if a wallet only needs half the space for an honest fee bump, an attacker is bound to a 2x fee pin attack, maximum, instead of 200x without V3 rules.

    Might want to mention that it bounds the economic damage by ~100x in a digestible form.


    glozow commented at 3:19 pm on November 2, 2022:
    Added a note about 1/100
  74. in doc/policy/packages.md:158 in add1250fda outdated
    153+   original transactions.
    154+
    155+*Rationale*: Attempt to ensure that the replacement transactions are not less incentive-compatible
    156+to mine.
    157+
    158+4. The package fee pays an absolute fee of at least the sum paid by the original transactions.
    


    instagibbs commented at 7:24 pm on October 25, 2022:
    “The duduplicated package” just to make it abundantly clear.

    glozow commented at 3:18 pm on November 2, 2022:
    Added
  75. in src/policy/contract_policy.h:17 in 67cc7a25d3 outdated
    12+#include <txmempool.h>
    13+
    14+#include <string>
    15+
    16+// This module enforces rules for transactions with nVersion=3 ("V3 transactions") which are
    17+// intended for use in contracting protocols.
    


    instagibbs commented at 7:27 pm on October 25, 2022:
    or wallets that want robust RBF abilities…

    glozow commented at 3:36 pm on November 2, 2022:
    Changed, that does sound cooler
  76. in src/policy/contract_policy.h:30 in 67cc7a25d3 outdated
    25+/** Maximum number of transactions including a tx and all its mempool ancestors. */
    26+static constexpr unsigned int V3_ANCESTOR_LIMIT{DEFAULT_ANCESTOR_LIMIT};
    27+/** Maximum total virtual size of transactions, in KvB, including a tx and all its mempool ancestors. */
    28+static constexpr unsigned int V3_ANCESTOR_SIZE_LIMIT_KVB{DEFAULT_ANCESTOR_SIZE_LIMIT_KVB};
    29+
    30+/** Every transaction that spends an unconfirmed V3 transaction must also have V3. Check this rule
    


    instagibbs commented at 7:30 pm on October 25, 2022:
    s/must also have/must also be/

    glozow commented at 3:36 pm on November 2, 2022:
    Done
  77. in src/policy/contract_policy.h:38 in 67cc7a25d3 outdated
    33+ * Assumes the transactions are sorted topologically and have no conflicts, i.e.,
    34+ * CheckPackage(package) passed.
    35+ * @returns a pair of wtxids (parent, child) where the parent is V3 but the child is not V3, if at
    36+ * least one exists. Otherwise std::nullopt.
    37+ */
    38+std::optional<std::pair<uint256, uint256>> CheckV3Inheritance(const Package& package);
    


    instagibbs commented at 7:37 pm on October 25, 2022:
    suggestion: rename arg to checked_package

    glozow commented at 3:37 pm on November 2, 2022:
    Changed the function to take any list of transactions, so won’t rename
  78. in src/policy/contract_policy.h:54 in 67cc7a25d3 outdated
    49+ * 2. Tx with all of its ancestors must be within V3_ANCESTOR_LIMIT.
    50+ *
    51+ * If a V3 tx has V3 ancestors,
    52+ * 1. Each V3 ancestor and its descendants must be within V3_DESCENDANT_LIMIT.
    53+ * 2. The tx must be within V3_CHILD_MAX_SIZE.
    54+ */
    


    instagibbs commented at 7:40 pm on October 25, 2022:
    make explicit what is returned

    glozow commented at 3:38 pm on November 2, 2022:
    Done
  79. in src/policy/contract_policy.h:40 in 67cc7a25d3 outdated
    35+ * @returns a pair of wtxids (parent, child) where the parent is V3 but the child is not V3, if at
    36+ * least one exists. Otherwise std::nullopt.
    37+ */
    38+std::optional<std::pair<uint256, uint256>> CheckV3Inheritance(const Package& package);
    39+
    40+/** Every transaction that spends an unconfirmed V3 transaction must also have V3. */
    


    instagibbs commented at 7:43 pm on October 25, 2022:
    s/must also have/must also be/

    instagibbs commented at 7:45 pm on October 25, 2022:
    please make it explicit what is being returned

    glozow commented at 3:38 pm on November 2, 2022:
    Done and done
  80. in src/test/txvalidation_tests.cpp:105 in 4b8176b96b outdated
    100+        auto tx_v3_from_v2 = make_tx({COutPoint{mempool_tx_v3->GetHash(), 0}}, /*version=*/2);
    101+        pool.CalculateMemPoolAncestors(entry.FromTx(tx_v3_from_v2), ancestors, no_limit, no_limit, no_limit, no_limit, placeholder_str);
    102+        BOOST_CHECK(ancestors.size() == 1);
    103+        BOOST_CHECK(CheckV3Inheritance(tx_v3_from_v2, ancestors).has_value());
    104+        ancestors.clear();
    105+        auto tx_v3_from_v2_and_v3 = make_tx({COutPoint{mempool_tx_v3->GetHash(), 0}, COutPoint{mempool_tx_v2->GetHash(), 0}}, /*version=*/2);
    


    instagibbs commented at 8:32 pm on October 25, 2022:
    s/tx_v3_from_v2_and_v3/tx_v2_from_v2_and_v3/ ?

    glozow commented at 3:39 pm on November 2, 2022:
    Right, unsure why I flipped the name. Fixed.
  81. in src/test/txvalidation_tests.cpp:110 in 4b8176b96b outdated
    105+        auto tx_v3_from_v2_and_v3 = make_tx({COutPoint{mempool_tx_v3->GetHash(), 0}, COutPoint{mempool_tx_v2->GetHash(), 0}}, /*version=*/2);
    106+        pool.CalculateMemPoolAncestors(entry.FromTx(tx_v3_from_v2_and_v3), ancestors, no_limit, no_limit, no_limit, no_limit, placeholder_str);
    107+        BOOST_CHECK(ancestors.size() == 2);
    108+        auto v3_ancestors = GetV3Ancestors(ancestors);
    109+        BOOST_CHECK(v3_ancestors.size() == 1);
    110+        BOOST_CHECK(CheckV3Inheritance(tx_v3_from_v2, ancestors).has_value());
    


    instagibbs commented at 8:33 pm on October 25, 2022:
    we also need positive test cases of CheckV3Inheritance

    glozow commented at 3:39 pm on November 2, 2022:
    Added, thanks
  82. in src/test/txvalidation_tests.cpp:184 in 4b8176b96b outdated
    146+        ancestors.clear();
    147+    }
    148+
    149+    // V3 tx cannot have too large ancestor size
    150+    std::vector<COutPoint> large_mempool_outpoints;
    151+    large_mempool_outpoints.resize(10);
    


    instagibbs commented at 8:36 pm on October 25, 2022:
    why bother with the loop vs one huge tx?

    glozow commented at 4:20 pm on November 2, 2022:
    Because each parent needs to be below 100kvb and parent+child needs to be above 101KvB. If there’s only 1 parent, then the child needs to be >1000vB to get us across 101KvB, but that is violating a different v3 rule as well.
  83. in src/test/txvalidation_tests.cpp:206 in 4b8176b96b outdated
    156+    }
    157+    {
    158+        auto tx_v3_large_parents = make_tx(large_mempool_outpoints, /*version=*/3);
    159+        pool.CalculateMemPoolAncestors(entry.FromTx(tx_v3_large_parents), ancestors, no_limit, no_limit, no_limit, no_limit, placeholder_str);
    160+        BOOST_CHECK(ancestors.size() == 10);
    161+        BOOST_CHECK(ApplyV3Rules(tx_v3_large_parents, ancestors, empty_conflicts_set).has_value());
    


    instagibbs commented at 8:37 pm on October 25, 2022:
    let’s assert up here what size this is, since I’m unsure how big 10 200-input txs are, off-hand

    glozow commented at 4:38 pm on November 2, 2022:
    Done
  84. in src/validation.cpp:790 in c82f832ddd outdated
    751-                if (!m_pool.m_full_rbf && !SignalsOptInRBF(*ptxConflicting)) {
    752+                // All V3 transactions are considered replaceable.
    753+                //
    754+                // Replaceability signaling of the original transactions may be
    755+                // ignored due to node setting.
    756+                if (!m_pool.m_full_rbf && !SignalsOptInRBF(*ptxConflicting) && ptxConflicting->nVersion != 3) {
    


    instagibbs commented at 8:54 pm on October 25, 2022:
    thought: do we just want to make the version check a thing in SignalsOptInRBF?

    glozow commented at 9:46 am on November 1, 2022:
    I thought about that approach, but it’s used for RPC info “bip125-replaceable” (and somewhere in the wallet iirc). Though maybe that’s a good thing (and we should move towards renaming that field to “replaceable” or something).
  85. in test/functional/mempool_accept_v3.py:31 in c11fb499f1 outdated
    26+        self.log.info("Generate blocks to create UTXOs")
    27+        node = self.nodes[0]
    28+        self.wallet = MiniWallet(node)
    29+        self.generate(self.wallet, 10)
    30+        # Mature coinbase transactions
    31+        self.generate(self.wallet, 100)
    


    instagibbs commented at 8:58 pm on October 25, 2022:
    just do a single generate of 110 if you need 10 coins
  86. in test/functional/mempool_accept_v3.py:85 in c11fb499f1 outdated
    43+
    44+        self.log.info("Test a V3 transaction cannot have more than 1 descendant")
    45+        tx_grandchild_v3 = self.wallet.create_self_transfer(utxo_to_spend=tx_child_v3["new_utxo"], version=3)
    46+        assert_raises_rpc_error(-26, "would exceed descendant count limit", node.sendrawtransaction, tx_grandchild_v3["hex"])
    47+
    48+        self.log.info("Test V3 transactions may be replaced by V3 transactions")
    


    instagibbs commented at 9:00 pm on October 25, 2022:
    clear out mempool via miner, to make it clear what’s in flight in test

    glozow commented at 4:03 pm on November 3, 2022:
    Done
  87. in test/functional/mempool_accept_v3.py:64 in c11fb499f1 outdated
    59+            from_node=node,
    60+            fee_rate=DEFAULT_FEE * 2,
    61+            utxo_to_spend=utxo_v3_bip125,
    62+            version=3
    63+        )
    64+        assert node.getmempoolentry(tx_v3_bip125_rbf["txid"])
    


    instagibbs commented at 9:04 pm on October 25, 2022:
    change all these checks to getrawmempool for strict assertion there is a single tx, and the hash is as expected

    glozow commented at 4:03 pm on November 3, 2022:
    Agree that’s better, added check_mempool everywhere.

    glozow commented at 4:10 pm on November 3, 2022:
    Agree that’s better, added check_mempool that does this everywhere.
  88. in test/functional/mempool_accept_v3.py:74 in c11fb499f1 outdated
    69+            from_node=node,
    70+            fee_rate=DEFAULT_FEE * 3,
    71+            utxo_to_spend=utxo_v3_bip125,
    72+            version=2
    73+        )
    74+        assert node.getmempoolentry(tx_v3_bip125_rbf_v2["txid"])
    


    instagibbs commented at 9:05 pm on October 25, 2022:
    mine a block for simpler reading of test

    glozow commented at 5:20 pm on November 2, 2022:
    Split into subtests and mining in between
  89. in test/functional/mempool_accept_v3.py:100 in c11fb499f1 outdated
     95+            utxo_to_spend=tx_v3_parent["new_utxo"],
     96+            version=2
     97+        )
     98+        assert_raises_rpc_error(-26, "non-v3-tx-spends-v3", node.sendrawtransaction, tx_v3_child_rbf_v2["hex"])
     99+
    100+        self.log.info("Test V3 transactions may be replaced by V2 transactions")
    



    glozow commented at 4:06 pm on November 3, 2022:
    Removed
  90. in test/functional/mempool_accept_v3.py:141 in c11fb499f1 outdated
    104+            utxo_to_spend=utxo_v3_bip125,
    105+            version=2
    106+        )
    107+        assert node.getmempoolentry(tx_v3_bip125_rbf_v2["txid"])
    108+
    109+        self.log.info("Test V3 transactions that don't signal BIP125 are replaceable")
    


    instagibbs commented at 9:08 pm on October 25, 2022:
    assert that fullrbf is false in getmempoolinfo… just in case

    glozow commented at 5:06 pm on November 3, 2022:
    Added
  91. in test/functional/mempool_accept_v3.py:139 in c11fb499f1 outdated
    134+            utxo_to_spend=tx_v3_parent_normal["new_utxo"],
    135+            target_weight=4004,
    136+            version=3
    137+        )
    138+        assert_greater_than_or_equal(tx_v3_child_heavy["tx"].get_vsize(), 1000)
    139+        assert_raises_rpc_error(-26, "nonstandard", node.sendrawtransaction, tx_v3_child_heavy["hex"])
    


    instagibbs commented at 9:10 pm on October 25, 2022:
    can we get a better error description?

    glozow commented at 4:10 pm on November 3, 2022:
    Done, it’s now “v3-tx-nonstandard, v3 child tx is too big”
  92. in test/functional/mempool_accept_v3.py:132 in c11fb499f1 outdated
    127+        assert node.getmempoolentry(tx_v3_no_bip125_rbf["txid"])
    128+        assert_raises_rpc_error(-5, "Transaction not in mempool", node.getmempoolentry, tx_v3_no_bip125["txid"])
    129+
    130+        self.log.info("Test a child of a V3 transaction cannot be more than 1000vB")
    131+        self.restart_node(0, extra_args=["-datacarriersize=1000"])
    132+        tx_v3_parent_normal = self.wallet.send_self_transfer(from_node=node, version=3)
    


    instagibbs commented at 9:11 pm on October 25, 2022:
    let’s also have an example that just makes it in

    glozow commented at 4:10 pm on November 3, 2022:
    Changed tx_v3_almost_heavy to be just in
  93. in src/validation.cpp:1200 in dd8d0af4a2 outdated
    1196@@ -1182,7 +1197,7 @@ MempoolAcceptResult MemPoolAccept::AcceptSingleTransaction(const CTransactionRef
    1197     AssertLockHeld(cs_main);
    1198     LOCK(m_pool.cs); // mempool "read lock" (held through GetMainSignals().TransactionAddedToMempool())
    1199 
    1200-    Workspace ws(ptx);
    1201+    Workspace ws(ptx, m_limit_descendants, m_limit_descendant_size);
    


    instagibbs commented at 1:56 pm on October 26, 2022:
    note: a test that this commit fixes the purported future bug would be a good idea

    glozow commented at 5:33 pm on November 3, 2022:
    Dropped the commit and replaced with a big comment, since the bug is not possible when package RBF requires v3.
  94. in src/validation.cpp:1006 in bb1b7ee6c6 outdated
    1000@@ -1001,6 +1001,9 @@ bool MemPoolAccept::PackageMempoolChecks(const std::vector<CTransactionRef>& txn
    1001                        { return !m_pool.exists(GenTxid::Txid(tx->GetHash()));}));
    1002 
    1003     std::string err_string;
    1004+    // Note that the MemPoolAccept::m_limit_descendants, not Workspace::m_tx_limit_descendants, are
    1005+    // used as descendant limits. If any extra descendant carveouts were granted (see comments in
    1006+    // PreChecks), they do not apply here.
    


    instagibbs commented at 2:03 pm on October 26, 2022:
    the idea being, carveouts are only useful in the individual tx scenario, not package context? i.e., you wouldn’t need to propagate both anchor spends on a commit tx in a package, which busts original limits?

    glozow commented at 9:48 am on November 1, 2022:
    Yeah exactly. And that would never happen since 2 children + 1 parent is not an accepted package topology anyway.
  95. in src/validation.cpp:588 in 459a82c3a1 outdated
    568@@ -569,18 +569,11 @@ class MemPoolAccept
    569         std::set<uint256> m_conflicts;
    570         /** Iterators to mempool entries that this transaction directly conflicts with. */
    571         CTxMemPool::setEntries m_iters_conflicting;
    572-        /** Iterators to all mempool entries that would be replaced by this transaction, including
    


    instagibbs commented at 2:05 pm on October 26, 2022:

    to belabor the commit text:

    “Transactions could conflict with the same tx”

    meaning transactions in the same package can conflict with the same original tx, so for package evaluation, we only want to count these once?


    glozow commented at 9:53 am on November 1, 2022:
    Yes. I can elaborate in commit message
  96. in src/test/rbf_tests.cpp:235 in 5cc295b8ef outdated
    225@@ -225,6 +226,60 @@ BOOST_FIXTURE_TEST_CASE(rbf_helper_functions, TestChain100Setup)
    226 
    227     const auto spends_conflicting_confirmed = make_tx({m_coinbase_txns[0], m_coinbase_txns[1]}, {45 * CENT});
    228     BOOST_CHECK(HasNoNewUnconfirmed(*spends_conflicting_confirmed.get(), pool, {entry1, entry3}) == std::nullopt);
    229+
    230+    // Tests for CheckMinerScores
    231+    // Don't allow replacements with a low ancestor feerate.
    232+    BOOST_CHECK(CheckMinerScores(/*replacement_fees=*/entry1->GetFee(),
    233+                                 /*replacement_vsize=*/entry1->GetTxSize(),
    234+                                 /*ancestors=*/{entry5},
    


    instagibbs commented at 2:14 pm on October 26, 2022:
    can we “watermark” the entry names with if they’re low or high, or just grab a renamed copy?

    glozow commented at 9:51 am on November 1, 2022:
    sorry what do you mean? Like rename the variables to indicate fee e.g. entry5_low ?

    glozow commented at 3:54 pm on November 2, 2022:
    Done, renamed them all
  97. in src/validation.cpp:1024 in 7cbe926e13 outdated
    1019     // Note that the MemPoolAccept::m_limit_descendants, not Workspace::m_tx_limit_descendants, are
    1020     // used as descendant limits. If any extra descendant carveouts were granted (see comments in
    1021     // PreChecks), they do not apply here.
    1022     if (!m_pool.CheckPackageLimits(txns, m_limit_ancestors, m_limit_ancestor_size, m_limit_descendants,
    1023                                    m_limit_descendant_size, err_string)) {
    1024+        // TODO: When replacements exist, to avoid overestimating descendant counts, deduct the
    


    instagibbs commented at 2:19 pm on October 26, 2022:
    can you state in the comment what the practical effect(s) of not accounting for this difference is?

    glozow commented at 5:34 pm on November 3, 2022:
    Deleted this particular comment since the TODO is no longer necessary while package RBF requires v3.
  98. in src/validation.cpp:1033 in 7cbe926e13 outdated
    1033+
    1034+    // Further checks are all RBF-only.
    1035+    m_rbf = std::any_of(workspaces.cbegin(), workspaces.cend(), [](const auto& ws){return !ws.m_conflicts.empty();});
    1036+    if (!m_rbf) return true;
    1037+
    1038+    // Transactions must pay for their own replacements unless they are V3.
    


    instagibbs commented at 2:25 pm on October 26, 2022:
    Don’t understand this comment

    glozow commented at 5:01 pm on November 3, 2022:
    Rewritten
  99. in src/validation.cpp:1063 in 7cbe926e13 outdated
    1063+            return package_state.Invalid(PackageValidationResult::PCKG_POLICY,
    1064+                                         "package RBF failed: too many potential replacements", *err_string);
    1065+        }
    1066+    }
    1067+
    1068+    // Check that the union of all collective conflicts and ancestors is disjoint.
    


    instagibbs commented at 3:23 pm on October 26, 2022:
    to rephrase: make sure original conflicting txs are not being used by the replacement package itself?

    glozow commented at 5:11 pm on November 3, 2022:
    Exactly. Edited the comment.
  100. in test/functional/mempool_package_rbf.py:63 in f48315af9c outdated
    58+        for coin in parent_coins:
    59+            parent_result = self.wallet.create_self_transfer(
    60+                fee_rate=0,
    61+                fee=parent_fee,
    62+                utxo_to_spend=coin,
    63+                sequence=MAX_BIP125_RBF_SEQUENCE - self.ctr, # prevent identical txids
    


    instagibbs commented at 3:29 pm on October 26, 2022:
    prevent identical txids between different packages, I presume

    glozow commented at 4:12 pm on November 3, 2022:
    added “between packages”
  101. instagibbs commented at 3:34 pm on October 26, 2022: member
    bunch of comments, didn’t comb through final functional tests, but I’ll wait for the latest rework of the proposal
  102. in src/validation.cpp:1408 in f48315af9c outdated
    1326+    std::transform(txns.cbegin(), txns.cend(), std::back_inserter(workspaces), [this](const auto& tx) {
    1327+        return Workspace(tx, m_limit_descendants, m_limit_descendant_size);
    1328+    });
    1329     std::map<const uint256, const MempoolAcceptResult> results;
    1330 
    1331+    if (const auto v3_violation{CheckV3Inheritance(txns)}) {
    


    instagibbs commented at 9:02 pm on October 27, 2022:
    I think this is broken, it’s only considering the deduplicated package, which “blinds” this evaluation from checking if the in-mempool parent is V3. Hence, the child as V2 would be accepted, even if parent in mempool was V3.

    glozow commented at 9:21 pm on October 27, 2022:
    Also see CheckV3Inheritance call on L932

    instagibbs commented at 9:26 pm on October 27, 2022:
    Ok is this a quick check that misses these scenarios, then PreCheck later calls it again with all ancestors?

    instagibbs commented at 9:28 pm on October 27, 2022:
    ah ok, the mempool lock comment. got it!
  103. in src/policy/contract_policy.cpp:78 in 67cc7a25d3 outdated
    73+        return strprintf("total vsize of tx with ancestors would be too big: %u virtual bytes", tx_vsize + ancestor_vsize);
    74+    }
    75+
    76+    CTxMemPool::setEntries v3_ancestors{GetV3Ancestors(ancestors)};
    77+
    78+    // Note that this code assumes that any V3 transaction can only have 1 descendant.
    


    instagibbs commented at 3:09 pm on October 28, 2022:
    Comment should probably note that the check happens just below. As is it sounds like it’s happening via caller

    glozow commented at 3:25 pm on November 2, 2022:
    Deleted, I confused myself too. Thanks
  104. in src/policy/contract_policy.cpp:86 in 67cc7a25d3 outdated
    81+        // This tx is a child of a V3 tx. To avoid RBF pinning, it can't be too large.
    82+        if (tx_vsize > V3_CHILD_MAX_SIZE) {
    83+            return strprintf("tx is too big: %u virtual bytes", tx_vsize);
    84+        }
    85+    }
    86+    for (const auto& entry : v3_ancestors) {
    


    instagibbs commented at 3:09 pm on October 28, 2022:
    Add an explicit comment here what this check is

    glozow commented at 3:34 pm on November 2, 2022:
    Done
  105. glozow force-pushed on Nov 2, 2022
  106. DrahtBot removed the label Needs rebase on Nov 2, 2022
  107. in src/validation.cpp:1370 in 18076ccda8 outdated
    1366@@ -1245,7 +1367,7 @@ PackageMempoolAcceptResult MemPoolAccept::AcceptMultipleTransactions(const std::
    1367     // because it's unnecessary. Also, CPFP carve out can increase the limit for individual
    1368     // transactions, but this exemption is not extended to packages in CheckPackageLimits().
    1369     std::string err_string;
    1370-    if (txns.size() > 1 && !PackageMempoolChecks(txns, package_state)) {
    1371+    if (txns.size() > 1 && !PackageMempoolChecks(args, txns, workspaces, package_state)) {
    


    instagibbs commented at 4:40 pm on November 2, 2022:
    (just noting to remember) This lets through un-paid-for RBFs for single tx packages, as well as bypasses conflict set creation, leading to internally-inconsistent mempool

    glozow commented at 5:20 pm on November 2, 2022:
    Yes, thank you :hugs:. Dumb size > 1 rule.
  108. glozow force-pushed on Nov 3, 2022
  109. in src/validation.cpp:1138 in 78333c1cde outdated
    1127+    // transaction that conflicts with A) because the directly conflicting transaction A, has a low
    1128+    // individual feerate.
    1129+    // As such, it's also important to ensure that we don't apply CheckMinerScores() to individual
    1130+    // transactions that were submitted as a package (e.g. if it's the only transaction left after
    1131+    // deduplication in AcceptPackage()).
    1132+    Assume(txns.size() > 1);
    


    instagibbs commented at 5:39 pm on November 3, 2022:
    errr, the same code path hits here :)

    glozow commented at 6:01 pm on November 3, 2022:
    Oops I lost a commit on the floor, thanks. I think this assumption should hold true now? AcceptPackage should now be calling AcceptSingleTransaction, not AcceptMultipleTransactions when there’s only 1 tx left. Which means there should always be more than 1 tx when we get here.
  110. glozow force-pushed on Nov 3, 2022
  111. in src/validation.cpp:1542 in ab5e507712 outdated
    1569@@ -1379,6 +1570,7 @@ PackageMempoolAcceptResult MemPoolAccept::AcceptPackage(const Package& package,
    1570                 // future.  Continue individually validating the rest of the transactions, because
    1571                 // some of them may still be valid.
    1572                 quit_early = true;
    1573+                package_state.Invalid(PackageValidationResult::PCKG_TX, "transaction failed");
    


    instagibbs commented at 6:57 pm on November 3, 2022:

    no results are being pushed here, which seems to mean we’ll hit NONFATAL_UNREACHABLE in the switch statement in submitpackage RPC with a tx that doesn’t hit the above two conditions.

    e.g., a non-standard script


    instagibbs commented at 7:05 pm on November 3, 2022:
    0                results.emplace(wtxid, single_res);
    1                package_state.Invalid(PackageValidationResult::PCKG_TX, "transaction failed");
    

    glozow commented at 2:54 pm on November 4, 2022:
    Yes good point, I think this is correct.

    instagibbs commented at 2:35 pm on November 29, 2022:
    (this has been addressed)
  112. in src/policy/rbf.cpp:185 in ab5e507712 outdated
    180@@ -181,3 +181,43 @@ std::optional<std::string> PaysForRBF(CAmount original_fees,
    181     }
    182     return std::nullopt;
    183 }
    184+
    185+std::optional<std::string> CheckMinerScores(CAmount replacement_fees,
    


    instagibbs commented at 5:19 pm on November 4, 2022:
    some related discussion on estimating mining score of the replacement: #26451 (comment)

    instagibbs commented at 5:07 pm on November 29, 2022:

    This incentive deficiency linked above should be a non-issue for CheckMinerScores since V3 package RBF is a restricted topology which disallows an multiple ancestors entirely.

    The two cases of “adding a new unconfirmed input” during RBF with this PR:

    1. Package size of two(no dedup), new CPFP child is RBF’ing another (set of) tx, goes through CheckMinerScores
    2. Package size of 1(deduped parent), transaction is rejected via single-tx-acceptance rules currently in master. Perhaps a constrained V3-only version of #26451 could remove the pin?

    The asymmetry above is annoying and a mild pinning vector.

    As of this PR, wallet authors can avoid this pinning scenario entirely by not moving fee utxos between different unconfirmed parents, but it would be nice if asymmetry was avoided to make wallet reasoning simpler.


    glozow commented at 4:45 pm on December 2, 2022:
    Do I understand correctly that you are pointing out that package RBF and single tx RBF have different rules? And suggesting we try to make them more symmetrical by applying something similar in the single tx rules?

    instagibbs commented at 4:49 pm on December 2, 2022:
    Ideally yes. Doesn’t have to be done now, it just reduces the utility of new unconfirmed inputs to randomly working sometimes, basically.
  113. glozow force-pushed on Nov 4, 2022
  114. JeremyRubin commented at 6:26 pm on November 4, 2022: contributor

    Adding a note here:

    if we’re doing nVersion 3, we should also fix the lack of discouragement for uninterpreted nSequence values and arguments to CSV/CLTV. see https://github.com/bitcoin/bitcoin/pull/22871

  115. glozow marked this as a draft on Nov 17, 2022
  116. glozow commented at 5:07 pm on November 18, 2022: member

    This is mentioned in a few other places, but Rule 3 pinning via additional ancestor is still not resolved with the multi-parent-1-child case. Quick diagram to illustrate - this was pointed out to me by @instagibbs.

    image

    Working on further restricting the rules to 1-parent-1-child since that eliminates the possibility of a conflict’s high fees diluted by ancestors. Will add more tests as well, and then will take out of draft when finished.

    if we’re doing nVersion 3, we should also fix the lack of discouragement for uninterpreted nSequence values and arguments to CSV/CLTV. see #22871

    Will consider adding this if people feel strongly. Wondering if others on this thread have thoughts on it?

  117. bitcoin deleted a comment on Nov 21, 2022
  118. DrahtBot added the label Needs rebase on Nov 28, 2022
  119. glozow force-pushed on Nov 28, 2022
  120. DrahtBot removed the label Needs rebase on Nov 28, 2022
  121. in doc/policy/packages.md:154 in a984c34c72 outdated
    149+*Rationale*: The fee-related rules are economically rational for ancestor packages, but not
    150+necessarily other types of packages. A child-with-parents package is a type of ancestor package. It
    151+may be fine to allow any ancestor package, but it's more difficult to account for all of the
    152+possibilities, e.g. descendant limits.
    153+
    154+2. All original transactions signal replaceability, i.e., either through BIP125 or through being V3.
    


    ariard commented at 0:17 am on November 29, 2022:
    In the light of the discrepancy between BIP125 and our mempool code, I think this should precise if we mean explicit signaling only or explicit signaling+inherited signaling.

    glozow commented at 4:30 pm on December 2, 2022:
    Added that they must signal explicitly
  122. in doc/policy/packages.md:142 in a984c34c72 outdated
    137+feerate](#Package-Fees-and-Feerate) may allow these transactions to be replaced.
    138+
    139+A package may replace mempool transaction(s) ("Package RBF") under the following conditions:
    140+
    141+1. All package transactions with mempool conflicts must be V3. This also means the "sponsoring"
    142+   child transaction must be V3.
    


    ariard commented at 0:23 am on November 29, 2022:

    This should precise if the mempool conflicts are required to be V3 themselves. From my memory of the mailing list discussions and elsewhere, I think the current state of thinking, directly conflicting transactions versions number are not considered.

    For the Lightning use-case, this is a significant security advantage as the ~80k of public channels, with all pre-signed revoked commitment transactions V2 can be replaced with newer version=3 package. Revoked commitment transactions, even if the honest counterparty is in knowledge of the revocation secret, are still consensus and policy valid transactions, and as such can be leveraged as pinning means.

    On the other hand, I think allowing V3 to replace V2 chain of transactions lead to the consequence than V2 users are implicitly subscribing to this new replacement regime. A receiver of a V2 transaction could apply the BIP125 rules to estimate the replacement odd of the transaction, and the application logic be broken when the actual rule 3 of minimum between package feerate and ancestor feerate is applied. I don’t know if we’re significantly interfering with any Bitcoin application here.


    glozow commented at 10:27 am on November 29, 2022:

    I think allowing V3 to replace V2 chain of transactions lead to the consequence than V2 users are implicitly subscribing to this new replacement regime. A receiver of a V2 transaction could apply the BIP125 rules to estimate the replacement odd of the transaction, and the application logic be broken when the actual rule 3 of minimum between package feerate and ancestor feerate is applied.

    I’ll try to be more vocal that V2 transactions can be replaced by V3 transactions. The cost of replacing a V2 transaction being replaced is identical (or higher, given the ancestor feerate rule) - the fee must be paid, but it can be by 2 transactions instead of 1. So, while there is a “new” way to replace, the “likelihood” of a V2 transaction being replaced is the same or lower.

  123. in doc/policy/version3_transactions.md:32 in a984c34c72 outdated
    28+
    29+2. Adding a high-fee descendant of B that also spends from a large, low-feerate mempool transaction,
    30+   C. The child may pay a very large fee but not actually be fee-bumping B if its overall ancestor
    31+feerate is still lower than B's individual feerate. For example, assuming the default ancestor size
    32+limit is 101KvB, B is 1000vB paying 2sat/vB, and C is 99KvB paying 1sat/vB, adding a 1000vB child of
    33+B and C increases the cost to replace B by 101Ksat.
    


    ariard commented at 0:39 am on November 29, 2022:

    If we would like to add more rational to the design of the rules, we could explain why a scorched earth approach would solve the Lightning case really imperfectly (discussed recently here: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-November/021176.html). Indeed, Lightning implementation could just blindly replace-by-fee the pinning transactions, paying up to the channel value. This wouldn’t work at all against an attacker pinning with one single malicious transaction multiple channel from different parties where the malicious transaction fee is above the value of each channel. E.g channel A 10000 sats, channel B 25000 sats and channel C 8000 sats. If the malicious pinning transactions fee equals 26000 sats, none of the honest party will rationally fee-bump more than their own channel value.

    Feerate-only replacement rules scales up to the maximum package size and worst historical mempool reduce consideraly the fee-bumping reserves requirement for a LN node, which is a notable advantage I think.

  124. in doc/policy/version3_transactions.md:97 in a984c34c72 outdated
    92+*Rationale*: This makes the rule very easily "tacked on" to existing logic for policy and wallets.
    93+A transaction may be up to 100KvB on its own (`MAX_STANDARD_TX_WEIGHT`) and 101KvB with descendants
    94+(`DEFAULT_DESCENDANT_SIZE_LIMIT_KVB`). If an existing V3 transaction in the mempool is 100KvB, its
    95+descendant can only be 1000vB, even if the policy is 10KvB.
    96+
    97+4. A V3 transaction cannot have more than 1 unconfirmed ancestor.
    


    ariard commented at 0:57 am on November 29, 2022:
    I think it could be precised if those rules are still applied in case of reorg and a V2 ancestor transaction comes back in the mempool, if the V3 child package should be evicted or conserved. In the case it’s evicted, you have to consider LN V3 package being evicted and as a LN node you should resuscitate the V2, fee-bump it again, wait for confirmation and then re-broadcast your V3 package.

    ariard commented at 1:30 am on November 29, 2022:
    Also, I think this could be a minor issue for LN implementation. If some of your counterparties have not upgraded their software to support V3 transactions you might have a subset of your channel transaction still under V2 during the transition period. If you have a common child for batched fee-bumping shared between a V2 commitment transaction and V3 commitment transaction, the whole operation is likely to fail. Of course, batch fee-bumping should be considered harmful for time-sensitive commitment transactions (i.e ones with HTLC outputs) though we might have further silent issues even when it’s not time-sensitive. In my opinion, this is a potential issue LN devs should be aware of, processing between V2 and V3 channels should be well isolated.

    glozow commented at 10:23 am on November 29, 2022:
    IIUC an LN channel would either only use V2 or only use V3. If the channel is using V3 commitment transactions, there shouldn’t be old state in V2 transactions. So there shouldn’t really be cases where V3 needs to replace V2 or vice versa. Does that sound correct?

    glozow commented at 10:24 am on November 29, 2022:
    Hopefully the LN wallet knows that it cannot bump a V2 and V3 together, since an unconfirmed V3 cannot spend an unconfirmed V2 and vice versa. I actually think this rule helps prevent risks associated with bumping multiple states, since batched bumping is not possible at all.

    instagibbs commented at 2:31 pm on November 29, 2022:

    It would be evicted as any package-limit-breaking transaction.

    In the case it’s evicted, you have to consider LN V3 package being evicted and as a LN node you should resuscitate the V2, fee-bump it again, wait for confirmation and then re-broadcast your V3 package.

    I’m not sure application-level detail like this is helpful in the document unless it’s a high-level principle.


    ajtowns commented at 5:20 pm on December 22, 2022:
    Should be number “6.” not a second “4.”
  125. in doc/policy/version3_transactions.md:40 in a984c34c72 outdated
    35+## Version 3 Rules
    36+
    37+All existing standardness rules apply to V3. The following set of additional restrictions apply to
    38+V3 transactions:
    39+
    40+1. A V3 transaction can be replaced, even if it does not signal BIP125 replaceability. Other
    


    ariard commented at 1:23 am on November 29, 2022:
    I think this implies that any 0conf acceptance softwares should be upgraded to tread V3 transactions by default as replaecable. I would say it could be valuable to notify the operators of such services.

    glozow commented at 10:22 am on November 29, 2022:
    Of course, will do my best to tell people that V3 should be treated similarly to BIP125 in software/services that use replaceability signaling.

    naumenkogs commented at 1:08 pm on January 6, 2023:
    Should LN users be aware of this as well? Imagine them forwarding channel closure directly towards a service claiming to work 0-conf, but for some reason (closure being v3) the experience is not really 0-conf. (Perhaps this is already the case with rbf-by-default? Indeed more like a UX challenge.)
  126. ariard commented at 1:50 am on November 29, 2022: member

    if we’re doing nVersion 3, we should also fix the lack of discouragement for uninterpreted nSequence values and arguments to CSV/CLTV. see #22871

    IIRC with #22871, the main standing concern was the risk of breaking applications actually using nSequence undefined value for any semantic. I don’t think this a concern anymore if we restrain only for nVersion=3 as those transactions are not standard today and no one should use them.

  127. in doc/policy/packages.md:54 in a984c34c72 outdated
    48-The following rules are only enforced for packages to be submitted to the mempool (not enforced for
    49-test accepts):
    50+* [CPFP Carve Out](./mempool-limits.md#CPFP-Carve-Out) is disabled. (#21800)
    51+
    52+   - *Rationale*: This carve out cannot be accurately applied when there are multiple transactions'
    53+     ancestors and descendants being considered at the same time.
    


    instagibbs commented at 2:17 pm on November 29, 2022:
    0     ancestors and descendants being considered at the same time. Single-transaction
    1     submission behavior is unchanged.
    

    glozow commented at 4:29 pm on December 2, 2022:
    Done, thanks
  128. in doc/policy/packages.md:51 in a984c34c72 outdated
    45      heavily connected, i.e. some transaction in the package is the ancestor or descendant of all
    46      the other transactions.
    47 
    48-The following rules are only enforced for packages to be submitted to the mempool (not enforced for
    49-test accepts):
    50+* [CPFP Carve Out](./mempool-limits.md#CPFP-Carve-Out) is disabled. (#21800)
    


    instagibbs commented at 2:17 pm on November 29, 2022:
    0* [CPFP Carve Out](./mempool-limits.md#CPFP-Carve-Out) is disabled in the package context. (#21800)
    

    glozow commented at 4:29 pm on December 2, 2022:
    Done, thanks
  129. in doc/policy/version3_transactions.md:41 in a984c34c72 outdated
    36+
    37+All existing standardness rules apply to V3. The following set of additional restrictions apply to
    38+V3 transactions:
    39+
    40+1. A V3 transaction can be replaced, even if it does not signal BIP125 replaceability. Other
    41+   conditions rules apply, see [RBF rules](./mempool-replacements.md) and [Package RBF
    


    instagibbs commented at 2:25 pm on November 29, 2022:
    0   conditions apply, see [RBF rules](./mempool-replacements.md) and [Package RBF
    

    glozow commented at 4:29 pm on December 2, 2022:
    Done, thanks
  130. in src/policy/v3_policy.h:24 in 000facb046 outdated
    19+/** Maximum virtual size of a tx which spends from an unconfirmed V3 transaction, in vB. */
    20+static constexpr unsigned int V3_CHILD_MAX_SIZE{1000};
    21+/** Maximum number of transactions including an unconfirmed tx and its descendants. */
    22+static constexpr unsigned int V3_DESCENDANT_LIMIT{2};
    23+
    24+// Define additional values in case we want V3 ancestor limits to diverge from default ancestor limits.
    


    instagibbs commented at 2:45 pm on November 29, 2022:
    this comment I think only applies to V3_ANCESTOR_SIZE_LIMIT_KVB?

    glozow commented at 5:21 pm on November 29, 2022:
    true, thanks
  131. in src/policy/v3_policy.cpp:23 in 000facb046 outdated
    18+    // If all transactions are V3, we can stop here.
    19+    if (std::all_of(package.cbegin(), package.cend(), [](const auto& tx){return tx->nVersion == 3;})) {
    20+        return std::nullopt;
    21+    }
    22+    // If all transactions are non-V3, we can stop here.
    23+    if (std::all_of(package.cbegin(), package.cend(), [](const auto& tx){return tx->nVersion != 3;})) {
    


    instagibbs commented at 2:52 pm on November 29, 2022:

    After this check, we know it’s a mixture(?) and therefore illicit, making hitting https://github.com/bitcoin/bitcoin/pull/25038/commits/000facb0460372cc320158189090e4d36862eb0e#diff-a19c07329799a164e1e3bd89e7cd07f1a8e5e97b88f2b24c9bef17df04479c74R51 impossible and the below loops a only useful for returning an example illicit tuple.

    Perhaps just grab one v3 example and one non-v3 example and return those to simplify the rest of the code?


    glozow commented at 5:20 pm on November 29, 2022:
    Yeah, the loops/maps are only used to return a more helpful string to the user. I don’t think it’s too much work? This function accepts a list of transactions that doesn’t necessarily need to be an ancestor packages, so just grabbing a V3 and non-V3 could be 2 transactions that aren’t parent-child.
  132. in src/policy/v3_policy.cpp:87 in 000facb046 outdated
    82+    if (ancestor_vsize + tx_vsize > V3_ANCESTOR_SIZE_LIMIT_KVB * 1000) {
    83+        return strprintf("total vsize of tx with ancestors would be too big: %u virtual bytes", tx_vsize + ancestor_vsize);
    84+    }
    85+
    86+    // Any two unconfirmed transactions with a dependency relationship must either both be V3 or both non-V3.
    87+    if (auto err_string{CheckV3Inheritance(ptx, ancestors)}) {
    



    glozow commented at 4:35 pm on December 2, 2022:
    Simplified ApplyV3Rules since it’s 1-parent-1-child and removed the V3-specific ancestor size limit which is now redundant.
  133. in src/validation.cpp:903 in a984c34c72 outdated
    898+        // This carve out is NOT granted in package RBF (see m_allow_carveouts in ATMPArgs ctors) because, during
    899+        // package acceptance, we may call PreChecks for multiple transactions that conflict with different mempool
    900+        // entries that don't share ancestry. It would be a bug to keep increasing the descendant limit each time.
    901+        // The carve out also does not apply to package RBF while it is restricted to V3 transactions, since an
    902+        // unconfirmed V3 transaction cannot have more than one descendant.
    903+        if (args.m_package_feerates && args.m_package_submission) {
    


    instagibbs commented at 3:39 pm on November 29, 2022:

    m_package_feerates currently is only true when m_allow_carveouts is false, no? The Assume should be dead code.

    this whole block is extremely verbose and I’m not sure what it’s supposed to be doing


    glozow commented at 5:45 pm on November 29, 2022:
    The hope is to explain + maybe prevent a potential bug if an assumption (package RBF is v3-only) is broken. Ifsomebody in the future says “it seems fine to relax package RBF to non-v3,” or does a “refactor no behavior change” that actually does change something, the currently-dead code would not be dead anymore and the Assume would hit. Don’t think it’s completely useless, though I’m very happy to make it less verbose and/or move it.

    glozow commented at 4:29 pm on December 2, 2022:
    Ok I’ve reduced the comment significantly and turned this into a non-dead Assume().
  134. in src/validation.cpp:1079 in a984c34c72 outdated
    1075+    // Further checks are all RBF-only.
    1076+    m_rbf = std::any_of(workspaces.cbegin(), workspaces.cend(), [](const auto& ws){return !ws.m_conflicts.empty();});
    1077+    if (!m_rbf) return true;
    1078+
    1079+    // Unless the transaction is V3, its own fees must meet the requirements for replacing its conflicts.
    1080+    if (args.m_package_feerates) {
    


    instagibbs commented at 4:10 pm on November 29, 2022:
    what is this switch doing?

    glozow commented at 4:26 pm on December 2, 2022:
    Yeah removed
  135. glozow force-pushed on Dec 2, 2022
  136. glozow force-pushed on Dec 2, 2022
  137. DrahtBot added the label Needs rebase on Dec 5, 2022
  138. glozow commented at 10:46 am on December 6, 2022: member
    This PR is open for review but leaving as draft, attempting to shave off the beginning commits in #26646 to make this smaller and v3-focused.
  139. in doc/policy/version3_transactions.md:75 in 0c089a327a outdated
    57+co-parent C (i.e. B spends from A and C). C also has another child, D. B is one of the original
    58+transactions and thus its ancestor feerate must be lower than the package's. However, this may be an
    59+underestimation because D can bump C without B's help. This is resolved if V3 transactions can only
    60+have V3 ancestors, as then C cannot have another child.
    61+
    62+4. A V3 transaction cannot have more than 1 unconfirmed descendant.
    


    pinheadmz commented at 9:49 pm on December 7, 2022:

    Question about this and sorry if it’s already been addressed:

    Does this require extra logic for reorg handling? I’m thinking of a chain of v3 TXs:

    tx1 (confirmed in block 100) -> tx2 (confirmed in block 101) -> tx3 (unconfirmed, currently in mempool)

    if block 101 is disconnected, that potentially puts tx2 back in the mempool meaning tx3 is now invalid and should be evicted.


    glozow commented at 10:11 am on December 8, 2022:
    Yes, v3 enforcement during reorg is implemented in MaybeUpdateMempoolForReorg here and tested in mempool_accept_v3.py here. Would it make sense for me to add a note in the docs?

    pinheadmz commented at 3:14 pm on December 8, 2022:
    Sweet 👍 Couldn’t hurt to mention that in a bullet point here I suppose.
  140. in doc/policy/version3_transactions.md:24 in 7acf63771a outdated
    20+### "Rule 3" Pinning
    21+
    22+RBF rules require the replacement transaction pay a higher absolute fee than the aggregate fees paid
    23+by all original transactions. This means Mallory may increase the fees required to replace B by:
    24+
    25+1. Adding transaction(s) that descend from B and pay a feerate too low to fee-bump B through CPFP.
    


    naumenkogs commented at 12:59 pm on January 6, 2023:

    too low to fee-bump B through CPFP

    I don’t get this sentence. Nobody wants to fee-bump B, no? Alice would want to replace B, what does it have to do with fee-bumping?


    glozow commented at 3:05 pm on January 11, 2023:
    Mallory’s goal is to pin the transaction in the mempool. That is, increase the fees required to replace B, but not make it confirm. Mallory can of course put 1BTC of fees in the descendant, thus making it extremely expensive to replace B, but then she’s also CPFP’d B and it’s probably going to be mined very soon.
  141. in doc/policy/version3_transactions.md:63 in 7acf63771a outdated
    48+signals replaceability this way does not require mempool traversal, and does not change based on
    49+what transactions are mined.
    50+
    51+*Note*: A V3 transaction can spend outputs from *confirmed* non-V3 transactions.
    52+
    53+3. A V3 transaction's unconfirmed ancestors must all be V3.
    


    naumenkogs commented at 1:41 pm on January 6, 2023:

    Do you ever discuss what will happen in the 2-block-reorg case? A chain of txs could end up being not-re-added to the mempool (due to rules 2/3), and then something becomes insecure.

    I think I should bring a concrete example here.


    naumenkogs commented at 1:42 pm on January 6, 2023:
    Ahhh too bad, the next comment asks the same question. Certainly worth documenting though :)

    glozow commented at 3:07 pm on January 11, 2023:
    I think a concrete example would help, yeah. For example, when you say “something becomes insecure” it’s not really clear to me what you mean - a v3 transaction gets censored? A mempool has a chain of more than 2 v3 transactions?
  142. glozow referenced this in commit 26002570ab on Jan 11, 2023
  143. glozow force-pushed on Jan 11, 2023
  144. glozow commented at 3:02 pm on January 11, 2023: member

    Last push is just a rebase. Still need to do:

    • Not allowing non-v3 below minrelayfee in package CPFP. Currently working on test util to raise mempool min feerate for this.
    • Removing below-minrelayfee-and-no-longer-bumped transactions after replacement.
  145. instagibbs commented at 3:08 pm on January 11, 2023: member

    Not allowing non-v3 below minrelayfee in package CPFP. Currently working on test util to raise mempool min feerate for this.

    Implication being a v1/2 transaction cannot be 0-fee and bumped by child?

  146. glozow commented at 3:11 pm on January 11, 2023: member

    Implication being a v1/2 transaction cannot be 0-fee and bumped by child?

    Yes unfortunately, unless there is a better alternative… see ~08:16 in https://gnusha.org/bitcoin-core-dev/2022-12-16.log

  147. sidhujag referenced this in commit f5d8392ffe on Jan 11, 2023
  148. DrahtBot removed the label Needs rebase on Jan 11, 2023
  149. glozow force-pushed on Jan 17, 2023
  150. glozow force-pushed on Jan 17, 2023
  151. glozow force-pushed on Jan 17, 2023
  152. glozow force-pushed on Jan 19, 2023
  153. ariard commented at 2:33 am on January 27, 2023: member

    Lightning legacy channel types (before BOLT9’s option_anchors_output / option_anchors_zero_fee_htlc_tx) presents the issue of counterparty’s second-stage HTLC transactons malleable enough to opt-out from BIP125 replacement rules (the preimage path on the local commitment transaction offered output and the timeout path on the local commitment transaction accepted output). This path can be leveraged by a counterarty not only to do opt-out pinning but also for rule #3 style of pinning, where the second-stage HTLC transaction is malleated to a high-fee/low-feerate.

    This repleaceability issue is moving away with option_anchors_zero_fee_htlc_tx, where all the second-stage spends must signal replacement due to the new CSV 1 to satisfy. This anchor states are currently signed under nversion=2, assuming we have dynamic upgrades, a new channel type with states committing to nversion=3 could be deployed in-flight (i.e without confirming on-chain new funding outputs). Those channels would be safe against rule #3 style of pinning, as if old RBF opt-in nversion=2 states are used for pinning, under the proposed policy rules (“All directly conflicting transactions signal replaceability explicitly, either through BIP125 or V3.”, the incentive-compatible enhanced RBF rules should apply.

    As of today, option_anchors_zero_fee_htlc_tx is enabled by default by Eclair (since v0.7.0) and LND (since v.0.12.0). This is not deployed by default by CLN and LDK. So we would still have probably tens of thousands of legacy channels forever exposed to pinning attacks mitigated by nversion=3. For this subset, this is a wonder if we would like to introduce some “retro-active” policy effect, where nversion=3 effects are granted to the legacy channel types with some at-broadcast opt-in mechanism. E.g a signature from a pubkey of the funding output committing to a package version bit. Of course, this would be more technical complexity on the Core-side, at the security and economic benefit of Lightning channels, so trade-off.

    Lightning node operators of legacy channel types could be also just force-close all channels, and re-open new safer ones. Assuming it’s done in an asynchronous fashion (to avoid accidental flood&loot situations) and mempool fees, this should be okay. On the other hand, I can see the operators incentives to keep unsafe channel open to preserve good reputation accumulated in routing algorithms.

    Personally, I think the deployment cost are low enough on the Lightning-side to not bother with dedicated “retro-active” policy mechanism on the Core-side, at least in this case. Though this is only my own appreciation of the deployment cost and it’s highly probable we’ll have newer policy to deploy in the future and interactions with channel types, so I think it can valuable to think about it.

  154. ajtowns commented at 6:38 am on January 27, 2023: contributor

    For this subset, this is a wonder if we would like to introduce some “retro-active” policy effect

    The alternative approach would be for early lightning adopters to update their channel on-chain to permanently invalidate any old states that might enable pinning (eg, by splicing some funds in or out)… Simple is better than complicated.

  155. ariard commented at 7:14 pm on January 27, 2023: member

    The alternative approach would be for early lightning adopters to update their channel on-chain to permanently invalidate any old states that might enable pinning (eg, by splicing some funds in or out)… Simple is better than complicated.

    On this present case, this is my thinking too (“I think the deployment cost are low enough on the Lightning-side to not bother with dedicated “retro-active” policy mechanism on the Core-side”). However, deployment workaround for new channel types have already been proposed in the past, and to avoid hard synchronisation issue (i.e provoking mempool backlogs) we might have to rely on “retro-active” police deployment in the future. That said, we probably still have one or two orders of magnitude of the number of Lightning channels growth before this starts to be a concern.

  156. in src/policy/v3_policy.h:51 in 44fcc4d391 outdated
    46+ * 1. Tx with all of its ancestors (including non-nVersion=3) must be within V3_ANCESTOR_SIZE_LIMIT_KVB.
    47+ * 2. Tx with all of its ancestors must be within V3_ANCESTOR_LIMIT.
    48+ *
    49+ * If a V3 tx has V3 ancestors,
    50+ * 1. Each V3 ancestor and its descendants must be within V3_DESCENDANT_LIMIT.
    51+ * 2. The tx must be within V3_CHILD_MAX_SIZE.
    


    theStack commented at 0:27 am on March 1, 2023:
    0 * 2. The tx must be within V3_CHILD_MAX_WEIGHT
    
  157. in src/test/txvalidation_tests.cpp:179 in 44fcc4d391 outdated
    175+        BOOST_CHECK(GetTransactionWeight(*tx_v3_child_big) > V3_CHILD_MAX_WEIGHT);
    176+        auto ancestors{pool.CalculateMemPoolAncestors(entry.FromTx(tx_v3_child_big), m_limits)};
    177+        BOOST_CHECK(ApplyV3Rules(tx_v3_child_big, *ancestors, empty_conflicts_set).has_value());
    178+    }
    179+
    180+    // Parent + child with v3 in the mempool. Child is allowed as long as it is under V3_CHILD_MAX_SIZE.
    


    theStack commented at 0:28 am on March 1, 2023:
    0    // Parent + child with v3 in the mempool. Child is allowed as long as it is under V3_CHILD_MAX_WEIGHT.
    
  158. in test/functional/mempool_package_rbf.py:128 in c27864fbca outdated
    123+        node.sendrawtransaction(tx_v3_no_bip125["hex"])
    124+        self.assert_mempool_contents(expected=[tx_v3_no_bip125["tx"]])
    125+
    126+        self.log.info("Test that non-V3 transactions signaling BIP125 are replaceable")
    127+        coin = self.coins[0]
    128+        del self.coins[0]
    


    theStack commented at 0:34 am on March 1, 2023:
    0        coin = self.coins.pop(0)
    
  159. in test/functional/mempool_package_rbf.py:87 in c27864fbca outdated
    82+        self.wallet = MiniWallet(node)
    83+        self.generate(self.wallet, 160)
    84+        self.coins = self.wallet.get_utxos(mark_as_spent=False)
    85+        # Mature coinbase transactions
    86+        self.generate(self.wallet, 100)
    87+        self.address = self.wallet.get_address()
    


    theStack commented at 0:37 am on March 1, 2023:

    Seems like this is not used anywhere:

  160. in test/functional/mempool_package_rbf.py:214 in c27864fbca outdated
    209+        # The RBFs should otherwise work.
    210+        submitres2 = node.submitpackage(package_hex2)
    211+        submitres2["replaced-transactions"] == [tx.rehash() for tx in package_txns1]
    212+        self.assert_mempool_contents(expected=package_txns2, unexpected=package_txns1)
    213+        submitres3 = node.submitpackage(package_hex3)
    214+        submitres3["replaced-transactions"] == [tx.rehash() for tx in package_txns2]
    


    theStack commented at 0:51 am on March 1, 2023:

    Missing assertions for the replaced-tx checks:

    0        assert submitres2["replaced-transactions"] == [tx.rehash() for tx in package_txns1]
    1        self.assert_mempool_contents(expected=package_txns2, unexpected=package_txns1)
    2        submitres3 = node.submitpackage(package_hex3)
    3        assert submitres3["replaced-transactions"] == [tx.rehash() for tx in package_txns2]
    
  161. in src/validation.cpp:1127 in 27881f0d82 outdated
    1069@@ -1066,6 +1070,20 @@ bool MemPoolAccept::PackageMempoolChecks(const ATMPArgs& args,
    1070                                      "package RBF failed: package conflicts with dependency", *err_string);
    1071     }
    1072 
    1073+    // CheckMinerScores is very conservative and should not be used for individual transactions.
    1074+    // For example, the mempool contains a large, low-feerate transaction A (99,000vB, 1sat/vB feerate) is
    1075+    // Transaction A has a small, high-feerate child B (1,000vB, 101sat/vB). The user wants to
    


    theStack commented at 0:56 am on March 1, 2023:
    0    // For example, the mempool contains a large, low-feerate transaction A (99,000vB, 1sat/vB feerate).
    1    // Transaction A has a small, high-feerate child B (1,000vB, 101sat/vB). The user wants to
    

    or maybe

    0    // For example, the mempool contains a large, low-feerate transaction A (99,000vB, 1sat/vB feerate)
    1    // which has a small, high-feerate child B (1,000vB, 101sat/vB). The user wants to
    
  162. theStack commented at 1:16 am on March 1, 2023: contributor

    Concept ACK

    Left some nit comments below, mostly docs and functional test related stuff.

  163. DrahtBot added the label Needs rebase on Mar 20, 2023
  164. glozow force-pushed on Apr 17, 2023
  165. DrahtBot removed the label Needs rebase on Apr 17, 2023
  166. in test/functional/mempool_package_rbf.py:86 in c1bd1e1f8e outdated
    81+        node = self.nodes[0]
    82+        self.wallet = MiniWallet(node)
    83+        self.generate(self.wallet, 160)
    84+        self.coins = self.wallet.get_utxos(mark_as_spent=False)
    85+        # Mature coinbase transactions
    86+        self.generate(self.wallet, 100)
    


    instagibbs commented at 3:05 pm on May 1, 2023:
    I don’t think this is necessary. get_utxos already filters for mature coinbases, and you’re making 60 of those above.
  167. DrahtBot added the label Needs rebase on May 9, 2023
  168. glozow force-pushed on Aug 7, 2023
  169. DrahtBot removed the label Needs rebase on Aug 7, 2023
  170. glozow commented at 11:56 am on August 7, 2023: member
    Rebased
  171. instagibbs commented at 8:50 pm on August 8, 2023: member

    Looks like this lost TrimToSize changes?

    0commit ed6045a8f99f14987977e13faa1865c3bf56890f
    1Author: glozow <gloriajzhao@gmail.com>
    2Date:   Tue Jan 17 13:43:27 2023 +0000
    3
    4    [mempool] evict everything below min relay fee in TrimToSize()
    5    
    6    At this point it's not expected that there are any such transactions,
    7    except from reorgs and possibly when loading from mempool.dat.
    

    I rely on this for ephemeral anchors, trying to rebase.

  172. in test/functional/mempool_accept_v3.py:220 in ba6cc7516e outdated
    211@@ -212,6 +212,23 @@ def test_nondefault_package_limits(self):
    212                 tx_v3_child_large2["hex"])
    213         self.check_mempool([tx_v3_parent_large2["txid"]])
    214 
    215+    @cleanup
    216+    def test_fee_dependency_replacements(self):
    217+        """
    218+        Since v3 introduces the possibility of 0-fee (i.e. below min relay feerate) transactions in
    219+        the mempool, it's possible for these transactions' sponsors to disappear due to RBF.
    220+        Document that the 0-fee transaction should be evicted along with the replacements.
    


    instagibbs commented at 8:56 pm on August 8, 2023:

    I don’t think this case is actually covered

    rebased ephemeral anchor branch is failing on this test: https://github.com/bitcoin/bitcoin/pull/26403/files#diff-d18bbdec91d0f4825b512a31f34666b000c7b7e2e05a3d43570c4b971532616fR409


    glozow commented at 8:55 am on August 9, 2023:
    Thanks for catching, should be fixed now
  173. glozow force-pushed on Aug 9, 2023
  174. glozow commented at 11:21 am on August 9, 2023: member
    Fixed accidental drop of TrimToSize commit
  175. glozow force-pushed on Aug 10, 2023
  176. glozow commented at 2:07 pm on August 10, 2023: member
    Rebased on top of #28251 for the issue posted in #26403
  177. DrahtBot added the label Needs rebase on Aug 31, 2023
  178. ariard commented at 0:08 am on September 9, 2023: member

    Looked on the proposed doc/policy/version3_transactions.md changes with other Lightning eyes on a whiteboard, and as far as I can tell it works well for Lightning usual flow and also upcoming dual-funding / splicing: https://github.com/lightning/bolts/pull/863 (to be checked again when the specs are finalized)

    The replacement rule for signed version=3 transactions is the following as of 06e41874, it can replaces nversion=2 signaling replaceability or opt-out transactions in a node with mempoolfullrbf=1. It cannot replace non-signaling nversion=2 (L785 in src/validation.cpp). In term of types of Lightning channels:

    Once nversion=3 is deployed, the new nversion=3 should be able to to replace all the anchor-outputs channels transactions without rejection. I think old legacy channels Lightning node operators would be advised to upgrade to anchor-outputs channels as soon as it’s ready in their Lightning implementations, and they’ve practical understanding of fee-bumping reserves management and swallow the bullet in term of on-chain fees. In the current state of things, I don’t think there is a need to introduce unilateral retroactive policy rules effect as suggested above, though this can be a welcome mechanism in the far future to massively save on on-chain fees for the Lightning ecosystem.

    I think missing test coverage for v3 transactions not replacing not-bip125-signaling v2 transaction

     0commit aec6ea3a378d532cd86f8812006e4eb42e185662
     1Author: Antoine Riard <dev@ariard.me>
     2Date:   Thu Sep 7 04:27:49 2023 +0100
     3
     4    Add non-v3 trasnsactions not signaling not replaceable test
     5
     6diff --git a/test/functional/mempool_package_rbf.py b/test/functional/mempool_package_rbf.py
     7index e969b0249b..dd1c55be05 100755
     8--- a/test/functional/mempool_package_rbf.py
     9+++ b/test/functional/mempool_package_rbf.py
    10@@ -143,6 +143,24 @@ class PackageRBFTest(BitcoinTestFramework):
    11         self.assert_mempool_contents(expected=package_txns_v3, unexpected=[tx_bip125_v2["tx"]])
    12         self.generate(node, 1)
    13 
    14+        self.log.info("Test that non-V3 transactions not signaling BIP125 are not replaceable")
    15+        coin = self.coins.pop()
    16+
    17+        tx_nobip125_v2 = self.wallet.create_self_transfer(
    18+            fee=DEFAULT_FEE,
    19+            utxo_to_spend=coin,
    20+            sequence=MAX_BIP125_RBF_SEQUENCE +1,
    21+            version=2
    22+        )
    23+        node.sendrawtransaction(tx_nobip125_v2["hex"])
    24+
    25+        self.assert_mempool_contents(expected=[tx_nobip125_v2["tx"]])
    26+        assert not node.getmempoolentry(tx_nobip125_v2["tx"].rehash())["bip125-replaceable"]
    27+        assert tx_bip125_v2["tx"].nVersion == 2
    28+        package_hex_v3, package_txns_v3 = self.create_simple_package(coin, parent_fee=0, child_fee=DEFAULT_FEE * 3, version=3)
    29+        assert all([tx.nVersion == 3 for tx in package_txns_v3])
    30+        assert_raises_rpc_error(-26, "txn-mempool-conflict", node.submitpackage, package_hex_v3)
    31+
    32     def test_package_rbf_additional_fees(self):
    33         self.log.info("Check Package RBF must increase the absolute fee")
    34         node = self.nodes[0]
    
  179. in src/policy/rbf.cpp:206 in 06e4187468 outdated
    201+        // high-feerate descendants or are themselves higher feerate than this package/transaction.
    202+        // For now, as a conservative estimate, use the minimum between the transaction's individual
    203+        // feerate and ancestor feerate.
    204+        const CFeeRate replacement_miner_score = std::min(replacement_individual_feerate, replacement_ancestor_feerate);
    205+        for (const auto& entry : direct_conflicts) {
    206+            const CFeeRate original_individual_feerate(entry->GetModifiedFee(), entry->GetTxSize());
    


    glozow commented at 11:23 am on September 20, 2023:

    Note to self from Greg

    For v3, where we can ask the exact miner score, use that instead of individual, otherwise absurd brutal fee when replacing child + switching sponsees. https://github.com/bitcoin/bitcoin/pull/26403/files#diff-d18bbdec91d0f4825b512a31f34666b000c7b7e2e05a3d43570c4b971532616fR415


    glozow commented at 3:52 pm on November 27, 2023:
    @instagibbs I’ve changed this to be min(individual, ancestor) when it’s v3
  180. glozow force-pushed on Oct 3, 2023
  181. DrahtBot removed the label Needs rebase on Oct 3, 2023
  182. DrahtBot added the label Needs rebase on Oct 5, 2023
  183. cacrowley approved
  184. cacrowley approved
  185. glozow force-pushed on Nov 10, 2023
  186. DrahtBot added the label CI failed on Nov 10, 2023
  187. DrahtBot removed the label Needs rebase on Nov 10, 2023
  188. glozow force-pushed on Nov 10, 2023
  189. glozow force-pushed on Nov 10, 2023
  190. [mempool] evict everything paying 0 fees in TrimToSize()
    At this point it's not expected that there are any such transactions,
    except from reorgs.
    3bca679849
  191. [policy] policy rules for nVersion=3 18406ecbd5
  192. glozow commented at 3:26 pm on November 27, 2023: member

    if we’re doing nVersion 3, we should also fix the lack of discouragement for uninterpreted nSequence values and arguments to CSV/CLTV. see #22871

    (cc @instagibbs) decided not to do this for a few different reasons:

    • It’s not intended for everyone to stop using v2 and switch to v3; this isn’t really a “standardness rules version bump” so making something nonstandard here doesn’t have the effect of discouraging it.
    • From speaking to a few folks, it doesn’t seem like there are any short- or long-term plans to upgrade using these fields. So the work would likely not pay off.
    • It seems the branch as it’s written may have the effect of making some LN transactions nonstandard.
  193. instagibbs commented at 3:34 pm on November 27, 2023: member
    @glozow I think at best you’d get ~7 bits reserved(without colliding with well-known protocols), and only for v3, which as you note isn’t a general version bump
  194. [policy] allow V3 transactions under certain conditions 1b2cd22c86
  195. [policy] allow individual v3 txns to be below min relay feerate
    As long as they are otherwise paid for, i.e. through package CPFP.
    If a v3 transaction loses its sponsor, we can evict them with no trouble
    because it will not have other descendants or ancestors to make the
    feerate assessment more complicated.
    c74c8262d7
  196. [test framework] parameterize version in create_self_transfer 3ad3ea5dc3
  197. [functional test] v3 transaction submission b828ea09ab
  198. [doc] v3 policy f82c9fdfce
  199. [doc] cpfp carveout is excluded in packages
    The behavior is not new, but this rule exits earlier than before.
    Previously, a carve out could have been granted in PreChecks() but then
    nullified in PackageMempoolChecks() when CheckPackageLimits() is called
    with the default limits.
    d6bd8dc8a6
  200. [refactor] make some members MemPoolAccept-wide
    No change in behavior.
    
    For single transaction acceptance, this is a simple refactor:
    Workspace::m_all_conflicting         -> MemPoolAccept::m_all_conflicts
    Workspace::m_replacement_transaction -> MemPoolAccept::m_rbf
    Workspace::m_conflicting_fees        -> MemPoolAccept::m_conflicting_fees
    Workspace::m_conflicting_size        -> MemPoolAccept::m_conflicting_size
    Workspace::m_replaced_transactions   -> MemPoolAccept::m_replaced_transactions
    
    And local variables m_total_vsize and m_total_modified_fees are now
    MemPoolAccept members so they can be accessed from PackageMempoolChecks.
    
    We want these to be package-wide variables because
    - Transactions could conflict with the same tx (just not the same
    prevout), or their conflicts could share descendants.
    - We want to compare conflicts with the package fee rather than
    individual transaction fee.
    75a1efd77e
  201. [policy] check whether replacement is more incentive compatible
    Avoid accepting replacements that are not more incentive compatible to
    mine.  For now, as a conservative estimate, require that the minimum
    between the transaction's individual feerate and ancestor feerate is
    higher than the individual feerates of directly conflicting transactions
    and the ancestor feerates of all original transactions.
    
    Note that a package/transaction's ancestor feerate is not perfectly
    representative of its incentive compatibility; it may overestimate (some
    subset of the ancestors could be included by itself if it has other
    high-feerate descendants or are themselves higher feerate than this
    package/transaction). This is a conservative estimate and works for now.
    
    Co-authored-by: Suhas Daftuar <sdaftuar@chaincode.com>
    25f528a60c
  202. [refactor] rename entries to be more descriptive 0933d4f5bc
  203. [unit test] for CheckAncestorScores d629f29fc8
  204. [packages/policy] implement package RBF for v3 transactions 4785582f88
  205. [test util] CreateValidMempoolTransaction version parameter, always signal bip125 678f56d23f
  206. [test] package rbf 0bc6375272
  207. glozow force-pushed on Nov 27, 2023
  208. glozow commented at 4:42 pm on November 27, 2023: member
    Reviewers: note that the first half (7 commits) of this PR has been opened separately, in #28948. This has been rebased and current outstanding comments addressed.
  209. DrahtBot removed the label CI failed on Nov 27, 2023
  210. DrahtBot added the label Needs rebase on Dec 1, 2023
  211. DrahtBot commented at 9:26 pm on December 1, 2023: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

  212. glozow commented at 3:18 pm on December 11, 2023: member
    Closing this as superseded by #28948 and #28984 (v3 and package RBF PRs respectively). Note that the new package RBF has a slightly different approach than the one implemented here, permitting txns based on topology instead of v3-only. The discussion on this PR may still be helpful for context, but the volume of comments makes it pretty hard to do any review here.
  213. glozow closed this on Dec 11, 2023

  214. achow101 referenced this in commit 7143d43884 on Feb 10, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 07:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me