rpc: Allow to ignore specific policy reject reasons #20753

pull MarcoFalke wants to merge 4 commits into bitcoin:master from MarcoFalke:2012-policyRpcIgnore changing 21 files +152 −95
  1. MarcoFalke commented at 4:01 pm on December 23, 2020: member

    The -acceptnonstdtxn footgun is only enabled for regtest/testnet due to several shortcomings:

    • It can’t be changed without restarting the node
    • It applies to all txs and all reject reasons

    This pull adds a new optional RPC parameter to ignore specific reject reasons for a specific tx.

  2. in src/bench/block_assemble.cpp:52 in fa381a1507 outdated
    48@@ -49,7 +49,7 @@ static void AssembleBlock(benchmark::Bench& bench)
    49 
    50         for (const auto& txr : txs) {
    51             TxValidationState state;
    52-            bool ret{::AcceptToMemoryPool(*test_setup.m_node.mempool, state, txr, nullptr /* plTxnReplaced */, false /* bypass_limits */)};
    53+            bool ret{::AcceptToMemoryPool(*test_setup.m_node.mempool, state, txr, nullptr /* plTxnReplaced */, {} /* ignore_rejects */, false /* bypass_limits */)};
    


    luke-jr commented at 5:06 pm on December 23, 2020:
  3. in src/policy/policy.cpp:131 in fa381a1507 outdated
    134         if (whichType == TxoutType::NULL_DATA)
    135             nDataOut++;
    136         else if ((whichType == TxoutType::MULTISIG) && (!permit_bare_multisig)) {
    137-            reason = "bare-multisig";
    138-            return false;
    139+            MAYBE_REJECT(reason, "bare-multisig", ignore_rejects);
    



    DrahtBot commented at 4:25 pm on December 24, 2020:
    Thanks, also fixed the test issue in #20760
  4. luke-jr changes_requested
  5. luke-jr commented at 5:11 pm on December 23, 2020: member
    What’s the goal of reinventing this rather than just reopening and reviewing #7533 (which is still maintained)?
  6. DrahtBot added the label GUI on Dec 23, 2020
  7. DrahtBot added the label P2P on Dec 23, 2020
  8. DrahtBot added the label RPC/REST/ZMQ on Dec 23, 2020
  9. DrahtBot added the label TX fees and policy on Dec 23, 2020
  10. DrahtBot added the label Validation on Dec 23, 2020
  11. MarcoFalke removed the label GUI on Dec 23, 2020
  12. MarcoFalke removed the label P2P on Dec 23, 2020
  13. MarcoFalke removed the label Validation on Dec 23, 2020
  14. DrahtBot commented at 9:41 pm on December 23, 2020: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #20833 (rpc/validation: enable packages through testmempoolaccept by glozow)
    • #20828 (fuzz: Introduce CallOneOf helper to replace switch-case by MarcoFalke)
    • #20012 (rpc: Remove duplicate name and argNames from CRPCCommand by MarcoFalke)
    • #19935 (Move SaltedHashers to separate file and add some new ones by achow101)
    • #19602 (wallet: Migrate legacy wallets to descriptor wallets by achow101)
    • #19438 (Introduce deploymentstatus by ajtowns)
    • #19168 (Refactor: Improve setup_clean_chain semantics by fjahr)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  15. MarcoFalke force-pushed on Dec 24, 2020
  16. MarcoFalke force-pushed on Dec 24, 2020
  17. jnewbery commented at 10:14 am on December 29, 2020: member

    I think this needs more motivation. Without a strong reason to do this, I tend to agree with the comments on #7533:

  18. MarcoFalke commented at 10:25 am on December 29, 2020: member
    The reasons can only change if the server version changes. And the policy reject reasons may already change in all other RPCs (e.g. sendrawtransaction), so I don’t see how this pull makes any difference. If you have specific concerns, it would be good to elaborate them with an example, so that it is easier to understand them.
  19. MarcoFalke commented at 11:00 am on December 29, 2020: member

    To extend my reply a bit: Currently it is only possible to override the baked-in behavior for some very specific policy checks via command line (e.g. -dustrelayfee) for all network and rpc transactions. The goal of this change is to allow to override policy reject reasons while also giving the user more flexibility than an “on-off” -acceptnonstdtxn setting. The reason that -acceptnonstdtxn is disabled is that it wouldn’t be safe to turn on in normal operation for the default user. So I think the flexibility to have a per-tx override without restarting the node is useful. Maybe even outside the primary use case of one-off manual sends.

    Simply having broader categories (or even a single category) doesn’t solve the problem that policy changes between versions.

  20. luke-jr commented at 4:08 pm on December 29, 2020: member
    (Concept ACK, will review code in detail later)
  21. sipa commented at 7:02 pm on December 29, 2020: member

    Concept NACK on exposing this for mainnet. That seems like a huge footgun, even with the warning. What is the use case? We should never treat our own transactions differently than ones coming from the network.

    I think it’s pretty ugly in any case, but can imagine it’s useful for testing.

  22. MarcoFalke commented at 8:53 am on December 30, 2020: member
    One use case is to accept a transaction as a miner that has a large fee, but violates a policy limit (e.g. tx-version, dust, …) and thus has issues to propagate the network and be accepted to the likely-default mempool the miner is running. If someone is using this without thinking, it might lead to issues, just like invalidateblock may lead to issues when used without thinking.
  23. luke-jr commented at 5:01 pm on December 30, 2020: member

    Where is the footgun? Many policies are based on DoS risk, which doesn’t exist if there’s a user actively trying to add a transaction.

    As for use cases, I know one real-world case where a user accidentally used an uncompressed key for a segwit address. He’s figured out his mistake, and in theory he should be able to recover his coins, but he hasn’t convinced any miners (last I checked) to modify their software to accept it. (This PR might not be enough to get that one past, but it demonstrates that there are legitimate needs to override policies.)

  24. glozow commented at 11:55 pm on December 31, 2020: member
    I can imagine a user wanting to look at a transaction just based on consensus rules and not policy… testmempoolaccept will currently tell you exactly which policy a tx is failing; if a user wants to know “does my transaction pass just consensus rules otherwise?” I think ability to toggle on/off might make sense. But if a user is trying to get a nonstandard tx included in a block and can’t get a miner to accept it directly, how does this help? Core nodes still wouldn’t (and shouldn’t) relay it. Unless someone is planning to mine the tx themselves (not the typical user?) it would just be garbage sitting in their local mempool right?
  25. MarcoFalke commented at 10:59 am on January 1, 2021: member

    Correct, the transaction won’t relay typically. (Unless you know that you are running an old version of Bitcoin Core to relay a tx that future versions of Bitcoin Core consider policy-“valid”, but this is probably an edge case). Thus, I think the primary user of this feature is a miner. (And maybe some power users that know exactly what they are doing).

    I think ability to toggle on/off might make sense.

    I am happy to explore this direction. Though, testmempoolaccept will only tell you the first policy-failure reason, not the second one. So I was thinking that it might be safer to supply all failure reasons explicitly instead of hoping there was only one.

    Another option would be for testmempoolaccept to continue and collect all policy-failures on the way and return all of them instead of only the first? This would make an on/off switch safe again, I believe.

  26. luke-jr commented at 2:42 pm on January 1, 2021: member

    But if a user is trying to get a nonstandard tx included in a block and can’t get a miner to accept it directly, how does this help?

    This is how a miner would accept it directly.

    Another option would be for testmempoolaccept to continue and collect all policy-failures on the way and return all of them instead of only the first?

    This seems like a good idea regardless.

  27. glozow commented at 7:46 pm on January 1, 2021: member

    This is how a miner would accept it directly.

    Ah ok. Then, wrt maintainability, it still seems simpler to just set policy on/off entirely?

    Another option would be for testmempoolaccept to continue and collect all policy-failures on the way and return all of them instead of only the first? This would make an on/off switch safe again, I believe.

    That sounds nice. Although I’d say it’s tricky to implement - what we might consider individual policies aren’t really mutually exclusive and we usually stop after the first violation because later checks would break / say the same thing but be more expensive. For example too small or scriptSig size. Would we run EvalScript multiple times for the standard flags? I suppose this is where inputs specifying which policies to exclude would make sense, but still requires policy to be well-enumerated/defined.

  28. jnewbery commented at 10:47 pm on January 1, 2021: member

    Although I’d say it’s tricky to implement - what we might consider individual policies aren’t really mutually exclusive and we usually stop after the first violation because later checks would break / say the same thing but be more expensive.

    Absolutely agree with this. Mempool acceptance logic should be optimized towards simplicity and performance. Making the internal logic more complex so that we can return additional debugging information over an interface that we can’t guarantee to be stable doesn’t seem like the right decision to me.

  29. refactor: Pass reason out of policy checks
    The policy rejection reasons should live in ./src/policy, just like the
    other policy rejection reasons (e.g. IsStandardTx)
    13c15be575
  30. refactor: Pass ignore_rejects from RPCs to policy
    This is a pure refactor to pass ignore_rejects from the
    sendrawtransaction and testmempoolaccept RPCs to policy.
    
    Only default-constructed (empty) sets are passed and never used, so this
    doesn't change behavior.
    9d70043ba2
  31. rpc: Allow to ignore specific reject reasons af9b76a3d4
  32. test: Replace acceptnonstdtxn=1 with specific ignore in wallet_basic 28760050fa
  33. MarcoFalke force-pushed on Jan 3, 2021
  34. MarcoFalke commented at 6:16 pm on January 3, 2021: member

    Although I’d say it’s tricky to implement - what we might consider individual policies aren’t really mutually exclusive and we usually stop after the first violation because later checks would break / say the same thing but be more expensive.

    Absolutely agree with this. Mempool acceptance logic should be optimized towards simplicity and performance. Making the internal logic more complex so that we can return additional debugging information over an interface that we can’t guarantee to be stable doesn’t seem like the right decision to me.

    We already return the failure reason, and it is already not stable, so this wouldn’t change anything fundamentally. Also, this doesn’t change performance of mempool acceptance logic if not enabled. Yes, it might run checks that aren’t mutually exclusive, but what is the problem with that if the user explicitly asks to do that. A user might also rescan the wallet in a loop, even though it has already been rescanned.

  35. DrahtBot added the label Needs rebase on Jan 13, 2021
  36. DrahtBot commented at 9:11 am on January 13, 2021: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

    Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a “draft”.

  37. DrahtBot commented at 11:21 am on December 15, 2021: contributor
    • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
    • Is it no longer relevant? ➡️ Please close.
    • Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
  38. DrahtBot commented at 1:06 pm on March 21, 2022: contributor
    • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
    • Is it no longer relevant? ➡️ Please close.
    • Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
  39. MarcoFalke closed this on Jul 22, 2022

  40. MarcoFalke deleted the branch on Jul 22, 2022
  41. bitcoin locked this on Jul 22, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-21 15:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me