test: fix inconsistency in fundrawtransaction weight limits test #30353

pull furszy wants to merge 1 commits into bitcoin:master from furszy:2024_test_fix_max_weight_test changing 1 files +3 −3
  1. furszy commented at 7:50 pm on June 27, 2024: member

    Fix #30309 (review) inconsistency.

    Currently, the test is passing due to a mistake in the test inputs selection process. We are selecting the parent transaction change output as one of the inputs of the transaction to fund, which helps to surpass the target amount when it shouldn’t due to the fee reduction.

    The failure arises when the test behaves as intended by its coder; that is, when it does not select the change output. In this case, the pre-selected inputs aren’t enough to cover the target amount.

    Fix this by excluding the parent transaction’s change output from the inputs selection and including an extra input to cover the tx fee.

    The CI failure can be replicated with the following patch in master:

     0diff --git a/test/functional/wallet_fundrawtransaction.py b/test/functional/wallet_fundrawtransaction.py
     1--- a/test/functional/wallet_fundrawtransaction.py	(revision 9b480f7a25a737c9c4ebc33401e94d66c2da9ec3)
     2+++ b/test/functional/wallet_fundrawtransaction.py	(date 1720652934739)
     3@@ -1322,7 +1322,7 @@
     4         outputs = []
     5         for _ in range(1472):
     6             outputs.append({wallet.getnewaddress(address_type="legacy"): 0.1})
     7-        txid = self.nodes[0].send(outputs=outputs)["txid"]
     8+        txid = self.nodes[0].send(outputs=outputs, change_position=0)["txid"]
     9         self.generate(self.nodes[0], 1)
    10 
    11         # 272 WU per input (273 when high-s); picking 1471 inputs will exceed the max standard tx weight.
    12@@ -1330,7 +1330,7 @@
    13 
    14         # 1) Try to fund transaction only using the preset inputs
    15         input_weights = []
    16-        for i in range(1471):
    17+        for i in range(1, 1472):  # skip first output as it is the parent tx change output
    18             input_weights.append({"txid": txid, "vout": i, "weight": 273})
    19         assert_raises_rpc_error(-4, "Transaction too large", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
    
  2. DrahtBot commented at 7:50 pm on June 27, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK ismaelsadeeq, achow101
    Stale ACK rkrux, fjahr

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #27286 (wallet: Keep track of the wallet’s own transaction outputs in memory by achow101)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Tests on Jun 27, 2024
  4. maflcko added this to the milestone 28.0 on Jun 27, 2024
  5. rkrux approved
  6. rkrux commented at 8:06 am on June 28, 2024: none

    tACK 4f571fb

    make, test/functional are successful.

    Verified that the default value of add_inputs is True in fundrawtransaction that might have caused the inconsistency in the CI: https://github.com/bitcoin/bitcoin/blob/master/src/wallet/rpc/spend.cpp#L751

    Also verified that the corresponding scenario in wallet_send.py has add_inputs: False passed already here: https://github.com/bitcoin/bitcoin/blob/master/test/functional/wallet_send.py#L600

  7. fjahr commented at 11:26 am on June 30, 2024: contributor

    Code review ACK 4f571fb73526b55a83af3c04fbc662ccdd1307ae

    This looks like the right fix for the observed test failure to me.

  8. maflcko added the label Wallet on Jul 9, 2024
  9. ismaelsadeeq commented at 5:17 pm on July 10, 2024: member

    Concept ACK

    I could not recreate this failure locally. Looking closely at the code, 4f571fb73526b55a83af3c04fbc662ccdd1307ae might not solve the issue. For us to load coins from AutomaticCoinSelection and reach the failure in the C.I. selection_target in SelectCoins must be greater than 0.

    But when selection_target is > 0 and we prevent adding input with m_allow_other_inputs=False, just as it’s done in 4f571fb73526b55a83af3c04fbc662ccdd1307ae we will encounter another issue: https://github.com/bitcoin/bitcoin/blob/9adebe145557ef410964dd48a02f3d239f488cd0/src/wallet/spend.cpp#L769-L771

    I wonder why selection_target is not even positive in this test and cause failure like it did in the C.I because; The selected inputs values should be 14710000000 sats. Output value is also same 14710000000 sats. nTargetValue (output value + fees) should be greater than the selected inputs values making selection_target positive?


    I think to fix to prevent this failure, we should add more inputs to the input_weight to cover for the fees and ensure the selection_target is negative.

     0diff --git a/test/functional/wallet_fundrawtransaction.py b/test/functional/wallet_fundrawtransaction.py
     1index 3c1b2deb1d7..fdc7d8e80a2 100755
     2--- a/test/functional/wallet_fundrawtransaction.py
     3+++ b/test/functional/wallet_fundrawtransaction.py
     4@@ -1326,11 +1326,12 @@ class RawTransactionsTest(BitcoinTestFramework):
     5         self.generate(self.nodes[0], 1)
     6 
     7         # 272 WU per input (273 when high-s); picking 1471 inputs will exceed the max standard tx weight.
     8+        # Add 1 input to cover for fees
     9         rawtx = wallet.createrawtransaction([], [{wallet.getnewaddress(): 0.1 * 1471}])
    10 
    11         # 1) Try to fund transaction only using the preset inputs
    12         input_weights = []
    13-        for i in range(1471):
    14+        for i in range 1472):
    15             input_weights.append({"txid": txid, "vout": i, "weight": 273})
    16         assert_raises_rpc_error(-4, "Transaction too large", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
    

    Or even better, we should fail early without needing to perform all the redundant operations whenever pre-selected inputs weight alone exceed the maximum standard weight.

     0diff --git a/src/wallet/rpc/spend.cpp b/src/wallet/rpc/spend.cpp
     1index ac2a4826f05..d3c1349553e 100644
     2--- a/src/wallet/rpc/spend.cpp
     3+++ b/src/wallet/rpc/spend.cpp
     4@@ -670,6 +670,7 @@ CreatedTransactionResult FundTransaction(CWallet& wallet, const CMutableTransact
     5         }
     6     }
     7 
     8+    int total_inputs_weight{0};
     9     if (options.exists("input_weights")) {
    10         for (const UniValue& input : options["input_weights"].get_array().getValues()) {
    11             Txid txid = Txid::FromUint256(ParseHashO(input, "txid"));
    12@@ -696,10 +697,13 @@ CreatedTransactionResult FundTransaction(CWallet& wallet, const CMutableTransact
    13             if (weight > MAX_STANDARD_TX_WEIGHT) {
    14                 throw JSONRPCError(RPC_INVALID_PARAMETER, strprintf("Invalid parameter, weight cannot be greater than the maximum standard tx weight of %d", MAX_STANDARD_TX_WEIGHT));
    15             }
    16-
    17+            total_inputs_weight += weight;
    18             coinControl.SetInputWeight(COutPoint(txid, vout), weight);
    19         }
    20     }
    21+    if (total_inputs_weight > MAX_STANDARD_TX_WEIGHT) {
    22+        throw JSONRPCError(RPC_INVALID_PARAMETER, strprintf("Selected inputs weight cannot be greater than the maximum standard tx weight of %d", MAX_STANDARD_TX_WEIGHT));
    23+    }
    24 
    25     if (recipients.empty())
    26         throw JSONRPCError(RPC_INVALID_PARAMETER, "TX must have at least one output");
    27diff --git a/test/functional/wallet_fundrawtransaction.py b/test/functional/wallet_fundrawtransaction.py
    28index 3c1b2deb1d7..727451fb79e 100755
    29--- a/test/functional/wallet_fundrawtransaction.py
    30+++ b/test/functional/wallet_fundrawtransaction.py
    31@@ -1332,7 +1332,7 @@ class RawTransactionsTest(BitcoinTestFramework):
    32         input_weights = []
    33         for i in range 1471):
    34             input_weights.append({"txid": txid, "vout": i, "weight": 273})
    35-        assert_raises_rpc_error(-4, "Transaction too large", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
    36+        assert_raises_rpc_error(-8, "Selected inputs weight cannot be greater than the maximum standard tx weight of 400000", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
    37 
    38         # 2) Let the wallet fund the transaction
    39         assert_raises_rpc_error(-4, "The inputs size exceeds the maximum weight. Please try sending a smaller amount or manually consolidating your wallet's UTXOs",
    
  10. furszy commented at 9:35 pm on July 10, 2024: member

    Nice writeup @ismaelsadeeq!. Thanks for the push, just understood the reason behind it.

    The issue is in the inputs selection. They are manually selected from the parent tx without excluding the change output. This output is not from the wallet and helps to surpass the selection amount (otherwise yes, the test would be failing all the time). So, the CI failure arises when the test excludes the change output which only occurs when the change output is at the end of the tx outputs vector.. whose probability is very low.

    This also presents an issue inside the wallet. Because it is not detecting that the pre-selected input is not solvable when the user provides the inputs weight. And this has an edge case explanation too.. it is because the wallet can know the not-owned change output internally (which is not so common) when it is an output of a transaction the wallet stores -> because another output(s) of this tx is owned by the wallet. Extra note: This is not a problem for fundrawtransaction() because the command does not sign the transaction. Only for send().

    Will fix the test and also this last issue.

  11. test: fix inconsistency in fundrawtransaction weight limits test
    Currently, the test is passing due to a mistake in the test inputs
    selection process. We are selecting the parent transaction change
    output as one of the inputs of the transaction to fund, which
    helps to surpass the target amount when it shouldn't due to the
    fee reduction.
    
    The failure arises when the test behaves as intended by its coder;
    that is, when it does not select the change output. In this case,
    the pre-selected inputs aren't enough to cover the target amount.
    
    Fix this by excluding the parent transaction's change output from
    the inputs selection and including an extra input to cover the tx
    fee.
    00b8e26bd6
  12. furszy force-pushed on Jul 10, 2024
  13. furszy commented at 11:12 pm on July 10, 2024: member

    Updated per feedback.

    The CI failure that originated this PR (https://github.com/bitcoin/bitcoin/pull/30309#discussion_r1657628378) can be replicated with the following patch in master:

     0diff --git a/test/functional/wallet_fundrawtransaction.py b/test/functional/wallet_fundrawtransaction.py
     1--- a/test/functional/wallet_fundrawtransaction.py	(revision 9b480f7a25a737c9c4ebc33401e94d66c2da9ec3)
     2+++ b/test/functional/wallet_fundrawtransaction.py	(date 1720652934739)
     3@@ -1322,7 +1322,7 @@
     4         outputs = []
     5         for _ in range(1472):
     6             outputs.append({wallet.getnewaddress(address_type="legacy"): 0.1})
     7-        txid = self.nodes[0].send(outputs=outputs)["txid"]
     8+        txid = self.nodes[0].send(outputs=outputs, change_position=0)["txid"]
     9         self.generate(self.nodes[0], 1)
    10 
    11         # 272 WU per input (273 when high-s); picking 1471 inputs will exceed the max standard tx weight.
    12@@ -1330,7 +1330,7 @@
    13 
    14         # 1) Try to fund transaction only using the preset inputs
    15         input_weights = []
    16-        for i in range(1471):
    17+        for i in range(1, 1472):  # skip first output as it is the parent tx change output
    18             input_weights.append({"txid": txid, "vout": i, "weight": 273})
    19         assert_raises_rpc_error(-4, "Transaction too large", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
    
  14. maflcko commented at 6:23 am on July 11, 2024: member

    The test failure looks unrelated:

    02024-07-10T23:36:07.2558178Z test_framework.authproxy.JSONRPCException: 'stop' RPC took longer than 1200.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
    

    https://github.com/bitcoin/bitcoin/actions/runs/9882674449/job/27296008093?pr=30353

     02024-07-10T23:36:12.6566166Z  test  2024-07-10T23:16:06.381000Z TestFramework (INFO): Stopping nodes 
     12024-07-10T23:36:12.6566491Z  test  2024-07-10T23:16:06.381000Z TestFramework.node0 (DEBUG): Stopping node 
     22024-07-10T23:36:12.6567186Z  node0 2024-07-10T23:16:06.382752Z [http] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:305] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:61436 
     32024-07-10T23:36:12.6567862Z  node0 2024-07-10T23:16:06.382862Z [httpworker.3] [D:\a\bitcoin\bitcoin\src\rpc\request.cpp:232] [parse] [rpc] ThreadRPCServer method=stop user=__cookie__ 
     42024-07-10T23:36:12.6568479Z  node0 2024-07-10T23:16:06.382950Z [init] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:498] [InterruptHTTPServer] [http] Interrupting HTTP server 
     52024-07-10T23:36:12.6569093Z  node0 2024-07-10T23:16:06.382994Z [init] [D:\a\bitcoin\bitcoin\src\httprpc.cpp:378] [InterruptHTTPRPC] [rpc] Interrupting HTTP RPC server 
     62024-07-10T23:36:12.6569629Z  node0 2024-07-10T23:16:06.383015Z [init] [D:\a\bitcoin\bitcoin\src\rpc\server.cpp:308] [operator ()] [rpc] Interrupting RPC 
     72024-07-10T23:36:12.6570126Z  node0 2024-07-10T23:16:06.383041Z [init] [D:\a\bitcoin\bitcoin\src\init.cpp:273] [Shutdown] Shutdown: In progress... 
     82024-07-10T23:36:12.6570693Z  node0 2024-07-10T23:16:06.383060Z [shutoff] [D:\a\bitcoin\bitcoin\src\httprpc.cpp:383] [StopHTTPRPC] [rpc] Stopping HTTP RPC server 
     92024-07-10T23:36:12.6571484Z  node0 2024-07-10T23:16:06.383086Z [shutoff] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:760] [UnregisterHTTPHandler] [http] Unregistering HTTP handler for / (exactmatch 1) 
    102024-07-10T23:36:12.6572255Z  node0 2024-07-10T23:16:06.383112Z [shutoff] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:760] [UnregisterHTTPHandler] [http] Unregistering HTTP handler for /wallet/ (exactmatch 0) 
    112024-07-10T23:36:12.6572774Z  node0 2024-07-10T23:16:06.383131Z [shutoff] [D:\a\bitcoin\bitcoin\src\rpc\server.cpp:320] [operator ()] [rpc] Stopping RPC 
    122024-07-10T23:36:12.6573277Z  node0 2024-07-10T23:16:06.383334Z [shutoff] [D:\a\bitcoin\bitcoin\src\init.cpp:440] [OnRPCStopped] [rpc] RPC stopped. 
    132024-07-10T23:36:12.6573860Z  node0 2024-07-10T23:16:06.383370Z [shutoff] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:510] [StopHTTPServer] [http] Stopping HTTP server 
    142024-07-10T23:36:12.6574524Z  node0 2024-07-10T23:16:06.383391Z [shutoff] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:512] [StopHTTPServer] [http] Waiting for HTTP worker threads to exit 
    152024-07-10T23:36:12.6575470Z  node0 2024-07-10T23:16:06.383437Z [addcon] [D:\a\bitcoin\bitcoin\src\util\thread.cpp:22] [TraceThread] addcon thread exit 
    162024-07-10T23:36:12.6576169Z  node0 2024-07-10T23:16:06.383557Z [shutoff] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:526] [StopHTTPServer] [http] Waiting for 1 connections to stop HTTP server 
    172024-07-10T23:36:12.6576732Z  node0 2024-07-10T23:16:06.383658Z [http] [D:\a\bitcoin\bitcoin\src\httpserver.cpp:354] [ThreadHTTP] [http] Exited http event loop 
    182024-07-10T23:36:12.6577232Z  node0 2024-07-10T23:16:06.404069Z [net] [D:\a\bitcoin\bitcoin\src\util\thread.cpp:22] [TraceThread] net thread exit 
    192024-07-10T23:36:12.6577770Z  node0 2024-07-10T23:16:06.419772Z [msghand] [D:\a\bitcoin\bitcoin\src\util\thread.cpp:22] [TraceThread] msghand thread exit 
    202024-07-10T23:36:12.6578383Z  node0 2024-07-10T23:16:08.104417Z [scheduler] [D:\a\bitcoin\bitcoin\src\wallet\bdb.cpp:663] [PeriodicFlush] [walletdb] Flushing wallet.dat 
    212024-07-10T23:36:12.6579062Z  node0 2024-07-10T23:16:08.106925Z [scheduler] [D:\a\bitcoin\bitcoin\src\wallet\bdb.cpp:671] [PeriodicFlush] [walletdb] Flushed wallet.dat 2ms 
    222024-07-10T23:36:12.6579686Z  node0 2024-07-10T23:16:08.106958Z [scheduler] [D:\a\bitcoin\bitcoin\src\wallet\bdb.cpp:663] [PeriodicFlush] [walletdb] Flushing wallet.dat 
    232024-07-10T23:36:12.6580288Z  node0 2024-07-10T23:16:08.108697Z [scheduler] [D:\a\bitcoin\bitcoin\src\wallet\bdb.cpp:671] [PeriodicFlush] [walletdb] Flushed wallet.dat 1ms 
    242024-07-10T23:36:12.6580980Z  node1 2024-07-10T23:16:45.638808Z [scheduler] [D:\a\bitcoin\bitcoin\src\net.cpp:2369] [StartExtraBlockRelayPeers] [net] enabling extra block-relay-only peers 
    252024-07-10T23:36:12.6581662Z  node2 2024-07-10T23:16:46.802204Z [scheduler] [D:\a\bitcoin\bitcoin\src\net.cpp:2369] [StartExtraBlockRelayPeers] [net] enabling extra block-relay-only peers 
    262024-07-10T23:36:12.6582341Z  node0 2024-07-10T23:16:51.102814Z [scheduler] [D:\a\bitcoin\bitcoin\src\net.cpp:2369] [StartExtraBlockRelayPeers] [net] enabling extra block-relay-only peers 
    272024-07-10T23:36:12.6583076Z  node1 2024-07-10T23:17:00.635058Z [scheduler] [D:\a\bitcoin\bitcoin\src\validation.cpp:2015] [IsInitialBlockDownload] Leaving InitialBlockDownload (latching to false) 
    282024-07-10T23:36:12.6583697Z  node1 2024-07-10T23:31:00.645441Z [scheduler] [D:\a\bitcoin\bitcoin\src\net.cpp:2332] [DumpAddresses] [net] Flushed 0 addresses to peers.dat  2ms 
    292024-07-10T23:36:12.6584292Z  node2 2024-07-10T23:31:01.787885Z [scheduler] [D:\a\bitcoin\bitcoin\src\net.cpp:2332] [DumpAddresses] [net] Flushed 0 addresses to peers.dat  1ms 
    302024-07-10T23:36:12.6584894Z  node0 2024-07-10T23:31:06.111388Z [scheduler] [D:\a\bitcoin\bitcoin\src\net.cpp:2332] [DumpAddresses] [net] Flushed 0 addresses to peers.dat  1ms 
    312024-07-10T23:36:12.6584902Z 
    322024-07-10T23:36:12.6584907Z 
    
  15. ismaelsadeeq approved
  16. ismaelsadeeq commented at 12:47 pm on July 11, 2024: member

    Code review and Tested ACK 00b8e26bd6a47a8207c7571ca7ef9643f0ee2668

    I was able to recreate the failure with the diff in the OP. I also decoded the transaction and confirmed that it’s selecting the change output of the parent transaction with a value of 2.79949465 BTC. That’s why the test was not failing.

    The commit 00b8e26bd6a47a8207c7571ca7ef9643f0ee2668 fixes this issue.

    This is not a problem for fundrawtransaction()

    Yes, it is not. The call will succeed, but when attempting to sign the transaction with signrawtransactionwithkey, the RPC will return an error: “Unable to sign input, invalid stack size (possibly missing key)”.

    Only for send()

    How is this an issue for send()?

    Will fix the test and also this last issue.

    In this PR? Also, should we consider returning this error early as I suggested?

  17. DrahtBot requested review from fjahr on Jul 11, 2024
  18. DrahtBot requested review from rkrux on Jul 11, 2024
  19. achow101 commented at 6:49 pm on July 11, 2024: member

    ACK 00b8e26bd6a47a8207c7571ca7ef9643f0ee2668

    Ran into a similar failure in #27286 which I fixed slightly differently. However this fix would work just as well, and aligns with my understanding of both failures.

  20. achow101 merged this on Jul 11, 2024
  21. achow101 closed this on Jul 11, 2024

  22. furszy deleted the branch on Jul 11, 2024
  23. furszy commented at 8:36 pm on July 11, 2024: member

    A bit late but here @ismaelsadeeq.

    Only for send()

    How is this an issue for send()?

    Mainly because of what users would expect from the send() command. Which, due to its description, is to actually craft a sendable transaction. As is now, if this happens, the command will return the incomplete transaction without telling the user what happened. But.. I don’t think will follow-up this thought. No one complained about it so far.

    should we consider returning this error early as I suggested?

    Yes sure. thats a possibility too. It does not catch all the scenarios (e.g. max weight can be exceeded with the sum of inputs and outputs sizes) but at least will promptly alert users when they set high custom inputs weights.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-21 09:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me