Allow fundrawtransaction and walletcreatefundedpsbt to take external inputs #17211

pull achow101 wants to merge 5 commits into bitcoin:master from achow101:fundtx-external-inputs changing 10 files +381 −50
  1. achow101 commented at 9:29 pm on October 21, 2019: member

    Currently fundrawtransaction and walletcreatefundedpsbt both do not allow external inputs as the wallet does not have the information necessary to estimate their fees.

    This PR adds an additional argument to both those RPCs which allows the user to specify solving data. This way, the wallet can use that solving data to estimate the size of those inputs. The solving data can be public keys, scripts, or descriptors.

  2. achow101 force-pushed on Oct 21, 2019
  3. fanquake added the label RPC/REST/ZMQ on Oct 21, 2019
  4. fanquake added the label Wallet on Oct 21, 2019
  5. DrahtBot commented at 10:25 pm on October 21, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #22019 (wallet: Introduce SelectionResult for encapsulating a coin selection solution by achow101)
    • #21283 (Implement BIP 370 PSBTv2 by achow101)
    • #20892 (tests: Run both descriptor and legacy tests within a single test invocation by achow101)
    • #20640 (wallet, refactor: return out-params of CreateTransaction() as optional struct by theStack)
    • #19602 (wallet: Migrate legacy wallets to descriptor wallets by achow101)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  6. achow101 force-pushed on Oct 21, 2019
  7. achow101 force-pushed on Oct 23, 2019
  8. in src/wallet/wallet.cpp:1472 in 3445b3b406 outdated
    1763     int nIn = 0;
    1764     for (const auto& txout : txouts)
    1765     {
    1766-        if (!DummySignInput(txNew.vin[nIn], txout, use_max_sig)) {
    1767-            return false;
    1768+        // Use max sig if watch only inputs were used or if this particular input is an external input
    


    instagibbs commented at 2:01 pm on October 24, 2019:
    “to ensure a sufficient feer is attained for the requested feerate” ?

    achow101 commented at 9:22 pm on October 24, 2019:
    Done
  9. in src/wallet/wallet.cpp:1764 in 3445b3b406 outdated
    1764     for (const auto& txout : txouts)
    1765     {
    1766-        if (!DummySignInput(txNew.vin[nIn], txout, use_max_sig)) {
    1767-            return false;
    1768+        // Use max sig if watch only inputs were used or if this particular input is an external input
    1769+        bool use_max_sig = coin_control && (coin_control->fAllowWatchOnly || (coin_control && coin_control->IsExternalSelected(txNew.vin[nIn].prevout)));
    


    instagibbs commented at 2:03 pm on October 24, 2019:
    nit: add a reference to txNew.vin[nIn] since it’s used 3 times here

    instagibbs commented at 2:03 pm on October 24, 2019:
    txNew.vin[nIn] is used three times here, just set a variable to reference it?

    achow101 commented at 9:22 pm on October 24, 2019:
    Done

    achow101 commented at 9:22 pm on October 24, 2019:
    Done
  10. in src/wallet/rpcwallet.cpp:3132 in 0095c01ffd outdated
    3128@@ -3094,6 +3129,18 @@ void FundTransaction(CWallet* const pwallet, CMutableTransaction& tx, CAmount& f
    3129         setSubtractFeeFromOutputs.insert(pos);
    3130     }
    3131 
    3132+    // Fetch specified UTXOs from the UTXO set
    


    instagibbs commented at 2:12 pm on October 24, 2019:

    Not immediately obvious to me why this change is necessary.

    Probably requires more extensive comment?


    instagibbs commented at 2:13 pm on October 24, 2019:
    In other words, how did selection of inputs work before this?

    achow101 commented at 5:47 pm on October 24, 2019:
    CTxOuts are needed for fee estimation and knowing the values of the outputs being selected. For inputs the wallet knows about, it would fetch the outputs from the wallet (obviously). Otherwise it would basically ignore them. So to support external inputs, either the user would have to enter the output info (bad UX IMO) or we can fetch it from the UTXO set.

    instagibbs commented at 5:59 pm on October 24, 2019:

    oh, duh pubkeys/descriptors aren’t enough :)

    Suggested text: “// Fetch specified UTXOs from the UTXO set for knowing the values of the outputs being selected // and to match with the given solving_data. These are only // used when the outputs are not stored in the wallet.”


    achow101 commented at 9:22 pm on October 24, 2019:
    Done
  11. instagibbs commented at 2:16 pm on October 24, 2019: member

    This needs tests for the scripts, descriptors options in RPC.

    Also a simple check that wrong pubkeys don’t also cause things to seem complete.

  12. instagibbs commented at 6:00 pm on October 24, 2019: member
    RPC help also needs to be updated to reflect the new functionality. For example: https://github.com/bitcoin/bitcoin/pull/17211/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR3167
  13. achow101 force-pushed on Oct 24, 2019
  14. achow101 force-pushed on Oct 24, 2019
  15. achow101 commented at 9:24 pm on October 24, 2019: member
    Updated the help
  16. achow101 force-pushed on Oct 25, 2019
  17. instagibbs commented at 6:47 pm on October 25, 2019: member
    can you also reflect the updated help into walletcreatefundedpsbt verbatim? It’s helpful for usage.
  18. DrahtBot added the label Needs rebase on Oct 29, 2019
  19. achow101 force-pushed on Oct 29, 2019
  20. achow101 force-pushed on Oct 29, 2019
  21. achow101 commented at 7:08 pm on October 29, 2019: member
    Also updated the help of walletcreatefundedpsbt
  22. DrahtBot removed the label Needs rebase on Oct 29, 2019
  23. instagibbs commented at 2:23 pm on October 31, 2019: member
    not compiling
  24. achow101 force-pushed on Oct 31, 2019
  25. in src/wallet/rpcwallet.cpp:3297 in 7f9aef6e6c outdated
    3205@@ -3157,6 +3206,25 @@ static UniValue fundrawtransaction(const JSONRPCRequest& request)
    3206                         "This boolean should reflect whether the transaction has inputs\n"
    3207                         "(e.g. fully valid, or on-chain transactions), if known by the caller."
    3208                     },
    3209+                    {"solving_data", RPCArg::Type::OBJ, RPCArg::Optional::OMITTED_NAMED_ARG, "Keys and scripts needed for producing a final transaction with a dummy signature. Used for fee estimation during coin selection.\n",
    


    luke-jr commented at 4:24 pm on November 4, 2019:
    This seems like an advanced feature, so maybe move to “options”?

    achow101 commented at 10:57 pm on November 18, 2019:
    I don’t think this really belongs as part of options. It’s not like a setting, just additional data needed.
  26. in src/wallet/wallet.cpp:1335 in 7f9aef6e6c outdated
    1331@@ -1332,14 +1332,12 @@ int64_t CWalletTx::GetTxTime() const
    1332 
    1333 // Helper for producing a max-sized low-S low-R signature (eg 71 bytes)
    1334 // or a max-sized low-S signature (e.g. 72 bytes) if use_max_sig is true
    1335-bool CWallet::DummySignInput(CTxIn &tx_in, const CTxOut &txout, bool use_max_sig) const
    1336+static bool DummySignInput(const SigningProvider* provider, CTxIn &tx_in, const CTxOut &txout, bool use_max_sig)
    


    luke-jr commented at 8:39 pm on November 13, 2019:

    Why not a reference instead of a pointer?

    Is there any reason not to move this to SigningProvider as a method?


    achow101 commented at 10:58 pm on November 18, 2019:

    I changed this to a reference.

    I don’t think this really makes sense as part of SigningProvider. SigningProvider is to provide data for signing and is agnostic of what is being signed and how it is being signed. To add this to SigningProvider would violate that. Also, that would imply adding every other signing function to SigningProvider as well, and I don’t think we should do that.

  27. achow101 force-pushed on Nov 18, 2019
  28. DrahtBot added the label Needs rebase on Nov 22, 2019
  29. achow101 force-pushed on Nov 22, 2019
  30. DrahtBot removed the label Needs rebase on Nov 22, 2019
  31. DrahtBot added the label Needs rebase on Nov 24, 2019
  32. achow101 force-pushed on Nov 25, 2019
  33. DrahtBot removed the label Needs rebase on Nov 25, 2019
  34. DrahtBot added the label Needs rebase on Dec 1, 2019
  35. achow101 force-pushed on Dec 2, 2019
  36. DrahtBot removed the label Needs rebase on Dec 2, 2019
  37. instagibbs referenced this in commit 526e802d69 on Dec 4, 2019
  38. DrahtBot added the label Needs rebase on Dec 10, 2019
  39. achow101 force-pushed on Dec 10, 2019
  40. DrahtBot removed the label Needs rebase on Dec 10, 2019
  41. DrahtBot added the label Needs rebase on Jan 29, 2020
  42. achow101 force-pushed on Jan 29, 2020
  43. DrahtBot removed the label Needs rebase on Jan 29, 2020
  44. DrahtBot added the label Needs rebase on Jan 30, 2020
  45. achow101 force-pushed on Jan 30, 2020
  46. DrahtBot removed the label Needs rebase on Jan 30, 2020
  47. DrahtBot added the label Needs rebase on Feb 25, 2020
  48. achow101 force-pushed on Feb 25, 2020
  49. DrahtBot removed the label Needs rebase on Feb 25, 2020
  50. DrahtBot added the label Needs rebase on Mar 9, 2020
  51. achow101 force-pushed on Mar 10, 2020
  52. DrahtBot removed the label Needs rebase on Mar 10, 2020
  53. DrahtBot added the label Needs rebase on Apr 19, 2020
  54. achow101 force-pushed on Apr 19, 2020
  55. DrahtBot removed the label Needs rebase on Apr 19, 2020
  56. DrahtBot added the label Needs rebase on May 4, 2020
  57. achow101 force-pushed on May 4, 2020
  58. DrahtBot removed the label Needs rebase on May 4, 2020
  59. DrahtBot added the label Needs rebase on Jun 21, 2020
  60. achow101 force-pushed on Jun 22, 2020
  61. DrahtBot removed the label Needs rebase on Jun 22, 2020
  62. DrahtBot added the label Needs rebase on Sep 3, 2020
  63. achow101 force-pushed on Sep 3, 2020
  64. DrahtBot removed the label Needs rebase on Sep 3, 2020
  65. achow101 force-pushed on Sep 30, 2020
  66. in src/wallet/wallet.cpp:1549 in 1cd3840f7b outdated
    1555-        if (!DummySignInput(txNew.vin[nIn], txout, use_max_sig)) {
    1556-            return false;
    1557+        CTxIn& txin = txNew.vin[nIn];
    1558+        // Use max sig if watch only inputs were used or if this particular input is an external input
    1559+        // to ensure a sufficient fee is attained for the requested feerate.
    1560+        bool use_max_sig = coin_control && (coin_control->fAllowWatchOnly || (coin_control && coin_control->IsExternalSelected(txin.prevout)));
    


    meshcollider commented at 8:55 am on October 7, 2020:
    This is a bit more complicated than it needs to be, you can drop the second coin_control &&

    achow101 commented at 2:00 pm on October 8, 2020:
    Done
  67. in src/wallet/rpcwallet.cpp:3158 in 6b87667529 outdated
    3153+                if (!pubkey.IsFullyValid()) {
    3154+                    throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, strprintf("%s is not a valid public key", pubkey_strs[i].get_str()));
    3155+                }
    3156+                coinControl.m_external_provider.pubkeys.emplace(pubkey.GetID(), pubkey);
    3157+                // Add witnes script for pubkeys
    3158+                CScript wit_script = GetScriptForDestination(WitnessV0KeyHash(pubkey.GetID()));
    


    meshcollider commented at 9:04 am on October 7, 2020:
    unnecessary .getID()

    achow101 commented at 2:00 pm on October 8, 2020:
    Done
  68. meshcollider commented at 9:07 am on October 7, 2020: contributor
    Code review ACK e92720e44caac789d1784fbf35700da5ca86dfbc
  69. achow101 force-pushed on Oct 8, 2020
  70. meshcollider commented at 7:27 pm on October 8, 2020: contributor

    Re-utACK 3cdd95425123b9c59d7e5d2c18e5698f837e4220

    Only changes were the two minor above.

  71. in test/functional/rpc_psbt.py:579 in 3cdd954251 outdated
    496@@ -497,5 +497,21 @@ def test_psbt_input_keys(psbt_input, keys):
    497 
    498         assert_raises_rpc_error(-25, 'Inputs missing or spent', self.nodes[0].walletprocesspsbt, 'cHNidP8BAJoCAAAAAkvEW8NnDtdNtDpsmze+Ht2LH35IJcKv00jKAlUs21RrAwAAAAD/////S8Rbw2cO1020OmybN74e3Ysffkglwq/TSMoCVSzbVGsBAAAAAP7///8CwLYClQAAAAAWABSNJKzjaUb3uOxixsvh1GGE3fW7zQD5ApUAAAAAFgAUKNw0x8HRctAgmvoevm4u1SbN7XIAAAAAAAEAnQIAAAACczMa321tVHuN4GKWKRncycI22aX3uXgwSFUKM2orjRsBAAAAAP7///9zMxrfbW1Ue43gYpYpGdzJwjbZpfe5eDBIVQozaiuNGwAAAAAA/v///wIA+QKVAAAAABl2qRT9zXUVA8Ls5iVqynLHe5/vSe1XyYisQM0ClQAAAAAWABRmWQUcjSjghQ8/uH4Bn/zkakwLtAAAAAAAAQEfQM0ClQAAAAAWABRmWQUcjSjghQ8/uH4Bn/zkakwLtAAAAA==')
    499 
    500+        # Test that we can fund psbts with external inputs specified
    501+        ext_utxo = self.nodes[0].listunspent()[0]
    502+        addr_info = self.nodes[0].getaddressinfo(ext_utxo['address'])
    503+
    504+        # An external input without solving data should result in an error
    505+        assert_raises_rpc_error(-4, "Missing solving data for estimating transaction size", self.nodes[1].walletcreatefundedpsbt, [{"txid": ext_utxo['txid'], "vout": ext_utxo['vout']}], {self.nodes[0].getnewaddress(): 10 + ext_utxo['amount']}, 0, {'add_inputs': True})
    


    adamjonas commented at 3:39 pm on October 19, 2020:

    This is causing intermittent failures for me. It’s failing about 20%. Here’s the traceback:

    0File "/bitcoin/test/functional/rpc_psbt.py", line 505, in run_test
    1    assert_raises_rpc_error(-4, "Missing solving data for estimating transaction size", self.nodes[1].walletcreatefundedpsbt, [{"txid": ext_utxo['txid'], "vout": ext_utxo['vout']}], {self.nodes[0].getnewaddress(): 10 + ext_utxo['amount']}, 0, {'add_inputs': True})
    2File "/bitcoin/test/functional/test_framework/util.py", line 124, in assert_raises_rpc_error
    3    assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
    4File "/bitcoin/test/functional/test_framework/util.py", line 139, in try_rpc
    5    raise AssertionError(
    6AssertionError: Expected substring not found in error message:
    7substring: 'Missing solving data for estimating transaction size'
    8error message: 'Insufficient funds'
    

    achow101 commented at 8:22 pm on November 5, 2020:
    I think I’ve fixed this.

    adamjonas commented at 3:34 pm on November 6, 2020:
    I ran it 20 times and could not trigger any failures. Looks good.
  72. DrahtBot added the label Needs rebase on Nov 4, 2020
  73. achow101 force-pushed on Nov 4, 2020
  74. DrahtBot removed the label Needs rebase on Nov 4, 2020
  75. achow101 force-pushed on Nov 5, 2020
  76. achow101 force-pushed on Nov 5, 2020
  77. in src/wallet/rpcwallet.cpp:3256 in 8674f2b6eb outdated
    3231@@ -3184,7 +3232,8 @@ static RPCHelpMan fundrawtransaction()
    3232                 "Note that inputs which were signed may need to be resigned after completion since in/outputs have been added.\n"
    3233                 "The inputs added will not be signed, use signrawtransactionwithkey\n"
    3234                 " or signrawtransactionwithwallet for that.\n"
    3235-                "Note that all existing inputs must have their previous output transaction be in the wallet.\n"
    3236+                "Note that all existing inputs must either have their previous output transaction be in the wallet\n"
    3237+                "or be in the utxo set. Solving data must be provided for non-wallet inputs.\n"
    


    luke-jr commented at 4:13 pm on November 9, 2020:
    Maybe we should require the user to provide the TXO too?

    achow101 commented at 5:23 pm on November 12, 2020:
    I don’t think it should be required. It could be an option like for the signrawtransaction* RPCs.

    jonatack commented at 2:47 pm on November 13, 2020:

    6bb343c

    • s/Note that all/All/ -> the very next sentence also begins with “Note that all” (both uses of “Note that” could be omitted)

    • s/utxo/UTXO/


    achow101 commented at 7:32 pm on January 12, 2021:
    Done
  78. in src/wallet/rpcwallet.cpp:3049 in 8674f2b6eb outdated
    3045@@ -3046,7 +3046,7 @@ static RPCHelpMan listunspent()
    3046     };
    3047 }
    3048 
    3049-void FundTransaction(CWallet* const pwallet, CMutableTransaction& tx, CAmount& fee_out, int& change_position, const UniValue& options, CCoinControl& coinControl)
    3050+void FundTransaction(CWallet* const pwallet, CMutableTransaction& tx, CAmount& fee_out, int& change_position, const UniValue options, CCoinControl& coinControl, const UniValue& solving_data)
    


    luke-jr commented at 8:06 pm on November 9, 2020:
    Why is options losing its explicit ref typing?

    achow101 commented at 6:41 pm on November 12, 2020:
    Oops. should be fixed.
  79. in src/wallet/wallet.cpp:2479 in 2825a72a51 outdated
    2476@@ -2467,7 +2477,13 @@ bool CWallet::SelectCoins(const std::vector<COutput>& vAvailableCoins, const CAm
    2477             }
    2478             setPresetCoins.insert(coin);
    2479         } else {
    


    apoelstra commented at 5:32 pm on November 11, 2020:
    This else is for an if(txid is in wallet) … but what we actually want to check is if(outpoint is in wallet). If you try to spend an output you don’t know from a wallet transaction (e.g. by sending coins to a peer then trying to spend their output), this logic will simply fail – it will return false; a few lines above when trying to compute wtx.GetSpendSize on the outpoint.

    achow101 commented at 6:41 pm on November 12, 2020:
    I think I’ve resolved this.
  80. in src/wallet/wallet.cpp:2483 in 2825a72a51 outdated
    2476@@ -2467,7 +2477,13 @@ bool CWallet::SelectCoins(const std::vector<COutput>& vAvailableCoins, const CAm
    2477             }
    2478             setPresetCoins.insert(coin);
    2479         } else {
    2480-            return false; // TODO: Allow non-wallet inputs
    2481+            CTxOut txout;
    2482+            if (coin_control.GetExternalOutput(outpoint, txout)) {
    2483+                nValueFromPresetInputs += txout.nValue;
    2484+                setPresetCoins.insert(CInputCoin(outpoint, txout));
    


    apoelstra commented at 7:47 pm on November 11, 2020:
    I think you need to subtract the coin value from value_to_select. This looks like a rebasing error.

    apoelstra commented at 8:00 pm on November 11, 2020:
    Would be good to add a functional test which funds a transaction using external inputs, asserting that the wallet balance would be insufficient without the external inputs. (As in https://github.com/ElementsProject/elements/pull/755/ which is a rough analogue of this PR.)

    achow101 commented at 6:42 pm on November 12, 2020:
    Should be fixed. Also added a test.
  81. achow101 force-pushed on Nov 12, 2020
  82. in src/wallet/coincontrol.h:15 in cd75670167 outdated
     9@@ -10,6 +10,8 @@
    10 #include <policy/feerate.h>
    11 #include <policy/fees.h>
    12 #include <primitives/transaction.h>
    13+#include <script/keyorigin.h>
    14+#include <script/signingprovider.h>
    15 #include <script/standard.h>
    16 
    


    jonatack commented at 10:36 am on November 13, 2020:

    e2c50aa11 missing headers?

    0 #include <script/signingprovider.h>
    1 #include <script/standard.h>
    2 
    3+#include <algorithm>
    4+#include <map>
    5+#include <set>
    6+
    

    achow101 commented at 7:31 pm on January 12, 2021:
    Done
  83. in src/wallet/wallet.cpp:1532 in cd75670167 outdated
    1558-        if (!DummySignInput(txNew.vin[nIn], txout, use_max_sig)) {
    1559-            return false;
    1560+        CTxIn& txin = txNew.vin[nIn];
    1561+        // Use max sig if watch only inputs were used or if this particular input is an external input
    1562+        // to ensure a sufficient fee is attained for the requested feerate.
    1563+        bool use_max_sig = coin_control && (coin_control->fAllowWatchOnly || coin_control->IsExternalSelected(txin.prevout));
    


    jonatack commented at 11:27 am on November 13, 2020:

    e2c50aa

    0        const bool use_max_sig = coin_control && (coin_control->fAllowWatchOnly || coin_control->IsExternalSelected(txin.prevout));
    

    achow101 commented at 7:31 pm on January 12, 2021:
    Done
  84. in src/wallet/wallet.cpp:1607 in cd75670167 outdated
    1624         const auto mi = wallet->mapWallet.find(input.prevout.hash);
    1625         // Can not estimate size without knowing the input details
    1626-        if (mi == wallet->mapWallet.end()) {
    1627+        if (mi != wallet->mapWallet.end()) {
    1628+            assert(input.prevout.n < mi->second.tx->vout.size());
    1629+            txouts.emplace_back(mi->second.tx->vout[input.prevout.n]);
    


    jonatack commented at 11:37 am on November 13, 2020:

    e2c50aa do you want this to be lookup only?

    0            txouts.emplace_back(mi->second.tx->vout.at(input.prevout.n));
    

    achow101 commented at 7:31 pm on January 12, 2021:
    Done
  85. in src/wallet/wallet.cpp:2458 in cd75670167 outdated
    2480-            if (coin_selection_params.use_bnb) {
    2481-                value_to_select -= coin.effective_value;
    2482-            } else {
    2483-                value_to_select -= coin.txout.nValue;
    2484+            input_bytes = wtx.GetSpendSize(outpoint.n, false);
    2485+            txout = wtx.tx->vout[outpoint.n];
    


    jonatack commented at 11:51 am on November 13, 2020:

    e2c50aa do we want lookup only or also insertion?

    0            txout = wtx.tx->vout.at(outpoint.n);
    

    achow101 commented at 7:31 pm on January 12, 2021:
    Done
  86. in src/wallet/wallet.cpp:2451 in cd75670167 outdated
    2459@@ -2445,6 +2460,8 @@ bool CWallet::SelectCoins(const std::vector<COutput>& vAvailableCoins, const CAm
    2460     coin_control.ListSelected(vPresetInputs);
    2461     for (const COutPoint& outpoint : vPresetInputs)
    2462     {
    2463+        int input_bytes = -1;
    2464+        CTxOut txout;
    2465         std::map<uint256, CWalletTx>::const_iterator it = mapWallet.find(outpoint.hash);
    2466         if (it != mapWallet.end())
    2467         {
    


    jonatack commented at 12:13 pm on November 13, 2020:

    e2c50aa if you retouch, while here

    0-    for (const COutPoint& outpoint : vPresetInputs)
    1-    {
    2+    for (const COutPoint& outpoint : vPresetInputs) {
    3
    4...
    5
    6-        if (it != mapWallet.end())
    7-        {
    8+        if (it != mapWallet.end()) {
    

    achow101 commented at 7:31 pm on January 12, 2021:
    Done
  87. in src/wallet/rpcwallet.cpp:4487 in cd75670167 outdated
    4454+                            {"scripts", RPCArg::Type::ARR, /* default */ "empty array", "A json array of scripts.\n",
    4455+                                {
    4456+                                    {"script", RPCArg::Type::STR_HEX, RPCArg::Optional::OMITTED, "A script"},
    4457+                                },
    4458+                            },
    4459+                            {"descriptors", RPCArg::Type::ARR, /* default */ "empty array", "A json array of descriptors.\n",
    


    jonatack commented at 2:39 pm on November 13, 2020:

    6bb343c for each of the new help entries:

    • can you line break these two sentences and remove the extra line break at the end?
    0                    {"solving_data", RPCArg::Type::OBJ, RPCArg::Optional::OMITTED_NAMED_ARG, "Keys and scripts needed for producing a final transaction with a dummy signature.\n"
    1                        "Used for fee estimation during coin selection.",
    
    • could you omit all 6 of the “A json array of " text descriptions–the type “json array” is already displayed by the type declaration–and maybe use the gained space to better describe the entry, e.g. a recent help improvement in this file changed a json array of “Outputs” to “Outputs to subtract the fee from, specified as integer indices.”

    • please remove the “\n” from the end of each entry; this should only be added to intermediary lines, otherwise it causes extra line breaks in the output to be printed


    achow101 commented at 7:31 pm on January 12, 2021:
    Done
  88. in src/wallet/rpcwallet.cpp:4418 in cd75670167 outdated
    4384@@ -4317,7 +4385,9 @@ static RPCHelpMan walletcreatefundedpsbt()
    4385 {
    4386     return RPCHelpMan{"walletcreatefundedpsbt",
    4387                 "\nCreates and funds a transaction in the Partially Signed Transaction format.\n"
    4388-                "Implements the Creator and Updater roles.\n",
    4389+                "Implements the Creator and Updater roles.\n"
    4390+                "Note that all existing inputs must either have their previous output transaction be in the wallet\n"
    4391+                "or be in the utxo set. Solving data must be provided for non-wallet inputs.\n",
    


    jonatack commented at 2:44 pm on November 13, 2020:
    6bb343c s/Note that all/All/ and s/utxo/UTXO/

    achow101 commented at 7:31 pm on January 12, 2021:
    Done
  89. in src/wallet/rpcwallet.cpp:3254 in cd75670167 outdated
    3231@@ -3184,7 +3232,8 @@ static RPCHelpMan fundrawtransaction()
    3232                 "Note that inputs which were signed may need to be resigned after completion since in/outputs have been added.\n"
    3233                 "The inputs added will not be signed, use signrawtransactionwithkey\n"
    3234                 " or signrawtransactionwithwallet for that.\n"
    


    jonatack commented at 2:48 pm on November 13, 2020:
    6bb343c while making changes here, can you remove this extra space? E.g. s/ or/or/

    achow101 commented at 7:32 pm on January 12, 2021:
    Done
  90. in src/wallet/rpcwallet.cpp:4199 in cd75670167 outdated
    4195@@ -4128,7 +4196,7 @@ static RPCHelpMan send()
    4196             // Automatically select coins, unless at least one is manually selected. Can
    4197             // be overridden by options.add_inputs.
    4198             coin_control.m_add_inputs = rawTx.vin.size() == 0;
    4199-            FundTransaction(pwallet, rawTx, fee, change_position, options, coin_control);
    4200+            FundTransaction(pwallet, rawTx, fee, change_position, options, coin_control, NullUniValue);
    


    jonatack commented at 2:58 pm on November 13, 2020:

    6bb343c for easier code comprehension, here and the other FundTransaction calls

    0            FundTransaction(pwallet, rawTx, fee, change_position, options, coin_control, /* solving_data */ NullUniValue);
    

    achow101 commented at 7:32 pm on January 12, 2021:
    Done
  91. in src/wallet/rpcwallet.cpp:3325 in cd75670167 outdated
    3178+        if (solving_data.exists("descriptors")) {
    3179+            UniValue desc_strs = solving_data["descriptors"].get_array();
    3180+            for (unsigned int i = 0; i < desc_strs.size(); ++i) {
    3181+                FlatSigningProvider desc_out;
    3182+                std::string error;
    3183+                std::unique_ptr<Descriptor> desc = Parse(desc_strs[i].get_str(), desc_out, error, true);
    


    jonatack commented at 3:10 pm on November 13, 2020:
    6bb343c is desc used?

    achow101 commented at 7:32 pm on January 12, 2021:
    Made some fixes that use desc
  92. in src/wallet/rpcwallet.cpp:3340 in cd75670167 outdated
    3182+                std::string error;
    3183+                std::unique_ptr<Descriptor> desc = Parse(desc_strs[i].get_str(), desc_out, error, true);
    3184+                coinControl.m_external_provider = Merge(coinControl.m_external_provider, desc_out);
    3185+            }
    3186+        }
    3187+    }
    


    jonatack commented at 3:13 pm on November 13, 2020:

    6bb343c

     0     if (!solving_data.isNull()) {
     1         if (solving_data.exists("pubkeys")) {
     2-            UniValue pubkey_strs = solving_data["pubkeys"].get_array();
     3+            const UniValue pubkey_strs = solving_data["pubkeys"].get_array();
     4             for (unsigned int i = 0; i < pubkey_strs.size(); ++i) {
     5-                std::vector<unsigned char> data(ParseHex(pubkey_strs[i].get_str()));
     6+                const std::vector<unsigned char> data(ParseHex(pubkey_strs[i].get_str()));
     7                 CPubKey pubkey(data.begin(), data.end());
     8                 if (!pubkey.IsFullyValid()) {
     9                     throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, strprintf("%s is not a valid public key", pubkey_strs[i].get_str()));
    10                 }
    11                 coinControl.m_external_provider.pubkeys.emplace(pubkey.GetID(), pubkey);
    12-                // Add witnes script for pubkeys
    13-                CScript wit_script = GetScriptForDestination(WitnessV0KeyHash(pubkey));
    14+                // Add witness script for pubkeys
    15+                const CScript wit_script = GetScriptForDestination(WitnessV0KeyHash(pubkey));
    16                 coinControl.m_external_provider.scripts.emplace(CScriptID(wit_script), wit_script);
    17             }
    18         }
    19 
    20         if (solving_data.exists("scripts")) {
    21-            UniValue script_strs = solving_data["scripts"].get_array();
    22+            const UniValue script_strs = solving_data["scripts"].get_array();
    23             for (unsigned int i = 0; i < script_strs.size(); ++i) {
    24-                CScript script = ParseScript(script_strs[i].get_str());
    25+                const CScript script = ParseScript(script_strs[i].get_str());
    26                 coinControl.m_external_provider.scripts.emplace(CScriptID(script), script);
    27             }
    28         }
    29 
    30         if (solving_data.exists("descriptors")) {
    31-            UniValue desc_strs = solving_data["descriptors"].get_array();
    32+            const UniValue desc_strs = solving_data["descriptors"].get_array();
    

    achow101 commented at 7:32 pm on January 12, 2021:
    Done
  93. in src/wallet/rpcwallet.cpp:3361 in cd75670167 outdated
    3202@@ -3168,6 +3203,19 @@ void FundTransaction(CWallet* const pwallet, CMutableTransaction& tx, CAmount& f
    3203         setSubtractFeeFromOutputs.insert(pos);
    3204     }
    3205 
    3206+    // Fetch specified UTXOs from the UTXO set to get the scriptPubKeys and values of the outputs being selected
    3207+    // and to match with the given solving_data. Only used for non-wallet outputs.
    3208+    std::map<COutPoint, Coin> coins;
    


    jonatack commented at 3:15 pm on November 13, 2020:
    6bb343c add header #include <map>

    achow101 commented at 7:32 pm on January 12, 2021:
    Done
  94. in test/functional/rpc_fundrawtransaction.py:927 in cd75670167 outdated
    891+        # An external input without solving data should result in an error
    892+        raw_tx = wallet.createrawtransaction([{"txid": ext_utxo['txid'], "vout": ext_utxo['vout']}], {self.nodes[0].getnewaddress(): 15})
    893+        assert_raises_rpc_error(-4, "Insufficient funds", wallet.fundrawtransaction, raw_tx)
    894+
    895+        # But funding should work when the solving data is provided
    896+        funded_tx = wallet.fundrawtransaction(raw_tx, {}, False, {"pubkeys": [addr_info['pubkey']]})
    


    jonatack commented at 3:26 pm on November 13, 2020:
    cd7567016 in both rpc_fundrawtransaction.py and rpc_psbt.py, can you please add similar tests using a script and a descriptor, plus test the behavior with invalid ones of each, and a test that hits the “Missing solving data for estimating transaction size” case?

    achow101 commented at 7:32 pm on January 12, 2021:

    Done.

    The Missing solving data for estimating transaction size probably can’t actually be reached.

  95. jonatack commented at 3:37 pm on November 13, 2020: member
    Approach ACK cd75670167f. Feel free to pick and choose among the comments. Running clang-format on the changes could be helpful. I’d feel more comfortable if the test coverage mentioned below was added.
  96. in src/wallet/wallet.cpp:2471 in e2c50aa112 outdated
    2486+        }
    2487+        if (input_bytes == -1) {
    2488+            // The input is external. We either did not find the tx in mapWallet, or we did but couldn't compute the input size with wallet data
    2489+            if (!coin_control.GetExternalOutput(outpoint, txout)) {
    2490+                // Not ours, and we don't have solving data.
    2491+                return false;
    


    apoelstra commented at 1:50 am on November 14, 2020:

    This line will prevent you from adding the “Missing solving data for estimating transaction size” test that @jonatack suggests, since SelectCoins fails and the user will instead see “Insufficient funds”.

    One hacky fix may be to just return true here without selecting any coins, trusting that CreateTransaction will fail shortly thereafter. Proper fixes might be to have SelectCoins return a richer value (gross) or to use #20040 approach which moves this effective-value computation out of SelectCoins entirely.


    achow101 commented at 4:10 am on November 14, 2020:
    I have some ongoing work to refactor CreateTransaction and SelectCoins which will change some order here as well as the return values, so it might be better to push this (entire PR?) back to after that.
  97. DrahtBot added the label Needs rebase on Nov 17, 2020
  98. achow101 force-pushed on Nov 17, 2020
  99. DrahtBot removed the label Needs rebase on Nov 17, 2020
  100. in src/rpc/client.cpp:111 in 6a9730ebad outdated
    107@@ -108,11 +108,13 @@ static const CRPCConvertParam vRPCConvertParams[] =
    108     { "combinerawtransaction", 0, "txs" },
    109     { "fundrawtransaction", 1, "options" },
    110     { "fundrawtransaction", 2, "iswitness" },
    111+    { "fundrawtransaction", 3, "solving_data" },
    


    apoelstra commented at 4:19 am on November 29, 2020:
    I suppose you ought to add these to send from #16378 as well

    achow101 commented at 8:18 pm on January 12, 2021:
    I’ve added solving_data to send via options.
  101. achow101 force-pushed on Jan 12, 2021
  102. achow101 force-pushed on Jan 12, 2021
  103. achow101 force-pushed on Jan 12, 2021
  104. in src/wallet/rpcwallet.cpp:4709 in ac88a7942b outdated
    4705@@ -4587,7 +4706,7 @@ static const CRPCCommand commands[] =
    4706     { "wallet",             "signrawtransactionwithwallet",     &signrawtransactionwithwallet,  {"hexstring","prevtxs","sighashtype"} },
    4707     { "wallet",             "unloadwallet",                     &unloadwallet,                  {"wallet_name", "load_on_startup"} },
    4708     { "wallet",             "upgradewallet",                    &upgradewallet,                 {"version"} },
    4709-    { "wallet",             "walletcreatefundedpsbt",           &walletcreatefundedpsbt,        {"inputs","outputs","locktime","options","bip32derivs"} },
    4710+    { "wallet",             "walletcreatefundedpsbt",           &walletcreatefundedpsbt,        {"inputs","outputs","locktime","options","bip32derivs","solving_data"} },
    


    MarcoFalke commented at 6:43 pm on January 28, 2021:
    can remove the diffs for commands, and then rebase

    achow101 commented at 6:55 pm on January 28, 2021:
    Done
  105. achow101 force-pushed on Jan 28, 2021
  106. DrahtBot added the label Needs rebase on Feb 5, 2021
  107. achow101 force-pushed on Feb 5, 2021
  108. DrahtBot removed the label Needs rebase on Feb 5, 2021
  109. DrahtBot added the label Needs rebase on Mar 8, 2021
  110. achow101 force-pushed on Mar 10, 2021
  111. DrahtBot removed the label Needs rebase on Mar 10, 2021
  112. stevenroose referenced this in commit c7bf5baf96 on Mar 16, 2021
  113. DrahtBot added the label Needs rebase on Mar 17, 2021
  114. achow101 force-pushed on Mar 17, 2021
  115. DrahtBot removed the label Needs rebase on Mar 17, 2021
  116. DrahtBot added the label Needs rebase on Apr 19, 2021
  117. achow101 force-pushed on Apr 19, 2021
  118. DrahtBot removed the label Needs rebase on Apr 19, 2021
  119. DrahtBot added the label Needs rebase on Apr 26, 2021
  120. achow101 force-pushed on Apr 29, 2021
  121. DrahtBot removed the label Needs rebase on Apr 29, 2021
  122. DrahtBot added the label Needs rebase on May 10, 2021
  123. achow101 force-pushed on May 10, 2021
  124. DrahtBot removed the label Needs rebase on May 10, 2021
  125. DrahtBot added the label Needs rebase on May 25, 2021
  126. achow101 force-pushed on May 25, 2021
  127. DrahtBot removed the label Needs rebase on May 25, 2021
  128. ghost commented at 7:32 pm on May 25, 2021: none

    Concept ACK

    Compiled on Ubuntu. Not able to test because using wrong syntax or missing something else:

    0bitcoin-cli fundrawtransaction "0200000001f7739b1278e8e0df75a1f692fbface31b09d4dc7b4ddb2f20dec698e7617cd310000000000ffffffff0150c30000000000001600147c5ac840b455af9f9b3da2c327275c0b0e985e7600000000" {} false '{"pubkeys" : "02b1f9f65bdf8e686333be0ac8979908ae8aeeee33e396c446ba2b17348a32335c"}'
    1error code: -1
    2error message:
    3JSON value is not an array as expected
    
  129. DrahtBot added the label Needs rebase on May 26, 2021
  130. achow101 force-pushed on May 26, 2021
  131. achow101 commented at 4:39 pm on May 26, 2021: member

    Concept ACK

    Compiled on Ubuntu. Not able to test because using wrong syntax or missing something else:

    0bitcoin-cli fundrawtransaction "0200000001f7739b1278e8e0df75a1f692fbface31b09d4dc7b4ddb2f20dec698e7617cd310000000000ffffffff0150c30000000000001600147c5ac840b455af9f9b3da2c327275c0b0e985e7600000000" {} false '{"pubkeys" : "02b1f9f65bdf8e686333be0ac8979908ae8aeeee33e396c446ba2b17348a32335c"}'
    1error code: -1
    2error message:
    3JSON value is not an array as expected
    

    pubkeys is an array, not a string. The correct command would be:

    bitcoin-cli fundrawtransaction "0200000001f7739b1278e8e0df75a1f692fbface31b09d4dc7b4ddb2f20dec698e7617cd310000000000ffffffff0150c30000000000001600147c5ac840b455af9f9b3da2c327275c0b0e985e7600000000" {} false '{"pubkeys" : ["02b1f9f65bdf8e686333be0ac8979908ae8aeeee33e396c446ba2b17348a32335c"]}'
  132. DrahtBot removed the label Needs rebase on May 26, 2021
  133. ghost commented at 2:00 pm on May 28, 2021: none

    pubkeys is an array, not a string. The correct command would be:

    0bitcoin-cli fundrawtransaction "0200000001f7739b1278e8e0df75a1f692fbface31b09d4dc7b4ddb2f20dec698e7617cd310000000000ffffffff0150c30000000000001600147c5ac840b455af9f9b3da2c327275c0b0e985e7600000000" {} false '{"pubkeys" : ["02b1f9f65bdf8e686333be0ac8979908ae8aeeee33e396c446ba2b17348a32335c"]}'
    

    Thanks. That worked.

    0{
    1  "hex": "0200000001f7739b1278e8e0df75a1f692fbface31b09d4dc7b4ddb2f20dec698e7617cd310000000000ffffffff02637e0e000000000016001461c2e2d7dbcfcaa92b686fe90d0dd91870f6e0ce50c30000000000001600147c5ac840b455af9f9b3da2c327275c0b0e985e7600000000",
    2  "fee": 0.00000141,
    3  "changepos": 0
    4}
    
  134. unknown approved
  135. DrahtBot added the label Needs rebase on May 30, 2021
  136. achow101 force-pushed on May 30, 2021
  137. DrahtBot removed the label Needs rebase on May 30, 2021
  138. DrahtBot added the label Needs rebase on Jun 9, 2021
  139. achow101 force-pushed on Jun 9, 2021
  140. DrahtBot removed the label Needs rebase on Jun 9, 2021
  141. DrahtBot added the label Needs rebase on Jun 22, 2021
  142. achow101 force-pushed on Jun 22, 2021
  143. DrahtBot removed the label Needs rebase on Jun 23, 2021
  144. t-bast commented at 1:47 pm on August 12, 2021: member
    If a follow-up PR (or this one) can also enable external inputs in bumpfee and psbtbumpfee you would be my hero :pray:
  145. DrahtBot added the label Needs rebase on Aug 16, 2021
  146. achow101 force-pushed on Aug 17, 2021
  147. DrahtBot removed the label Needs rebase on Aug 18, 2021
  148. DrahtBot added the label Needs rebase on Sep 3, 2021
  149. achow101 force-pushed on Sep 3, 2021
  150. DrahtBot removed the label Needs rebase on Sep 3, 2021
  151. DrahtBot added the label Needs rebase on Sep 28, 2021
  152. achow101 force-pushed on Sep 28, 2021
  153. DrahtBot removed the label Needs rebase on Sep 29, 2021
  154. Allow CInputCoin to also be constructed with COutPoint and CTxOut a00eb388e8
  155. Allow Coin Selection be able to take external inputs d5cfb864ae
  156. achow101 force-pushed on Sep 29, 2021
  157. t-bast commented at 8:23 am on October 1, 2021: member

    Could you please clarify if something is blocking that PR from being merged? Does it depend on a separate feature or refactoring to be accepted? Or is it just a lack of reviews?

    This PR would be invaluable for L2 protocols (at least lightning and vaults), as they are currently forced to re-implement a bitcoin wallet internally to work around this limitation (which is quite a waste since bitcoin core already does it well!). This is particularly true for RBF, where L2 protocols currently must re-implement all the RBF logic outside of bitcoin core because transactions have external inputs, which is very sub-optimal.

    It would be very helpful to have this available in bitcoin core someday (and similar arguments for the bumpfee RPCs).

    I don’t have a good enough knowledge of the wallet code to provide a meaningful review, but I can help test this E2E from a lightning implementation when you feel it’s ready for that step.

  158. fanquake requested review from meshcollider on Oct 1, 2021
  159. fanquake requested review from instagibbs on Oct 1, 2021
  160. in src/wallet/rpcwallet.cpp:3297 in c37d244f61 outdated
    3289@@ -3289,6 +3290,47 @@ void FundTransaction(CWallet& wallet, CMutableTransaction& tx, CAmount& fee_out,
    3290         coinControl.fAllowWatchOnly = ParseIncludeWatchonly(NullUniValue, wallet);
    3291     }
    3292 
    3293+    if (!solving_data.isNull()) {
    3294+        if (solving_data.exists("pubkeys")) {
    3295+            const UniValue pubkey_strs = solving_data["pubkeys"].get_array();
    3296+            for (unsigned int i = 0; i < pubkey_strs.size(); ++i) {
    3297+                const std::vector<unsigned char> data(ParseHex(pubkey_strs[i].get_str()));
    


    meshcollider commented at 6:01 am on October 3, 2021:
    Check IsHex before calling ParseHex or use ParseHexV? (Here and below)

    achow101 commented at 4:57 pm on October 3, 2021:
    Done
  161. in src/wallet/rpcwallet.cpp:4370 in c37d244f61 outdated
    4366@@ -4291,7 +4367,7 @@ static RPCHelpMan send()
    4367             // Automatically select coins, unless at least one is manually selected. Can
    4368             // be overridden by options.add_inputs.
    4369             coin_control.m_add_inputs = rawTx.vin.size() == 0;
    4370-            FundTransaction(*pwallet, rawTx, fee, change_position, options, coin_control, /* override_min_fee */ false);
    4371+            FundTransaction(*pwallet, rawTx, fee, change_position, options, coin_control, /* override_min_fee */ false, NullUniValue /* solving_data */);
    


    meshcollider commented at 6:10 am on October 3, 2021:
    Nit: comment before the variable (here and elsewhere)? (Can’t remember if this is preferred style)


    achow101 commented at 4:57 pm on October 3, 2021:
    No longer relevant.
  162. in src/wallet/rpcwallet.cpp:4645 in c37d244f61 outdated
    4640+                                    {"script", RPCArg::Type::STR_HEX, RPCArg::Optional::OMITTED, "A script"},
    4641+                                },
    4642+                            },
    4643+                            {"descriptors", RPCArg::Type::ARR, RPCArg::Default{UniValue::VARR}, "Descriptors that provide solving data for this transaction.",
    4644+                                {
    4645+                                    {"descriptor", RPCArg::Type::STR_HEX, RPCArg::Optional::OMITTED, "A descriptor"},
    


    meshcollider commented at 6:11 am on October 3, 2021:
    This isn’t STR_HEX (same in fundrawtransaction and send)

    achow101 commented at 4:57 pm on October 3, 2021:
    Done
  163. meshcollider commented at 6:21 am on October 3, 2021: contributor
    Question: why not put solving_data in options for fundrawtransaction and walletcreatefundedpsbt like you did with send? Preferable to use that unless there’s a good reason?
  164. in src/wallet/rpcwallet.cpp:3297 in 37ebd04649 outdated
    3290@@ -3289,6 +3291,47 @@ void FundTransaction(CWallet& wallet, CMutableTransaction& tx, CAmount& fee_out,
    3291         coinControl.fAllowWatchOnly = ParseIncludeWatchonly(NullUniValue, wallet);
    3292     }
    3293 
    3294+    if (!solving_data.isNull()) {
    3295+        if (solving_data.exists("pubkeys")) {
    3296+            const UniValue pubkey_strs = solving_data["pubkeys"].get_array();
    3297+            for (unsigned int i = 0; i < pubkey_strs.size(); ++i) {
    


    yanmaani commented at 10:00 am on October 3, 2021:
    Nit: is there any reason not to use a C++-style loop here? It’s used further on on line 3359 and so on.

    achow101 commented at 4:57 pm on October 3, 2021:
    Done
  165. in src/wallet/rpcwallet.cpp:3312 in 37ebd04649 outdated
    3307+            }
    3308+        }
    3309+
    3310+        if (solving_data.exists("scripts")) {
    3311+            const UniValue script_strs = solving_data["scripts"].get_array();
    3312+            for (unsigned int i = 0; i < script_strs.size(); ++i) {
    


    yanmaani commented at 10:21 am on October 3, 2021:
    Ditto.
  166. in src/wallet/rpcwallet.cpp:3321 in 37ebd04649 outdated
    3316+            }
    3317+        }
    3318+
    3319+        if (solving_data.exists("descriptors")) {
    3320+            const UniValue desc_strs = solving_data["descriptors"].get_array();
    3321+            for (unsigned int i = 0; i < desc_strs.size(); ++i) {
    


    yanmaani commented at 10:21 am on October 3, 2021:
    Ditto.
  167. in test/functional/rpc_fundrawtransaction.py:1019 in 37ebd04649 outdated
    1014+        ext_utxo = self.nodes[0].listunspent(addresses=[addr])[0]
    1015+
    1016+        # An external input without solving data should result in an error
    1017+        raw_tx = wallet.createrawtransaction([ext_utxo], {self.nodes[0].getnewaddress(): 15})
    1018+        assert_raises_rpc_error(-4, "Insufficient funds", wallet.fundrawtransaction, raw_tx)
    1019+
    


    yanmaani commented at 10:23 am on October 3, 2021:
    Some error conditions are not tested, like transactions with no outputs, invalid descriptors, or invalid public keys.

    achow101 commented at 4:57 pm on October 3, 2021:
    Added tests for some failure conditions.
  168. yanmaani commented at 10:24 am on October 3, 2021: none
    utACK, very good initiative. Here are some minor comments.
  169. allow fundtx rpcs to work with external inputs 38f5642ccc
  170. achow101 force-pushed on Oct 3, 2021
  171. achow101 commented at 4:58 pm on October 3, 2021: member

    Question: why not put solving_data in options for fundrawtransaction and walletcreatefundedpsbt like you did with send? Preferable to use that unless there’s a good reason?

    Done

  172. Tests for funding with external inputs e39b5a5e7a
  173. allow send rpc take external inputs and solving data 928af61cdb
  174. achow101 force-pushed on Oct 3, 2021
  175. meshcollider commented at 8:25 pm on October 3, 2021: contributor

    Re-utACK 928af61cdb2c4de1c3d10e6fda13bbba5ca0bba9

    Only changes are those suggested above.

  176. in src/wallet/spend.cpp:477 in d5cfb864ae outdated
    488+            input_bytes = CalculateMaximumSignedInputSize(txout, &coin_control.m_external_provider, /* use_max_sig */ true);
    489+        }
    490+
    491+        CInputCoin coin(outpoint, txout, input_bytes);
    492+        nValueFromPresetInputs += coin.txout.nValue;
    493+        if (coin.m_input_bytes <= 0) {
    


    instagibbs commented at 1:10 am on October 4, 2021:
    nit: we should probably keep error checking conditions the same, input_bytes == -1 as per above
  177. in src/wallet/coincontrol.h:96 in 928af61cdb
    91     void Select(const COutPoint& output)
    92     {
    93         setSelected.insert(output);
    94     }
    95 
    96+    void Select(const COutPoint& outpoint, const CTxOut& txout)
    


    instagibbs commented at 1:16 am on October 4, 2021:
    Would rather it be named the more obvious SelectExternal for grepping
  178. instagibbs approved
  179. instagibbs commented at 1:19 am on October 4, 2021: member
    crACK 928af61cdb2c4de1c3d10e6fda13bbba5ca0bba9
  180. in src/wallet/rpcwallet.cpp:3295 in 928af61cdb
    3290@@ -3289,6 +3291,54 @@ void FundTransaction(CWallet& wallet, CMutableTransaction& tx, CAmount& fee_out,
    3291         coinControl.fAllowWatchOnly = ParseIncludeWatchonly(NullUniValue, wallet);
    3292     }
    3293 
    3294+    if (options.exists("solving_data")) {
    3295+        UniValue solving_data = options["solving_data"].get_obj();
    


    yanmaani commented at 4:50 am on October 4, 2021:
    Any reason for this not to be const?
  181. in src/wallet/rpcwallet.cpp:3303 in 928af61cdb
    3298+                const std::string& pk_str = pk_univ.get_str();
    3299+                if (!IsHex(pk_str)) {
    3300+                    throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, strprintf("'%s' is not hex", pk_str));
    3301+                }
    3302+                const std::vector<unsigned char> data(ParseHex(pk_str));
    3303+                CPubKey pubkey(data.begin(), data.end());
    


    yanmaani commented at 4:51 am on October 4, 2021:
    Ditto.
  182. ghost commented at 4:55 am on October 4, 2021: none
  183. in src/wallet/spend.cpp:816 in 928af61cdb
    810@@ -788,10 +811,10 @@ static bool CreateTransactionInternal(
    811     }
    812 
    813     // Calculate the transaction fee
    814-    TxSize tx_sizes = CalculateMaximumSignedTxSize(CTransaction(txNew), &wallet, coin_control.fAllowWatchOnly);
    815+    TxSize tx_sizes = CalculateMaximumSignedTxSize(CTransaction(txNew), &wallet, &coin_control);
    816     int nBytes = tx_sizes.vsize;
    817     if (nBytes < 0) {
    


    yanmaani commented at 4:57 am on October 4, 2021:
    nit: Shouldn’t that likewise be -1, then?
  184. in test/functional/rpc_fundrawtransaction.py:1020 in 928af61cdb
    1015+
    1016+        # An external input without solving data should result in an error
    1017+        raw_tx = wallet.createrawtransaction([ext_utxo], {self.nodes[0].getnewaddress(): 15})
    1018+        assert_raises_rpc_error(-4, "Insufficient funds", wallet.fundrawtransaction, raw_tx)
    1019+
    1020+        # Error conditions
    


    yanmaani commented at 5:02 am on October 4, 2021:
    I think you forgot “TX must have at least one output”?

    instagibbs commented at 1:24 pm on October 5, 2021:
    doesn’t seem related to the feature?
  185. yanmaani commented at 5:04 am on October 4, 2021: none
  186. meshcollider commented at 9:08 am on October 4, 2021: contributor
    Remaining review comments are non-blocking and can be left for a followup.
  187. meshcollider merged this on Oct 4, 2021
  188. meshcollider closed this on Oct 4, 2021

  189. sidhujag referenced this in commit 1f3bb2e4f6 on Oct 4, 2021
  190. apoelstra commented at 3:28 pm on October 4, 2021: contributor
    Thank you for merging this!!
  191. t-bast commented at 4:35 pm on October 4, 2021: member

    Thanks for the quick reviews and integration guys! I’m trying to test this E2E and I’m having issues though…

    I’m trying to call fundrawtransaction with a tx that spends an external p2wsh input. I’m providing the witness script in solving_data.scripts. But I just get an Insufficient funds (code: -4) error returned. Reading the code, it feels like I didn’t provide the right solving data, but what else should I provide then? Do I need to provide a complete witness stack with dummy signatures?

    Also, for my understanding, how does the size evaluation work at a high-level for p2wsh inputs? If I provide the witness script in solving_data.scripts that lets bitcoind know this part of the witness stack. But depending on the actual witness script, bitcoind cannot know how many other items will be pushed to the witness stack to satisfy the script, does it? Does bitcoind simply ignore that (which I can easily take into account by asking for a slightly bigger feerate)?

  192. achow101 commented at 4:54 pm on October 4, 2021: member

    @t-bast This feature currently only works for scripts that Bitcoin Core would be able to sign too, so you are limited to the set of standard scripts. You also need to provide pubkeys.

    We would need to be able to understand arbitrary scripts in order for this to work with any script. In order for that to happen, I think we will need Miniscript to be implemented.

    I suppose an alternative would be to allow the user to supply their own input size?

  193. t-bast commented at 4:58 pm on October 4, 2021: member

    Damn, ok, thanks for the clarification, that makes sense.

    I suppose an alternative would be to allow the user to supply their own input size?

    That would be perfect for apps that rely on more complex scripts (LN / vaults). As an external application, I know the size of my final witness stack, so if I could simply provide this size in the solving_data and have bitcoind ignore validation of my inputs, that would be great!

  194. in test/functional/rpc_fundrawtransaction.py:1027 in 928af61cdb
    1022+        assert_raises_rpc_error(-5, "'01234567890a0b0c0d0e0f' is not a valid public key", wallet.fundrawtransaction, raw_tx, {"solving_data": {"pubkeys":["01234567890a0b0c0d0e0f"]}})
    1023+        assert_raises_rpc_error(-5, "'not a script' is not hex", wallet.fundrawtransaction, raw_tx, {"solving_data": {"scripts":["not a script"]}})
    1024+        assert_raises_rpc_error(-8, "Unable to parse descriptor 'not a descriptor'", wallet.fundrawtransaction, raw_tx, {"solving_data": {"descriptors":["not a descriptor"]}})
    1025+
    1026+        # But funding should work when the solving data is provided
    1027+        funded_tx = wallet.fundrawtransaction(raw_tx, {"solving_data": {"pubkeys": [addr_info['pubkey']], "scripts": [addr_info["embedded"]["scriptPubKey"]]}})
    


    MarcoFalke commented at 9:21 am on October 5, 2021:

    This sometimes fails with “insufficient funds”. I wonder if the following diff fixes it:

     0diff --git a/test/functional/rpc_fundrawtransaction.py b/test/functional/rpc_fundrawtransaction.py
     1index f1215915a6..e92d5c8886 100755
     2--- a/test/functional/rpc_fundrawtransaction.py
     3+++ b/test/functional/rpc_fundrawtransaction.py
     4@@ -1009,13 +1009,13 @@ class RawTransactionsTest(BitcoinTestFramework):
     5         addr = self.nodes[0].deriveaddresses(desc)[0]
     6         addr_info = self.nodes[0].getaddressinfo(addr)
     7 
     8-        self.nodes[0].sendtoaddress(addr, 10)
     9-        self.nodes[0].sendtoaddress(wallet.getnewaddress(), 10)
    10+        self.nodes[0].sendtoaddress(addr, .10)
    11+        self.nodes[0].sendtoaddress(wallet.getnewaddress(), .10)
    12         self.nodes[0].generate(6)
    13         ext_utxo = self.nodes[0].listunspent(addresses=[addr])[0]
    14 
    15         # An external input without solving data should result in an error
    16-        raw_tx = wallet.createrawtransaction([ext_utxo], {self.nodes[0].getnewaddress(): 15})
    17+        raw_tx = wallet.createrawtransaction([ext_utxo], {self.nodes[0].getnewaddress(): .15})
    18         assert_raises_rpc_error(-4, "Insufficient funds", wallet.fundrawtransaction, raw_tx)
    19 
    20         # Error conditions
    

    katesalazar commented at 9:34 am on October 5, 2021:
    Forced push https://github.com/bitcoin/bitcoin/pull/23100/commits/86c66206bba0ce6cf1452fd77ceed971b4ab120c (https://github.com/bitcoin/bitcoin/pull/23100) on top of 446b7066, which doesn’t include this change added in e39b5a5e… 🤔

    meshcollider commented at 9:47 am on October 5, 2021:
    @MarcoFalke if I had to guess, that would fail due to paying extremely high fee?

    MarcoFalke commented at 9:49 am on October 5, 2021:
    fundrawtransaction adjusts the feerate by adjusting the change output

    katesalazar commented at 11:43 am on October 5, 2021:

    Forced push 86c6620 (#23100) on top of 446b706, which doesn’t include this change added in e39b5a5… 🤔

    Right now the only successful run of the last 15 or 20 runs is that one. 😄

  195. ryanofsky commented at 9:55 pm on October 5, 2021: member

    I’m seeing this new test_external_inputs test fail on windows in 3 different PRs with an “Insufficient funds” error. Maybe a bug in the new test?

    https://cirrus-ci.com/task/5958524342632448?logs=functional_tests#L2395 https://cirrus-ci.com/task/6233803560583168?logs=functional_tests#L2255 https://cirrus-ci.com/task/6423598886813696?logs=functional_tests#L2533

     0Traceback (most recent call last):
     1  File "C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\test\functional\test_framework\test_framework.py", line 132, in main
     2    self.run_test()
     3  File "C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\test\functional\rpc_fundrawtransaction.py", line 137, in run_test
     4    self.test_external_inputs()
     5  File "C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\test\functional\rpc_fundrawtransaction.py", line 1027, in test_external_inputs
     6    funded_tx = wallet.fundrawtransaction(raw_tx, {"solving_data": {"pubkeys": [addr_info['pubkey']], "scripts": [addr_info["embedded"]["scriptPubKey"]]}})
     7  File "C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\test\functional\test_framework\coverage.py", line 49, in __call__
     8    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
     9  File "C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\test\functional\test_framework\authproxy.py", line 144, in __call__
    10    raise JSONRPCException(response['error'], status)
    11test_framework.authproxy.JSONRPCException: Insufficient funds (-4)
    
  196. MarcoFalke referenced this in commit 66d11b1435 on Oct 6, 2021
  197. sidhujag referenced this in commit 98fe03bf77 on Oct 6, 2021
  198. fanquake referenced this in commit 99e53967c8 on Oct 8, 2021
  199. domob1812 referenced this in commit b59aa1cec2 on Oct 11, 2021
  200. domob1812 referenced this in commit fa1eef35b6 on Oct 12, 2021
  201. domob1812 referenced this in commit 9e0680acbf on Oct 18, 2021
  202. DrahtBot locked this on Oct 30, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-04 13:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me