wallet: Pass in transactions and messages for signing instead of exporting the private keys #18115

pull achow101 wants to merge 9 commits into bitcoin:master from achow101:sign-in-spkman changing 19 files +427 −270
  1. achow101 commented at 5:23 am on February 11, 2020: member

    Following #17261, the way to sign transactions, PSBTs, and messages was to use GetSigningProvider() and get a SigningProvider containing the private keys. However this may not be feasible for future ScriptPubKeyMans, such as for hardware wallets. Instead of exporting a SigningProvider containing private keys, we need to pass these things into the ScriptPubKeyMan (via CWallet) so that they can do whatever is needed internally to sign them. This is largely a refactor as the logic of processing transactions, PSBTs, and messages for is moved into LegacyScriptPubKeyMan and CWallet instead of being handled by the caller (e.g. signrawtransaction).

    To help with this, I’ve refactored the 3(!) implementations of a SignTransaction() function into one generic one. This function will be called by signrawtransactionwithkey and LegacyScriptPubKeyMan::SignTransaction(). CWallet::CreateTransaction() is changed to call CWallet::SignTransaction() which in turn, calls LegacyScriptPubKeyMan::SignTransaction(). Other ScriptPubKeyMans may implement SignTransaction() differently.

    FillPSBT() is moved to be a member function of CWallet and the psbtwallet.cpp/h files removed. It is further split so that CWallet handles filling the UTXOs while the ScriptPubKeyMan handles adding keys, derivation paths, scripts, and signatures. In the end LegacyScriptPubKeyMan::FillPSBT still calls SignPSBTInput, but the SigningProvider is internal to LegacyScriptPubKeyMan. Other ScriptPubKeyMans may do something different.

    A new SignMessage() function is added to both CWallet and ScriptPubKeyMan. Instead of having the caller (i.e. signmessage or the sign message dialog) get the private key, hash the message, and sign, ScriptPubKeyMan will now handle that (CWallet passes through to the ScriptPubKeyMans as it does for many functions). This signing code is thus consolidated into LegacyScriptPubKeyMan::SignMessage(), though other ScriptPubKeyMans may implement it differently. Additionally, a SigningError enum is introduced for the different errors that we expect to see from SignMessage().

    Lastly, GetSigningProvider() is renamed to GetPublicSigningProvider(). It will now only provide pubkeys, key origins, and scripts. LegacySigningProvider has it’s GetKey and HaveKey functions changed to only return false. Future implementations should return HidingSigningProviders where private keys are hidden.

    Other things like dumpprivkey and dumpwallet are not changed because they directly need and access the LegacyScriptPubKeyMan so are not relevant to future changes.

  2. fanquake added the label Wallet on Feb 11, 2020
  3. DrahtBot commented at 5:47 am on February 11, 2020: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #17938 (Disallow automatic conversion between disparate hash types by Empact)
    • #17211 (Allow fundrawtransaction and walletcreatefundedpsbt to take external inputs by achow101)
    • #16910 (wallet: reduce loading time by using unordered maps by achow101)
    • #16549 ([WIP] UI external signer support (e.g. hardware wallet) by Sjors)
    • #16546 ([WIP] External signer support - Wallet Box edition by Sjors)
    • #16528 (Native Descriptor Wallets using DescriptorScriptPubKeyMan by achow101)
    • #16440 (BIP-322: Generic signed message format by kallewoof)
    • #12911 (wallet: Show fee in results for signrawtransaction* for segwit inputs by kallewoof)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  4. achow101 force-pushed on Feb 11, 2020
  5. fanquake renamed this:
    [wallet] Pass in transactions and messages for signing instead of exporting the private keys
    wallet: Pass in transactions and messages for signing instead of exporting the private keys
    on Feb 12, 2020
  6. Sjors commented at 8:57 am on February 12, 2020: member
    @kallewoof may find this interesting too in light of BIP-322 (Generic signed message format) #16440
  7. in src/rpc/rawtransaction_util.h:27 in 0eb285f714 outdated
    23@@ -24,6 +24,7 @@ class SigningProvider;
    24  * @param result         JSON object where signed transaction results accumulate
    25  */
    26 void SignTransaction(CMutableTransaction& mtx, const SigningProvider* keystore, const std::map<COutPoint, Coin>& coins, const UniValue& hashType, UniValue& result);
    27+void MakeSignTransactionResult(CMutableTransaction& mtx, bool complete, const std::map<COutPoint, Coin>& coins, std::map<int, std::string>& input_errors, UniValue& result);
    


    Sjors commented at 9:20 am on February 12, 2020:

    In 0eb285f7146ae9beec3a19829cc4420ad9f251ae, this trips up AppVeyor:

    0:\projects\bitcoin\src\rpc\rawtransaction_util.h(27,132): error C2039: 'string': is not a member of 'std' [C:\projects\bitcoin\build_msvc\libbitcoin_common\libbitcoin_common.vcxproj]
    163C:\projects\bitcoin\src\rpc\rawtransaction_util.h(27,138): error C2065: 'string': undeclared identifier [C:\projects\bitcoin\build_msvc\libbitcoin_common\libbitcoin_common.vcxproj]
    264C:\projects\bitcoin\src\rpc\rawtransaction_util.h(27,118): error C2923: 'std::map': 'string' is not a valid template type argument for parameter '_Ty' [C:\projects\bitcoin\build_msvc\libbitcoin_common\libbitcoin_common.vcxproj]
    365C:\projects\bitcoin\src\rpc\rawtransaction_util.h(27,138): fatal error C1903: unable to recover from previous error(s); stopping compilation [C:\projects\bitcoin\build_msvc\libbitcoin_common\libbitcoin_common.vcxproj]
    

    Sjors commented at 9:40 am on February 12, 2020:
    Maybe rename it to SignTransactionResultToJSON().

    achow101 commented at 5:35 pm on February 12, 2020:
    Fixed (I think) and renamed
  8. in src/qt/signverifymessagedialog.cpp:146 in e4714e05ca outdated
    146+    QString error;
    147+    switch (err) {
    148+        case SigningError::OK:
    149+            error = tr("No error");
    150+        case SigningError::PRIVATE_KEY_NOT_AVAILABLE:
    151+            error = tr("Private key for the entered address is not available.");
    


    vasild commented at 9:34 am on February 12, 2020:
    Missing break;?

    Sjors commented at 10:31 am on February 12, 2020:
    Also these strings can be obtained fromSigningErrorString in error.cpp (the wording here seems better)

    achow101 commented at 5:20 pm on February 12, 2020:
    The strings from SigningErrorString won’t be translated (we don’t translate things for RPC), so that logic had to be duplicated here. Also, the strings are different to maintain compatibility.

    achow101 commented at 5:34 pm on February 12, 2020:
    Added the breaks in.
  9. in src/script/sign.cpp:497 in 0eb285f714 outdated
    492+
    493+        UpdateInput(txin, sigdata);
    494+
    495+        // amount must be specified for valid segwit signature
    496+        if (amount == MAX_MONEY && !txin.scriptWitness.IsNull()) {
    497+            input_errors[i] = "Missing amount";
    


    Sjors commented at 9:47 am on February 12, 2020:
    In 0eb285f7146ae9beec3a19829cc4420ad9f251ae this seems a bit brittle: input_errors[i] = "Missing amount"; (which is then matched by string to throw an exception later). But at least this error message is covered by rpc_rawtransaction.py

    achow101 commented at 5:07 pm on February 12, 2020:
    It is brittle, and I’m not sure why only that error specifically throws an exception. But keeping it for compatibility.
  10. vasild commented at 9:51 am on February 12, 2020: member
    https://github.com/achow101/bitcoin/blob/e4714e0/src/rpc/misc.cpp#L340 still calls the old signing code key.SignCompact(); directly. Maybe that needs to be updated to also use the newly added function SignMessage()?
  11. in src/script/sign.cpp:505 in 0eb285f714 outdated
    500+
    501+        ScriptError serror = SCRIPT_ERR_OK;
    502+        if (!VerifyScript(txin.scriptSig, spk, &txin.scriptWitness, STANDARD_SCRIPT_VERIFY_FLAGS, TransactionSignatureChecker(&tx_const, i, amount), &serror)) {
    503+            if (serror == SCRIPT_ERR_INVALID_STACK_OPERATION) {
    504+                // Unable to sign input and verification failed (possible attempt to partially sign).
    505+                input_errors[i] = "Unable to sign input, invalid stack size (possibly missing key)";
    


    Sjors commented at 9:52 am on February 12, 2020:
    This is a good time to add tests for the three error strings used here (SCRIPT_ERR_INVALID_STACK_OPERATION, SCRIPT_ERR_SIG_NULLFAIL, else). Not sure how though.
  12. in src/wallet/wallet.cpp:2428 in e496e29816 outdated
    2440-        nIn++;
    2441+bool CWallet::SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const
    2442+{
    2443+    // Sign the tx with ScriptPubKeyMans
    2444+    bool result = false;
    2445+    for (const auto& spk_man_pair : m_spk_managers) {
    


    Sjors commented at 10:06 am on February 12, 2020:
    In e496e2981699ac2938b74297d9b0bce355d26ad8 Implement CWallet::SignTransaction using ScriptPubKeyMan::SignTransaction at // Sign the tx with ScriptPubKeyMans, check CanProvide() first?

    achow101 commented at 6:56 pm on February 12, 2020:

    I’ve updated it to use GetScriptPubKeyMan which calls CanProvide

    This change is slightly more complex but it does reduce the number of repeated signings.

  13. in src/wallet/wallet.cpp:2429 in e496e29816 outdated
    2441+bool CWallet::SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const
    2442+{
    2443+    // Sign the tx with ScriptPubKeyMans
    2444+    bool result = false;
    2445+    for (const auto& spk_man_pair : m_spk_managers) {
    2446+        result |= spk_man_pair.second->SignTransaction(tx, coins, sighash, input_errors);
    


    Sjors commented at 10:08 am on February 12, 2020:

    The way input_errors are set, if there are multiple matching ScriptPubKeyMans* they would override each-others results. The last commit e4714e05ca9f3000384f9ed8f44e91a485f88bee clears the error for any given input, but only if SPKman SignTransaction() returns true, which requires all input errors to be empty, so that seems useless. At minimum this is confusing.

    * = not the case with legacy, and in practice shouldn’t happen with descriptor wallets, but that’s not enforced


    achow101 commented at 5:14 pm on February 12, 2020:
    The expectation is that all of the errors will be the same across all failed ScriptPubKeyMans because they don’t have the private keys. The ScriptPubKeyMan which fully signs an input will clear the error for that input. This clearing is not dependent on input_errors.empty().
  14. in src/wallet/wallet.cpp:2466 in a90a98c3aa outdated
    2486-        if (!provider) {
    2487-            complete = false;
    2488-            continue;
    2489+    // Fill in information from ScriptPubKeyMans
    2490+    for (const auto& spk_man_pair : m_spk_managers) {
    2491+        TransactionError res = spk_man_pair.second->FillPSBT(psbtx, sighash_type, sign, bip32derivs);
    


    Sjors commented at 10:20 am on February 12, 2020:
    In a90a98c3aa90ed4dc4f995fac0859e58f177a79a under // Fill in information from ScriptPubKeyMans, add CanProvide()?

    achow101 commented at 6:56 pm on February 12, 2020:

    I’ve updated it to use GetScriptPubKeyMan which calls CanProvide

    This change is slightly more complex but it does reduce the number of repeated signings.

  15. in src/wallet/scriptpubkeyman.cpp:526 in a90a98c3aa outdated
    521+        // Get the Sighash type
    522+        if (sign && input.sighash_type > 0 && input.sighash_type != sighash_type) {
    523+            return TransactionError::SIGHASH_MISMATCH;
    524+        }
    525+
    526+        // Get the scriptPubKey to know which SigningProvider to use
    


    Sjors commented at 10:26 am on February 12, 2020:
    In a90a98c3aa90ed4dc4f995fac0859e58f177a79a: this comment is outdated, because you’re no longer calling GetSigningProvider(script, sigdata). Also the CScript script below is not used.

    achow101 commented at 5:36 pm on February 12, 2020:
    Fixed the comment and simplified this block to just check for correct non_witness_utxo or continue if there is no utxo.
  16. in src/wallet/scriptpubkeyman.h:218 in e4714e05ca outdated
    216     virtual bool CanProvide(const CScript& script, SignatureData& sigdata) { return false; }
    217 
    218+    /** Creates new signatures and adds them to the transaction. Returns whether all inputs were signed */
    219+    virtual bool SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const { return false; }
    220+    /** Sign a message with the given script */
    221+    virtual SigningError SignMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) const { return SigningError::OK; };
    


    vasild commented at 10:35 am on February 12, 2020:

    It would be better to return some error from here (like the neighboring methods do). In case somebody forgets to override this method, it would be unexpected to return OK and leave the output str_sig unmodified.

    Maybe even use a pure virtual method ...str_sig) const = 0; to cause a compilation error if somebody forgets to override SignMessage(). I checked that:

    • it compiles if changed to pure virtual and
    • it does not compile if changed to pure virtual and the LegacyScriptPubKeyMan child class forgets to override SignMessage().

    achow101 commented at 5:35 pm on February 12, 2020:
    Changed it to return SigningError::SIGNING_FAILED. I’m avoiding pure virtual functions in ScriptPubKeyMan because there is no requirement for every ScriptPubKeyMan to implement every function.
  17. Sjors commented at 10:51 am on February 12, 2020: member

    Concept ACK. I roughly reviewed code for e4714e05ca9f3000384f9ed8f44e91a485f88bee and left some comments.

    0eb285f7146ae9beec3a19829cc4420ad9f251ae Refactor rawtransaction's SignTransaction into generic SignTransaction function would be easier to review (--color-moved=dimmed-zebra -w) if you renamed the variables in separate commit.

    It may be better to combine 4595326628bd4c91c753e923535178fca9ade2fd with be7991f63de2c0cce7d1e622a6ee12dd8f7204c9 so the signing code is moved, rather than just appear (though it’s short enough to compare manually).

    Lastly, GetSigningProvider() is renamed to GetPublicSigningProvider(). It will now only provide pubkeys, key origins, and scripts. LegacySigningProvider has it’s GetKey and HaveKey functions changed to only return false. Future implementations should return HidingSigningProviders where private keys are hidden.

    This bit is fairly confusing.

    This PR is fairly large, and only deduplicates a subset of @vasild’s #17577 (only what’s needed to move forward with the wallet refactor). I didn’t study this other PR in detail, but if it’s possible to build on top of that, it would split the review burden and remove more duplication.

  18. achow101 force-pushed on Feb 12, 2020
  19. achow101 force-pushed on Feb 12, 2020
  20. achow101 commented at 5:53 pm on February 12, 2020: member

    0eb285f Refactor rawtransaction's SignTransaction into generic SignTransaction function would be easier to review (--color-moved=dimmed-zebra -w) if you renamed the variables in separate commit.

    I’ve undone the renames.

    It may be better to combine 4595326 with be7991f so the signing code is moved, rather than just appear (though it’s short enough to compare manually).

    Squashed them together.

    Lastly, GetSigningProvider() is renamed to GetPublicSigningProvider(). It will now only provide pubkeys, key origins, and scripts. LegacySigningProvider has it’s GetKey and HaveKey functions changed to only return false. Future implementations should return HidingSigningProviders where private keys are hidden.

    This bit is fairly confusing.

    How so? I’m just saying that GetPublicSigningProvider() will just return a SigningProvider where GetKey() and HaveKey() will fail.

    This PR is fairly large, and only deduplicates a subset of @vasild’s #17577 (only what’s needed to move forward with the wallet refactor). I didn’t study this other PR in detail, but if it’s possible to build on top of that, it would split the review burden and remove more duplication.

    I’ll take a look at that.

    https://github.com/achow101/bitcoin/blob/e4714e0/src/rpc/misc.cpp#L340 still calls the old signing code key.SignCompact(); directly. Maybe that needs to be updated to also use the newly added function SignMessage()?

    Done

  21. achow101 force-pushed on Feb 12, 2020
  22. in src/util/error.h:37 in 01839f3797 outdated
    33@@ -34,6 +34,14 @@ enum class TransactionError {
    34 
    35 std::string TransactionErrorString(const TransactionError error);
    36 
    37+enum class SigningError {
    


    kallewoof commented at 10:52 am on February 13, 2020:
    SigningResult seems more friendly, considering this can be OK.

    achow101 commented at 1:23 am on February 14, 2020:
    Renamed to SigningResult
  23. kallewoof commented at 11:33 am on February 13, 2020: member

    Looked at code a bit, but not extensively.

    The SignMessage for each scriptpubman is a step backwards considering BIP322 will unify these all into a single place. It would be great if you reviewed that PR (if you already have, and still feel this is the right approach, I’d love to know why).

    I’ll review/test this PR as well and comment more generally but wanted to put this out there now.

  24. kallewoof commented at 1:08 pm on February 13, 2020: member

    I am trying to merge BIP322 #16440 on top of this, and am running into an issue demonstrated by the below patch:

     0diff --git a/src/wallet/rpcwallet.cpp b/src/wallet/rpcwallet.cpp
     1index bde8a6099..dde7819fa 100644
     2--- a/src/wallet/rpcwallet.cpp
     3+++ b/src/wallet/rpcwallet.cpp
     4@@ -559,6 +559,14 @@ static UniValue signmessage(const JSONRPCRequest& request)
     5     }
     6
     7     const PKHash *pkhash = boost::get<PKHash>(&dest);
     8+
     9+    auto provider = pwallet->GetPublicSigningProvider(GetScriptForDestination(dest));
    10+    auto keyid = GetKeyForDestination(*provider, dest);
    11+    CKey secret;
    12+    if (!provider->GetKey(keyid, secret)) {
    13+        assert(0);
    14+    }
    15+
    16     if (!pkhash) {
    17         throw JSONRPCError(RPC_TYPE_ERROR, "Address does not refer to key");
    18     }
    

    The above code hits the assert. I.e. despite getting a signing provider for the destination, it still fails to get the key. This code (i.e. getting signing provider and then getting privkey from that) in current master.

  25. achow101 commented at 3:40 pm on February 13, 2020: member

    The SignMessage for each scriptpubman is a step backwards considering BIP322 will unify these all into a single place. It would be great if you reviewed that PR (if you already have, and still feel this is the right approach, I’d love to know why).

    You can still have the message signing code unified to a single place, and in fact, this PR does that (it consolidates the signmessage RPC and the Sign Message dialog signing implementations). Each ScriptPubKeyMan can still call that single unified function from within it’s own SignMessage function. But each one having it’s own SignMessage also allows us to implement it differently depending on the ScriptPubKeyMan. In particular, for hardware wallets, we can’t use the unified SignMessage function because that would require producing a SigningProvider containing private keys, which is not possible for hardware wallets. In order to sign anything with hardware wallets, we need to pass in the thing to be signed to the device and let it handle the signing rather than signing it ourselves.

    This change means that a future HardwareScriptPubKeyMan can do exactly that: the message is passed into HardwareScriptPubKeyMan::SignMessage which then passes it along to the hardware wallet. This way, nothing external to the wallet needs to know that there is a hardware wallet or how to handle it. That can all be done within the HardwareScriptPubKeyMan.

    The above code hits the assert. I.e. despite getting a signing provider for the destination, it still fails to get the key. This code (i.e. getting signing provider and then getting privkey from that) in current master.

    That’s because you are still using the old code and way of signing messages. There is no need for signmessage to get a SigningProvider anymore. Instead you should be implementing either within the SignMessage function that is now in sign.cpp. You’ll probably need to change the arguments, but you shouldn’t need to go to the specific call sites like signmessage or the Sign Message dialog.

    Specifically the reason this hits the assert is that the SigningProvider that is returned only returns public keys and scripts. GetKey and HaveKey have been changed to always fail. Notice that the function named GetSigningProvider was renamed to GetPublicSigningProvider.

  26. achow101 force-pushed on Feb 13, 2020
  27. achow101 commented at 11:05 pm on February 13, 2020: member
    I’ve based this on top of #17577
  28. kallewoof commented at 4:03 am on February 14, 2020: member

    That’s because you are still using the old code and way of signing messages. There is no need for signmessage to get a SigningProvider anymore. Instead you should be implementing either within the SignMessage function that is now in sign.cpp. You’ll probably need to change the arguments, but you shouldn’t need to go to the specific call sites like signmessage or the Sign Message dialog.

    Ahh. In that case PublicSigningProvider is a bad name for it, I think? It’s not providing any signing at all, after all. I assume this is used to verify signatures? Maybe GetPublicKeyProvider.

  29. achow101 force-pushed on Feb 14, 2020
  30. achow101 commented at 6:07 pm on February 14, 2020: member

    Ahh. In that case PublicSigningProvider is a bad name for it, I think? It’s not providing any signing at all, after all. I assume this is used to verify signatures? Maybe GetPublicKeyProvider.

    It’s used primarily for checking solvability. So I’ve renamed it to GetSolvingProvider

  31. achow101 force-pushed on Feb 14, 2020
  32. Sjors commented at 7:50 pm on February 14, 2020: member
    GetSolutionProvider? :-P
  33. instagibbs commented at 5:13 pm on February 19, 2020: member
    concept ACK
  34. in src/script/sign.cpp:514 in a9f1c58ccd outdated
    509+            } else {
    510+                input_errors[i] = ScriptErrorString(serror);
    511+            }
    512+        }
    513+    }
    514+    return input_errors.empty();;
    


    Sjors commented at 4:19 pm on February 20, 2020:
    double semicolumn

    achow101 commented at 5:13 pm on February 21, 2020:
    Fixed
  35. in src/wallet/wallet.cpp:2474 in 2c6db38744 outdated
    2472+        if (spk_man->SignTransaction(tx, coins, sighash, input_errors)) {
    2473+            return true;
    2474+        }
    2475+
    2476+        // Add this spk_man to visited_spk_mans so we can skip it later
    2477+        visited_spk_mans.insert(spk_man->GetID());
    


    Sjors commented at 4:48 pm on February 20, 2020:
    Rather than having this visited_spk_mans bookkeeping, maybe just add spk_man->SignInput(tx, input, coins, sighash, error).

    achow101 commented at 4:18 pm on February 21, 2020:
    I considered that, but not all ScriptPubKeyMan implementations will be able to sign just one input without potentially signing all inputs. E.g. a hardware wallet spkman. With such spkmans, we would end up signing the entire tx multiple times which is both slow and bad UX.

    Sjors commented at 5:13 pm on February 21, 2020:
    That makes sense. It’s worth emphasizing in the comments that SignTransaction() will try to sign all inputs, not just txin.

    achow101 commented at 5:19 pm on February 21, 2020:
    I’ve added some comments.
  36. in src/wallet/wallet.cpp:2554 in 886002c66b outdated
    2547+        if (res != TransactionError::OK) {
    2548+            return res;
    2549         }
    2550+
    2551+        // Add this spk_man to visited_spk_mans so we can skip it later
    2552+        visited_spk_mans.insert(spk_man->GetID());
    


    Sjors commented at 5:20 pm on February 20, 2020:

    Same suggestion as with regular signing: adding FillPSBT(PartiallySignedTransaction& psbt, int& index, ... to LegacyScriptPubKeyMan() would avoid this bookkeeping and be easier to reason about than this loop-in-loop behavior.

    Such per input control could also make its way into the RPC, where e.g. coinjoin software could tell the wallet what to sign while ignoring the other inputs. That might save some (negligible?) time doing spk_man lookups on a thousand inputs.


    Sjors commented at 5:14 pm on February 21, 2020:
    Nvm, see #18115 (review)
  37. in src/wallet/scriptpubkeyman.cpp:564 in 886002c66b outdated
    537+        SignPSBTInput(HidingSigningProvider(this, !sign, !bip32derivs), psbtx, i, sighash_type);
    538+    }
    539+
    540+    // Fill in the bip32 keypaths and redeemscripts for the outputs so that hardware wallets can identify change
    541+    for (unsigned int i = 0; i < psbtx.tx->vout.size(); ++i) {
    542+        UpdatePSBTOutput(HidingSigningProvider(this, true, !bip32derivs), psbtx, i);
    


    Sjors commented at 5:35 pm on February 20, 2020:
    I tested that BIP32 info is correctly added to watch-only wallets (see also my comment #16528 (review) and earlier(?) version of this code).
  38. Sjors commented at 7:51 pm on February 20, 2020: member

    ACK ba91cc878ad18e457c79ca3f84be3b53f1182887 (pending parent PR)

    I made a commit that introduces SolvingProvider as the SigningProvider base class, which I think makes things more clear. But it’s lot of code changes: https://github.com/Sjors/bitcoin/commit/7f97d83ccbc291d0a69c7c8aa5b8f9d217d6badf

  39. instagibbs commented at 4:42 pm on February 21, 2020: member
    note that this PR is built on https://github.com/bitcoin/bitcoin/pull/17577
  40. achow101 force-pushed on Feb 21, 2020
  41. achow101 force-pushed on Feb 21, 2020
  42. in src/rpc/rawtransaction_util.cpp:288 in 5cab890b91 outdated
    323+{
    324+    // Make errors UniValue
    325+    UniValue vErrors(UniValue::VARR);
    326+    for (const auto& err_pair : input_errors) {
    327+        if (err_pair.second == "Missing amount") {
    328+            // This particular error needs to be an exception for some reason
    


    instagibbs commented at 3:05 pm on February 24, 2020:
    This comment isn’t really helpful for future code readers, just reviewers.

    ProofOfKeags commented at 7:33 pm on February 27, 2020:
    Is the reason known, or is this a “rain dance” and all we know is that it doesn’t work if it isn’t an Exception?

    achow101 commented at 7:42 pm on February 27, 2020:
    It’s preserving previous behavior of the API which would throw an exception for that particular error instead of putting it into the errors array.
  43. in src/wallet/wallet.h:924 in 1bb7939d3e outdated
    915@@ -916,7 +916,8 @@ class CWallet final : public WalletStorage, private interfaces::Chain::Notificat
    916      * calling CreateTransaction();
    917      */
    918     bool FundTransaction(CMutableTransaction& tx, CAmount& nFeeRet, int& nChangePosInOut, std::string& strFailReason, bool lockUnspents, const std::set<int>& setSubtractFeeFromOutputs, CCoinControl);
    919-    bool SignTransaction(CMutableTransaction& tx) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
    920+    bool SignTransaction(CMutableTransaction& tx) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
    921+    bool SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const;
    


    instagibbs commented at 3:10 pm on February 24, 2020:
    Please document when these two functions are used.

    achow101 commented at 5:15 pm on February 24, 2020:
    Added a comment.
  44. in src/wallet/wallet.h:1179 in 1bb7939d3e outdated
    1150@@ -1150,6 +1151,7 @@ class CWallet final : public WalletStorage, private interfaces::Chain::Notificat
    1151 
    1152     //! Get the ScriptPubKeyMan for a script
    1153     ScriptPubKeyMan* GetScriptPubKeyMan(const CScript& script) const;
    1154+    ScriptPubKeyMan* GetScriptPubKeyMan(const CScript& script, SignatureData& sigdata) const;
    


    instagibbs commented at 3:10 pm on February 24, 2020:
    Please document how this is used.

    achow101 commented at 5:18 pm on February 24, 2020:
    Added a comment.
  45. in src/wallet/wallet.cpp:2455 in 1bb7939d3e outdated
    2455         }
    2456-        UpdateInput(input, sigdata);
    2457-        nIn++;
    2458+
    2459+        // Input needs to be signed, find the right ScriptPubKeyMan
    2460+        ScriptPubKeyMan* spk_man = GetScriptPubKeyMan(coin->second.out.scriptPubKey, sigdata);
    


    instagibbs commented at 3:21 pm on February 24, 2020:
    Will one SPKM always be sufficient for any input even going forward? Seems to me the same SPKM will always be sourced for a particular input?

    instagibbs commented at 3:44 pm on February 24, 2020:
    5cab890b915fd9f25cada7ddab7cc00eea22c80b suggests no, there may be multiple.

    achow101 commented at 4:54 pm on February 24, 2020:

    The expectation is that one spkman will be sufficient. But I suppose it is possible that multiple spkmans could sign for a particular input. I don’t think we handle that case.

    5cab890 was from a previous iteration where all spkmans signed the transaction. Perhaps it isn’t needed anymore.


    instagibbs commented at 4:57 pm on February 24, 2020:
    I think it makes sense to remove that commit then, and just state the assumption here.

    achow101 commented at 5:14 pm on February 24, 2020:
    Dropped the commit. Added a comment about the assumption.

    achow101 commented at 5:43 pm on February 25, 2020:

    I’ve had to restore the commit. It actually is needed. It actually doesn’t imply that the different ScriptPubKeyMans can successfully sign the same input.

    It is still needed because each time the wrong ScriptPubKeyMan signs an input, it will add an error. Once that input becomes signed by the correct ScriptPubKeyMan, we need to clear away those errors otherwise they will incorrectly persist. Thus that commit is still needed.


    Sjors commented at 6:52 pm on February 25, 2020:

    Shouldn’t this loop over all matching GetScriptPubKeyMans? Otherwise the comment above should be changed to “We assume that each input is matched (and signed) by only one ScriptPubKeyMan.”.

    Right now:

    • if non-matching ScriptPubKeyMan tries and fails to sign this input, we’ll try with the matching ScriptPubKeyMan, but;
    • if we match, but can’t sign, that error can only be cleared if another input happens to match a ScriptPubKeyMan that then happens to be able for the failed input as well.

    In practice this shouldn’t matter because two ScriptPubKeyMans should never overlap, but we don’t have a way to enforce that.


    achow101 commented at 7:29 pm on February 25, 2020:
    Fine. I’ve changed it to check all ScriptPubKeyMans that CanProvide.
  46. in src/wallet/wallet.cpp:2542 in c5ae320bc6 outdated
    2533             continue;
    2534         }
    2535 
    2536-        complete &= SignPSBTInput(HidingSigningProvider(provider.get(), !sign, !bip32derivs), psbtx, i, sighash_type);
    2537-    }
    2538+        // If we've already been signed by this spk_man, skip it
    


    instagibbs commented at 3:37 pm on February 24, 2020:
    Will one SPKM always be sufficient for any input even going forward? Seems to me the same SPKM will always be sourced for a particular input?

    achow101 commented at 5:14 pm on February 24, 2020:
    Documented the assumption.
  47. in src/wallet/wallet.h:1185 in f2bd919cfd outdated
    1177@@ -1178,8 +1178,8 @@ class CWallet final : public WalletStorage, private interfaces::Chain::Notificat
    1178     ScriptPubKeyMan* GetScriptPubKeyMan(const uint256& id) const;
    1179 
    1180     //! Get the SigningProvider for a script
    1181-    std::unique_ptr<SigningProvider> GetSigningProvider(const CScript& script) const;
    1182-    std::unique_ptr<SigningProvider> GetSigningProvider(const CScript& script, SignatureData& sigdata) const;
    1183+    std::unique_ptr<SigningProvider> GetSolvingProvider(const CScript& script) const;
    


    instagibbs commented at 3:43 pm on February 24, 2020:
    A little confused by this commit. Should SigningProvider itself be renamed as well?

    achow101 commented at 4:55 pm on February 24, 2020:
    We should probably refactor SigningProvider at some point to have a parent SolvingProvider that doesn’t have private keys. But for now, this just returns a SigningProvider without private keys.

    Sjors commented at 5:05 pm on February 24, 2020:
    I have commit for exactly that, see my earlier comment: #18115#pullrequestreview-362040407
  48. instagibbs approved
  49. instagibbs commented at 3:49 pm on February 24, 2020: member
    reviewed to 5cab890b915fd9f25cada7ddab7cc00eea22c80b , just a few points, otherwise looks good
  50. achow101 force-pushed on Feb 24, 2020
  51. achow101 force-pushed on Feb 24, 2020
  52. DrahtBot added the label Needs rebase on Feb 25, 2020
  53. achow101 force-pushed on Feb 25, 2020
  54. DrahtBot removed the label Needs rebase on Feb 25, 2020
  55. Sjors commented at 6:58 pm on February 25, 2020: member
    utACK b26777cb7b3fa59c4b518d953165fdb80ca444a9, modulo the below issue
  56. achow101 force-pushed on Feb 25, 2020
  57. Sjors commented at 7:41 pm on February 25, 2020: member
    utACK 1f9a806a0184ebc4fe7b7b99df6341e0a00fef80
  58. in src/script/sign.cpp:514 in 1f9a806a01 outdated
    508@@ -509,6 +509,9 @@ bool SignTransaction(CMutableTransaction& mtx, const SigningProvider* keystore,
    509             } else {
    510                 input_errors[i] = ScriptErrorString(serror);
    511             }
    512+        } else {
    513+            // If this input succeeds, make sure there is no error set for it
    514+            input_errors.erase(i);
    


    instagibbs commented at 8:24 pm on February 25, 2020:

    Help me understand:

    1. Two SPKM: A and B
    2. Two input tx: input 1: key resides in A input 2: key resides in B
    3. A asked to sign 1, successfully
    4. A asked to sign 2, unsuccessful, appends error C
    5. B asked to sign 1, unsuccessful, appends error D
    6. B asked to sign 2, successful, deletes C

    We’re left with error D?


    achow101 commented at 8:26 pm on February 25, 2020:
    1. B asked to sign 1. 1 comes with existing signature. Skips signing because already signed, no error.

    instagibbs commented at 8:34 pm on February 25, 2020:
    CWallet::SignTransaction outer loop over inputs has the complete check, not the inner SPKM loop?

    Sjors commented at 8:50 pm on February 25, 2020:

    @achow101 CWallet skips B when considering input 1, because it’s already signed. But then when it’s considering input 2 it will tell B to sign the transaction, and the sign code doesn’t seem skip already signed inputs.

    It should be easy to reproduce this scenario in a test on the descriptor wallet PR. E.g. spend from a wallet with one legacy and one native SegWit UTXO.


    achow101 commented at 10:13 pm on February 25, 2020:

    To be more specific, the generic SignTransaction function uses DataFromTransaction on each input to pull out any existing signatures, pubkeys, and scripts into a SignatureData object. When this SignatureData object gets passed into ProduceSignature, ProduceSignature first looks at the SignatureData finds an existing signatures, pubkeys, and scripts, and fills those into the transaction rather than asking for the SigningProvider (which is produced by the ScriptPubKeyMan). So even if that SigningProvider can’t sign, because we already had signatures, pubkeys, and scripts in that transaction, no error occurs.

    There is an additional optimization already in ProduceSignature which skips signing when it finds that SignatureData.complete == true, which is determined by doing a VerifyScript within DataFromTransaction. Hence signing is skipped for that input.

    All of this is pre-existing code which allows us to sign a transaction with external inputs without error. The same process that we do internally with multiple ScriptPubKeyMans is almost the same as if you signed a transaction that has inputs from multiple wallets with those different wallets. Errors are not given for the inputs that are already signed.


    instagibbs commented at 10:17 pm on February 25, 2020:
    Ok, I agree with what you’re saying, though it’s indirect.

    Sjors commented at 8:39 am on February 26, 2020:
    That makes sense and will do for now. Would be good to add a test to #15764 for this type of stuff, because it seems brittle.

    achow101 commented at 6:00 pm on February 26, 2020:
    The nature of descriptor wallets as implemented in #16528 requires this commit or tests fail. We are already testing this case. I added back this commit because tests were failing without it in #16528.
  59. instagibbs commented at 10:19 pm on February 25, 2020: member

    utACK https://github.com/bitcoin/bitcoin/pull/18115/commits/1f9a806a0184ebc4fe7b7b99df6341e0a00fef80

    Though I wish the code regarding the last commit was easier to grok about.

  60. laanwj added this to the "Blockers" column in a project

  61. fanquake requested review from meshcollider on Feb 28, 2020
  62. in src/wallet/wallet.cpp:3030 in f4b3fbd70b outdated
    2940-                } else {
    2941-                    UpdateInput(txNew.vin.at(nIn), sigdata);
    2942-                }
    2943-
    2944-                nIn++;
    2945+            if (!SignTransaction(txNew)) {
    


    Empact commented at 11:28 pm on February 28, 2020:
    nit: Might also collapse the outer if (sign)

    achow101 commented at 0:35 am on February 29, 2020:
    Will collapse if rebase or other changes are needed.

    achow101 commented at 4:27 pm on March 8, 2020:
    had to rebase, collapsed.
  63. in src/primitives/transaction.h:131 in b525f73852 outdated
    123@@ -124,6 +124,11 @@ class CTxIn
    124         return !(a == b);
    125     }
    126 
    127+    friend bool operator<(const CTxIn& a, const CTxIn& b)
    128+    {
    129+        return a.prevout < b.prevout;
    130+    }
    131+
    


    Empact commented at 11:28 pm on February 28, 2020:
    nit: this isn’t required for this commit - maybe move it to later where it is required?

    achow101 commented at 0:34 am on February 29, 2020:
    It looks like this isn’t needed at all… But I would like to avoid invalidating ACKs, so I’ll leave it in unless I have to rebase or make other changes.

    achow101 commented at 4:27 pm on March 8, 2020:
    Had to rebase, removed
  64. in src/qt/signverifymessagedialog.cpp:143 in 1f9a806a01 outdated
    146+    SigningResult res = model->wallet().signMessage(message, *pkhash, signature);
    147+
    148+    QString error;
    149+    switch (res) {
    150+        case SigningResult::OK:
    151+            error = tr("No error");
    


    Empact commented at 11:52 pm on February 28, 2020:
    nit: unnecessary assignment

    achow101 commented at 0:26 am on February 29, 2020:
    Meh. Just in case some other change results in error being shown.
  65. Empact approved
  66. achow101 force-pushed on Mar 8, 2020
  67. Refactor rawtransaction's SignTransaction into generic SignTransaction function 2c52b59d0a
  68. Add SignTransaction function to ScriptPubKeyMan and LegacyScriptPubKeyMan d999dd588c
  69. Implement CWallet::SignTransaction using ScriptPubKeyMan::SignTransaction f37de92744
  70. Use CWallet::SignTransaction in CreateTransaction and signrawtransactionwithwallet
    Instead of duplicating signing code, just use the function we already
    have.
    a4af324d15
  71. achow101 force-pushed on Mar 8, 2020
  72. achow101 commented at 4:28 pm on March 8, 2020: member
    There was a hidden merge conflict that required a rebase.
  73. achow101 force-pushed on Mar 8, 2020
  74. achow101 force-pushed on Mar 8, 2020
  75. meshcollider commented at 7:51 am on March 9, 2020: contributor

    Light code review / build / test run ACK 08b9b68528b4644bce6bb925d6b0cbcaae62c5a9

    If Empact, instagibbs, or Sjors could post-rebase re-ACK I’ll merge this

  76. Sjors commented at 10:38 am on March 9, 2020: member

    Will do after @achow101 fixes the Travis build failure :-) https://travis-ci.org/bitcoin/bitcoin/jobs/659844515#L2418

    Marking fillPSBT() as const in src/interfaces/wallet.{h,cpp} might help, and would at least be consistent.

  77. Move FillPSBT to be a member of CWallet 3d70dd99f9
  78. Move key and script filling and signing from CWallet::FillPSBT to ScriptPubKeyMan::FillPSBT
    Instead of fetching a SigningProvider from ScriptPubKeyMan in order
    to fill and sign the keys and scripts for a PSBT, just pass that
    PSBT to a new FillPSBT function that does all that for us.
    82a30fade7
  79. Move direct calls to MessageSign into new SignMessage functions in CWallet and ScriptPubKeyMan
    Instead of getting a SigningProvider and then going to MessageSign,
    have ScriptPubKeyMan handle the message signing internally.
    6a9c429084
  80. Replace GetSigningProvider with GetSolvingProvider
    Not all ScriptPubKeyMans will be able to provide private keys,
    but pubkeys and scripts should be. So only provide public-only
    SigningProviders, i.e. ones that can help with Solving.
    dc174881ad
  81. Clear any input_errors for an input after it is signed
    Make sure that there are no errors set for an input after it is signed.
    This is useful for when there are multiple ScriptPubKeyMans. Some may
    fail to sign, but one may be able to sign, and after it does, we don't
    want there to be any more errors there.
    d2774c09cf
  82. achow101 force-pushed on Mar 9, 2020
  83. achow101 commented at 3:29 pm on March 9, 2020: member

    Will do after @achow101 fixes the Travis build failure :-) https://travis-ci.org/bitcoin/bitcoin/jobs/659844515#L2418

    Marking fillPSBT() as const in src/interfaces/wallet.{h,cpp} might help, and would at least be consistent.

    I did that, but I don’t think it will help. I’m not sure what would cause that travis failure; it’s failing on something that the other targets are compiling too. I think travis may have gotten confused?

  84. Sjors commented at 6:15 pm on March 9, 2020: member
    re-utACK d2774c09cfcc6c5c967d40bb094eabc8c0bdb6bf
  85. meshcollider commented at 7:56 pm on March 9, 2020: contributor
    re-utACK d2774c09cfcc6c5c967d40bb094eabc8c0bdb6bf
  86. meshcollider merged this on Mar 9, 2020
  87. meshcollider closed this on Mar 9, 2020

  88. sidhujag referenced this in commit 98c8a84b4c on Mar 10, 2020
  89. laanwj removed this from the "Blockers" column in a project

  90. sidhujag referenced this in commit 248ed9dbbc on Mar 28, 2020
  91. jasonbcox referenced this in commit dabfedaea0 on Oct 24, 2020
  92. jasonbcox referenced this in commit 588d25004d on Oct 24, 2020
  93. jasonbcox referenced this in commit dd2c6cebcd on Oct 24, 2020
  94. jasonbcox referenced this in commit 27fe47e7af on Oct 24, 2020
  95. jasonbcox referenced this in commit 7f2483ac94 on Oct 25, 2020
  96. jasonbcox referenced this in commit f0ce817308 on Oct 25, 2020
  97. jasonbcox referenced this in commit bee3cb09b3 on Oct 25, 2020
  98. jasonbcox referenced this in commit 67435e511a on Oct 25, 2020
  99. jasonbcox referenced this in commit 206346fce4 on Oct 25, 2020
  100. sidhujag referenced this in commit cf35de8d51 on Nov 10, 2020
  101. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 13:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me