Refactor PSBT signing logic to enforce invariant and fix signing bug #14588

pull gwillen wants to merge 7 commits into bitcoin:master from gwillen:feature-psbt-sign-fix changing 7 files +80 −68
  1. gwillen commented at 2:11 am on October 27, 2018: contributor

    As discussed in the comments on #14473, I think that bug was caused primarily by failure to adhere to the invariant that a PSBTInput always has exactly one of the two utxo fields present – an invariant that is already enforced by PSBTInput::IsSane, but which we were temporarily suspending during signing.

    This refactor repairs the invariant, also fixing the bug. It also simplifies some other code, and removes redundant parameters from some related functions.

    fixes #14473

  2. gwillen commented at 2:11 am on October 27, 2018: contributor
    @achow101: As original author of most of this code, you should look at this and make sure I have not done anything crazy (and that you agree that by and large my changes are improvements.)
  3. gwillen commented at 2:13 am on October 27, 2018: contributor
    Arguably, we may want to call PartiallySignedTransaction::IsSane as the first step in every PSBT RPC, so we can give a thoughtful error message if it fails (or at least an error message.)
  4. fanquake added the label Wallet on Oct 27, 2018
  5. in src/rpc/rawtransaction.cpp:1543 in e6277c90f6 outdated
    1541     bool complete = true;
    1542     for (unsigned int i = 0; i < psbtx.tx->vin.size(); ++i) {
    1543         PSBTInput& input = psbtx.inputs.at(i);
    1544-
    1545-        complete &= SignPSBTInput(DUMMY_SIGNING_PROVIDER, *psbtx.tx, input, i, 1);
    1546+        complete &= PSBTInputSigned(input);
    


    sipa commented at 2:18 am on October 27, 2018:
    I think the SignPSBTInput call here was necessary to perform finalization (constructing the actual scriptSig/witness; there is no guarantee that the previous step actually did this, for example if it was a combiner that didn’t understand the script).

    achow101 commented at 3:50 am on October 27, 2018:
    Agree with @sipa. SignPSBTInput is what does the finalization.

    gwillen commented at 6:39 pm on October 29, 2018:
    Aha, thanks, not sure how I misunderstood that so badly, it makes much more sense to me now. I will change it back and add a comment.
  6. in src/wallet/rpcwallet.cpp:3759 in f22df4eb3d outdated
    3757-        if (sign && input.sighash_type > 0 && input.sighash_type != sighash_type) {
    3758-            throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "Specified Sighash and sighash in PSBT do not match.");
    3759+        // Get the sighash_type (if we're signing).
    3760+        if (sign) {
    3761+            if (input.sighash_type == 0) {
    3762+                input.sighash_type = sighash_type;
    


    sipa commented at 2:21 am on October 27, 2018:
    Will this sighash_type value get serialized into the PSBT? If so, I believe that’s incorrect; if no sighash type is specified in the PSBT file, signers can choose their own, but the choice of one signer shouldn’t affect the type used by future/other signers.

    gwillen commented at 6:38 pm on October 29, 2018:
    Ahh, I see. Ok, will fix, thanks.
  7. DrahtBot commented at 5:47 am on October 27, 2018: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #14380 (fix assert crash when specified change output spend size is unknown by instagibbs)
    • #13932 (Additional utility RPCs for PSBT by achow101)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  8. gwillen commented at 6:40 pm on October 29, 2018: contributor

    @achow101 I think this probably supersedes #14197? It has 14197’s effect as a side-effect, but having done so, that also allows removing all the code to handle having both types of utxos at the same time (since the non_witness_utxo is now sufficient.)

    If you agree, I would like to steal your test from 14197 though. :-)

  9. achow101 commented at 4:21 am on October 30, 2018: member
    @gwillen Go ahead.
  10. gwillen force-pushed on Oct 30, 2018
  11. gwillen force-pushed on Oct 30, 2018
  12. gwillen force-pushed on Oct 30, 2018
  13. gwillen commented at 8:17 am on October 30, 2018: contributor

    Ok: Fixed @sipa’s issues, stole @achow101’s test, also added a regression test for #14473, verified that it fails on master and passes here.

    Please take another look! :-)

  14. DrahtBot added the label Needs rebase on Nov 1, 2018
  15. More concise conversion of CDataStream to string
    Use .str() instead of .data() and .size() when converting CDataStream to
    a string. Uses std::string, avoiding conversion to a C string.
    fe5d22bc67
  16. Remove redundant txConst parameter to FillPSBT 4f3f5cb4b1
  17. New PartiallySignedTransaction constructor from CTransction
    New constructor that creates a PartiallySignedTransaction from a
    CTransaction, automatically sizing the inputs and outputs vectors for
    convenience.
    65166d4cf8
  18. Add bool PSBTInputSigned
    Refactor out a "PSBTInputSigned" function to check if a PSBT is signed,
    for use in subsequent commits.
    
    Also improve a related comment.
    53e6fffb8f
  19. Simplify arguments to SignPSBTInput
    Remove redundant arguments to SignPSBTInput -- since it needs several
    bits of the PartiallySignedTransaction, pass in a reference instead of
    doing it piecemeal. This saves us having to pass in both a PSBTInput and
    its index, as well as having to pass in the CTransaction. Also avoid
    redundantly passing the sighash_type, which is contained in the
    PSBTInput already.
    0f5bda2bd9
  20. Refactor PSBTInput signing to enforce invariant
    Refactor the process of PSBTInput signing to enforce the invariant that
    a PSBTInput always has _either_ a witness_utxo or a non_witness_utxo,
    never both.
    
    This simplifies the logic of SignPSBTInput slightly, since it no longer
    has to deal with the "both" case. When calling it, we now give it, in
    order of preference: (1) whichever of the utxo fields was already
    present in the PSBT we received, or (2) if neither, the
    non_witness_utxo field, which is just a copy of the input transaction,
    which we get from the wallet.
    
    SignPSBTInput no longer has to remove one of the two fields; instead, it
    will check if we have a witness signature, and if so, it will replace
    the non_witness_utxo with the witness_utxo (which is smaller, as it is
    just a copy of the output being spent.)
    
    Add PSBTInput::IsSane checks in two more places, which checks for
    both utxo fields being present; we will now give an RPC error early on
    if we are supplied such a malformed PSBT to fill in.
    
    Also add a check to FillPSBT, to avoid touching any input that is
    already signed. (This is now redundant, since we should no longer
    potentially harm an already-signed input, but it's harmless.)
    
    fixes #14473
    565500508a
  21. Add regression test for PSBT signing bug #14473 e13fea975d
  22. gwillen force-pushed on Nov 1, 2018
  23. gwillen commented at 7:16 pm on November 1, 2018: contributor
    It looks like #14197 got merged – I just rebased over it, which effectively erases it on this branch since it does the same thing as this PR. (It does have an assert that I didn’t include in mine – I could if desired.)
  24. DrahtBot removed the label Needs rebase on Nov 1, 2018
  25. achow101 commented at 6:17 pm on November 2, 2018: member
    utACK e13fea975d5e4ae961faba36379a1cdaf9e50c1c
  26. in src/wallet/rpcwallet.h:33 in 0f5bda2bd9 outdated
    29@@ -30,5 +30,5 @@ bool EnsureWalletIsAvailable(CWallet *, bool avoidException);
    30 
    31 UniValue getaddressinfo(const JSONRPCRequest& request);
    32 UniValue signrawtransactionwithwallet(const JSONRPCRequest& request);
    33-bool FillPSBT(const CWallet* pwallet, PartiallySignedTransaction& psbtx, int sighash_type = 1, bool sign = true, bool bip32derivs = false);
    34+bool FillPSBT(const CWallet* pwallet, PartiallySignedTransaction& psbtx, int sighash_type = 1 /* SIGHASH_ALL */, bool sign = true, bool bip32derivs = false);
    


    sipa commented at 7:36 pm on November 2, 2018:
    Why not int sighash_type = SIGHASH_ALL directly?

    gwillen commented at 3:24 am on November 10, 2018:

    Well, I wouldn’t want to do that without #including script/interpreter.h for the definition of SIGHASH_ALL. (As it turns out, it works without doing that, but that means it’s counting on other files to #include specific things in a specific order, which is gross and wrong.) And I didn’t want to add a dependency to a header file just for a constant.

    But it seems like everything in the universe already includes interpreter.h anyway, so I’m willing to make the change if you prefer.


    sipa commented at 3:25 am on November 10, 2018:
    Oh, I see. I was just wondering if there was a reason - seems there is. No need to hold this PR up for.
  27. sipa commented at 7:41 pm on November 2, 2018: member
    utACK e13fea975d5e4ae961faba36379a1cdaf9e50c1c
  28. sipa merged this on Nov 10, 2018
  29. sipa closed this on Nov 10, 2018

  30. sipa referenced this in commit b30c62d4b9 on Nov 10, 2018
  31. gmaxwell commented at 3:33 am on November 20, 2018: contributor
    Backport me?
  32. gwillen deleted the branch on Nov 20, 2018
  33. sipa added the label Needs backport on Nov 22, 2018
  34. sipa commented at 1:08 am on November 22, 2018: member
    I’ve marked it for backport, but this may not be trivial as some of the code it builds upon has changed since 0.17?
  35. fanquake removed the label Needs backport on Nov 30, 2018
  36. fanquake commented at 2:29 pm on November 30, 2018: member
    This will be backported in #14780.
  37. sipa referenced this in commit cfdd6b2f6c on Dec 3, 2018
  38. sipa referenced this in commit a9eab081d5 on Dec 3, 2018
  39. sipa referenced this in commit 70ee1f8709 on Dec 3, 2018
  40. sipa referenced this in commit 39ece4fc28 on Dec 3, 2018
  41. sipa referenced this in commit ad94165db9 on Dec 3, 2018
  42. sipa referenced this in commit db445d4e5a on Dec 3, 2018
  43. sipa referenced this in commit ff56bb9b44 on Dec 3, 2018
  44. in src/wallet/test/psbt_wallet_tests.cpp:63 in e13fea975d
    58@@ -59,12 +59,8 @@ BOOST_AUTO_TEST_CASE(psbt_updater_test)
    59     CDataStream ssData(ParseHex("70736274ff01009a020000000258e87a21b56daf0c23be8e7070456c336f7cbaa5c8757924f545887bb2abdd750000000000ffffffff838d0427d0ec650a68aa46bb0b098aea4422c071b2ca78352a077959d07cea1d0100000000ffffffff0270aaf00800000000160014d85c2b71d0060b09c9886aeb815e50991dda124d00e1f5050000000016001400aea9a2e5f0f876a588df5546e8742d1d87008f000000000000000000"), SER_NETWORK, PROTOCOL_VERSION);
    60     ssData >> psbtx;
    61 
    62-    // Use CTransaction for the constant parts of the
    63-    // transaction to avoid rehashing.
    


    Sjors commented at 8:31 am on December 5, 2018:
    For posterity, can you explain what the “rehashing” problem was and why it’s not actually a problem / why this didn’t solve it?

    gwillen commented at 10:05 am on December 5, 2018:

    The explanation I received was that the comment was a cut-and-paste from elsewhere and didn’t ever actually apply here.

    The underlying issue is that calling GetHash on a CMutableTransaction repeatedly is a waste of effort, since it will not cache the transaction hash but recompute it every time. So one is well-advised to copy a CMutableTransaction to a CTransaction if one is done modifying it, and will be calling GetHash more than once. But that doesn’t apply here, as we only make extremely limited used of the underlying transaction in FillPSBT, and we never call GetHash on it.

  45. in src/rpc/rawtransaction.cpp:1547 in 53e6fffb8f outdated
    1544     bool complete = true;
    1545     for (unsigned int i = 0; i < psbtx.tx->vin.size(); ++i) {
    1546         PSBTInput& input = psbtx.inputs.at(i);
    1547 
    1548-        complete &= SignPSBTInput(DUMMY_SIGNING_PROVIDER, *psbtx.tx, input, i, 1);
    1549+        complete &= SignPSBTInput(DUMMY_SIGNING_PROVIDER, *psbtx.tx, input, i, SIGHASH_ALL);
    


    Sjors commented at 8:37 am on December 5, 2018:
    This is the kind of thing that makes me want to shill for linters (or better use of the type system) :-) cc @practicalswift
  46. Sjors commented at 9:58 am on December 5, 2018: member
    Post merge ACK
  47. MarcoFalke referenced this in commit 5d12143c73 on Dec 5, 2018
  48. deadalnix referenced this in commit f55f0c3e08 on Apr 15, 2020
  49. ftrader referenced this in commit f66417876f on Aug 17, 2020
  50. MarcoFalke locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-04 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me