Bitcoin Core 0.20.1 not finalizing my psbt (not recognizing as complete)

Fonta1n3 commented at 7:16 AM on August 3, 2020: none

bitcoin-cli finalizepsbt cHNidP8BAHECAAAAAVQXYcBrEVxdFSiKjgN7apFrbALHH6FE+a/880gdWg+0AQAAAAD9////AqCGAQAAAAAAFgAUqWr4NqXfkkV9VKlah8i6LLHjtuw1QBIAAAAAABYAFPo+csLEb7S4W2Kt/U0a3ionBpKyAAAAAAABAHECAAAAAV9eHHMoKiY4WZNqMSOSuHihwGtXU4pdsieAoRJrgqPEAAAAAAD9////AlIn5wIAAAAAFgAUhHfCQ3vybJC7XRfYliLjXjWjS7pixxMAAAAAABYAFFTgWpDKPYC4Ef677073UzsbnjyoAAAAAAEBH2LHEwAAAAAAFgAUVOBakMo9gLgR/rvvTvdTOxuePKgiAgPxa1S5laFXdrX3va8PjzRmVE1Okh4VkeM5l3FBQb60pUcwRAIgM/2kG4Q04ilDCzgDCaRlE1g0kicet1QlkvS87FAkDo4CIFOVDAEaVIEuv58lnPza6gARh10LHbe4F4r9vMVpY8oXASIGA/FrVLmVoVd2tfe9rw+PNGZUTU6SHhWR4zmXcUFBvrSlGMJwW35UAACAAQAAgAAAAIAAAAAABwAAAAAAIgIDIiGKJQ7NYfjNy1cDnOVbUazTFnvFOjUWIG1/kh0CsukYwnBbflQAAIABAACAAAAAgAEAAAARAAAAAA==

Does not finalize the psbt, it returns:

result =     {
        complete = 0;
        psbt = "cHNidP8BAHECAAAAAVQXYcBrEVxdFSiKjgN7apFrbALHH6FE+a/880gdWg+0AQAAAAD9////AqCGAQAAAAAAFgAUqWr4NqXfkkV9VKlah8i6LLHjtuw1QBIAAAAAABYAFPo+csLEb7S4W2Kt/U0a3ionBpKyAAAAAAABAHECAAAAAV9eHHMoKiY4WZNqMSOSuHihwGtXU4pdsieAoRJrgqPEAAAAAAD9////AlIn5wIAAAAAFgAUhHfCQ3vybJC7XRfYliLjXjWjS7pixxMAAAAAABYAFFTgWpDKPYC4Ef677073UzsbnjyoAAAAAAEBH2LHEwAAAAAAFgAUVOBakMo9gLgR/rvvTvdTOxuePKgiAgPxa1S5laFXdrX3va8PjzRmVE1Okh4VkeM5l3FBQb60pUcwRAIgM/2kG4Q04ilDCzgDCaRlE1g0kicet1QlkvS87FAkDo4CIFOVDAEaVIEuv58lnPza6gARh10LHbe4F4r9vMVpY8oXASIGA/FrVLmVoVd2tfe9rw+PNGZUTU6SHhWR4zmXcUFBvrSlGMJwW35UAACAAQAAgAAAAIAAAAAABwAAAAAAIgIDIiGKJQ7NYfjNy1cDnOVbUazTFnvFOjUWIG1/kh0CsukYwnBbflQAAIABAACAAAAAgAEAAAARAAAAAA==";
    }

There is only one input and there is one signature for that input, it is a single signature psbt. Using Bitcoin Core 0.20.0 works fine.

To return a signed raw transaction.

Actual behavior

See above.

To reproduce

See above.

I can reproduce this every single time, totally breaks my single sig wallet 😱

System information

0.20.1, I got it by downloading the tarball from bitcoincore.org, for macOS.

macbook

Fonta1n3 added the label Bug on Aug 3, 2020

Fonta1n3 commented at 7:55 AM on August 3, 2020: none

perhaps #19215 is causing this?

sipa assigned achow101 on Aug 3, 2020

sipa commented at 8:02 AM on August 3, 2020: member

@achow101 Want to have a look? The input has both witness_utxo and non_witness_utxo, and a partial_signatures for the right key, but somehow doesn't get finalized. analyzepsbt says next is an updater step (?)

MarcoFalke added this to the milestone 0.20.2 on Aug 3, 2020

Fonta1n3 commented at 10:04 AM on August 3, 2020: none

@achow101 Want to have a look? The input has both witness_utxo and non_witness_utxo, and a partial_signatures for the right key, but somehow doesn't get finalized. analyzepsbt says next is an updater step (?)

Worth mentioning the wallet on the node was created with a combo prefixed descriptor which may explain why there are both witness_utxo and non_witness_utxo? I am not sure about that but it seemed odd to see "2" inputs in the non_witness_utxo when in fact there is only one input.

achow101 commented at 3:52 PM on August 3, 2020: member

It's most likely that the signature is invalid. How did you sign this transaction?

The changes from #19215 is to address a potential attack that some hardware wallets decided to address. It doesn't have anything to do with combo descriptors. However because both UTXOs are being provided, it is possible that the software you are using doesn't know how to handle that and is signing the segwit input with a non-segwit signature.

Fonta1n3 commented at 2:50 AM on August 4, 2020: none

It's most likely that the signature is invalid. How did you sign this transaction?

The changes from #19215 is to address a potential attack that some hardware wallets decided to address. It doesn't have anything to do with combo descriptors. However because both UTXOs are being provided, it is possible that the software you are using doesn't know how to handle that and is signing the segwit input with a non-segwit signature.

Signing with LibWally 0.7.7

The thing is there is only one utxo, I do not understand why Bitcoin Core shows two?

it is possible that the software you are using doesn't know how to handle that and is signing the segwit input with a non-segwit signature

It is possible but I do not want to hack LibWally that much...

Again the signing works perfectly fine in conjunction with 0.20.0 (same wallet, same utxo), which is why I raised the issue here.

sipa commented at 4:16 PM on August 4, 2020: member

I think we're missing some steps here. What is the exact procedure you used to obtain the input PSBT?

Does the exact same PSBT finalize in 0.20? That would surprise me as it contains both the full creating transactions and the UTXO, which used to be illegal in PSBT.

sipa commented at 4:21 PM on August 4, 2020: member

Here is what I suspect happened:

You create a PSBT with 0.20, signed it using libwally, and then try to finalize it with 0.20. This works.
You create a PSBT with 0.20.1, sign it using libwally (which now creates an invalid signature as it misinterprets the PSBT), which 0.20.1 now fails to finalize because the signature is invalid.

I suspect:

If you create a PSBT with 0.20, sign with libwally, and then finalize with 0.20.1, it will still work.
If you create a PSBT with 0.20.1, sign with libwally, and then finalize with 0.20, it will fail in the last step, saying the PSBT file is invalid.

It seems libwally isn't compatible with the change that had to made to PSBT to support hardware wallets rolling out mitigations for the segwit value issue (CVE-2020-14199). I believe this will be fixed in an upcoming version of libwally.

Fonta1n3 commented at 10:10 AM on August 5, 2020: none

Here is what I suspect happened:

You create a PSBT with 0.20, signed it using libwally, and then try to finalize it with 0.20. This works.

You create a PSBT with 0.20.1, sign it using libwally (which now creates an invalid signature as it misinterprets the PSBT), which 0.20.1 now fails to finalize because the signature is invalid.

I suspect:

If you create a PSBT with 0.20, sign with libwally, and then finalize with 0.20.1, it will still work.

If you create a PSBT with 0.20.1, sign with libwally, and then finalize with 0.20, it will fail in the last step, saying the PSBT file is invalid.

It seems libwally isn't compatible with the change that had to made to PSBT to support hardware wallets rolling out mitigations for the segwit value issue (CVE-2020-14199). I believe this will be fixed in an upcoming version of libwally. @sipa I think you have hit the nail on the head. I can definitely provide a detailed list of commands to reproduce the issue if that will be helpful, however I think it is just simply that LibWally psbt signing is not compatible with having both types of utxo's so I am not sure it is really necessary... Let me know if you would still like me to include detailed steps to reproduce the issue and I will happily do so.

How do you all recommend I be more in tune with changes that may break my apps in the future? All of the apps I develop utilize Bitcoin Core as a backend and I very much want to avoid surprises like this moving forward. IRC?

Fonta1n3 commented at 3:48 PM on August 11, 2020: none

I managed to fix this thanks to some guidance from @jgriffiths, you can see details https://github.com/ElementsProject/libwally-core/issues/213

fwiw I have also noticed this breaks Electrum's psbt as it rejects anything with both utxo types.

Correction, you can build Electrum from master branch and it no longer rejects psbt with both utxo types.

Fonta1n3 commented at 9:11 AM on August 17, 2020: none

I am going to close this as it was totally unrelated to Bitcoin Core.

Fonta1n3 closed this on Aug 17, 2020

DrahtBot locked this on Feb 15, 2022

Bitcoin Core 0.20.1 not finalizing my psbt (not recognizing as complete) - breaks my wallet #19650