signrawtransactionwithwallet fails with signed non-wallet inputs with many witness stack elements

t-bast commented at 8:26 am on February 11, 2021: member

I’m seeing a very strange behavior with signrawtransactionwithwallet when I provide signed non-wallet inputs. The following sample transaction (regtest) is a lightning HTLC transaction with added bitcoind wallet inputs/outputs to bump the fee:

 0020000000001027697c418d3c56138f1ea0469026d081705290bd0317ad9a133319fa7b4ac59b10200000000010000001cd5376d42e12bad24911ae73b13c40b530a18c3da34de8d16fcf356bf2e32180000000000fdffffff02bb080000000000002200209357506c5a4a2df7c685481aad0302a3ccfe44c4e41122e33818861a37e2e6cccdfbea020000000016001487c7624537496328484f590d04a805f0c2a213e30500483045022100aef712b8d806077872e894343f917473497841b6d859bb7cf20473597353e7d8022030c39a0dc43b742ad0b746e8968b763e3053ec1001bbae4f5368d15d20d1152683483045022100a71dba3a0ed7d005c5e99b3a3e603026e60cd2708e7590162892bae9391965ba0220179d15465950863401956d0d4bbad0f413ce465b111dc71fe511d9bbf4200d0f0120b27cf03ab42eb16a475e289df7878d8c36d9465ba6b4f22ce1d21ef217e4f3d88d76a914c333ee02e6633f9bce0281ae0dc09e29b1b2761f8763ac672103bcdf028d4e53f7fbddb817091a60118b1d2fb7ab8cbab4b14ebeb981450d6ff77c8201208763a91487520d9fc7939063f7244ce842e4a5d2507d93d188527c21036df69837ccd9b8b353bdd37a8c616e3ea094ffee4c9512fec902ae8250ffb9f952ae6775022701b175ac6851b275680000000000
 1{
 2  "txid": "d922b3e02d77e7c522fcf46a308996df764dc61ef708a5702dc3dce6c2f58882",
 3  "hash": "1f558b7fd77d1339c48772f6ef12a0bf6cdbc05e371c6871d885afc89e6a5e3c",
 4  "version": 2,
 5  "size": 492,
 6  "vsize": 248,
 7  "weight": 990,
 8  "locktime": 0,
 9  "vin": [
10    {
11      "txid": "b159acb4a79f3133a1d97a31d00b290517086d026904eaf13861c5d318c49776",
12      "vout": 2,
13      "scriptSig": {
14        "asm": "",
15        "hex": ""
16      },
17      "txinwitness": [
18        "",
19        "3045022100aef712b8d806077872e894343f917473497841b6d859bb7cf20473597353e7d8022030c39a0dc43b742ad0b746e8968b763e3053ec1001bbae4f5368d15d20d1152683",
20        "3045022100a71dba3a0ed7d005c5e99b3a3e603026e60cd2708e7590162892bae9391965ba0220179d15465950863401956d0d4bbad0f413ce465b111dc71fe511d9bbf4200d0f01",
21        "b27cf03ab42eb16a475e289df7878d8c36d9465ba6b4f22ce1d21ef217e4f3d8",
22        "76a914c333ee02e6633f9bce0281ae0dc09e29b1b2761f8763ac672103bcdf028d4e53f7fbddb817091a60118b1d2fb7ab8cbab4b14ebeb981450d6ff77c8201208763a91487520d9fc7939063f7244ce842e4a5d2507d93d188527c21036df69837ccd9b8b353bdd37a8c616e3ea094ffee4c9512fec902ae8250ffb9f952ae6775022701b175ac6851b27568"
23      ],
24      "sequence": 1
25    },
26    {
27      "txid": "18322ebf56f3fc168dde34dac3180a530bc4133be71a9124ad2be1426d37d51c",
28      "vout": 0,
29      "scriptSig": {
30        "asm": "",
31        "hex": ""
32      },
33      "sequence": 4294967293
34    }
35  ],
36  "vout": [
37    {
38      "value": 0.00002235,
39      "n": 0,
40      "scriptPubKey": {
41        "asm": "0 9357506c5a4a2df7c685481aad0302a3ccfe44c4e41122e33818861a37e2e6cc",
42        "hex": "00209357506c5a4a2df7c685481aad0302a3ccfe44c4e41122e33818861a37e2e6cc",
43        "reqSigs": 1,
44        "type": "witness_v0_scripthash",
45        "addresses": [
46          "bcrt1qjdt4qmz6fgkl0359fqd26qcz50x0u3xyusgj9cecrzrp5dlzumxq6sxuyz"
47        ]
48      }
49    },
50    {
51      "value": 0.48954317,
52      "n": 1,
53      "scriptPubKey": {
54        "asm": "0 87c7624537496328484f590d04a805f0c2a213e3",
55        "hex": "001487c7624537496328484f590d04a805f0c2a213e3",
56        "reqSigs": 1,
57        "type": "witness_v0_keyhash",
58        "addresses": [
59          "bcrt1qslrky3fhf93jsjz0tyxsf2q97rp2yylr3fd72n"
60        ]
61      }
62    }
63  ]
64}

Notice that the first input (the non-wallet one) has a stack with 5 elements (the first of which is empty). It is correctly signed and passes script validation.

When I send this transaction to signrawtransactionwithwallet, bitcoind returns the following error: Unable to sign input, invalid stack size (possibly missing key) (code: -1) referencing the non-wallet input.

But it does correctly sign the wallet input and returns the following transaction:

 0020000000001027697c418d3c56138f1ea0469026d081705290bd0317ad9a133319fa7b4ac59b10200000000010000001cd5376d42e12bad24911ae73b13c40b530a18c3da34de8d16fcf356bf2e32180000000000fdffffff02bb080000000000002200209357506c5a4a2df7c685481aad0302a3ccfe44c4e41122e33818861a37e2e6cccdfbea020000000016001487c7624537496328484f590d04a805f0c2a213e3018d76a914c333ee02e6633f9bce0281ae0dc09e29b1b2761f8763ac672103bcdf028d4e53f7fbddb817091a60118b1d2fb7ab8cbab4b14ebeb981450d6ff77c8201208763a91487520d9fc7939063f7244ce842e4a5d2507d93d188527c21036df69837ccd9b8b353bdd37a8c616e3ea094ffee4c9512fec902ae8250ffb9f952ae6775022701b175ac6851b275680247304402201566b009b4e392ca16d0e5658e7fd2257e511e8974f198fb8ac265473d492949022001c9b41d261f8eacc19c41f179aa36a449200fe66abb8f3aab45c5d49e280f2901210271f8ed45eddce4cb9f990998f1d28f91fe4f747a7d67900b5382484276ae4bfd00000000
 1{
 2  "txid": "d922b3e02d77e7c522fcf46a308996df764dc61ef708a5702dc3dce6c2f58882",
 3  "hash": "06722d22306472337f1229ad9642c3530f161b7ef3d54e061acf7ca09fb8ed76",
 4  "version": 2,
 5  "size": 418,
 6  "vsize": 229,
 7  "weight": 916,
 8  "locktime": 0,
 9  "vin": [
10    {
11      "txid": "b159acb4a79f3133a1d97a31d00b290517086d026904eaf13861c5d318c49776",
12      "vout": 2,
13      "scriptSig": {
14        "asm": "",
15        "hex": ""
16      },
17      "txinwitness": [
18        "76a914c333ee02e6633f9bce0281ae0dc09e29b1b2761f8763ac672103bcdf028d4e53f7fbddb817091a60118b1d2fb7ab8cbab4b14ebeb981450d6ff77c8201208763a91487520d9fc7939063f7244ce842e4a5d2507d93d188527c21036df69837ccd9b8b353bdd37a8c616e3ea094ffee4c9512fec902ae8250ffb9f952ae6775022701b175ac6851b27568"
19      ],
20      "sequence": 1
21    },
22    {
23      "txid": "18322ebf56f3fc168dde34dac3180a530bc4133be71a9124ad2be1426d37d51c",
24      "vout": 0,
25      "scriptSig": {
26        "asm": "",
27        "hex": ""
28      },
29      "txinwitness": [
30        "304402201566b009b4e392ca16d0e5658e7fd2257e511e8974f198fb8ac265473d492949022001c9b41d261f8eacc19c41f179aa36a449200fe66abb8f3aab45c5d49e280f2901",
31        "0271f8ed45eddce4cb9f990998f1d28f91fe4f747a7d67900b5382484276ae4bfd"
32      ],
33      "sequence": 4294967293
34    }
35  ],
36  "vout": [
37    {
38      "value": 0.00002235,
39      "n": 0,
40      "scriptPubKey": {
41        "asm": "0 9357506c5a4a2df7c685481aad0302a3ccfe44c4e41122e33818861a37e2e6cc",
42        "hex": "00209357506c5a4a2df7c685481aad0302a3ccfe44c4e41122e33818861a37e2e6cc",
43        "reqSigs": 1,
44        "type": "witness_v0_scripthash",
45        "addresses": [
46          "bcrt1qjdt4qmz6fgkl0359fqd26qcz50x0u3xyusgj9cecrzrp5dlzumxq6sxuyz"
47        ]
48      }
49    },
50    {
51      "value": 0.48954317,
52      "n": 1,
53      "scriptPubKey": {
54        "asm": "0 87c7624537496328484f590d04a805f0c2a213e3",
55        "hex": "001487c7624537496328484f590d04a805f0c2a213e3",
56        "reqSigs": 1,
57        "type": "witness_v0_keyhash",
58        "addresses": [
59          "bcrt1qslrky3fhf93jsjz0tyxsf2q97rp2yylr3fd72n"
60        ]
61      }
62    }
63  ]
64}

Notice how it drops all witness stack elements from the non-wallet input except the last one: this seems to be the culprit. If I ignore bitcoind’s error and restore the witness stack of the non-wallet input, the transaction becomes valid and can be correctly published.

I tested this with Bitcoin Core 0.20.1 and 0.21.0, let me know if you want me to test against other versions.

t-bast added the label Bug on Feb 11, 2021

achow101 commented at 5:18 pm on February 11, 2021: member

The signing should be ignoring fully signed inputs. Can you provide a sample that I can check with testnet? I would like to run this with gdb to see what it is doing.

t-bast commented at 8:28 am on February 12, 2021: member

Sure, here are the same kind of transactions made on testnet:

this one is what I send to signrawtransaction (the non-wallet input is signed): 02000000000102519769cd1236d254a359be0b7e6346249964e7c924652c53d88900ce7f10e3ba02000000000100000032567cc9550e68c2e781231af49f3b72671e72f31101d0c212233d845525e07f0100000000fdffffff027e74000000000000220020744423a7acd7005ef45f8c96931d1b0477d2892cb5efaa12a8a6a3458fdf171d8069110000000000160014b4189c012a436718ca20aa2133d4d1cd9be65a2a050047304402200aa02bbe36e751a8edd67e0980a79c93bdb94a1a9032e7f2317e1570912402780220180be3d8b2312e2ad77069fc44994ad3ef143fdeefad377699e4c4bee6e58cea83483045022100d2dc1e98630f42dc9ac1908b5db9031a2da4af46a6bc46ca44ea4c1c4e947afa02206bdb9b04ae671305e6708fbef81f0120c41c274bb71508f11001418c3f8ca4870120aab09210ace4b6134b01aa03e755cdb8610cf8c60a513124e535984d66c3f6668e76a914d0953f591363f638bec2acddd6325aff8c98956c8763ac67210219d34adfe9d4d41c4a1234ed2fda35f9e7d51c002b4c3e99a03a3ee4fa467b437c8201208763a9143b6649e2e706814918c700b54a9b34df495662b588527c2102a01bf50ab395f125b3a0c4d26e3dc4d4b97c0f1044f91a9362e21b86651feb2652ae67750303891db175ac6851b275680000000000
this is what signrawtransaction returns (along with the error and complete=false, note that it modified the witness stack of my non-wallet input): 02000000000102519769cd1236d254a359be0b7e6346249964e7c924652c53d88900ce7f10e3ba02000000000100000032567cc9550e68c2e781231af49f3b72671e72f31101d0c212233d845525e07f0100000000fdffffff027e74000000000000220020744423a7acd7005ef45f8c96931d1b0477d2892cb5efaa12a8a6a3458fdf171d8069110000000000160014b4189c012a436718ca20aa2133d4d1cd9be65a2a018e76a914d0953f591363f638bec2acddd6325aff8c98956c8763ac67210219d34adfe9d4d41c4a1234ed2fda35f9e7d51c002b4c3e99a03a3ee4fa467b437c8201208763a9143b6649e2e706814918c700b54a9b34df495662b588527c2102a01bf50ab395f125b3a0c4d26e3dc4d4b97c0f1044f91a9362e21b86651feb2652ae67750303891db175ac6851b275680247304402206e637a522dd98b58952caa0b6d41a949daf6787d8d23250a4ee9caf3498bf14002205c16a5c26032b85eb47de8fb64d6a2ae176b681e4da1c431fb116c3563cb9a0b0121028f99fc31a955bdc85fdb90410f4e930e2c2a4dea400da6f9993b717b5db20e6c00000000
when I then restore the correct witness stack, the transaction passes testmempoolaccept: 02000000000102519769cd1236d254a359be0b7e6346249964e7c924652c53d88900ce7f10e3ba02000000000100000032567cc9550e68c2e781231af49f3b72671e72f31101d0c212233d845525e07f0100000000fdffffff027e74000000000000220020744423a7acd7005ef45f8c96931d1b0477d2892cb5efaa12a8a6a3458fdf171d8069110000000000160014b4189c012a436718ca20aa2133d4d1cd9be65a2a050047304402200aa02bbe36e751a8edd67e0980a79c93bdb94a1a9032e7f2317e1570912402780220180be3d8b2312e2ad77069fc44994ad3ef143fdeefad377699e4c4bee6e58cea83483045022100d2dc1e98630f42dc9ac1908b5db9031a2da4af46a6bc46ca44ea4c1c4e947afa02206bdb9b04ae671305e6708fbef81f0120c41c274bb71508f11001418c3f8ca4870120aab09210ace4b6134b01aa03e755cdb8610cf8c60a513124e535984d66c3f6668e76a914d0953f591363f638bec2acddd6325aff8c98956c8763ac67210219d34adfe9d4d41c4a1234ed2fda35f9e7d51c002b4c3e99a03a3ee4fa467b437c8201208763a9143b6649e2e706814918c700b54a9b34df495662b588527c2102a01bf50ab395f125b3a0c4d26e3dc4d4b97c0f1044f91a9362e21b86651feb2652ae67750303891db175ac6851b275680247304402206e637a522dd98b58952caa0b6d41a949daf6787d8d23250a4ee9caf3498bf14002205c16a5c26032b85eb47de8fb64d6a2ae176b681e4da1c431fb116c3563cb9a0b0121028f99fc31a955bdc85fdb90410f4e930e2c2a4dea400da6f9993b717b5db20e6c00000000

I made sure the non-wallet input stays unspent, so you should be able to test it on vanilla testnet without any issues. Let me know if I can help with anything.

achow101 commented at 7:22 pm on February 12, 2021: member

@t-bast Can you try #21166

t-bast commented at 9:21 am on February 15, 2021: member

Thanks @achow101! I tested #21166 for both HTLC-success and HTLC-timeout txs, and it works as we’d expect: complete is now correctly set to true, wallet inputs are signed and the non-wallet input is untouched.

fanquake closed this on Apr 7, 2021

sidhujag referenced this in commit 976ed65d4d on Apr 7, 2021

DrahtBot locked this on Aug 18, 2022

signrawtransactionwithwallet fails with signed non-wallet inputs with many witness stack elements #21151