non-mandatory-script-verify-flag (Witness program hash mismatch) when trying to spend Taproot output with 32x 0x00 tapbranch #24765

<!-- This issue tracker is only for technical issues related to Bitcoin Core. General bitcoin questions and/or support requests are best directed to the Bitcoin StackExchange at https://bitcoin.stackexchange.com. For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/. If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue! -->

<!-- Describe the issue -->

i made a taproot output with an internal key, a tapscript in a leaf and a nonexistent tapscript with a SHA256 of 32x 0x00 spending it doesnt work, but when i change the hash of the nonexistent tapscript to 31x 0x00 + 0x01 is does work. there is no reason why with 0 it wouldnt work as this theoretically is a valid SHA256 hash

Expected behavior

<!--- What behavior did you expect? -->

the transaction got accepted

Actual behavior

<!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->

it got not

To reproduce

<!--- How reliably can you reproduce the issue, what are the steps to do so? -->

100% reliable with https://github.com/antonilol/btc_stuff/blob/461391d6ecbba98d361bca60b0e5f7d17349494a/p2tr_tapscript.ts

if you dont have time to make one yourself here is my TX: taproot output (this one is in the testnet blockchain eed3cb854b1faaf3a342d2738b356ba4c176cb0c96e06a378b7ee47d8d68a12f)

02000000000101eb9ed40b43fc676e2d33d6059338b75850611ea7ec7480a1cc030d7997cc7e730000000000feffffff021027000000000000225120c730e130570fb56367fb31d9ab9d492e38be719d31a0b4602fda344caa4b3d03fee468000000000016001494634ec1184c82d6a12b984f0e5efab9f6b053ec0247304402203ba26bbd89ddb641e19bab67773652ea09512ed7996a591d94d197109bf794ea0220375b5614eb4eb6a5f123aa5abce257f0273a9b70a9d87e6087378fc3fd452b68012102ca71026e93dc7183350a3c7af0c38ced30fea24dd230ccb2ffa365c55ec3e5da71762100

trying to spend:

02000000000101b294960b0f1c6d185882abab322dab58cba995b77fac80afc13b5595a0add7810000000000ffffffff017a2600000000000016001411b07a052ffdb815bb7eb017631b23d13eba5af904401a5f24e4d351e59eb64b213e5ce5dbe8e610eca7ea9128fd2f99bcc72eae122b6517a4a689122301dfa21f31c002cd3e55289be36be80bfd910e3f5649529f54401a5f24e4d351e59eb64b213e5ce5dbe8e610eca7ea9128fd2f99bcc72eae122b6517a4a689122301dfa21f31c002cd3e55289be36be80bfd910e3f5649529f54462042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ac2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ba528741c15be1beaf62c286f2eae1b60ee1f1f6dcc9b96104ede0f09f1497c5bae82ea5aa000000000000000000000000000000000000000000000000000000000000000000000000

System information

What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)?

bitcoin.org binary v22

What type of machine are you observing the error on (OS/CPU and disk type)?

not needed, this is consensus related

<!-- GUI-related issue? What is your operating system and its version? If Linux, what is your desktop environment and graphical shell? -->

<!-- Any extra information that might be useful in the debugging process. -->

<!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

There should not be any special casing of the all-zeroes hash in the consensus rules. Is it possible that the library you're using for constructing the signature/tweak/output key/control block treats the all-zero hash specially (and wrong)?

diff of two spending transactions the difference is they use a different TapBranch, therefore another prev out, i needed a different output and that gives a different txid, hash and sighash which gave a different signature the top one this the second transaction of #24765#issue-1192292067 the bottom one is in the testnet blockchain (i tried spending it with all zeros except last byte and it worked)

Is it possible that the library you're using for constructing the signature/tweak/output key/control block treats the all-zero hash specially (and wrong)?

i think not, i checked all output of my script with btcdeb and things like tapleaf, tapbranch and taptweak worked as expected. but i will work it through completely from generated keys to taproot output with btcdeb and my javascript function + bitcoinlib-js

internal pub: 035be1beaf62c286f2eae1b60ee1f1f6dcc9b96104ede0f09f1497c5bae82ea5aa nonexistent leaf hash: 0000000000000000000000000000000000000000000000000000000000000000 (call this leaf1) leaf2: 2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ac2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ba5287 leaf2 hash:

btcdeb> tf tagged-hash TapLeaf c0 prefix_compact_size(2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ac2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ba5287)
msg = c0462042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ac2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ba5287
dac59a19ed030a45e6b20e14c5e88934429213369703b24aeeaf1e623b58df58

> tapLeaf(Buffer.from('2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ac2042c68328c229ddef4f0588bd952e05eaca009bee43d01ff6bcbb0b681c15c894ba5287', 'hex')).toString('hex')
'dac59a19ed030a45e6b20e14c5e88934429213369703b24aeeaf1e623b58df58'

tapbranch(leaf1, leaf2) (note that in btcdeb you must enter the branches in the correct order (lexicographically smallest first), my tapBranch function sorts them before hashing)

btcdeb> tf tagged-hash TapBranch 0000000000000000000000000000000000000000000000000000000000000000 dac59a19ed030a45e6b20e14c5e88934429213369703b24aeeaf1e623b58df58
msg = 0000000000000000000000000000000000000000000000000000000000000000dac59a19ed030a45e6b20e14c5e88934429213369703b24aeeaf1e623b58df58
8af589ab15530ce5ce1d0ca79f5d1c2a7db576c6802bb0b200904b7ea0272618

> zeros = Buffer.alloc(32)
<Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
> tapBranch(Buffer.from('dac59a19ed030a45e6b20e14c5e88934429213369703b24aeeaf1e623b58df58', 'hex'), zeros).toString('hex')
'8af589ab15530ce5ce1d0ca79f5d1c2a7db576c6802bb0b200904b7ea0272618'

tweak(pub, tapTweak(internal_pub, branch))

btcdeb> tf tagged-hash TapTweak 5be1beaf62c286f2eae1b60ee1f1f6dcc9b96104ede0f09f1497c5bae82ea5aa 8af589ab15530ce5ce1d0ca79f5d1c2a7db576c6802bb0b200904b7ea0272618
msg = 5be1beaf62c286f2eae1b60ee1f1f6dcc9b96104ede0f09f1497c5bae82ea5aa8af589ab15530ce5ce1d0ca79f5d1c2a7db576c6802bb0b200904b7ea0272618
cd537bb4434ce4ca7b7aefc20ed0ffcce55bbab5ef805ef4bd9df2dd87a4c869
btcdeb> tf taproot-tweak-pubkey 5be1beaf62c286f2eae1b60ee1f1f6dcc9b96104ede0f09f1497c5bae82ea5aa cd537bb4434ce4ca7b7aefc20ed0ffcce55bbab5ef805ef4bd9df2dd87a4c869
02c730e130570fb56367fb31d9ab9d492e38be719d31a0b4602fda344caa4b3d03

> tapTweak(Buffer.from('5be1beaf62c286f2eae1b60ee1f1f6dcc9b96104ede0f09f1497c5bae82ea5aa', 'hex'), Buffer.from('8af589ab15530ce5ce1d0ca79f5d1c2a7db576c6802bb0b200904b7ea0272618', 'hex')).toString('hex')
'cd537bb4434ce4ca7b7aefc20ed0ffcce55bbab5ef805ef4bd9df2dd87a4c869'
> curve.xOnlyPointAddTweak(Buffer.from('5be1beaf62c286f2eae1b60ee1f1f6dcc9b96104ede0f09f1497c5bae82ea5aa', 'hex'), Buffer.from('cd537bb4434ce4ca7b7aefc20ed0ffcce55bbab5ef805ef4bd9df2dd87a4c869', 'hex'))
{
  parity: 0,
  xOnlyPubkey: Uint8Array(32) [
    199,  48, 225,  48,  87,  15, 181, 99,
    103, 251,  49, 217, 171, 157,  73, 46,
     56, 190, 113, 157,  49, 160, 180, 96,
     47, 218,  52,  76, 170,  75,  61,  3
  ]
}
> Buffer.from(_.xOnlyPubkey)
<Buffer c7 30 e1 30 57 0f b5 63 67 fb 31 d9 ab 9d 49 2e 38 be 71 9d 31 a0 b4 60 2f da 34 4c aa 4b 3d 03>

_ in js means previous output parity 0 = 02 and parity 1 = 03 prefix

this c730e130570fb56367fb31d9ab9d492e38be719d31a0b4602fda344caa4b3d03 that came out of both btcdeb and js is the taproot key

the js line are reproducible by git cloning antonilol/btc_stuff cd into it type node to enter nodejs repl run exports={} and then .load p2tr_tapscript.js it will give some errors but my js functions tapLeaf, tapBranch and tapTweak should work

was anyone able to reproduce this?

I tried to recreate your scenario in Bitcoin Core's test framework: https://github.com/sipa/bitcoin/commits/202204_try24765

I do seem to get very different results (perhaps the way hash outputs are treated as private keys is different?), but it does work. In case you're interested, the output is:

Internal privkey: 99cc9f872553025f99977238bba5ff60053a8d19b676bf84cfab7dce40a7ede5
Internal pubkey: 6354f7dcabb8d506321bf2ed98f37d0058f93096c0205159f54f25810417dd07
Script privkey: 40cf24c0a04be8a0fcf9976c97e617a260420970066026d5b42fc95cb1625db5
Script pubkey: d2055541cd829ec122ff03233920a6d403b2ad1058293f00e2216828a4d2cb97
Merkle root: be28fd1d09cab5b9e1f2bd87fe452451d23e784e84d46c61d000b69b7934cc31
Tweak: e22993c6b6f74a486151785d437c09419142452d5752b48808935aa2c0cf973d
Output pubkey: d408be1569a4eb76ba8d2ac8fb69e0138d08ebf903579dd64bbbb506df398386
ScriptPubKey: 5120d408be1569a4eb76ba8d2ac8fb69e0138d08ebf903579dd64bbbb506df398386
Leaf: script: 20d2055541cd829ec122ff03233920a6d403b2ad1058293f00e2216828a4d2cb97ac20d2055541cd829ec122ff03233920a6d403b2ad1058293f00e2216828a4d2cb97ba5287
      controlblock: c06354f7dcabb8d506321bf2ed98f37d0058f93096c0205159f54f25810417dd070000000000000000000000000000000000000000000000000000000000000000

I do seem to get very different results

yeah i used different privs before, so thats expected

i was able to create a valid spend with 32 zero bytes as tapleaf hash by using random keys and running the program over and over again. sometimes it failed and other times it worked, i still dont know why, but will go to bitcoin SE for that. it does seem to be related to the input values, and apparently when first playing with this it failed when using all zeros and worked with 2 different values, so basically i made the wrong conclusion based on 3 attempts.

i was able to narrow down the problem, by looking at public key parities and i noticed that it works when key1 Y is even and the output key Y is even i am 99% sure this is my fault and not of bitcoin core