bitcoind accepts invalid hex. #4458

While playing with mining on btcd, cgminer (@ckolivas) occasionally issued a submitblock request with invalid data (odd length):

09:28:03 2014-07-02 [INF] RPCS: SUBMITBLOCK LEN=7731
09:28:03 2014-07-02 [INF] RPCS: 02000000f89abacb6af25c9a98ec444a34b9b807c098a4b230b45bef17b6000000000000c72e2509244412e76ba897ea097ea3b50e6b00ca306fd5d0c6cd
82e642432fb1a508b4536431011b011fb51b0701000000010000000000000000000000000000000000000000000000000000000000000000ffffffff2a03f30f0407062f503253482ffe8d08b453
fe4f1304000963676d696e6572343208740b00000000000000ffffffff01e0672295000000001976a91485eb47fe98f349065d6f044e27a4ac541af79ee288ac00000000010000000148e435fa4f
77ce56a0c64a5310a8993aa7c97d48b351f92c364b6d3b6da9d974000000006a473044022023549634c9dc0459c3cab3dd6601b72594d8bdb107b1036968895d3bf23a02780220317b09c915e2f0
d2c0cd76c81e204dc29083d5c8b34bed440aa33b1a294f3173012102eab397e5fc97916f40010d79cb217d4b982afb27c4cdf6baf5af3a0d83d1bd8cffffffff02002d31010000000017a9145617
1509a86104afb2643560c02e7e300d1fffc587a0556f31000000001976a914a0705cbefa38510f2b0c56b47ae58f323c30842288ac000000000100000001a0ed6a97b1634e35cfb8cc66c878413c
feda127545c8d411a0c511c63b1f2605000000006b48304502210081516b709b10c7e48fe7901502849b8d090595f8c4989378cb7601a588e9e31402200bbd0922738e03f5f7dc36baaff9933224
7a41cc84cbc76bb15e55d03d7756310121026b19a8d55e839bef7e5ec0c9061b8dde18d31caf576b0c24ef9e6e542a0b5786ffffffff02809698000000000017a91456171509a86104afb2643560
c02e7e300d1fffc587309e8b00000000001976a9148ede9033b0c270e6e7356ca3e10e02217670b35e88ac000000000100000001563980317a997dc27ab52bd669589c74108e99fa69e0845b61ae
f82f8dba5183010000008a47304402205fdfed478fce15f491b6a1b7b3d8a23b268f0ecc464752072963cb8d7a80af920220296606a15956d4447c86f9a0dd8b1f4b592341a9adfc2f337aa64a52
e0eb15a60141040cfa3dfb357bdff37c8748c7771e173453da5d7caa32972ab2f5c888fff5bbaeb5fc812b473bf808206930fade81ef4e373e60039886b51022ce68902d96ef70ffffffff02a086
0100000000001976a91449d179fd48ad3e16f88ee13b27082eadb28161a888aca0595727000000001976a91461b469ada61f37c620010912a9d5d56646015f1688ac000000000100000001ba99d6
ddfc1b132f37e6a76ca377da52980d14e00571f92b5a2b67d86c5d4c8b010000008c493046022100b3369737705e92a8d04cafb3356df92dc0148e89ee6999f6b9fe526999afe483022100efe606
a2c77f74d11ace2fb47c1e016f463a657988b9727785fba3cd4cffe8050141040cfa3dfb357bdff37c8748c7771e173453da5d7caa32972ab2f5c888fff5bbaeb5fc812b473bf808206930fade81
ef4e373e60039886b51022ce68902d96ef70ffffffff02a0860100000000001976a9141ccb6cdf8532053a64ca373af6937e30e6c777c588acf0ab5527000000001976a91461b469ada61f37c620
010912a9d5d56646015f1688ac000000000100000001f447cee6d838a27925255da2a6d7860e444f5756030f6a56627516daa22e4943010000008a4730440220149c03c3c103cdb357880c233f19
086227c7cf86f742685422af9d90da765613022065b65d0b0fe902e2855f22efd46b98314a438a953e275ce9699d72c83a36e4a10141040cfa3dfb357bdff37c8748c7771e173453da5d7caa3297
2ab2f5c888fff5bbaeb5fc812b473bf808206930fade81ef4e373e60039886b51022ce68902d96ef70ffffffff02a0860100000000001976a91440e5a9d4d0a234b4a4650d5cebdb34c8cef64c01
88ac40fe5327000000001976a91461b469ada61f37c620010912a9d5d56646015f1688ac000000000100000010a79b31b2f26ca4b14f74bb59e43100e96a322b1fbfb137803ed8532cf7e2afd801
0000006a4730440220103378427919d1bc487387778f55c72bb493f6aa499acc4cb32d8f4786c7457402200ede74cc94eb7fb0741f5fc3ede7198df8596713e80a8ce725ec3920b255650e012103
7cdd27d2b0629a985ffd30582dd002fb11ea4b0c6dace88d88058b58b42c8f19ffffffffff9ff404c85b55466686181072302d4a7275fbac44ae69170fb99c4aa4d6259d000000006a4730440220
52bf62231e5559ddd577304191f7f08afdba35d3f46461a802fd77c4a25e0f550220725f9ecc9e14806859df4edcaa262568a2ca19ccffb89bc4731619171e13aad80121032b8575df5ce09b9d7a
f6d4963e659784024c687c136141c2e88989fdfb1f7289ffffffff88dced12aca197e0f60c3c60c975b4ff568311f70a034d260196a5ddf3297cf5000000006b483045022100810c8f9cbb39b171
6d6b3eb590940fae9b9019f28d171ff434fd160f6a744d9202200438b52d8d533597ebd335672f9a08925f5784c7c8340baf9663b803ff6d4cf50121028267c1d5835efb33cc44dae34fb43943dc
7c9fa61cd401e5528fef7721bd1ca8ffffffff4580997e49afc8f71a82cac1eb1d8d5053d0bb1c65e33cd10412dbbb4a91fbdf000000006a4730440220587cef4257cdd5935a8d1eb6527c33016c
68d8926052f400da561785d3b3586602205a6bd45afd2796e9d9b21bb0ec007058cbf203331cad9c1e6eb170739f74e65a0121032b8575df5ce09b9d7af6d4963e659784024c687c136141c2e889
89fdfb1f7289ffffffff73a5e7a93247445d433e5a142de8cfd78804a77849cae953e137406231feb51a000000006b483045022100c5e785f3b9ba8f75d2e01b547fbb3311bb7e5ed3a7db3d2b94
6c1f08eb1502d402204cf5e3e4dfbd9865c627985bc463a2875e082a1f237ed7c7a1363b7b151bb0cb0121032b8575df5ce09b9d7af6d4963e659784024c687c136141c2e88989fdfb1f7289ffff
ffffa3b21377e02d7ce0499054f32d6ca249ef5d83b81ed9bd501e27b102067efb74010000006b483045022100c5fd8839e6fed697c2962218d94f1013046e246a8527ddebb44165dc11dcae5e02
2055de74532525ebbb2a526d26621e1742fe3a203d16116b690e41acd3062c78ee0121028267c1d5835efb33cc44dae34fb43943dc7c9fa61cd401e5528fef7721bd1ca8ffffffff824289a402d4
f294241cfd16e9c903916976a8bdc7eca3159efacee541b27e5b020000006b4830450221008fb29bc5a0747e1355dc7abe008e2198e2fc340072b42dbaba0a9a86fa338497022028172cfbc8c5c4
79aca8a60a4ea8893c5b27ab32bee6f7418dc6ea615ba523fc01210263eb6ca14b17ffc533c4e198baa87b7d8e050c221229af0967fc826c1a062e68ffffffffcab2718b3e8f933366c2e2b5e357
72408c92f4fd2bdf36c69dd9fd8fe498f273010000006b483045022100e4b37ca9cb4295a18f7eb077c530f75963e31dc51e37c75c95fcbfae63be27ff02201da58447ab05e4c299ae3cf3554b32
e5ba1c8c08e2223f97e0f90d01dc68bde70121028267c1d5835efb33cc44dae34fb43943dc7c9fa61cd401e5528fef7721bd1ca8ffffffffd169e16e12f3fec972938809b6442972c39d06db8b1d
43b1a971b23c9cf60454010000006a47304402206f622d8685c526e1fe9a03ba63c095c3d46d0f700cdd7df6574ae3631b11c7f60220457ed1eeaa0f9f95efa81a567f0f209812d2be3bb2fdafb7
b057896b227217740121028267c1d5835efb33cc44dae34fb43943dc7c9fa61cd401e5528fef7721bd1ca8fffffffff0adf93c931132b867e5c99d803fb701b518bcc4fea1649916252f325af1c1
eb000000006a4730440220786a9e78be7fde24380f809f34da364750bc476681e62a5f23f8560b935f0eb1022038aa1b7027f95ce526521a736ef8e1eada065b3ca45192ad0a3e60f0dfc9879501
21028267c1d5835efb33cc44dae34fb43943dc7c9fa61cd401e5528fef7721bd1ca8ffffffffb9163749af6a6415166b376c85e004eea9c217abe50c68576385fc735f294752000000006a473044
02206921eaa40f8b082cbf7c57dadfd39c4ccdaed4a63805e0b7ffd26dc2cfde33d80220234c030a9116cab6e017f262017c92cbc755238c4e817143fe038c974ace6b2e0121028267c1d5835efb
33cc44dae34fb43943dc7c9fa61cd401e5528fef7721bd1ca8ffffffff2ef2d0d07d0d418c5edac06ff13148d5d328d3e0905cbe7e9677cf4a0996ece5010000006a47304402205645ac1bb77c02
5e14df186ffbfae0576b16a9e49af14a11fa652d5d2cf34e9802202fdb31dacd20fb1580165f6b46b7769a95b98110b858fc6b88e39436bdbcdf5801210292f4384445aa0d75f08fd41bd9b5c45b
d2164050cf5afcdb2eb87612d16989c0ffffffff87203006da6cc95fc05061dcee1c1f520639aa6c5ff1e3871bab5522068f38d4010000006a473044022005a4ff3e626f086924355eed38026d50
d4104363ce7e78e1b473686e13c5337a0220470b9ab0681ffaed8df9e06fd3b0473dc16963fc97a21145d2739def96440b60012102ea16c2a1fa4ecfa89a8912d2d15e91fd589c6313d90a69eb0e
00a53519067e1affffffffb6c7530cb9f24e7f2e274d8dfa919ec8edbe2bcf77aeda03b74d2c5375734af6010000006a47304402200732099e6060f5bea2248a7516ae82c77d3f1012f8ca708f4f
71f131e5185f2f02200435f70f317a065900fc6cf8746b6b197a7782bc2efe9176282b7637c19016a1012102ea16c2a1fa4ecfa89a8912d2d15e91fd589c6313d90a69eb0e00a53519067e1affff
ffffc6e2790723d84e92f2dc9016175aaf96997db755ae6468d68adca647bfb8fee4010000006b483045022100ca7039ed6753318e2cd291ed2e321d9748d344719584e94ba6cb66d35c38605402
207b3f7fcf7b0399ba7fed72e96ee629377ab088c42c75d860d696751a696dab26012102ea16c2a1fa4ecfa89a8912d2d15e91fd589c6313d90a69eb0e00a53519067e1affffffffd9a3d7ad7791
bf17fcd6857fe1007a19e97202788e169dd102af579a0d09b083010000006a47304402205d7fc9d2b3153d922e75b1f4cf7d44dabcea94ec4e89a3b951296a81c15b980202202029bb445e3db282
3dfb73dc9f878acac71c73f04fdc2b65ddce434538ed278d012102ea16c2a1fa4ecfa89a8912d2d15e91fd589c6313d90a69eb0e00a53519067e1affffffff0240420f00000000001976a914f37d
cc18906ffe8b766d9ed3ac3576b5c32c653e88ac80969800000000001976a914a54a325280fa5901fa05894dabd51876028a897a88ac00000000�
09:28:03 2014-07-02 [ERR] RPCS: SubmitBlock DecodeString: encoding/hex: invalid byte: U+00EF 'ï'

If we remove the last byte on odd lengths in our submitblock handler, it worked fine, but that hides a bug. bitcoind currently would have accepted that cgminer submitblock.

I am not sure if the problem is in ParseHex or jsonspirit get_str, but I think bitcoind shouldn't accept bad hex.

Perhaps a fail test should be added as well.

I added a LogPrintf to bitcoind's submitblock to show it accepts odd length submissions.

2014-07-02 12:18:46 SUBMITBLOCK LEN=5321
02000000a3090a2471a37945c3e4e60c69374d78aa7e9fa3a969d8b0e237d67a00000000d88fe8048ae99d58bdcb446a68a431d3fd5f4432410f80f5e0fdb2fe7c93cb0086f8b3536431011b234e16850601000000010000000000000000000000000000000000000000000000000000000000000000ffffffff2a03e20f0407062f503253482ffe87f8b353fe24aa00000963676d696e6572343208270b00000000000000ffffffff0150bc0395000000001976a91485eb47fe98f349065d6f044e27a4ac541af79ee288ac00000000010000000208d222cfe5b858a3033c4a7b2423054a0ffac8c7dd515956647618449d79dc8a00000000fd5e01004730440220641c6fa710bcaa50671c2b3e0be0943c8be660e09fda12544786f87e0ac8742402204293a65d56d85170925b568060d50637402ee598a79c9bd5e87f352e63812f2401493046022100ae004945da84c33a8b496ee44deb9b6847aadda1d44069ecfdce4fc8496da097022100b18a688fa9cd6af907855fef834ade30a8db9fbf96bd1e2ac4f9c1ccd6cf2e14014cc95241041fea7862a56ef665df0bffc86345936a631cb4b54ee3c2ae9a497681d3bc31d6427bc43ffca99ddcdb0c14e5414fdfffd4e6daeee596c1514338c091bf9303fa41045309a7280c3947b7ec2d44127d0b3b607f70566e36e980c948e370d6415a33c586c35eed1b6aec672c9dd50560a3ab1e4f8ef9cd9a1e7f2f31338b90448ab31d4104960e28cae8f3848c840a518427d86529b14632f33ecbb8aac7b71ebc7eaf88a171170244a25cb9e3f4b39e8a19cef55bbaaf319f3b8a3b1cb9a3ef693012b0db53aeffffffff3468c34d8b7c0ca54e98a1d9b8ba76ad2e8da332ee359a20d5b04041ac9baa7d00000000fd5e01004830450221008309fb7b0dcb9e693326bf1be4b2f2a3d70100b42d0679e0915d57129e7dd3aa02201482906920b0f136d5c0fb70444fc6f553a11b0d3723015e628d16f174e15b4101483045022053203cbb3018a0632732478431810033d534a4bfcdb60300aa5b7300f2283dbb022100bc189944efd3dca0ec8ebddbf8b9bf7cc298de73712126f3cd564a65e7179a8f014cc95241041fea7862a56ef665df0bffc86345936a631cb4b54ee3c2ae9a497681d3bc31d6427bc43ffca99ddcdb0c14e5414fdfffd4e6daeee596c1514338c091bf9303fa41045309a7280c3947b7ec2d44127d0b3b607f70566e36e980c948e370d6415a33c586c35eed1b6aec672c9dd50560a3ab1e4f8ef9cd9a1e7f2f31338b90448ab31d4104960e28cae8f3848c840a518427d86529b14632f33ecbb8aac7b71ebc7eaf88a171170244a25cb9e3f4b39e8a19cef55bbaaf319f3b8a3b1cb9a3ef693012b0db53aeffffffff0200c2eb0b000000001976a9147c6bfd1ba181169adf0ccd21dcfa41ff0284216a88acf02e21830000000017a91405f5e7e132752a5c59838e62052e72a430c380a68700000000010000000181590891946dbc30c967e503ded6ca300a5b4bb58958a57609078f27e89fbe32010000008b483045022100ac070699c7e29100b5e2e4ab0d422ba623f6dbb5119d30619a0d494da0e423c902200c640e548bf4281c7c6794156e9c17390dc0a1245356c29c64220cfc1a4771700141040cfa3dfb357bdff37c8748c7771e173453da5d7caa32972ab2f5c888fff5bbaeb5fc812b473bf808206930fade81ef4e373e60039886b51022ce68902d96ef70ffffffff02a0860100000000001976a914ef46ce8577717e9615a117916195938eb5b4291b88ac10997a27000000001976a91461b469ada61f37c620010912a9d5d56646015f1688ac000000000100000001cd9f6bfd6c43525fae7d016b6f4641b2cca6250e6a9846bfed5a19eca35ddfed010000008b483045022100eacf488f71d039170d6b1bcb2f60d21fc09e3302acb8005b13953109c019ed0702204c363661dcf3b85db1c4ef020717e87c5894f41abd660877e8c883151b09980d0141040cfa3dfb357bdff37c8748c7771e173453da5d7caa32972ab2f5c888fff5bbaeb5fc812b473bf808206930fade81ef4e373e60039886b51022ce68902d96ef70ffffffff02a0860100000000001976a9144214fe42050f045d584ddbf4f180f7f5ca7dde1c88ac60eb7827000000001976a91461b469ada61f37c620010912a9d5d56646015f1688ac000000000100000001dfdbddf55ec8faec45bce3b92506aea504f6e04ab5fffbfeadacfda607d7a4d8010000008c493046022100cc730d44ea1213a13ae422a60e744a9de131f007e5c89a651cadd74844073575022100d94f6d0755b9259f03170ae191acffa6ed9570d03e7d21e586084d2bec2b7b610141040cfa3dfb357bdff37c8748c7771e173453da5d7caa32972ab2f5c888fff5bbaeb5fc812b473bf808206930fade81ef4e373e60039886b51022ce68902d96ef70ffffffff02a0860100000000001976a9144ad1aa8528ef993ded8a3ed222ddf372a21dbad888acb03d7727000000001976a91461b469ada61f37c620010912a9d5d56646015f1688ac00000000010000000542a1fa05adaca5dd13299d9e9071bb91df0d39f43fcbbf8e5271582657d97859000000006a4730440220717162ed652f00c3a41656429e39b3a20b590a2718dbb57e847abda4304f5c23022036ecef3f0d66ef7bcf51953469288c4ce3c63cd38a2e8d35b73bb63361939537012102d4d1e01947df603d2677d3273125a268fbc6a2aad030ca768a0bb73aa41cbfd1ffffffffda801e41b63d2adfa33027219202319eac905e80d9203b4bbac1341462c67bd6010000006b483045022100f129aade1f7af8a137ac41fb0dab4250c7931b21dea537dc3fa9391f250ca91b02204dd06ac68c37a30339a17cf91f0c1991dc5ac9392b8c9f39fd5ae03ec4925094012103c926c2443ab2699b8079ddc6a1a9d69e0000c7cc80f8976dee30b75a22edad54ffffffff28b33d2d9868062c6aa317d9e194a80b2e2dd0e3bdd379fe16b9d7bd9d6b57fe010000006b483045022100b739d38a76b4b2a72a98815700daeff06b94a577eb48abc7f0773956de8b38ad022010937e93932a14a8b5de9510e651fce80bdf21915a0b8cd767ebc1522228d781012102507f689f6f182c0618cd1b503cebcbcc755f873de1ac8a2118e12fe378bafe4bffffffffcaacff9dded54a393baebfc34a869926190f5bed361315164b0fc308c9043ba7010000006a47304402203e15222e2f6bbb90c4d666885037ce82316fb3baff7566deb422f27150c1cfda02201164bbad394c0170341ec4752a2f9f39f951c373013977cfda6c47c1779928b4012102c9a69adb30ee1de0fde9b4a6da2045b0af05ecdf0740f6de8cf7a7fc6630ecb6ffffffff015ab4577347a8dbf5b151ef33d69e5546ebb005d302f10d7132d92ee5667c09010000006a4730440220096662843ee6789af94ea32174b4b0d1a547182de2367d65ee4bb85e70e3350a0220524ad3b788abf4e71048d6554b8c8cfd2db00c9beca7b2a0655602179b65725e012102b8cd59b8dfdf53300a7f3100d8ec4f42c6e591c7a384c27432e552dd2956f0e6ffffffff02d0401705000000001976a9149cf1cf53e89437db05364faab2ce5d0028cbe80b88ac4afb1500000000001976a914790e75f9ab937e9e91c086280a596842b93e37f288ac000000000
2014-07-02 12:18:47 UpdateTip: new best=000000000000c2c2c67fe0ea231aaf265e145392b43829de7912b42cad0cbb15  height=266210  log2_work=60.015838  tx=1811865  date=2014-07-02 12:18:14 progress=1.000000

I think perhaps more importantly than it simply accepting an odd length (which there is nothing wrong with) is the fact it's accepting hex with invalid characters in it. In particular, the character at the end of the submission is 0xa0. Note: That's not 0x0a (\n).

As a test, I submitted the byte sequence 0x22, 0x30, 0x32, 0xa0, 0x22 which is "02�" to both decodescript and submitblock. Obviously the two " will be stripped during the JSON unmarshal.

The call to decodescript gives: -8: argument must be hexadecimal string (not '02�') which is what one would expect when submitting invalid hex. However, the same data passed to submitblock gives: -22: Block decode failed meaning it's accepting the invalid hex and passing it on block decoding.

decodescript uses ParseHexV (from rpcserver.cpp), which first calls IsHex, throws an exception if invalid hex, or parses the string if valid. submitblock uses ParseHex directly, without verifying that the string is valid, and ParseHex ignores all bad input so failure cannot be detected.

For reference, ckolivas/cgminer#621 fixes the cgminer bug.

While I'm glad the cgminer bug is fixed, I still think submitblock should reject invalid hex.

cgminer has released a new stable version that includes the fix.

https://bitcointalk.org/index.php?topic=28402.msg7887585#msg7887585

- Fixed a corruption when solo mining (ALL SOLO MINERS URGED TO UPGRADE!)

Not sure since when, but this is no longer an issue. DecodeHexBlk, called by submitblock calls IsHex(), which returns false when there are non-hex characters or the number of characters is not an even number.