Raw transaction format not error robust #1626

The current raw transaction format is not error-robust. Its trivial to corrupt transactions accidentally (e.g. accidentally dropping characters or adding characters when copy/pasting).

There are some use-case patterns where this could be quite dangerous. I prepare a txn for you to pay me some coins you control... I give it to you.. You decode and from your perspective it looks fine, but a character got dropped. You sign and transmit. Coins gone forever.

The hex strings are already inconveniently enormous. Sticking a CRC or a 64bits of a truncated hash on the end wouldn't be much additional burden (and if we cared about size at all we could make them much easier to handle if they used base64 instead like our signatures).

This would be a minor burden on application code, but its easily solved with a library and would close off a lot of mistakes and mischief.

I'd stick the checksum at the front, after a type octet (so we don't confuse transactions with signatures, etc). Maybe even some kind of could-replace-base58?

not that it matters for data this small, but checksum at the front means you can't emit any data until you have all of it.

How so? If you're checking the checksum, you need to wait until you have all the data no matter where the checksum is. If you're not checking it, you can emit the data as soon as you get it...

Because it requires buffering on the transmitter side as well as on the receiver, and buffering of the whole txn which may be hundreds of KB. In either case the receiver only needs to buffer the crc state (e.g. if it could process the txn in a streaming fashion).

This does have some really vague applicability here: if I build some 8bit microcontroller based secure hardware signing dohicky... I may not have enough memory to buffer the whole output txn before printing it. Though I admit that this is somewhat fringe, but not entirely It's not uncommon for small secure devices to only have a few kb of memory.

Oh okay, that makes sense (and I agree justifies post-checksum).

The JSON-RPC interface is meant for machines, not people. I think it would be a mistake to add error-detection at the raw transaction API, it will just make it harder for machines to interact with it.

Besides, the Bitcoin transaction serialization format is somewhat robust to random errors anyway, and if you're signing a transaction without decoding it and making sure all the txouts are going to where you think they should go... well, then I have little sympathy for you.

It's really easy to have a subtle error that would be unnoticeable, a 3 for an E in the middle of an address, etc... The fact that the first time I tried to do it for anything 'real' I ended up with corruption makes me a bit concerned.

I'm not too convinced with 'harder for machines', since just validating an address takes the same work, so everyone should already have code for that. I guess I'll ponder more and see if my opinion changes after the shock of how easy it was to corrupt in practice wears off.

I still feel a bit uneasy about this; but lets see if it turns out to blow up in the field. If it does we can reconsider— what good is it to call bitcoin beta if every decision is an irreversible suicide pact?

Is it really easy to have a subtle error? One-byte-changed errors seem extremely unlikely to me; what's the scenario where those creep into the JSON stream?

Much more likely are truncation/additional-character errors due to copy/paste. And those are extremely unlikely to produce valid transactions, since the serialization format is "number of things to expect" followed by "that number of things strung together."

(I assume your corruption resulted in an invalid transaction...)