Right now the most common way to sign general messages is still with a private key that corresponds to a legacy P2PKH address. Many hardware and software projects have implemented this including Ledger, Trezor, Coldcard, and Sparrow, as well as a few multisig companies. One reason why they are so useful is they help users test that the hardware controlling their keys has not succumbed to bit rot.
This technique for message signing comes from Bitcoin Core code that Satoshi wrote (see src/util/message.cpp [link]). This was created before the BIP process, so the best technical documentation I have found, aside from the actual implementation, is this page on the Bitcoin Wiki.
While comprehensive, the Bitcoin Wiki page doesn’t make some very important technical details as obvious as they should be. Examples include:
The “Bitcoin Signed Message:\n” magic bytes. I think this is alluded to in the Displaying signed messages section. But the actual magic bytes are never stated on the page.
This wiki page also covers the rules for the signature header byte (see src/key.cpp::SignCompact()) but I found this post on Bitcoin Stack Exchange did a much better job explaining this tricky part.
Related work & message signing in the wild
There has been lots of good work to address message signing for other address types, including BIP-137, BIP-notatether-messageverify, and BIP-322, but I have yet to find a single reliable source of documentation on the “Satoshi format” of message signing described above.
This baffles me because there are plenty of projects that have implemented this. Apart from the examples listed earlier, we also have things like:
It seems like plenty of people know how to do this, so either I’m bad at reading the material we have today, or those that have come before me have spent a substantial amount of time and effort figuring it out. Can someone help me tease apart the information I have been able to gather here? Is the “Satoshi format” of message signing fully documented anywhere other than the code?
The message signing feature and encoding wasn’t designed by Satoshi, but by me, so you can direct all blame for quirks and lack of documentation here. You’re right that it predates the BIP process, so the code is really the specification I’m afraid, though with many reimplementations there are probably several languages to choose from.
It is roughly:
Serialize the string “Bitcoin Signed Message:\n” plus the message being signed in the P2P protocol serialization format (which means: prepending each with a CompactSize-encoding of the number of bytes that follow)
Double-SHA256 hash that serialization.
Construct an ECDSA signature with that double-SHA256 as hashed message z, and the address’ key as private key d (with corresponding public key Q), resulting in two integers (r, s).
That signature is encoded as a “recoverable signature”, which consists of a header byte plus 32-byte big-endian encodings of r and s. This header bytes is there to assist the verifier in recovering the public key from the signature. It can be computed by running the verification algorithm:
Let u_1 = s^{-1}z\, (\operatorname{mod}\, n)
Let u_2 = r^{-1}z\, (\operatorname{mod}\, n)
Let R = u_1G + u_2Q
From the result the “recid” can be derived:
The X coordinate of R must now either be r (recid=0 or 1) or r + n (recid=2 or 3).
The Y coordinate of R must be even (recid=0 or 2) or odd (recid=1 or 3)
The header byte equals the recid + 27 for uncompressed Q and recid + 31 for compressed Q.
Wow!! Thank you for such a fast and detailed reply @sipa! That list of steps is exactly the clarification I was trying to find. But there goes the idea of calling it “Satoshi format” (I swear I didn’t make that up!)
Thanks for digging up those resources @ajtowns! I will assign myself the task of updating the wiki. This is really great information that should be shared.
