BIP 322: Generic Signed Message Format

kallewoof commented at 8:08 am on September 10, 2018: member

Formatted version: https://github.com/kallewoof/bips/blob/bip-generic-signmessage/bip-0322.mediawiki

kallewoof force-pushed on Sep 10, 2018

sipa commented at 5:18 pm on September 10, 2018: member

Does the scriptPubKey need to be included? That can be derived from the address to check it against.

in bip-generic-signmessage.mediawiki:108 in 58368027e4 outdated

103+
104+The specification can be extended to handle proof of funds in the following manner:
105+
106+* Let the message be prefixed with "POF:", followed by a newline-terminated string<ref><strong>Why not just the UTXO data?</strong> We want the verifier to be able to challenge the prover with a custom message to sign, or anyone can reuse the POF proof for a set of UTXO:s once they have seen it, and the funds have not yet been spent</ref>, followed by [entries] series of hex-encoded transaction ID:vout pairs, separated by a single space (" ") character
107+* Fail if the number of txid:vout pairs is not exactly equal to [entries]
108+* Retain the message as is for all sighash operations (i.e. all sign and verify operations sign and verify for the entire list of UTXO:s)<ref><strong>Why use same sighash?</strong> The prover is proving that they have a set of UTXO:s at their disposal. Taking a sub-set of the proofs and turning into a new proof should not be valid.</ref>

odarboe commented at 6:44 pm on September 10, 2018:

Suggested improvement: Line 108: Retain the message as is for all sighash operations (i.e. all sign and verify operations should sign and verify the entire list of UTXO:s ………..Taking a sub-set of the proofs and turning them into a new proof should not be valid.

or Line 108: Retain the message as is for all sighash operations (i.e. all sign and verify operations should validate the entire list of UTXO:s ………..Taking a sub-set of the proofs and turning them into a new proof should not be valid.

kallewoof commented at 3:21 am on September 11, 2018:

Thanks, will tweak!

jl2012 commented at 6:47 pm on September 10, 2018: contributor

This scheme might not be compatible with OP_CODESEPARATOR. While it is rarely used, it changes scriptCode and affects the signature validity. So you need to concatenate scriptCode with the message in order to fully support OP_CODESEPARATOR

A related problem is since your message does not commit to the public key, it is possible to “recover” the public key from any random signature and random message, with no one has the private key. (I think the existing message system has the same problem)

I also have a related idea: turn an unused opcode (e.g. 0xf0) to OP_MESSAGEONLY. The code remains invalid in bitcoin txs and requires no change in consensus. In message system, however, it is interpreted as OP_NOP. So this allows people separating their message keys and money-holding keys. With MAST, the use of OP_MESSAGEONLY could be totally invisible except to the message validators.

in bip-generic-signmessage.mediawiki:75 in 58368027e4 outdated

70+=== Signing ===
71+
72+The "Sign" action takes as input a scriptPubKey and a message (e.g. "hello world"). It succeeds or fails.
73+
74+# FAIL if scriptPubKey already exists in scriptPubKeys set, otherwise insert it<ref><strong>Why track duplicates?</strong> Because a 3-entry proof is not proving 3 scriptPubKeys unless they are all distinct, or unless they are proving different UTXO:s (see Future Extensions)</ref>
75+# Derive the private key privkey for the scriptPubKey, or FAIL

instagibbs commented at 7:00 pm on September 10, 2018:

privkeys?

kallewoof commented at 3:23 am on September 11, 2018:

There is one privkey for each scriptPubKey I assume (this is in the individual case, even though it is tracking the SPK’s in the SPK set for early abort).

maaku commented at 7:45 pm on September 10, 2018: contributor

Why not construct a bitcoin-like transaction instead of something custom? Make a fake tx with the address as the output and a fake tx spending that output with the hash of the message in an OP_RETURN. That way existing hardware signers and other infrastructure could sign messages without changes.

jl2012 commented at 7:58 pm on September 10, 2018: contributor

Why not construct a bitcoin-like transaction instead of something custom? Make a fake tx with the address as the output and a fake tx spending that output with the hash of the message in an OP_RETURN.

I agree with @maaku , except that the SignatureHash() must be slightly modified to make sure that the “fake tx” couldn’t be valid on the mainnet. For example, replace nSigHash with nSigHash|0xff000000

That way existing hardware signers and other infrastructure could sign messages without changes.

This is very dangerous as the hardware wallet couldn’t tell if they are signing a message or a tx. Changes are needed, just as minimal as possible.

in bip-generic-signmessage.mediawiki:77 in 58368027e4 outdated

72+The "Sign" action takes as input a scriptPubKey and a message (e.g. "hello world"). It succeeds or fails.
73+
74+# FAIL if scriptPubKey already exists in scriptPubKeys set, otherwise insert it<ref><strong>Why track duplicates?</strong> Because a 3-entry proof is not proving 3 scriptPubKeys unless they are all distinct, or unless they are proving different UTXO:s (see Future Extensions)</ref>
75+# Derive the private key privkey for the scriptPubKey, or FAIL
76+# Define the message pre-image as the sequence "Bitcoin Message:" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD)
77+# Let sighash = sha256(sha256(pre-image))

jl2012 commented at 8:11 pm on September 10, 2018:

using plain double-sha256 here is very dangerous, as attackers could trick people to sign real txs this way. You need to either pad something before or after the message, or simply use a different hash function.

for example, it could be sighash = sha256(sha256(msg)|0xff).

If the size of msg is small enough to be shown on the wallet screen, the wallet will ask user to confirm the message and will do the whole double hash

If the size of msg is too big, the untrusted computer will send sha256(msg) to the wallet, and the wallet will only do the second hash. Careful users will use a trusted machine to verify the hash (they don’t need special software because it’s simple sha256). Even not verified, the worst case would be signing a wrong message, not sending bitcoin to attacker

kallewoof commented at 3:25 am on September 11, 2018:

The msg is prefixed with the string sequence “Bitcoin Message:”, so that should not be a problem.

I initially went for single sha256 but realized they could then potentially pass the single sha256 of a transaction and it would thus become the actual txid. But that relies again on the “Bitcoin Message:” part being ignored.

kallewoof commented at 3:26 am on September 11, 2018:

As for a long message, perhaps there should be a limit on message size. I don’t think I understand the benefits of single-sha256’ing and showing that. It doesn’t seem like it would convey anything at all to the user.

jl2012 commented at 8:38 pm on September 11, 2018:

The msg is prefixed with the string sequence “Bitcoin Message:”, so that should not be a problem.

This is 16 bytes at the beginning. To attack it, a 12 bytes (96 bit) collision is needed. Bitcoin mining is now 82bit/day, so it is 45 years for 96 bit. Not likely, but also not impossible.

The only way to make it completely safe (in terms of legacy sighash and BIP143), is to put a non-zero byte at the end of the message.

I don’t think I understand the benefits of single-sha256’ing and showing that. It doesn’t seem like it would convey anything at all to the user.

Maybe, but this is what Ledger Wallet is currently doing.

kallewoof commented at 5:50 am on September 12, 2018:

@jl2012

This is 16 bytes at the beginning. To attack it, a 12 bytes (96 bit) collision is needed. Bitcoin mining is now 82bit/day, so it is 45 years for 96 bit. Not likely, but also not impossible.

The only way to make it completely safe (in terms of legacy sighash and BIP143), is to put a non-zero byte at the end of the message.

Right now, the sighash is defined as sha256(sha256(scriptPubKey || pre-image)) where pre-image is “Bitcoin Message:” for a regular signmessage call. The added scriptPubKey should address the problem, right?

Maybe, but this is what Ledger Wallet is currently doing.

Interesting. I didn’t realize this. I will have to read up on why they do that.

maaku commented at 9:40 pm on September 10, 2018: contributor

Modifying the signature hash defeats the point of having drop in hardware signer support. Make the setup fake transaction a coinbase or otherwise give it a fake input that cannot conform.

luke-jr commented at 10:04 pm on September 10, 2018: member

The point of hardware signers is to confirm (and perhaps track) spending amounts. A signed message, however, should not present as a spend. So either way, the hardware needs to explicitly support signing messages - and therefore IMO a modified signature hash is fine.

in bip-generic-signmessage.mediawiki:104 in 58368027e4 outdated

 99+
100+== Future Extensions ==
101+
102+=== Proof of Funds ===
103+
104+The specification can be extended to handle proof of funds in the following manner:

luke-jr commented at 10:06 pm on September 10, 2018:

This should just be part of the specification IMO, providing a clear distinction between proof-of-funds (which only verifies an amount, not address(es)) and proof-of-receiving-at-address.

kallewoof commented at 3:27 am on September 11, 2018:

I was aiming for a simple and lean proposal and this was only meant as a “you CAN do this, by extending this in the future”, but it turned into a more complete proposal than I intended. You’re probably right.

kallewoof commented at 5:50 am on September 12, 2018:

I’ve rewritten the proposal to include this as part of the proposal.

in bip-generic-signmessage.mediawiki:26 in 58368027e4 outdated

21+
22+== Specification ==
23+
24+A new structure <code>SignatureProof</code> is added, which is a simple serializable scriptPubKey, scriptSig & witnessProgram container.
25+
26+Two actions "Sign" and "Verify" are defined.

luke-jr commented at 10:06 pm on September 10, 2018:

Should elaborate on what is being signed/verified.

kallewoof commented at 3:28 am on September 11, 2018:

The two sections further down are meant to do that. Is something missing?

luke-jr renamed this:
~~BIP: Generic Signed Message Format~~
BIP 322: Generic Signed Message Format
on Sep 10, 2018

luke-jr added the label New BIP on Sep 10, 2018

kallewoof commented at 3:32 am on September 11, 2018: member

@sipa

Does the scriptPubKey need to be included? That can be derived from the address to check it against.

The idea here is to only include the scriptPubKey. The conversion from address to scriptPubKey would be done by the UI/IX. This means a signmessage implementer could let users sign for a specific address (“message”) or a txid:vout (“POF”) or something else from which scriptPubKeys can be derived.

kallewoof commented at 3:37 am on September 11, 2018: member

@jl2012

This scheme might not be compatible with OP_CODESEPARATOR. While it is rarely used, it changes scriptCode and affects the signature validity. So you need to concatenate scriptCode with the message in order to fully support OP_CODESEPARATOR

The implementation (for Bitcoin Core) would literally add a new BaseSignatureChecker class that takes a predefined sighash (the message sighash) and executes VerifyScript as normal. Would that actually be affected by OP_CODESEPARATOR in any way? The sighash is the message only, so it shouldn’t matter, right?

A related problem is since your message does not commit to the public key, it is possible to “recover” the public key from any random signature and random message, with no one has the private key. (I think the existing message system has the same problem)

So, include the scriptPubKey in the sighash, then?

I also have a related idea: turn an unused opcode (e.g. 0xf0) to OP_MESSAGEONLY. The code remains invalid in bitcoin txs and requires no change in consensus. In message system, however, it is interpreted as OP_NOP. So this allows people separating their message keys and money-holding keys. With MAST, the use of OP_MESSAGEONLY could be totally invisible except to the message validators.

I love the idea, but not sure if it’s worth using an entire op code when you can probably solve it with simple message prefix and/or (as above) a custom sighash format (including scriptPubKey).

kallewoof commented at 3:40 am on September 11, 2018: member

@maaku

Why not construct a bitcoin-like transaction instead of something custom? Make a fake tx with the address as the output and a fake tx spending that output with the hash of the message in an OP_RETURN. That way existing hardware signers and other infrastructure could sign messages without changes.

It feels like unnecessary overhead, when all you really want to do is

for a given scriptPubKey P
given a message M
- create a signature S that signs M for P
- prove/verify that S signs M for P

Why would you even bother with transactions in this case? It adds unnecessary data to the proof, and unnecessary complexity to the prover/verifier.

kallewoof force-pushed on Sep 11, 2018

sipa commented at 3:52 am on September 11, 2018: member

@kallewoof But I don’t understand why you’re including the scriptPubKey in the signature. Transactions don’t have that either. You verify them against a scriptPubKey.

kallewoof commented at 3:55 am on September 11, 2018: member

@sipa Oh, I see what you mean. Yeah, I think it makes sense to require the verifier/prover to retain the order of the SPKs which means they are not needed in the proof. Fixing.

kallewoof force-pushed on Sep 11, 2018

maaku commented at 4:11 am on September 11, 2018: contributor

@kallewoof It shouldn’t add any overhead as far as I can tell. It’s just a different convention: this BIP adds “Bitcoin Message:” to the beginning and hashes. What I suggest above would have you deterministically create a sequence of two bitcoin transactions, the first containing an output with the scriptPubKey, and the second “spending” it. In a slight revision of my earlier suggestion, the salted hash of the message to be signed would be placed in the input.txid of the first transaction, thereby preventing it from ever being interpreted as a valid transaction.

Yes this is more data structures constructed in memory to fake something that looks like a real transaction, but it’s all deterministically constructed using fixed rules, and isn’t included in the proof any more than the string “Bitcoin Message:” is in yours.

kallewoof commented at 4:22 am on September 11, 2018: member

I see what you’re saying, and I’m not really against it, but I would love if someone could explain why it is better than just a sighash.

kallewoof force-pushed on Sep 11, 2018

maaku commented at 4:46 am on September 11, 2018: contributor

@kallewoof how do you get your hardware wallet or corporate HSM to sign your scheme, without a firmware update from the manufacturer?

kallewoof commented at 4:49 am on September 11, 2018: member

@maaku I flat out assumed a firmware update would be necessary to support this. I’m definitely intrigued if it is possible to support this with NLS’es and Trezor’s without an actual firmware update!

maaku commented at 4:58 am on September 11, 2018: contributor

The point is you make it look like a bitcoin transaction, you sign it like a bitcoin transaction, you verify it like a bitcoin transaction, and nothing in the signing path even needs to be aware it is not actually a bitcoin transaction.

It also works well with proof of reserve: the proof of reserve is a bitcoin transaction spending all the funds, but with an additional input (covered by SIGHASH_ALL) that points to a fake/invalid tx. This has the additional benefit of working in a forward compatible way with any future bitcoin extension, like confidential transactions or mimblewimble: your proof of reserve could have blinded inputs and outputs as well, or whatever else the bitcoin protocol is made to allow. As long as the spends are tangled up with the fake input (via SIGHASH_ALL or a mimblewimble kernel, or whatever), it doesn’t matter.

kallewoof commented at 5:17 am on September 11, 2018: member

@maaku Thanks a lot for the explanation. That definitely sounds worth it!

kallewoof commented at 5:39 am on September 11, 2018: member

@maaku After discussion with others on IRC, the forward compatibility part is pretty nice, but there are also dangers with a specification where the signers can be potentially fooled into signing one thing thinking it’s something else.

With the approach you are suggesting, it would be impossible for existing HW wallets (without a firmware upgrade) to distinguish between messages and transactions, meaning there is no way for the HW wallet to inform the user that they’re signing a message vs signing a transaction.

maaku commented at 6:53 am on September 11, 2018: contributor

Yes I’m aware of that. I’m not suggesting it as a desirable end-goal in the sense that hardware wallets should be blind to message signing. Rather it is a particularly nice feature to have now as there are hardware wallets and HSMs already deployed which won’t be receiving field upgrades, either from lack of upstream support or because they are explicitly designed NOT to in the corporate HSM setting. Signing things not understood at the time of firmware burn-in and key loading would otherwise be an attack vector for emptying funds held by the HSM. (But there are enough ways to make a transaction invalid that the HSM would see it as a signed transaction which never confirms and is double-spent according to whatever protocol governs the HSM.) There is even greater need for this with respect to corporate HSMs doing proof of reserve, then raw message signing.

Beyond being a good transition mechanism now, the same argument holds for any future bitcoin protocol upgrade. If you add confidential transactions, whatever general signing/proof-of-reserve system you come up with now would have to be updated to support CT. If you use (fake) bitcoin transactions, you get it for “free,” on any device that gets the CT upgrade.

maaku commented at 7:14 am on September 11, 2018: contributor

Oh I probably should mention this explicitly: to address the “sign something that looks like a transaction” concerns you can make the transaction pay to the same owner, so any signing interface shows a “0 btc” transaction.

This is probably needed anyway to get around any signing logic restrictions enforced by the HSM.

jl2012 commented at 7:48 am on September 11, 2018: contributor

@maaku: being unable to upgrade is a feature, not a bug. If the firmware designer wanted to allow message signing, it would have been already supported. If not, I’d assume that’s intentional.

The whole point of this BIP is to define a generic format, which could cover any previous and future scripts. For an unupgradable device, however, everything are set in stone so forward compatibility makes no sense to them

maaku commented at 8:55 am on September 11, 2018: contributor

Not on some of the commercial HSM systems I’ve actually worked on at least, where an essential part of the security model is the inability to change the signing logic of the device once it is switched on and the keys generated — signing logic governing what sorts of signatures the device is allowed to create. Otherwise you’re just one firmware update away from signing anything, including a steal-all-the-monies transaction, which introduces a central point of failure with however manages the firmware update process.

And I simply disagree on forward compatibility making no sense, and this BIP points to examples which disprove that—proof of reserve messages. An exchange with an HSM rate limiting withdraws is exactly the situation where in you might want to add this message signing feature to an already deployed high-value HSM which takes the extra security precautions of dis-allowing live updates to its signing authorization logic. So you masquerade the proof-of-reserve message to look like a zero-fee transaction to itself, which it IS authorized to sign.

On Sep 11, 2018, at 12:48 AM, Johnson Lau notifications@github.com wrote:

@maaku https://github.com/maaku: being unable to upgrade is a feature, not a bug. If the firmware designer wanted to allow message signing, it would have been already supported. If not, I’d assume that’s intentional.

The whole point of this BIP is to define a generic format, which could cover any previous and future scripts. For an unupgradable device, however, everything are set in stone so forward compatibility makes no sense to them

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bips/pull/725#issuecomment-420180567, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEOIov1q8QujVZ7j_AmQtx7LDO3WfEaks5uZ2regaJpZM4Wg57j.

prusnak commented at 9:33 am on September 11, 2018: contributor

First, thanks for the draft.

Second, there is already a way how to sign/verify messages via Segwit-in-P2SH and Native-Segwit addresses: see https://github.com/bitcoin/bitcoin/issues/10542#issuecomment-316032523 We are using this solution in Trezor for almost a year now and other wallets use this too already.

Third, I am against implementing generic SignMessage via a specially crafted transaction. This will severely cripple UX for hardware wallets that actually show the contents of the transaction during the signing. If the usecase is proof-of-funds, that might be the option, though, but for nothing else.

jl2012 commented at 9:51 am on September 11, 2018: contributor

@maaku If the HSM designer wanted the proof-of-reserve feature, they would/should have done it already (e.g. opendime). If they did not include this feature at the design stage, I’d assume it’s intentional.

Also, there is no guarantee that whatever you propose would be compatible with every existing HSM. It totally depends on the policy of the HSM. For example, it might not allow inclusion of an OP_RETURN or any non-whitelisted scriptPubKey, even if it is 0 value, so you have nowhere to hide your message.

in bip-0322.mediawiki:24 in 61156b66d8 outdated

19+
20+The current message signing standard only works for P2PKH (1...) addresses. By extending it to use a Bitcoin Script based approach, it could be made more generic without causing a too big burden on implementers, who most likely have access to Bitcoin Script interpreters already.
21+
22+== Specification ==
23+
24+A new structure <code>SignatureProof</code> is added, which is a simple serializable scriptSig & witnessProgram container.

kallerosenbaum commented at 5:47 pm on September 11, 2018:

I thought the witness program was the 2-40 bytes after the witness version in the scriptPubKey. It should be the witness, or possibly witness field here, no? If so, several places from here on should change.

kallewoof commented at 3:44 am on September 12, 2018:

I’m not entirely sure what you are suggesting, but the scriptPubKey is <version> <32 byte hash> where the latter is a hash of the witness program. The witness program itself is in the spending transaction’s input, not in the scriptPubKey. The scriptPubKey simply commits to it.

kallerosenbaum commented at 6:34 am on September 12, 2018:

BIP141 says the hash IS the witness program. I asked about this on twitter (https://twitter.com/kallerosenbaum/status/949630404302196736) and @sipa explained it well in that thread:

“I think of the P2WPKH or P2WSH hash as a program itself. It’s certainly an unusual one, but that hash defines the semantics entirely. Would you agree it’s a program if it we extended to allow including actual opcodes in the 2-40 bytes?”

kallewoof commented at 6:48 am on September 12, 2018:

Okay, I see what you’re saying. So basically, what I call witness program is the witness, and the scriptPubKey is the witness program. In fact, I may even wanna call it (witness) redeem script, since it could potentially refer to a P2SH as well, I think.

Does that match your view?

kallerosenbaum commented at 7:18 am on September 12, 2018:

Not really. I think you should simply call it “witness”. That’s how it’s defined in BIP141. The witness in turn is (in case of p2wsh) composed of input data and a witnessScript. The witnessScript corresponds to the redeemScript in the p2sh case.

To clarify, I suggest the following rewording: “A new structure SignatureProof is added, which is a simple serializable scriptSig & witness container.”

kallewoof commented at 7:21 am on September 12, 2018:

Sounds good. Thanks!

kallewoof commented at 7:22 am on September 12, 2018:

2f15277

in bip-0322.mediawiki:70 in 61156b66d8 outdated

65+
66+=== Signing ===
67+
68+The "Sign" action takes as input a scriptPubKey and a message (e.g. "hello world"). It succeeds or fails.
69+
70+# FAIL if scriptPubKey already exists in scriptPubKeys set, otherwise insert it<ref><strong>Why track duplicates?</strong> Because a 3-entry proof is not proving 3 scriptPubKeys unless they are all distinct, or unless they are proving different UTXO:s (see Future Extensions)</ref>

kallerosenbaum commented at 5:56 pm on September 11, 2018:

It’d help if the scriptPubKeys set was defined in the previous paragraph. Or say “… already exists in (the initially empty) scriptPubKeys set, …”. The same goes for the Verifying algorithm.

kallewoof commented at 3:45 am on September 12, 2018:

It felt obvious, but I can clarify, sure.

in bip-0322.mediawiki:105 in 61156b66d8 outdated

100+The specification can be extended to handle proof of funds in the following manner:
101+
102+* Let the message be prefixed with "POF:", followed by a newline-terminated string<ref><strong>Why not just the UTXO data?</strong> We want the verifier to be able to challenge the prover with a custom message to sign, or anyone can reuse the POF proof for a set of UTXO:s once they have seen it, and the funds have not yet been spent</ref>, followed by [entries] series of hex-encoded transaction ID:vout pairs, separated by a single space (" ") character
103+* Fail if the number of txid:vout pairs is not exactly equal to [entries]
104+* Retain the message as is for all sighash operations (i.e. all sign and verify operations should sign and verify the entire list of UTXO:s)<ref><strong>Why use same sighash?</strong> The prover is proving that they have a set of UTXO:s at their disposal. Taking a sub-set of the proofs and turning them into a new proof should not be valid.</ref>
105+* Add a verification that the txid/vout is a valid UTXO according to a synced up Bitcoin node, and that its corresponding scriptPubKey matches the one given by the proof. Return ERROR if scriptPubKey mismatch, and SPENT error if spent

kallerosenbaum commented at 6:31 pm on September 11, 2018:

You can’t verify that the corresponding scriptPubKey matches the one given by the proof. The scriptPubKey is not in the proof.

kallewoof commented at 3:47 am on September 12, 2018:

For a funds proof, the scriptPubKey is actually derived from the transaction, and the witness program commitment (the second push in the SPK) may differ from the witness program provided in the proof. I will clarify.

in bip-0322.mediawiki:129 in 61156b66d8 outdated

82+
83+# Return ERROR if scriptPubKey already exists in scriptPubKeys set, otherwise insert it
84+# If one or more of the standard flags are unknown, return INCONCLUSIVE
85+# Define the message pre-image as the sequence "Bitcoin Message:" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD).
86+# Let sighash = sha256(sha256(scriptPubKey || pre-image))
87+# Verify Script with flags=standard flags, scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness program, and sighash=sighash

kallerosenbaum commented at 6:49 pm on September 11, 2018:

How do you know which proof entry belongs to this particular scriptPubKey? I suppose these algorithms require that the verifyer first gives a list of scriptPubKeys to the signer, and that list must be kept and processed in order. If not, please clarify how ordering is managed. Or should one simply test the proof entries one by one until success? Maybe you want to leave this up to the user how to handle?

kallewoof commented at 3:49 am on September 12, 2018:

Yes, the ordering is given by the verifier and preserved by the prover.

in bip-0322.mediawiki:128 in 61156b66d8 outdated

79+
80+The "Verify" action takes as input a standard flags value, a scriptPubKey, a message, a script sig, and a witness program.
81+It emits one of INCONCLUSIVE, VALID, INVALID, or ERROR.
82+
83+# Return ERROR if scriptPubKey already exists in scriptPubKeys set, otherwise insert it
84+# If one or more of the standard flags are unknown, return INCONCLUSIVE

kallerosenbaum commented at 8:44 pm on September 11, 2018:

Isn’t it also important for the verifier to return INCONCLUSIVE for an unknown witness version?

kallewoof commented at 3:51 am on September 12, 2018:

The standard flags include a “do not allow upgradable flags” flag, which will trigger an error if a future segwit version is used (similar to how it will trigger a warning if a NOP is used).

jl2012 commented at 8:51 pm on September 11, 2018: contributor

@kallewoof

I love the idea, but not sure if it’s worth using an entire op code when you can probably solve it with simple message prefix and/or (as above) a custom sighash format (including scriptPubKey).

What I’m suggesting is orthogonal to your proposal. A OP_MESSAGEONLY = 0xf0 in the script means the script following the code would never be valid. For example, a scriptPubKey:

OP_IF OP_MESSAGEONLY <key_m> OP_ELSE <key_s> OP_ENDIF OP_CHECKSIG

For messaging purpose, OP_MESSAGEONLY is considered as OP_NOP and is ignored. A message could be signed with either key_m or key_s.

For spending, only key_s is valid.

I don’t think it is a big problem to consume a op_code. If this is a real concern, I could modify it as follow: in message system, OP_RETURN will pop the top stack. If top stack is msg in hex, it is ignored. Otherwise, the script fails.

kallewoof commented at 5:56 am on September 12, 2018: member

@prusnak

Second, there is already a way how to sign/verify messages via Segwit-in-P2SH and Native-Segwit addresses: see bitcoin/bitcoin#10542 (comment) We are using this solution in Trezor for almost a year now and other wallets use this too already.

That seems like a good temporary workaround but I think the proposal here is a better long term solution. Your feedback on the subject is very much desired, of course!

BIP: Generic Signed Message Format b925137d5f

kallewoof force-pushed on Sep 12, 2018

fix witness program -> witness 2f152773e6

in bip-0322.mediawiki:111 in 2f152773e6

106+# Return SPENT if the txid/vout is not a valid UTXO according to a Bitcoin node<ref><strong>Synced up or not?</strong> A normal verifier would use a synced up node. An auditor checking records from a client that were submitted in the past want to use a node that is synced up to the block corresponding to the proof, or the proof will fail, even if it may have been valid at the time of creation.</ref>
107+# Extract scriptPubKey from transaction output
108+# Define the message pre-image as the concatenation of the following components:<ref><strong>Why not just the UTXO data?</strong> We want the verifier to be able to challenge the prover with a custom message to sign, or anyone can reuse the POF proof for a set of UTXO:s once they have seen it, and the funds have not yet been spent</ref>
109+#* the string "POF:"
110+#* the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD), including the null terminating character (i.e. write strlen(message) + 1 bytes, for a C string)
111+#* all transactions being proven for, as binary txid (little endian uint256) followed by index (little endian uint32), each separated by a single `0x00` byte

jimpo commented at 4:40 pm on September 12, 2018:

Why have a separator character since the sizes of the TxID + Index entries are known? If a separator character is needed, U+001e or U+001f seems like a better choice than the null byte.

kallewoof commented at 3:38 am on September 13, 2018:

Cool, I didn’t know about those unicode characters. I think you’re right though. I wanted to include a separator character as I do so in the other cases, but since the entries are indeed fixed size it seems pointless.

in bip-0322.mediawiki:110 in 2f152773e6

105+# Return INVALID if the txid:vout pair already exists in `inputs` set, otherwise insert it
106+# Return SPENT if the txid/vout is not a valid UTXO according to a Bitcoin node<ref><strong>Synced up or not?</strong> A normal verifier would use a synced up node. An auditor checking records from a client that were submitted in the past want to use a node that is synced up to the block corresponding to the proof, or the proof will fail, even if it may have been valid at the time of creation.</ref>
107+# Extract scriptPubKey from transaction output
108+# Define the message pre-image as the concatenation of the following components:<ref><strong>Why not just the UTXO data?</strong> We want the verifier to be able to challenge the prover with a custom message to sign, or anyone can reuse the POF proof for a set of UTXO:s once they have seen it, and the funds have not yet been spent</ref>
109+#* the string "POF:"
110+#* the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD), including the null terminating character (i.e. write strlen(message) + 1 bytes, for a C string)

jimpo commented at 4:41 pm on September 12, 2018:

Exclude the null terminator. That seems fairly C-specific.

kallewoof commented at 3:41 am on September 13, 2018:

I was going back and forth on this. By including it, you can actually “print” the string starting with “POF:”. Without it, you need to sort of figure out where the string ends and the txid entries begin (e.g. take entry count * 36 and say string ends at character total len - that).

Perhaps this should work the same way the signature proof container works, i.e. a varint of the message length followed by the message sans NULL term char.

in bip-0322.mediawiki:90 in 2f152773e6

85+|ERROR||An error was encountered
86+|}
87+
88+== Signing and Verifying ==
89+
90+Let there be an empty set `inputs` which is populated and tested at each call to one of the actions below.

jimpo commented at 4:43 pm on September 12, 2018:

I don’t understand this. What tracks the state? Are the SignMessage/ProveFunds requests packaged into a single unit from which inputs is derived? Could this be stateless instead?

kallewoof commented at 3:43 am on September 13, 2018:

It’s simply a set that is used to ensure that no input is given twice. It is defined once and included/updated in each verify/proof call.

For example, I try to convince you that I have X bitcoin to send you, so I send you 150 proofs of individual UTXO’s. But I am sneakily including the same proof several times. I don’t think this is trivial to detect.

in bip-0322.mediawiki:92 in 2f152773e6

87+
88+== Signing and Verifying ==
89+
90+Let there be an empty set `inputs` which is populated and tested at each call to one of the actions below.
91+
92+=== Purpose: SignMessage ===

jimpo commented at 4:45 pm on September 12, 2018:

Would be helpful to see the serialization for SignMessage and ProveFunds? Or are these RPCs?

kallewoof commented at 3:57 am on September 13, 2018:

Adding serialization.

in bip-0322.mediawiki:120 in 2f152773e6

115+
116+The "Sign" action takes as input a purpose. It returns a signature or fails.
117+
118+# Obtain the sighash and scriptPubKey from the purpose; FAIL if not VALID
119+# Derive the private key privkey for the scriptPubKey; FAIL if not VALID
120+# Generate and return a signature sig with privkey=privkey, sighash=sighash

jimpo commented at 4:48 pm on September 12, 2018:

What does it mean to return a “signature sig”? Is this a SignatureProof as specified above?

kallewoof commented at 3:46 am on September 13, 2018:

Yes, will clarify.

jimpo commented at 5:02 pm on September 12, 2018: contributor

Concept ACK.

This should specify how CLTV/CSV are verified. Including an nLockTime in the SignatureProof container and an nSequence with each signature entry therein seems like it should work. Also, what is the sighash type byte on signatures produced? That should be specified.

I think the SignMessage purpose is fine as is. For the proof-of-funds purpose, constructing the sighash from a dummy, invalid transactions spending from all UTXOs that are to be signed for seems reasonable and should provide compatibility with existing hardware. As noted, the transaction should require an unspendable input to invalidate it.

kallewoof commented at 6:05 am on September 13, 2018: member

@jimpo Thanks for the valuable feedback. Unless I misunderstand what you’re saying, you are suggesting that the format for a proof of funds is an actual transaction while the format for SignMessage is not. It seems like this complicates things, but this is what @prusnak is saying as well (in that he’s opposed to a tx for signmessage but not for proof of funds), so perhaps it’s the way to go…

luke-jr merged this on Sep 19, 2018

luke-jr closed this on Sep 19, 2018

kallewoof deleted the branch on Dec 3, 2018

nodech cross-referenced this on Jun 21, 2019 from issue RPC signmessage by nodech

BIP 322: Generic Signed Message Format #725