It seems more people are leaning towards this being an update to BIP-322 over a new BIP, so I am swapping this to being an update instead of a new BIP.
I am not sure if I like the legacy format compatibility, but people asked for it elsewhere, so I'm keeping it for now.
No idea what's up with Travis.
936 | @@ -937,7 +937,7 @@ Those proposing changes should consider that ultimately consent may rest with th
937 | | Generic Signed Message Format
You'll have to add |- style="background-color: #ffcfcf" two lines above this one
Thanks, done
Regarding compatibility with legacy message signing,
I think at the moment when you are signing messages with any wallet it calculates the hash in the following way:
SHA256d(0x18 || "Bitcoin Signed Message:\n" || compactint(len(m)) || m) where 0x18 is the length of the magic prefix.
So digest for the message Hello will be:
SHA256d(0x18 || "Bitcoin Signed Message:\n" || 0x05 || "Hello")
If this digest is used for legacy p2pkh signing I think it will be easier to integrate this BIP because it will be 100% backward compatible with currently used message signing.
@stepansnigirev So the current approach is that people may use the legacy signing if they want to, in which case they will do exactly the same way (including digest derivation). If they don't, then the thing won't be backwards compatible anyway, even if we reuse the same digest derivation method. Or am I misunderstanding you?
Pushed a to-squash that overwrites BIP-322 instead of making this a new BIP. Will drop or squash once decided.
So the current approach is that people may use the legacy signing if they want to, in which case they will do exactly the same way (including digest derivation). If they don't, then the thing won't be backwards compatible anyway, even if we reuse the same digest derivation method. Or am I misunderstanding you?
I think it's ok to have a different digest (tagged hash) for the new message signing, I am just saying that legacy message signing is described in the document a bit unclear (I think).
Does this: SHA56d("Bitcoin Signed Message:\n"||m) mean I need to simply concatenate magic and the message, or do I need to prepend it with it's length?
I was not able to find any documentation for legacy signing standard so I had to look up in the code of the wallets:
Bitcoin Core: https://github.com/bitcoin/bitcoin/blob/a12d9e5fd24a25bef476c10620317e43a5905754/src/util/message.cpp#L75
Bitcoinlib: https://github.com/petertodd/python-bitcoinlib/blob/master/bitcoin/signmessage.py#L48-L63
I am not 100% sure but it looks like the magic and the message are serialized with their lengths. It would be nice to make this a bit more clear in the bip.
Regarding the new standard to sign messages - I really like it.
Concept ACK: @BlockchainCommons and a number of our patrons are very interested in supporting new message signing approach.
I think it's ok to have a different digest (tagged hash) for the new message signing, I am just saying that legacy message signing is described in the document a bit unclear (I think).
OH! That is definitely an error on my end. I'll fix the legacy description.
Dropped 992c5e3 tentatively. Will cherry-pick if we decide this should replace BIP322 after all.
23 | +* Let <code>V</code> generate a message <code>M</code> and hand this to <code>P</code>. 24 | +* <code>P</code> generates a signature <code>S</code> by signing the message <code>M</code> using <code>k</code>. Given <code>S</code>, <code>V</code> can prove that <code>P</code> has the private key associated with <code>A</code>. 25 | + 26 | +The astute reader will notice that the above is missing a critical part, namely the pubkey <code>kG</code>, without which the verifier cannot actually verify the message. The current message signing standard solves this via a cryptographic trick, wherein the signature <code>S</code> above is a special "recoverable signature" type. Given the message <code>M</code> and the signature <code>S</code>, it is then possible to recover the pubkey <code>kG</code>. The system thus derives the address for the pubkey <code>kG</code>, and if it does not match <code>A</code>, the proof is deemed invalid. 27 | + 28 | +While this is a neat trick, it unnecessarily restricts and complicates the message signing mechanism; for instance, it is currently not possible to sign a message for a P2SH address, because there is no pubkey to recover from the resulting signature.
Isn't it "there's no standard way to get from pubkey to the address" rather than "no pubkey to recover from the resulting signature" ?
Surely the signature could be of recoverable type, and if a prover and verifier use the same script template, the verifier can reconstruct the address and check (if all the other info such as other pubkeys are known to verifier and the template can be filled completely)
With a p2pkh there is a pubkey hash, and with p2sh there isn't. Yes, there may be one or multiple keys embedded in the script hash, but there's no standardized way of determining that, as opposed to the p2pkh case.
Edit: more importantly, the recoverable ECDSA signature can recover only one pubkey, so a multisig key, for example, doesn't actually work.
That was exactly my point in regards to the phrase "because there is no pubkey to recover from the resulting signature." -- It does not convey the correct meaning. The correct meaning is along the lines of "there's no standardized way of determining ...", "there's no standard way to get from recovered pubkey to the address"
I see what you mean, changing.
19 | +== Background == 20 | + 21 | +* Assume two actors, a prover <code>P</code> and a verifier <code>V</code>. 22 | +* <code>P</code> wants to prove that they own the private key <code>k</code> associated with a given address <code>A</code> (which in turn is derived from the pubkey <code>kG</code>). 23 | +* Let <code>V</code> generate a message <code>M</code> and hand this to <code>P</code>. 24 | +* <code>P</code> generates a signature <code>S</code> by signing the message <code>M</code> using <code>k</code>. Given <code>S</code>, <code>V</code> can prove that <code>P</code> has the private key associated with <code>A</code>.
I think it should read "V can check" rather than "V can prove", as it is the prover who proves, and the verifier who verifies (checks)
Good point, yeah. Changing.
67 | + 68 | +1. Valid, if it is a successful spend according to the current consensus rules (sometimes called "policy"). 69 | +2. Inconclusive, if it is a successful spend according to consensus rules, but NOT according to policy rules 70 | +3. Invalid, if it is not a successful spend according to consensus rules 71 | + 72 | +A proof is the base64-encoding of the message_signature as is. A validator takes the to be proven pubkey and the proof and transforms it to virtual transactions as described above.
It seems to me that it should be "to be proven script and the message" rather than "pubkey".
The "to_spend" transaction contains a hash of the message in the input and the script "message_challenge" in the ouptut. If no standard script templates are defined that would receive only one pubkey, then the verifier will need to know the script and how to insert the pubkey there, along with a way to construct witness to spend the script (miniscript could help, although won't cover all possible scripts).
Yeah, you're right. Added 'message'.
I'm not sure what you mean by the second part though -- I think the spending transaction's witness data should be enough, no? I need to write the reference implementation to verify though.
Maybe I do not fully get what data the prover would provide. Do they provide the full script (including the pubkeys) and the witness data to satisfy this script, that will contain any needed signatures ? Will it be the verifier's job to check that the script matches some template that prover and verifier agree beforehand ?
The prover provides the message_signature:
message and address.
To be even more clear, the way this is meant to be done is exactly like the Signet approach, except I'm disallowing scriptSigs and only allowing script witness, where Signet allows both types.
I.e. the signet block signature is the equivalent of the proof by the prover here, the block hash in Signet is the equivalent of the message hash here, and the block challenge in Signet is the equivalent of the prover's address.
Ah, I believe I understand now:
The witness data will include the script, if the scriptPubKey in the to_spend transaction is p2sh or p2wsh. And the because the prover and verifier agreed on a certain address beforehand, they agreed on the exact script, too.
Yup!
61 | + vout[0].nValue = 0 62 | + vout[0].scriptPubKey = message_signature 63 | + 64 | +It may include other inputs, to facilitate a proof of funds. 65 | + 66 | +A message signature is considered valid, inconclusive, or invalid based on whether the to_sign transaction is a valid spend of the to_spend transaction or not, according to the following steps (also see Consensus and standard flags section further down):
If the signature check is performed by the script interpreter, it will honor sighash flags. I think the document should specify that the signature must include SIGHASH_ALL flag, otherwise signature with SIGHASH_NONE would verify with any message_challenge
Also, scriptSig is not included in segwit signature hash, rather the script is included. It may not be clear for the implementors how they should make the signature commit to message_hash. I think the spec should say that the signature should be over sighash calculated with:
script = OP_0 PUSH32[ message_hash ]
hash_type = SIGHASH_ALL
amount = 0
sigversion = SIGVERSION_WITNESS_V0
(witness sigversion will avoid stripping CODESEPARATOR that happens with sighash calculating with sigversion = SIGVERSION_BASE)
Thanks, yep. I think requiring it to be SIGHASH_ALL is beneficial, since this also supports proof of funds.
For some reason I didn't see your 2nd response until now. I added a note that they must use SIGHASH_ALL, which commits to the message hash as expected. What do you think?
Moved the conversation about the rest to the vin[0].scriptSig = OP_0 PUSH32[ message_hash ] line.
42 | + nVersion = 0 43 | + nLockTime = 0 44 | + vin[0].prevout.hash = 0000...000 45 | + vin[0].prevout.n = 0xFFFFFFFF 46 | + vin[0].nSequence = 0 47 | + vin[0].scriptSig = OP_0 PUSH32[ message_hash ]
The SignatureHash function in Core does not take anything from scriptSig. It takes the scriptCode as the supplied argument, and later uses it in the hashing.
AFAIU this scriptSig = OP_0 PUSH32[ message_hash ] is intended to convey that the message hash will be committed to as part of the script supplied to SignatureHash. But as SignatureHash does not use the contents of scriptSig in the input at all, and only uses the script that is supplied as argument, I believe that it would be more clear to state this explicitly. This is what I meant in an earlier message. The document should just specify all the relevant arguments to SignatureHash, roughly: SignatureHash(CScript([OP_0, message_hash]), tx_to_spend, 0 /* nIn */, SIGHASH_ALL, 0 /* amount */, SigVersion::WITNESS_V0)
(edited the above message, added sigversion as the parameter to SignatureHash)
The way to calculate the hash that will be signed need to be unambiguous. If we choose to use SigVersion::WITNESS_V0 for sigversion, then amount should be fixed, because it will affect the hash.
We do not actually sign this transaction, we sign the transaction spending it.
So it will be included (committed to) in the sighash via txid in the prevout of to_sign transaction. And all the arguments to the SignatureHash except signature type can be derived from the message_challenge script to_spend transaction structure.
It is clear to me now, thanks.
57 | + nLockTime = 0 58 | + vin[0].prevout.hash = to_spend.txid 59 | + vin[0].prevout.n = 0 60 | + vin[0].nSequence = 0 61 | + vout[0].nValue = 0 62 | + vout[0].scriptPubKey = message_signature
Why is message_signature is put in the output ?
IIUC, message_signature is supposed to be provided as base64-encoded proof, not included in the transaction itself.
If the signature is over this exact "to_sign" transaction, it cannot be included at the time of signing, because there will be a circular dependency (the sighash will be dependent on the output, which have to include the signature)
That looks like an oopsie (that is also repeated in BIP-325). Good catch, fixing!
This seems like it should just be a revision to BIP 322?
P.S. I still think this whole concept is misguided since it doesn't address the conceptual problems with legacy signmessage.
This seems like it should just be a revision to BIP 322?
People seem to have differing opinions, but I am inclined to agree with you.
P.S. I still think this whole concept is misguided since it doesn't address the conceptual problems with legacy signmessage.
Yeah, I know. I haven't seen any concrete suggestions on how to make it to your liking, though. My channels are open, as the Lightning folks would say..
@kallewoof Something like this https://dpaste.com/5P6QXZGD7
@luke-jr Going through your diff and commenting on stuff:
OP_TRUE for proof of funds is a good idea, yep. They're proving by signing the inputs after all, as long as the signatures are actually committing to the bip-322 input (is this guaranteed?).+When a proof of funds is being created, additional outputs should be included for virtually spending transaction outputs of desired value.Pushing a to-squash for comparison.
I removed the background because it was wrong and I didn't see an obvious way to fix it. Current signed messages don't prove you have the private key.
I don't care strongly how the signatures are serialised, but I don't think more than two elements (message+signature) should be required. Any other serialisation I could think of quickly, however, seemed to increase complexity excessively.
Oh, and note that serialising the entire transactions leaves a lot of room for future extension... Minimising it might not.
Serializing the entire thing means you need to do that extra step as the verifier to ensure it actually contains the necessary stuff, but it seems like it's probably worth it.
I have removed the Background for now.
With everything included, there are several foot-gun points, so I clarified what validators need to check, beyond the actual tx validation.
Note: a discussion on how to handle OP_CLTV/CSV was made on IRC, resulting in a pushed additional section and two added outcomes (valid_in_future and inconclusive_in_future).
There are some parts that may need discussion, in particular:
to_spend transaction.Right now,
~Pushed a change to the to_spend transaction: the first output's nValue was originally set to 0, but is now set to the sum of all inputs (which is zero when no additional inputs exist).~
To determine if 'in the future' flag should be present will require knowledge of whether CLTV/CSV opcodes will actually be executed in the script given particular witness data.
Executing the script with CLTV/CSV will require the to_sign transaction to have locktime / sequence to be set accordingly, and the signature would need to be made over the transaction with these fields set.
Checking that CLTV/CSV was actually executed would require additional instrumentation of the interpreter, or running the interpreter two times for each opcode: first with transaction locktime / sequence set for CLTV/CSV to be always successful, and then to always fail (and also likely to need distinct signatures for each variant).
I doubt that handling CLTV/CSV is needed at all for the simple "signature for the address" case. Just say that if your script has these opcodes, you should use "proof of funds" feature.
The "need to set relevant tx fields in to_sign" issue is still there for "proof of funds" case, so I think it should be specified how checking "the CLTV and/or CSV conditions are not yet met" means that nLockTime on to_sign transaction has to be set for CLTV and that "proof of funds" UTXOs should specify correct sequences.
Writing code for this, I realize it's really foot-gunnery, so I added a validation section specifying what a validator must do. @dgpv Fair points. What if the wording was changed to say the in_future part is optional? I.e. the verifier may check if there is a time lock encumbrance, and if it does, it may reflect this using the in_future flag. Would love to hear Murch's thoughts as he brought this up initially.
What if the wording was changed to say the in_future part is optional?
Even if it is optional, it still have to be specified.
For example, current spec says nLockTime = 0 on to_sign transaction, but for proof-of-funds when scripts has CLTV, nLockTime has to be set accordingly. For the spec to be consistent, it has to account for this.
It seems that the prover need to to set certain transaction fields on to_sign and Validator has to check them (is nLockTime in the future according to current chain ?)
For "signature for the address" case, if the verifier chooses to support the in_future flag, the prover has to set nLockTime/nSequence fields on to_sign before signing, and the verifier needs to set them to the same values, and these values need to be communicated. Of course they can be communicated as a full transaction, and then verifier would need to validate the transaction to check that other fields are as written in the spec. It seems as in this case the complexity gets closer to the 'proof of funds' case, and also the data to be exchanged is the same (except the to_sign transaction passed from prover to verifier does not contain real inputs).
It seems to me that it is easier to introduce a degenerate 'no inputs' proof-of-funds case where prover passes the signed transaction to the verifier, and the verifier validates and then verifies.
The simple "signature for the address" case then becomes a further simplified case where the to_sign transaction is static.
Looks like there are three cases:
The full "proof of funds": multiple dynamic to_spend (taken from blockchain) and a dynamic to_sign (provided by the prover)
The complex case of "signature for the address": single static to_spend and a dynamic to_sign (provided by the prover)
The simple case of "signature for the address": single static to_spend and single static to_sign, only the signature provided by the prover, and put by the verifier into the input of to_sign
It makes sense to make the case 3 mandatory and other two optional, but they need to be thoroughly specified anyway.
It might make sense to specify the case 3 in one BIP and the other two (more complex) cases in another BIP, so that a simple "signature for the address/script without fancy opcodes" spec can become complete and be adopted earlier
so that a simple "signature for the address/script without fancy opcodes" spec can become complete and be adopted earlier
I think this is the most practical case where the lack of modern "signature for invoice address" standard causes pain for people, and the "proof of funds" and "proof of invoice address with fancy script" are much less profound cases.
For example, current spec says
nLockTime = 0onto_signtransaction, but for proof-of-funds when scripts has CLTV,nLockTimehas to be set accordingly. For the spec to be consistent, it has to account for this.
Ah, yes you're right. I'll update the spec to reflect this.
For "signature for the address" case, if the verifier chooses to support the
in_futureflag, the prover has to setnLockTime/nSequencefields onto_signbefore signing, and the verifier needs to set them to the same values, and these values need to be communicated. Of course they can be communicated as a full transaction, and then verifier would need to validate the transaction to check that other fields are as written in the spec. It seems as in this case the complexity gets closer to the 'proof of funds' case, and also the data to be exchanged is the same (except theto_signtransaction passed from prover to verifier does not contain real inputs).
Currently, the values are communicated as the two transaction pairs serialized as normal, which means the signer can set an arbitrary nLockTime and the verifier can just adopt/check that.
It seems to me that it is easier to introduce a degenerate 'no inputs' proof-of-funds case where prover passes the signed transaction to the verifier, and the verifier validates and then verifies.
This was the initial approach. The reason it's now the two transactions is that it becomes a lot more complex to have multiple types. For instance, if I prove with no additional inputs ("signature for the address") you get a signature only, but when I prove with inputs, you get a signature and the inputs, and now you have to decide if those should include the virtual input, and you get complexities when signing where you have to put it in and then remove it, unless you keep it there.
Yes, proofs are a little bigger, and you need deserialization code even for the non-PoF case, but you need that anyway, cause you need to be able to independently generate the sighash.
Looks like there are three cases:
- The full "proof of funds": multiple dynamic
to_spend(taken from blockchain) and a dynamicto_sign(provided by the prover)- The complex case of "signature for the address": single static
to_spendand a dynamicto_sign(provided by the prover)- The simple case of "signature for the address": single static
to_spendand single staticto_sign, only the signature provided by the prover, and put by the verifier into the input ofto_signIt makes sense to make the case 3 mandatory and other two optional, but they need to be thoroughly specified anyway.
It might make sense to specify the case 3 in one BIP and the other two (more complex) cases in another BIP, so that a simple "signature for the address/script without fancy opcodes" spec can become complete and be adopted earlier
It feels like it wouldn't be useful to have 1-2 unless they were a non-optional part of the specification. Even 3 would require you to have most of the functionality that is required by 1 (serialization, signature verification, transactions, etc).
(As a sidenote, I'm still wondering if it should optionally allow for "help" transactions, i.e. taken from the (U)TXO set or wallet. This would help when proving for a spent outpoint, and when giving a proof to a verifier who doesn't have access to the UTXO set.)
Checking that CLTV/CSV was actually executed would require additional instrumentation of the interpreter, or running the interpreter two times for each opcode: first with transaction locktime / sequence set for CLTV/CSV to be always successful, and then to always fail (and also likely to need distinct signatures for each variant).
Btw, I may be mistaken, but I believe the CLTV/CSV checks will succeed, but the locktime/sequence values in the transaction itself will determine if it is in the future or not.
In other words, the check will succeed, but the lock time will be e.g. 100 blocks from now, or 7 days in the future.
Btw, I may be mistaken, but I believe the CLTV/CSV checks will succeed, but the locktime/sequence values in the transaction itself will determine if it is in the future or not.
In other words, the check will succeed, but the lock time will be e.g. 100 blocks from now, or 7 days in the future.
When the prover provides the to_sign transaction with locktime/sequence fields already set, the verifier that just executes the script will not know if the script is only spendable in the future. To know that, they will need a way to detect that CLTV/CSV in the script is executing. If the verifier only has one to_sign (already signed) transaction from the prover, then without instrumented script interpreter, they won't know if the script can fail with different locktime/sequence.
On the other hand, they can just assume that if the to_sign transaction has locktime or any input sequence set to non-zero, the script can conatin CLTV/CSV, and therefore they shall set in_the_future flag to true, regardless of whether these opcodes are actually present/executed.
There seems to be no need to set these fields on to_sign for any other purpose, so maybe the spec can say something like "if nLocktime or any of nSequence for the inputs are not zero on the to_spend virtual transaction, the in_future flag is set to true"
Currently, the values are communicated as the two transaction pairs serialized as normal, which means the signer can set an arbitrary nLockTime and the verifier can just adopt/check that.
For the simplest "sign for address" case where only message, script, and witness is required to be communicated it may be more convenient to send just that data, without sending redundant transaction data. The transaction data is redundant in this case because it can be constructed by the prover. Why spend space to send prevout.hash of to_spend transaction if it is known to be all-zeroes ? (edit: the simplest case I'm talking about here is also "no fancy opcodes" case, but even with CLTV/CSV, nLocktime/nSequence can be optionally added without other "static" tx parts)
There may be usecases when the number of characters to display the proof is limited by the medium
but when I prove with inputs, you get a signature and the inputs, and now you have to decide if those should include the virtual input
If the "sig for address" and "proof of funds" are distinct cases, there should be no confusion - in the first case you have virtual input, in the second case you'r not. Or alternatively, always have virtual input, but do not sign it for the "proof of funds" case. The verifier are concerned with proof for individual inputs, nothing bad happens if they would check the witness of the inputs when to_sign is "partially signed" (no sig for virtual input) - they won't be able to broadcast the tx anyway, so no need for it to be fully signed
Yes, proofs are a little bigger, and you need deserialization code even for the non-PoF case, but you need that anyway, cause you need to be able to independently generate the sighash.
To generate a sighash you only need serialization code, not deserialization. (re "this is the same conceptually" - yes, but unifying ser/deser usually requires a language powerful enough and some effort to engineer this)
When the prover provides the
to_signtransaction with locktime/sequence fields already set, the verifier that just executes the script will not know if the script is only spendable in the future.
I believe you've got it wrong: OP_CSV and OP_CLTV put constraints on the transaction's locktime. If they fail, it means the locktime is too low. The locktime is enforced outside of the transaction script check, because a transaction with a locktime in the future may not be put into a block. So simply looking at the locktime of the transaction is sufficient to know if it's spendable now or not.
You're right that providing the two transactions and everything for a simple address proof is overkill. I'll switch it to optionally allow only the script witness for cases without proof of funds.
It feels like it wouldn't be useful to have 1-2 unless they were a non-optional part of the specification. Even 3 would require you to have most of the functionality that is required by 1 (serialization, signature verification, transactions, etc).
1-2 add complexity because they have more dynamic parts. The case 3 may set the basis for the functionality requirements, but uses this functionality in fewer ways because more of the data is static.
Having the basic case specified in one spec, and then further extend it in a separate spec for other use-cases seems to me like an adequate way to manage complexity. I'm talking more about spec complexity than implementation complexity. The simple spec without extra details (details that are relevant only to non-mainstream cases) is likely to be better understood and implemented. For example, for the "case 3 only", the spec do not need to say anything about locktime/sequence other than "they are zero".
So simply looking at the locktime of the transaction is sufficient to know if it's spendable now or not.
That's right, but this only works if you know the current height of the blockchain.
If you don't know the current height, you can only assume that transaction may be unspendable. This may be sufficient, as I said in
On the other hand, they can just assume that if the to_sign transaction has locktime or any input sequence set to non-zero, the script can conatin CLTV/CSV, and therefore they shall set in_the_future flag to true, regardless of whether these opcodes are actually present/executed.
I'm talking more about spec complexity than implementation complexity.
So am I. The spec grew in a way I disliked which is why I aborted. I've now pivoted back, as you can see in the latest revision.
Splitting into two specs may be the best course of action in the end - for now, I've split the validation for the two types into two paragraphs..
115 | +* All signatures must use the SIGHASH_ALL flag. 116 | +* The proof is considered valid, inconclusive, or invalid based on whether the to_sign transaction is a valid spend of the to_spend transaction or not, according to the rules specified in the "Consensus and standard flags" section below. 117 | +* It may (optionally) be encumbered with the in_future flag, according to the rules specified in the "Locktime and Sequence" section below, in which case we refer to the result in text form as "valid_in_future", "inconclusive_in_future", etc. 118 | 119 | -A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input. 120 | +Proofs with additional inputs are the base64-encoding of the to_spend and to_sign transactions concatenated in standard network serialisation, and proofs without additional inputs (simple proofs) are the base64-encoding of the to_sign script witness.
Simple proofs are not only without inputs, but also without scripts that require nVersion, nLockTime and/or nSequence to be set to non-zero values
Proofs of funds are the base64-encoding of the to_spend and to_sign transactions concatenated in standard network serialisation, and proofs without additional inputs (simple proofs) are the base64-encoding of the to_sign script witness.
Let's use the same term for this concept consistently.
Added "or time locks".
135 | +# deserialize the to_spend and to_sign transactions from the proof, and fail if the proof contains extraneous bytes 136 | +# verify that any additional inputs being proven (proof of funds) are included as inputs in the to_sign transaction, exactly once 137 | +# reconstruct the to_spend' and to_sign' transactions, based on the specification above, copying the version, lock time, and sequence values 138 | +# verify that to_spend = to_spend', that to_sign has at least 1 input, has exactly 1 output, and that to_sign.vin[0] = to_sign'.vin[0] 139 | +# optional: set the "in_future" flag if the transaction's lock time is in the future according to consensus rules 140 | +# in "coins", a mapping of outpoints (hash, vout) to coins (scriptPubKey, amount), let coins(to_spend.txid, 0) = (to_spend.vout[0], 0)
The next sentence uses the phrase "coins map". I think using it here would make this easier to read: Establish a "coins map": a mapping of outpoints (hash, vout) to coins (scriptPubKey, amount), initialized to coins(to_spend.txid, 0) => (to_spend.vout[0].scriptPubKey, 0)
Sounds good
140 | +# in "coins", a mapping of outpoints (hash, vout) to coins (scriptPubKey, amount), let coins(to_spend.txid, 0) = (to_spend.vout[0], 0) 141 | +# for each proof of fund input, set the corresponding values in the coins map; abort if the input cannot be found 142 | +# check the signature of each input using consensus rules, then upgradable rules 143 | 144 | -While omitted below, ERROR is returned if an unforeseen error occurs at any point in the process. A concrete example of this is if a legacy proof is given as input to a non-legacy address; the deserialization of the proof will fail in this case, and this should result in an ERROR result. 145 | +To validate a simple proof, the following steps must be taken:
I think simple proof have to come first, so if the reader is only interested in basic "simple proof" usecase, they will be presented with it immediately, rather than read a long list of items and then realize "hey, I do not need all that, I only need these two items"
Of course, swapped.
Hey, I just wanted to clarify: I did bring up the question what would happen if the script contained CSV/CLTV, but it was more from a "what breaks this"-kinda perspective. My intention would be to have the proposal sufficiently robust that it doesn't mislead verifiers about spendability in the case of locktime, i.e. something like "may only be spendable in the future" seems reasonable. It may be interesting to chat with people at Lightning Labs that are working on Lightning Loop whether they have ideas. Being able to sign messages including HTLCs may be of interest to them.
23 | -* <code>P</code> generates a signature <code>S</code> by signing the message <code>M</code> using <code>k</code>. Given <code>S</code>, <code>V</code> can prove that <code>P</code> has the private key associated with <code>A</code>. 24 | - 25 | -The astute reader will notice that the above is missing a critical part, namely the pubkey <code>kG</code>, without which the verifier cannot actually verify the message. The current message signing standard solves this via a cryptographic trick, wherein the signature <code>S</code> above is a special "recoverable signature" type. Given the message <code>M</code> and the signature <code>S</code>, it is then possible to recover the pubkey <code>kG</code>. The system thus derives the address for the pubkey <code>kG</code>, and if it does not match <code>A</code>, the proof is deemed invalid. 26 | - 27 | -While this is a neat trick, it unnecessarily restricts and complicates the message signing mechanism; for instance, it is currently not possible to sign a message for a P2SH address, because there is no pubkey to recover from the resulting signature. 28 | +A standard for interoperable signed messages based on the Bitcoin Script format, either for proving fund availability, or committing to a message as a recipient of an invoice address.
A standard for interoperable signed messages based on the Bitcoin Script format, either for proving fund availability, or committing to a message as the owner of an invoice address.
"Recipient" feels a bit odd in the context, because one could sign a message before ever receiving a payment to an address. I must admit that "owner" doesn't feel great either, but it's the best I have for "person in control of the private key corresponding to the invoice address in question".
A custodian might allow customers to sign messages with their designated deposit addresses, but the customers may not control the keys and just receive the promise of being able to withdraw equivalent amount in the future (potentially from completely unrelated UTXO).
It seems to me that "committing to a message as the intended recipient of the funds to be sent to the invoice address" would be more fitting description
Excellent point! I like the rephrasing, but think it could be put a bit more succinctly: "committing to a message as the intended recipient of funds sent to the invoice address".
Changed!
30 | == Motivation == 31 | 32 | -The current message signing standard only works for P2PKH (1...) addresses. By extending it to use a Bitcoin Script based approach, it could be made more generic without causing a too big burden on implementers, who most likely have access to Bitcoin Script interpreters already. 33 | - 34 | -== Specification == 35 | +The current message signing standard only works for P2PKH (1...) invoice addresses. By extending it to use a Bitcoin Script based approach, it could be made more generic without causing a too big burden on implementers, who most likely have access to Bitcoin Script interpreters already.
The current message signing standard only works for P2PKH (1...) invoice addresses. We propose to extend and generalize the standard by using a Bitcoin Script based approach. This approach minimizes the burden for implementers as message signing can be expected to be part of a library or project that includes Bitcoin Script interpreters already.
34 | -== Specification == 35 | +The current message signing standard only works for P2PKH (1...) invoice addresses. By extending it to use a Bitcoin Script based approach, it could be made more generic without causing a too big burden on implementers, who most likely have access to Bitcoin Script interpreters already. 36 | 37 | -A new structure <code>SignatureProof</code> is added, which is a simple serializable scriptSig & witness container. 38 | +Additionally, the current message signing only proves that the message has been committed to by the recipient of a given invoice address. 39 | +It does not prove anything about the invoice address itself, nor that the signer has access to the private keys used to implement this invoice.
I'm not sure I understand this sentence. My impression was that the signer has to have access to the private key to create a signature for the message. Since the current standard only supports messages for P2PKH, would that then not be equivalent to being able to spend funds locked to the address invoice? I'm also not sure what you are trying to express with "implement this invoice". If that just means "spend funds from the address", it would be best to repeat the same terminology for the same procedure. Being a bit repetitive in a technical text is acceptable for clarity's sake.
I can forward the message to Fred, who owns the keys, and he signs it and hands it back to me. Now I don't have the keys myself, but I am able to produce that signature for you, because Fred allows me. He may not allow me to send from a UTXO associated with that invoice address though, but that's another story.
106 | + vin[0].scriptWitness = message_signature 107 | + vout[0].nValue = 0 108 | + vout[0].scriptPubKey = OP_RETURN 109 | 110 | -For both cases, generate a sighash based on the given scriptPubKey and message as follows: 111 | +When a proof of funds is being created, additional inputs should be included for virtually spending transaction outputs of desired value.
When a proof of funds is being created, additional inputs MUST be included for virtually spending transaction outputs of desired value.
It is not clear to me how the proof of funds would work if the inclusion of the additional funds is optional. Wouldn't that have to be a requirement?
Well, to turn it around, if I give you a proof and it has inputs, it's a proof of funds for those inputs. If you ask me for a proof of funds for input A and I give you one for input B, it's the same as if I gave you a proof for bc1qA when you asked for bc1qB.
In that sense, I think "should" is correct, since it's up to the prover to put in what the verifier is asking for.
Still a bit on the fence for this one. Obviously, the challenged does not have to fulfill a challenge, but responding with other information instead seems a bit pointless. :)
118 | 119 | -A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input. 120 | +Proofs with additional inputs are the base64-encoding of the to_spend and to_sign transactions concatenated in standard network serialisation, and proofs without additional inputs (simple proofs) are the base64-encoding of the to_sign script witness. 121 | 122 | -=== Signing === 123 | +A validator must verify it is valid and meets the description of virtual transactions as specified above. See "Validation" below.
A validator must verify that the to_sign transaction is valid and meets the description of virtual transactions as specified above. See "Validation" below.
Let's avoid "it" and err on repeating the exact data that needs to be valid.
No they also have to make sure both of them are valid, because the prover could give a to_spend transaction that e.g. doesn't include the challenge.
It seems to me that it was not completely clear what is encompassed by "it" in that sentence. "It" is singular, so if this "it" refers to more than one item that needs to be validated, it may be helpful to explicitly list all the items again.
127 | 128 | -# Derive the private key privkey for the scriptPubKey; FAIL if not VALID 129 | -# Generate and return a signature sig with privkey=privkey, sighash=sighash 130 | - 131 | -=== Verifying === 132 | +To validate a proof with additional inputs, the following steps must be taken:
To validate a proof of funds, the following steps must be taken:
We should always use the same term to refer to the same concept.
Good point, yeah
131 | -=== Verifying === 132 | +To validate a proof with additional inputs, the following steps must be taken: 133 | 134 | -Verify a proof, given a standard flags value, a script sig, an optional witness, and a derived sighash as described above. 135 | +# deserialize the to_spend and to_sign transactions from the proof, and fail if the proof contains extraneous bytes 136 | +# verify that any additional inputs being proven (proof of funds) are included as inputs in the to_sign transaction, exactly once
# verify that the to_sign transaction uses all inputs covered by the proof of funds exactly once
134 | -Verify a proof, given a standard flags value, a script sig, an optional witness, and a derived sighash as described above. 135 | +# deserialize the to_spend and to_sign transactions from the proof, and fail if the proof contains extraneous bytes 136 | +# verify that any additional inputs being proven (proof of funds) are included as inputs in the to_sign transaction, exactly once 137 | +# reconstruct the to_spend' and to_sign' transactions, based on the specification above, copying the version, lock time, and sequence values 138 | +# verify that to_spend = to_spend', that to_sign has at least 1 input, has exactly 1 output, and that to_sign.vin[0] = to_sign'.vin[0] 139 | +# optional: set the "in_future" flag if the transaction's lock time is in the future according to consensus rules
I would propose that we choose to either not cover locktime at all (i.e. have the spec fail to make a determination), or explicitly state how it should be treated. I would worry that optional features in a standard would otherwise lead to variable behavior that could cause compatibility issues later on.
I think it may make sense to not make this optional now that we've split out into two types of proofs (simple proofs and PoF).
155 | 156 | -The legacy format is restricted to the legacy P2PKH address format. 157 | +The legacy format is restricted to the legacy P2PKH invoice address format. 158 | 159 | -Any other input (i.e. non-P2PKH address format) must be signed using the new format described above. 160 | +Any other input (i.e. non-P2PKH invoice address format) must be signed using the new format described above.
Perhaps it should be recommended here to prefer the general format even for P2PKH going forth.
This was discussed in an earlier draft of this proposal, and then people were opposing the conclusion (that was to use legacy for P2PK(H)). I'm going to push for using the new format and see what people think, so:
New proofs should use the new format for all invoice address formats, including P2PKH.
The legacy format MAY be used, but must be restricted to the legacy P2PKH invoice address format.
Co-authored-by: Stepan Snigirev <stepan.snigirev@mpq.mpg.de>
Co-authored-by: Luke Dashjr <luke_github1@dashjr.org>