bip-325: change signature scheme to be tx-based #947

I'm not sure if there were perhaps reasons not to do this, but maybe it is desirable to exactly structure it like transaction spending, as that would mean PSBT infrastructure can be reused for block signing.

One possibility would be, construct 2 virtual transactions. The first is a coinbase tx (perhaps intentionally BIP34-invalid), with a single output, whose scriptPubKey is the signet's challenge script. The second tx spends that output, referring to it using its txid, and its scriptSig/witness is what is put in the block. The first tx could even be cached and constant for all blocks.

PSBT compatibility would be great, yeah!

I've updated the text and the sample implementation to use this approach, it seems much better!

One reason we might not have thought of it, is that an idea had been to allow signet users to opt-in to reorgs by allowing a different challenge: ie, normally signet might be checked against "2 KEY_A KEY_B 2 CHECKMULTISIG" and the solution is "0 SIG_A SIG_B"; but if you were to use "DROP 1 KEY_A 1 CHECKMULTISIG" as your script, and KEY_A alone was used to sign short reorgs as well as the main chain, you'd be able to test how well your software handled reorgs, while people who used the normal signet challenge wouldn't see any of the reorgs. By actually constructing a tx to spend that includes the challenge, this doesn't work so well anymore.

On the other hand, it probably doesn't work anyway, because at the p2p layer, the peer doing the reorgs would try forwarding the to-be-reorged blocks to its peers, who would see them as consensus-invalid and ban the peer. So I think it's fine to drop this idea in favour of PSBT support.

aside: I think it's kind of awesome you could get a ColdCard, put it into HSM mode, and have it blocksign!

On the other hand, it probably doesn't work anyway, because at the p2p layer, the peer doing the reorgs would try forwarding the to-be-reorged blocks to its peers, who would see them as consensus-invalid and ban the peer. So I think it's fine to drop this idea in favour of PSBT support.

Even if the different signet policies are run on different p2p networks, it would be nice if the actual blockchain was identical. This would allow to restart the node and select a different signet reorg policy without a reindex/redownload (or selecting a different datadir).

A really dumb way to keep the psbt nature, but allow node operators to choose the signet reorg policy is to ask them to supply multiple challenges. Then try all and continue with the first one that matches.

I never thought of actually having multiple networks with the same underlying blockchain. That should totally work -- unfortunately signet now derives the magic from the block challenge, so you can't tweak that easily (well, you can, but it becomes a little messy). Perhaps add a -signet_magic flag that lets you override the derived one..

I think you have to have a reindex if you're applying a different reorg policy -- otherwise, suppose your chain goes 100a 101a 102a 103a 104b 105b (for height 100-105) but the "b" blocks will eventually be reorged out and replaced by 104a 105a 106a but you've only seen 104a so far and that's a less-work tip. Switching to "no reorgs" means you want to move to 104a as your tip immediately. I guess you could do that with a specialised rewind instead of a reindex?

As the challenge can only be changed at startup, simply disconnecting blocks until the just removed block can be connected again should suffice.

Any push operations that do not start with either of the 4-byte signet headers are ignored. Multiple push operations with either 4-byte signet header are ignored except for the first instance of each header.

The <code>signet_merkle_root</code> hash is obtained by generating the merkle root of the block transactions, with the coinbase witness commitment as-is, without the signet solution (that is, the secondary commitment includes a 4-byte push of the Signet scriptSig header or scriptWitness header, without additional solution data). This means the merkle root of the block is different from the merkle root in the signet commitment. This is needed, because the signature can never be included in the very message (in this case, a block) that is being signed. Apart from the signature, to facilitate block generation (mining), the block nonce value is the only other component of the block that the signet signature does not commit to. When grinding proof of work, the extended nonce cannot be used as it would invalidate the signature. Instead, simply resigning the same (or an updated) block will give a new search space.

A block is considered fully validated only if the solution is valid. It is recommended that this verification be done directly before or after the witness commitment verification, as the data required to do both is approximately the same.

I slightly prefer the original here.

This virtual transaction is validated against a virtual UTXO being output 0 of txid <code>block.hashPrevBlock</code>, with scriptPubKey as the signet challenge, and amount as 20,999,999.99999999 (i.e. 1 satoshi less than 21 million).

I'm starting to wonder if we even need the scriptSig part at all. It seems that the scriptWitness is a superset of the features we get from scriptSig, and any systems incorporating block sig validation will still need to handle segwit stuff, since that's a possibility now.

Could do that, but the scriptWitness would be more data. With scriptSig, you can make the signet challenge be the actual script; but with scriptWitness, the challenge has to be p2wpkh or p2wsh and you have to then include either the pubkey or the script that matches the hash in every block solution.

That seems really backwards. It should be possible to put the pubkey/script into the challenge directly, IMO, but we can sort that out later.

Just make the witness pubkey/script implicit. The client knows it and can insert it where it belongs.

Wallets do this all the time(fill out preimage details), now that the block is a "signable transaction" this is even more the case.

Out of curiosity @ajtowns why use the witness commitment for this and not an additional OP_RETURN with a different magic header?

What are the choices this "either" is referring to?

An earlier draft had two different secondary commitments, one for scriptSig and a different one for scriptWitness. When they were combined the "either" wasn't removed.