BIP54: Add a footnote mentioning the rule-out of forward block #2344

This is a follow-up from this discussion on the mailing list: https://groups.google.com/g/bitcoindev/c/T9s8nflvlK0

It is unclear what is the process when a consensus technical change is rendering impractical a class of bitcoin protocols (or making their efficiency advantage), such as "forward block" in the present situation. I don't have immediately in mind a precedent like this, when most of the time consensus changes has been careful to introduce upgradeability hooks to rather increase the design surface offered by the bitcoin network for protocol designers.

However, in the present situation, we're facing a change where for one side of the technical community (the wide majority as expressed over the last years) it is considered as a technical fix and for another the risks of attacks mitigated by the fix as deemed as too low to be worthy of fixing, in view of the potential advantages brought by letting the technical design door open, even if it's years or decades down the road.

For my own technical view, see my email on the mailing list. To me, one technical disadvantage that is striking me as making the "forward block" idea as technically unworthy of its trade-offs (even if it's an intellectually stimulating line of research) is the necessity, if I'm understanding correctly the proposal, that the validating full-nodes have to verify (or dedicate processing ressources) twice some subset of transactions, the first time on the forward blockchain, the second time on the compatibility blockchain, whatever increase in their reward (though the proposal does not mention any fee reward increase for this duplicated processing work).

It should be noted that idea has been proposed for the first time in 2018 at Scaling Bitcoin in Tokyo, and since then it has not really attracted mindshare traction among the bitcoin technical experts. This is not a demonstration that the idea is unsound itself, though rather a hint that the advantages / disadvantages trade-offs for the networks are not appealing to many. With this caveat, I'm still inviting anyone to read the "forward block" paper to forge its own judgement on the idea.

I'm thinking it's good to add a footnote about the technical outlaw of this idea, even if I'm very scrupulous and conscientious here, both for (a) explicitly acknowledging the idea has been considered in the design process of the BIP54's time-warp mitigation, though (b) for historical purpose years or decades down the road to enlighten to the technical community of this future day the existence of the "forward block" idea, if other insights from this design proposal can be useful at this future time (I'm minding how much sometimes some consensus design choices done by Satoshi are sadly obscure to us, and we can only make conjecture). This is offer a one last time window of opportunity for any proponent of the "forward block" idea to substantiate a technical conversation about its merits.

cc @darosior as BIP author cc @maaku as "forward block" idea author.

(For more in-depth technical conversation, if people think strongly about it, inviting anyone to answer on the mailing list).

increase the technical complexity of a full-node by orders of magnitude

This is speculation, as far as I can tell. Unless there is an implementation somewhere I could review?

there are also significant uncertainties on the soundness of its game theory for the changes in the miners block reward it is proposing

This is an unsubstantiated claims.

it would require full-nodes to dedicate validation resources twice to all the transactions

This is an incorrect reading of the proposal. The same transaction appears twice, once in the forward block and once in the compatibility chain. There is no reason to validate it twice.

If it is valid in the forward block chain, it must also be valid in the compatibility chain, as the validation code is very nearly the same in the forward block chain, just with a more restricted view of the utxoset, with one difference noted in the paper. Upgraded nodes wouldn't even validate non-coinbase, non-block-final transactions the compatibility chain, just check the hashes against observed locations in the forward block chain. (Unless, as noted in the paper, lock time checks are imposed, in which case the forward block validation outputs a sidechar of small, trivial checks for when it hits the compatibility chain. This needs to be tracked, but validation is an integer comparison.)

Non-upgraded nodes only check the compatibility chain, same as before.

Addressing the comments in the mailing group post, I believe you have confused the ~3,600x limit in the paper with an actual proposed scaling curve. The paper works out that the timewarp + forward blocks combination can yield up to a +358,400x on-chain scaling factor. It is not a proposal to actually scale the bitcoin blockchain by that amount. An actual proposal could limit that scaling to 2x, or whatever scaling curve is deemed appropriate. BIP54, by contrast, would limit scaling to +0.6%, as you work out. That's a rounding error lost in the noise of regular hash rate fluctuations, so I don't think saying BIP54 cuts off forward block scaling is hyperbolic.

2 fixups for the typos CI:

s/ressources/resources/ s/adjustement/adjustment/

Concept ±0 on whether forward blocks should be mentioned more explicitly, the proposed text does not seem useful to me, though. Many readers may have no context, so the proposal should be linked and briefly introduced before detailed criticism would make sense.

Pushed an update to make CI happy.

@maaku Thanks for your feedback, as an overall note may I ask you to reply on the mailing list thread where it's discussed at length, generally it's more convenient than github, which is not necessarily the best forum for long-text format discussions. I'll answer you there.

Unless there is an implementation somewhere I could review?

May I say generally the burden is on the author of a novel idea to come with a demonstration of the soundness of said idea, that meaning coming up with a toy implementation of the idea as a matter of pedagogical explanation.

That said, I'm seeing few data structures and algorithms for which a "forward block" idea would need additional dedicated code, i.e validation code to process the "forward block" (which may have Confidential Transaction logic as you're proposing), some kind of memory pool for the transaction shared queue or also proof-of-work validation logic for a "novel proof-of-work" algorithm (again as your paper is proposing it in section VI). I'm not talking probably all the p2p protocol extensions that would have to be crafted too. All those code areas non-trivial for anyone who has experience writing code or reviewing / testing code in those areas.

You're free to sketch out an ASCII art style of a "forward block" implementation, or even use a LLM tool for that to clarify the point of this conversation.

This is an unsubstantiated claims.

It's true in that my mailing list post, there is no explanation in depth of the game theory issues with the transition of most-of-the-reward outcome from the "compatibility chain" to the "forward chain". Let's develop a bit that point, there is something troubling with the idea of (a) making the reorg-safety of the "forward chain" dependent of periodic commitments on the "compatibility chain" and then (b) miners on the "forward chain" getting most of the fee reward. Thermodynamically it doesn't make sense as the energy burnt to be the ultimate backstop of the network is less reward than the energy burnt to secure a hierarchically less ordered state of the bitcoin blockchain.

Said differently in the view of adversarial miners reorg attacks, the attacking cost for them might be inferior to target the "compatibility chain" than the cost for the miners to add new blocks on the "forward chain". There is some troubling asymmetry there, where you're proposing to use less expensive PoW to protect more expensive PoW, assuming I'm understanding what you're proposing with your system of commitment (from what is described in Section IV of your paper).

There is no also no mention at all of an analysis of how your scheme might or might not affect the Gambler's Ruin security advantage as laid out in the Satoshi paper, where the demonstrate is made than an adversarial chain won't be able to overcome the head start that has been accumulated by a honest or economically-rational chain majority. What if, if your scheme an adversarial mining coalition (e.g one censuring some types of transactions) starts, at the very birth of the "forward chain" to gain a long sequence of blocks advantage on the "forward chain".

This is an incorrect reading of the proposal. The same transaction appears twice, once in the forward block and once in the compatibility chain. There is no reason to validate it twice.

There is no explanation brought on how the full nodes are expected to solve the identity problem, i.e how do you ensure that transaction A included in the forward chain is the very same exact transaction A to be included in the compatibility chain, and not a malleated transaction A'. This is even a wider problem than purely adversarial malleation, as somehow transactions have to be dissociated with some kind of unique non-colliding identifier than A == A and not that A == B.

Somehow, I don't see how you can avoid this problem without some kind of validation (now it might be some cost-minimized validation), though there is some validation to be done and I'm arguing that this validation work will be redundant between the two chains.

with one difference noted in the paper.

That's at least one difference to be checked on its own, so some validation code that has to be supported.

This needs to be tracked, but validation is an integer comparison.

Tracking implies the consumption of some resources, be it memory to store said tracker or CPU-wise to iterate over the tracker and ensure there is no element duplication. None are free for an average full-node.

I'm laying out my argument on the sub section "C. Coinbase Confirmation on the Forward Block Chain" which explain how the validation happens between the two chains, though sure you can detail more how the full pipeline validation would work step by step from Start: a tx enters in the "forward "chain" mempools to End: a tx is irremediably committed in the "compatibllity chain". That would bring clarity to the discussion.

Addressing the comments in the mailing group post, I believe you have confused the ~3,600x limit in the paper with an actual proposed scaling curve. The paper works out that the timewarp + forward blocks combination can yield up to a +358,400x on-chain scaling factor.

Thanks for the clarification, still a ~3,600 max additional cost (be it measured in compute resource, storage, bandwidth) sounds to me an overblown order of magnitude on what would be acceptable resources requirements to run full-node.

2x is indeed far more reasonable, though we have already slow validation block problems, that BIP54 aims to fix (and it is not certain that legacy inputs are the only slowness vectors we still have here). We cannot necessarily assume that the hardware costs are going down with time for the marginal hobbyist user. E.g a simple checks sounds to put the price of 1GB of RAM in 2018 (time of the publication of the paper) at $6 and now it sounds to be around $12 (that's a very quick "let me google that for you" style of check, though it can be explained by the demand from technological trends exogenous from bitcoin in itself).

It is not a proposal to actually scale the bitcoin blockchain by that amount. An actual proposal could limit that scaling to 2x, or whatever scaling curve is deemed appropriate. BIP54, by contrast, would limit scaling to +0.6%, as you work out. That's a rounding error lost in the noise of regular hash rate fluctuations, so I don't think saying BIP54 cuts off forward block scaling is hyperbolic.

A +0.6% in transaction capacity would be still very meaningful in my view, if one accounts that's under this +0.60% of capacity increase one can sneak in compressed transactions e.g doing only aggregated ZK-proofs of them, where the non-upgraded nodes would not have to validate the "compatibility chain". Now, I concede you it's true that any ZK super compressed validation would be non-trivial technical plumbery, though (a) we're likely be able to do recursive proofs to validate a nearly-unbounded number of transaction and (b) we would still be able to leverage that "hidden capacity increase" that we can get in terms of scaling due to the structure of the difficulty adjustment algorithm, which in my view is the main intuition of the "forward block" idea.

So if one go to consider other trade-offs architecture still folding in the "forward block" family of scaling solutions, I don't think we can say that BIP54 cuts off forward block style of scaling approach.

@murchandamus

From the old consensus changes (BIP16, BIP30, BIP34), I don't remember there was already this situation where a consensus change would significantly close some scaling technical options. I guess there has been indeed some when Satoshi did restrain the DoSy Script interpreter opcode like OP_CAT and friends), though Satoshi era style of way consensus changes were done it's not likely the standard we're adhering now...More recently with BIP141, I think it did rule out other approach of fixing malleability (e.g thing like BIP140), though was BIP140 enabling some protocols than one cannot with BIP141, I don't think at all.

To me, though correct me if you see other historical examples, I think it's a bit of a novel situation. Therefore, yeah in my view it's better to add a succinct note mentioning the impact BIP54 has on the "forward block" idea. For sure I can update with a link referencing the "forward block" paper, introduce the idea in two-sentences and link to criticism and its sum-up, all in a succinct comment, as well it's a footnote.