I added explicit parameters for deployment at the beginning of the specification. I think this greatly clarifies the spec for someone approaching it for the first time.
We've decided for now not to consensus-enforce the bit states, making the RFC language that had been previously proposed a bit superfluous. The only thing that remains consensus-enforced is the moment of activation which should be clear from context.
24 | ==Specification== 25 | 26 | +Each soft fork deployment is specified by the following parameters (further elaborated below): 27 | + 28 | +# The '''bit''' determines which bit in the nVersion field of the block is to be used to signal the soft fork lock-in and activation. 29 | +# The '''threshold''' specifies how many blocks within a single retarget period (2016 blocks) MUST have the bit set before we lock in the deploment.
s/deploment/deployment/
ACK
55 | '''Success: Lock-in Threshold''' 56 | -If bit B is set in 1916 (1512 on testnet) or 57 | -more of the 2016 blocks within a retarget period, it is considered 58 | -''locked-in''. Miners should continue setting bit B, so uptake is 59 | -visible. 60 | +For each retarget interval, the number of blocks in which bit B is set is counted. If bit B is set in at least some threshold number of blocks (1916 on mainnet, 1512 on testnet) of the 2016 blocks within a retarget period, it is considered ''locked-in''. Miners SHOULD continue setting bit B so uptake is visible.
MUST
Is it MUST? I thought we agreed that until activation it was still considered optional or else lock-in itself would be a soft fork.
Eric Lombrozo notifications@github.com writes:
if (BState != activated && BState != failed) { SetBInBlock(); }'''Success: Lock-in Threshold''' -If bit B is set in 1916 (1512 on testnet) or -more of the 2016 blocks within a retarget period, it is considered -''locked-in''. Miners should continue setting bit B, so uptake is -visible. +For each retarget interval, the number of blocks in which bit B is set is counted. If bit B is set in at least some threshold number of blocks (1916 on mainnet, 1512 on testnet) of the 2016 blocks within a retarget period, it is considered ''locked-in''. Miners SHOULD continue setting bit B so uptake is visible.
Is it MUST? I thought we agreed that until activation it was still considered optional or else lock-in itself would be a soft fork.
No, that's enforcement. Setting the bit is compulsory, and was the intent of the last BIP change.
Cheers, Rusty.
60 | @@ -55,10 +61,8 @@ visible. 61 | } 62 | 63 | '''Success: Activation Delay''' 64 | -The consensus rules related to ''locked-in'' soft fork will be enforced in 65 | -the second retarget period; ie. there is a one retarget period in 66 | -which the remaining 5% can upgrade. At the that activation block and 67 | -after, miners should stop setting bit B, which may be reused for a different soft fork. 68 | +The consensus rules related to ''locked-in'' soft fork will be enforced in the second retarget period; ie. there is a one retarget period in 69 | +which the remaining 5% can upgrade. Upon reaching the end of the period, the soft fork is considered active and miners SHALL enforce the new rules beginning with the next block whose height is divisible by 2016. At this point, miners SHALL stop setting bit B which MAY now be reused for a different soft fork.
I prefer MUST over SHALL.
In the RFC they are synonymous, I have no strong preference either way. MUST is fine.
Eric Lombrozo notifications@github.com writes:
@@ -55,10 +61,8 @@ visible. }
'''Success: Activation Delay''' -The consensus rules related to ''locked-in'' soft fork will be enforced in -the second retarget period; ie. there is a one retarget period in -which the remaining 5% can upgrade. At the that activation block and -after, miners should stop setting bit B, which may be reused for a different soft fork. +The consensus rules related to ''locked-in'' soft fork will be enforced in the second retarget period; ie. there is a one retarget period in +which the remaining 5% can upgrade. Upon reaching the end of the period, the soft fork is considered active and miners SHALL enforce the new rules beginning with the next block whose height is divisible by 2016. At this point, miners SHALL stop setting bit B which MAY now be reused for a different soft fork.
In the RFC they are synonymous, I have no strong preference either way. MUST is fine.
Indeed; if you can stick with the same subset throughout it makes life easier for the reader.
Thanks, Rusty.
123 | @@ -120,7 +124,7 @@ adding three to the current calendar year when drafting the soft fork proposal): 124 | 125 | If the soft fork still not ''locked-in'' and the 126 | GetMedianTimePast() of a block following a retarget period is at or 127 | -past this timeout, miners should cease setting this bit. 128 | +past this timeout, miners SHOULD cease setting this bit.
MUST
I'm not so sure this is a MUST. If it is a MUST I think we need to specify for how long they need to cease setting the bit since eventually the bit might get reused at which point the correct behavior for a client unaware of the new rule change is to accept the block but warn the user.
I guess the bigger point here is that we need to draw a distinction between behaviors that will result in invalid blocks vs. behaviors that will still produce valid blocks although we discourage them.
Unfortunately, we don't really have a good way to enforce the correct application of rules by miners, so about the best we can hope for right now is to reduce as much as possible any incentives for miners to feed misleading signals and to detect when miners are in fact feeding misleading signals or not enforcing the rules to allow for intervention.
Eric Lombrozo notifications@github.com writes:
@@ -120,7 +124,7 @@ adding three to the current calendar year when drafting the soft fork proposal):
If the soft fork still not ''locked-in'' and the GetMedianTimePast() of a block following a retarget period is at or -past this timeout, miners should cease setting this bit. +past this timeout, miners SHOULD cease setting this bit.
I guess the bigger point here is that we need to draw a distinction between behaviors that will result in invalid blocks vs. behaviors that will still produce valid blocks although we discourage them.
Simple rules win. An implementer reading this doesn't need to know how bad things will break if they screw up, just what to do.
Unfortunately, we don't really have a good way to enforce the correct application of rules by miners, so about the best we can hope for right now is to reduce as much as possible any incentives for miners to feed misleading signals and to detect when miners are in fact feeding misleading signals or not enforcing the rules to allow for intervention.
All language is simply words. We have no enforcement ability :(
As an aside, I feel RFC 2119 reflected a reduction in flexibility and increase in dogmatism for IETF. Shouting at people feels wrong, even if the aim is clarity, and a turn away from the original peer-like humiity of "Request For Comments".
Cheers, Rusty.
Eric Lombrozo notifications@github.com writes:
@@ -120,7 +124,7 @@ adding three to the current calendar year when drafting the soft fork proposal):
If the soft fork still not ''locked-in'' and the GetMedianTimePast() of a block following a retarget period is at or -past this timeout, miners should cease setting this bit. +past this timeout, miners SHOULD cease setting this bit.
I'm not so sure this is a MUST. If it is a MUST I think we need to specify for how long they need to cease setting the bit since eventually the bit might get reused at which point the correct behavior for a client unaware of the new rule change is to accept the block but warn the user.
Oh, in the theoretical case where we assign the bit to mean something else back-to-back?
I don't want them to think they don't have to stop! We can add "... until reused for another soft fork" to be clearer?
Cheers, Rusty.
141 | +To support upgrade warnings, an extra "unknown upgrade" is tracked, using the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask will be non-zero whenever an unexpected bit is set in nVersion. Whenever lock-in for the unknown upgrade is detected, the software SHOULD warn loudly about the upcoming soft fork. It SHOULD warn even more loudly after the next retarget period. 142 | 143 | '''Forks''' 144 | -It should be noted that the states are maintained along block chain 145 | -branches, but may need recomputation when a reorganization happens. 146 | +It SHOULD be noted that the states are maintained along block chain
This is not a SHOULD. Try "Note that the states..."
You are absolutely correct - damn search-and-replace :p
142 | 143 | '''Forks''' 144 | -It should be noted that the states are maintained along block chain 145 | -branches, but may need recomputation when a reorganization happens. 146 | +It SHOULD be noted that the states are maintained along block chain 147 | +branches, but MAY need recomputation when a reorganization happens.
... and this is not a MAY. It's a might.
150 | 151 | The mechanism described above is very generic, and variations are possible for future soft forks. Here are some ideas that can be taken into account. 152 | 153 | '''Modified thresholds''' 154 | -The 95% threshold (based on in BIP 34) does not have to be maintained for eternity, but changes should take the effect on the warning system into account. In particular, having a lock-in threshold that is incompatible with the one used for the warning system may have long-term effects, as the warning system cannot rely on a permanently detectable condition anymore. 155 | +The 95% threshold (based on in BIP 34) does not have to be maintained for eternity, but changes SHOULD take the effect on the warning system into account. In particular, having a lock-in threshold that is incompatible with the one used for the warning system MAY have long-term effects, as the warning system cannot rely on a permanently detectable condition anymore.
This is not a MAY, it's a might.
154 | -The 95% threshold (based on in BIP 34) does not have to be maintained for eternity, but changes should take the effect on the warning system into account. In particular, having a lock-in threshold that is incompatible with the one used for the warning system may have long-term effects, as the warning system cannot rely on a permanently detectable condition anymore. 155 | +The 95% threshold (based on in BIP 34) does not have to be maintained for eternity, but changes SHOULD take the effect on the warning system into account. In particular, having a lock-in threshold that is incompatible with the one used for the warning system MAY have long-term effects, as the warning system cannot rely on a permanently detectable condition anymore. 156 | 157 | '''Conflicting soft forks''' 158 | -At some point, two mutually exclusive soft forks may be proposed. The naive way to deal with this is to never create software that implements both, but that is making a bet that at least one side is guaranteed to lose. Better would be to encode "soft fork X cannot be locked-in" as consensus rule for the conflicting soft fork - allowing software that supports both, but can never trigger conflicting changes. 159 | +At some point, two mutually exclusive soft forks MAY be proposed. The naive way to deal with this is to never create software that implements both, but that is making a bet that at least one side is guaranteed to lose. Better would be to encode "soft fork X cannot be locked-in" as consensus rule for the conflicting soft fork - allowing software that supports both, but can never trigger conflicting changes.
"could"?
Probably it should be clarified that even the strongest "rules" here are ultimately up to the softforking BIP, and can in theory be overridden?
I removed the RFC language for now but kept in the parameters at the beginning of the specification, which IMO was the more important change I had made.
your money means nothing for bitcoin XT
On 19 October 2015 at 04:16, Eric Lombrozo notifications@github.com wrote:
I removed the RFC language for now but kept in the parameters at the beginning of the specification, which IMO was the more important change I had made.
— Reply to this email directly or view it on GitHub #219 (comment).
I like the idea of retaining the flexibility to specify a different threshold for each fork or to change it for all forks in the future. The way I see it, the thresholds used for mainnet and testnet are recommendations.
Eric Lombrozo notifications@github.com writes:
I like the idea of retaining the flexibility to specify a different threshold for each fork or to change it for all forks in the future. The way I see it, the thresholds used for mainnet and testnet are recommendations.
Sure, you could do that but you'll break everyone's warning system. And it's hard enough to propose a good soft fork without worrying about even more parameters.
Cheers, Rusty.
I'm not saying we should be changing these parameters arbitrarily on an ad hoc basis - I just think the flexibility costs nothing and it makes the spec more clear...and the BIP document even talks about potentially modifying thresholds in the future, so might as well have that flexibility from the beginning.
Eric Lombrozo notifications@github.com writes:
I'm not saying we should be changing these parameters arbitrarily on an ad hoc basis - I just think the flexibility costs nothing and it makes the spec more clear...and the BIP document even talks about potentially modifying thresholds in the future, so might as well have that flexibility from the beginning.
I don't understand?
It's just a document, you can take or leave it. It does describe some of the considerations if you decide to do something different.
But the way these things actually work is:
So, yeah, the default language says "DO THIS. THEN THIS. THEN THIS. DONE". Because I guarantee that nobody will read the rest of it.
Cheers, Rusty.
@btcdrak This needs an ACK from at least one of @sipa @petertodd @gmaxwell @rustyrussell to be merged.
NAK, seems controversial (the consensus logic affected by a start date).
See @rustyrussell's comment and the discussion above.
@sipa regardless, the thrust of this PR was actually adding the deployment parameters to the top of the spec in an easy-to-reference list. I removed the start time, then readded it after talking to you about it and you agreeing it was necessary.
I'll get rid of the start time, if nobody wants it.
I'm not the only one to decide that (anymore).
Can we at least merge this for now pending resolution of the controversy?
Ack. Specifying the parameters up front is nice (though in practice the BIP recommends specific threshold and timeout values).
ACK
@rustyrussell Regarding the previous point brought up in this PR, I'd very much like to argue in favor of a start date regardless that is part of the BIP9 consensus logic. The reason is that:
Mhmm, I thought that we wanted a star time per deployment, but not a different activation threshold per development (the bip implicitly states that there's at least one per chain, by specifying testnet's threshold).
Pieter Wuille notifications@github.com writes:
@rustyrussell Regarding the previous point brought up in this PR, I'd very much like to argue in favor of a start date regardless that is part of the BIP9 consensus logic. The reason is that:
- Experience has shown that some miners and other nodes do in fact run pre-release code, and this does cause problems.
OK, I respectfully disagree, but I'm often wrong :)
- Using a start and end date is a much cleaner way of avoiding bits that depend on each other than defining them recursively (if that's ever needed).
I look forward to reading your BIP patch which shows BIP authors how to choose the start date.
Perhaps we should make this the only decision, and make "end date = start date + 3 years" (ie. 94608000 seconds)? That still gives us failed softfork every 6 weeks still.
Thanks, Rusty.
@rustyrussell BIP65 was deployed by 20% of the hash power before the stable release.
I am very much for a start date and I dont think it needs needs to be complicated. The idea is simply to set the start date reasonably to a time after you plan a stable release. NACK against automatically setting the end date. That would create a lack of flexibility and for the time being it's clear 3 year expiry is way too long considering all softforks have been enforced in just a few months. Unless mining decentralises a bit I see no reason for it to take longer unless they actually do reject it, in which case a year is probably a good window.
tl;dr, The start and end dates should be configurable per softfork. The way it is implemented in core PR 7575 is ideal.
I slightly prefer having possibility to set startdate and enddate manually.
If I understand @rustyrussell point, this is about not having to think about more parameters during the soft fork creation. But even if there was no startdate to configure programmatically in BIP9, the startdate will need to be debated anyway by deciding when the soft fork will be merged.
At the end of the day, I don't think this matter a lot.
฿tcDrak notifications@github.com writes:
@rustyrussell BIP65 was deployed by 20% of the hash power before the stable release.
I know; and that's a long way from 95%. But I've already conceded the start date.
I am very much for a start date and I dont think it needs needs to be complicated. The idea is simply to set the start date reasonably to a time after you plan a stable release.
As a BIP author, this means I need to know when it's likely to be merged, and what the release cycle timing will be. So some advice about how to select this would be nice. Perhaps suggest aiming for 3 months after an expected production release?
NACK against automatically setting the end date. That would create a lack of flexibility and for the time being it's clear 3 year expiry is way too long considering all softforks have been enforced in just a few months. Unless mining decentralises a bit I see no reason for it to take longer unless they actually do reject it, in which case a year is probably a good window.
OK, let's make it a year then, but let's choose something for BIP9. You could change it for each BIP, just like you could change the thresholds, but why would one softfork choose 1 year and one choose 3 years? How would I, as a random BIP author, tell?
We can always have a new BIP which changes that advice if necessary, but I feel that punting to each BIP author is reneging on our responsibilities.
Cheers, Rusty.
@rustyrussell I agree BIP9 could recommend some sane defaults. In any case, each softfork BIP already must specify the exact deployment details, for example which bit they intend to use. A typical deployment section for a BIP9 deployment might say "This will be deployed using BIP9 with bit 5 set with a start date of 2016-04-01 00:00:00 and a timeout of 1 year".
I would also point out that draft softfork BIPs usually have the deployment section blank as "TBD" until exact deployment methodology is known (which is the case for the current BIP68,112 and 113).
Additionally, we've tended to separate the softfork feature patch and the deployment code. In fact sometimes it's even been necessary to release mempool only code first as was the case of BIP113.
How about wording for BIP9 something to the effect of:-
"It is recommended that the '''starttime''' be at least 1 month after expected production release of the softfork proposal, with a '''timeout''' of 1 year."
฿tcDrak notifications@github.com writes:
How about wording for BIP9 something to the effect of:-
"It is recommended that the '''starttime''' be at least 1 month after expected production release of the softfork proposal, with a '''timeout''' of 1 year."
OK, here's a patch (also available at https://github.com/rustyrussell/bips/tree/bip9-recommendations ) which makes the recommendations more explicit. If people like it after bikeshedding, I'll open a new PR:
(Moved to a separate section immediately following ==Specification==):
===Selection guidelines===
The following guidelines are suggested for selecting these parameters for a soft fork:
A later deployment using the same bit is possible as long as the starttime is after the previous one's timeout or activation, though it is recommended to have a pause in between to detect buggy software.
Cheers, Rusty.
Typo: one month software release.
ACK otherwise
I think we should recommend that a previously unused bit is used if available, otherwise it becomes a bit annoying in regtest to figure out how to activate both soft forks.