Add BIP 100, as currently implemented.

jgarzik commented at 4:48 pm on March 8, 2017: contributor

Constrained-increment hard fork size adjustments.

Add BIP 100, as currently implemented. 5ad92f06de

TheBlueMatt commented at 4:55 pm on March 8, 2017: contributor

It seems a bit late to be adding something from two and a half years ago as a “Draft”. Are you intending to re-propose this (ie should probably, at this point, go through another round of mailing list or other community discussions) or are you adding this for historical reasons (in which case status should likely be “Withdrawn”)?

luke-jr added the label New BIP on Mar 8, 2017

luke-jr commented at 5:27 pm on March 8, 2017: member

Missing “Layer: Consensus (hard fork)”. Other preamble headers are out of order; see BIP 2.

Needs “Copyright” section.

Also, what do you mean “as currently implemented”? No implementation exists AFAIK…

MarcoFalke commented at 5:30 pm on March 8, 2017: member

@BlueMatt See http://bitcoinstats.com/irc/bitcoin-dev/logs/2016/01/07#l1452194490.0 for the discussion about having it archived here. Though, the current commit seems to differ from the “original” bip100.

jameshilliard commented at 8:18 am on March 14, 2017: contributor

@jgarzik You really should use a different BIP number to avoid confusion, I’ve brought this issue up previously.

dgenr8 commented at 8:37 pm on May 4, 2017: contributor

Updated, diffs here.

bip100: Improve header metadata. Vote search edge cases.

jgarzik/bip100.git commits:
b82d840733b09104fca3f8805ffa6f2459193201
4ed2a0e4523b06d97e32259e6e180734a794e24f

ac9c8b73ed

jgarzik commented at 10:50 pm on May 4, 2017: contributor

Pushed @dgenr8 changes to branch.

luke-jr commented at 0:26 am on May 5, 2017: member

Now it’s missing the Comments headers entirely…

bip100: restore Comments header 7ea5bfccde

in bip-0100.mediawiki:67 in ac9c8b73ed outdated

62+==Backward compatibility==
63+
64+The first block larger than 1M will create a network partition, as nodes with a fixed 1M hard limit reject that block.
65+
66+==Reference Implementation==
67+https://github.com/bitcoinxt/bitcoinxt/pull/188

zander commented at 7:59 am on May 5, 2017:

Might I suggest to avoid there to be a reference implementation?

Instead make the spec leading. The difference is that should two implementations disagree, we should follow the spec. With a reference implementation this is not the case, a reference implementation design means that the spec needs to be updated when the implementation changes.

See BIP 134 for example wording.

dgenr8 commented at 3:24 pm on May 6, 2017:

@zander Thanks, heading changed to Implementations in next push.

in bip-0100.mediawiki:33 in 7ea5bfccde outdated

28+
29+===Dynamic Maximum Block Size===
30+# Initial value of <code>hardLimit</code> is 1000000 bytes, preserving current system.
31+# Changing <code>hardLimit</code> is accomplished by encoding a proposed value, a vote, within a block's coinbase scriptSig, and by processing the votes contained in the previous retargeting period.<br /><br />
32+## Vote encoding
33+### A vote is represented as a megabyte value using the BIP100 pattern<br /><br /><code>/BIP100/B[0-9]+/</code><br /><br />Example: <code>/BIP100/B8/</code> is a vote for a 8000000-byte <code>hardLimit</code>.<br /><br />

luke-jr commented at 3:31 pm on May 5, 2017:

How does one vote for sizes that aren’t a round number of MB? For example, 0.5 MB blocks? (The original BIP 100 draft supported this by specifying the size vote in bytes.)

dgenr8 commented at 5:36 pm on May 5, 2017:

Votes denominated in megabytes, and no other resolution, are specified.

This BIP offers a more deterministic path to miners who otherwise might rely on non-algorithmic coordination which is by nature size-unlimited. Max growth is about 256%/year (not 1164%).

Next push will include README.mediawiki modification.

zander commented at 5:53 pm on May 5, 2017:

I agree, having one digit behind the decimal place would be useful.

in bip-0100.mediawiki:37 in 7ea5bfccde outdated

32+## Vote encoding
33+### A vote is represented as a megabyte value using the BIP100 pattern<br /><br /><code>/BIP100/B[0-9]+/</code><br /><br />Example: <code>/BIP100/B8/</code> is a vote for a 8000000-byte <code>hardLimit</code>.<br /><br />
34+### If the block height is encoded at the start of the coinbase scriptSig, as per BIP34, it is ignored.
35+### Only the first BIP100 pattern match is processed in "Maximum block size recalculation" below.
36+### A megabyte value is represented by consecutive base-ten digits.
37+### If no BIP100 pattern is matched, the first matching emergent consensus pattern <code>/EB[0-9]+/</code>, if any, is accepted as the megabyte vote.<br /><br />

luke-jr commented at 3:32 pm on May 5, 2017:

This byte sequence is small enough that it may appear in random data. Why not just ignore EB and expect miners to set BIP100 if they want it?

dgenr8 commented at 5:17 pm on May 5, 2017:

We take this risk after examining the blockchain for near-misses and not finding any.

in bip-0100.mediawiki:53 in 7ea5bfccde outdated

48+#### If the resultant <code>lower value</code> is less than (<code>current hardLimit</code> / 1.05) rounded down to the nearest byte, it is set to that value.
49+#### If the resultant <code>lower value</code> is less than <code>current hardLimit</code>, the <code>lower value</code> becomes the <code>new hardLimit</code> and the recalculation is complete.<br /><br />
50+### Otherwise, <code>new hardLimit</code> remains the same as <code>current hardLimit</code>.
51+
52+===Signature Hashing Operations Limits===
53+# The per-block signature hashing operations limit is scaled to (actual block size rounded up to nearest megabyte)/50.

luke-jr commented at 3:35 pm on May 5, 2017:

To clarify: The sigop limit is no longer fixed for all blocks [within the adjustment period], but varies from block to block based on its actual size?

This would seem to excessively complicate miner optimisation for no apparent gain…?

dgenr8 commented at 5:21 pm on May 5, 2017:

The reason is to allow miners to check the sigops limit without calculating the BIP100 limit. Miners have begun to set their block size limit individually.

luke-jr commented at 10:45 pm on May 5, 2017:

You mean nodes? Miners don’t check the sigop limit, they conform to it, and having it based on the actual size (which is unknown at the time of block generation) greatly complicates that.

luke-jr commented at 10:46 pm on May 5, 2017:

(And if you mean nodes, then either they’re light clients or full nodes. Light clients can’t verify sigops at all either way, and full nodes need to verify the size limit regardless so can certainly calculate a sigop limit based on that.)

dgenr8 commented at 3:40 am on May 6, 2017:

I meant nodes. There is a class of nodes out there that already implements this rule, but not BIP100. The change to miner sigop check is simple, see miner.cpp line 204. Proceeding MB-by-MB could handle high-sigop cases better.

luke-jr commented at 4:55 am on May 6, 2017:

Uh, that logic looks very broken. Pretty sure I could craft a transaction that closes your block template after only <1k…

zander commented at 9:48 am on May 6, 2017:

You would be wrong luke, seems you misread the bip line you commented on.

I concur with dgenr8, your complaint is not an issue in practice.

in bip-0100.mediawiki:54 in 7ea5bfccde outdated

49+#### If the resultant <code>lower value</code> is less than <code>current hardLimit</code>, the <code>lower value</code> becomes the <code>new hardLimit</code> and the recalculation is complete.<br /><br />
50+### Otherwise, <code>new hardLimit</code> remains the same as <code>current hardLimit</code>.
51+
52+===Signature Hashing Operations Limits===
53+# The per-block signature hashing operations limit is scaled to (actual block size rounded up to nearest megabyte)/50.
54+# A maximum serialized transaction size of 1000000 bytes is imposed.

luke-jr commented at 3:35 pm on May 5, 2017:

Does this include or exclude witness data?

dgenr8 commented at 4:59 pm on May 5, 2017:

This BIP does not build on segregated witness. In conjunction with segwit, it seems clear that BIP100 would apply to the ex-witness limit, which is otherwise fixed at 1MB. In that case we could expect the limit to grow more slowly.

luke-jr commented at 10:44 pm on May 5, 2017:

It doesn’t need to build on segwit, to clearly define how it interacts with segwit.
There is no “ex-witness limit” in segwit. The only limit is the weight limit at 4M units. Size is not limited directly at all.

AllanDoensen commented at 12:25 pm on May 6, 2017:

How the segwit ‘weight’ will scale should be defined in the segwit BIP.

in bip-0100.mediawiki:57 in 7ea5bfccde outdated

52+===Signature Hashing Operations Limits===
53+# The per-block signature hashing operations limit is scaled to (actual block size rounded up to nearest megabyte)/50.
54+# A maximum serialized transaction size of 1000000 bytes is imposed.
55+
56+===Publication of <code>hardLimit</code>===
57+# For the benefit of emergent consensus nodes, <code>hardLimit</code> is published.  Example: a complete coinbase string might read <br /><br /><code>/BIP100/B8/EB2.123456/</code><br /><br /> which indicates a vote for 8M maximum block size, and an enforced <code>hardLimit</code> of 2.123456 megabytes for the block containing the coinbase string.

luke-jr commented at 3:36 pm on May 5, 2017:

Is this required by BIP 100? (Are blocks without this data invalid? Note that if so, you will be rejecting any blocks that signal a different EB size…) Or merely a recommendation?

dgenr8 commented at 4:53 pm on May 5, 2017:

This is a recommendation, since BIP100 nodes don’t rely on reading EB from each other and the published value is current limit (never a change from current limit).

luke-jr commented at 4:58 pm on May 5, 2017:

Clarify it in the BIP text, not simply reply to the review.. :p

zander commented at 5:59 pm on May 5, 2017:

it already says “example”, but the word “complete” may indicate it is spec. Replace ‘complete’ with ’elaborate’ may clarify.

dgenr8 commented at 3:41 pm on May 6, 2017:

This is moved to a Recommendations section in next push.

in bip-0100.mediawiki:61 in 7ea5bfccde outdated

56+===Publication of <code>hardLimit</code>===
57+# For the benefit of emergent consensus nodes, <code>hardLimit</code> is published.  Example: a complete coinbase string might read <br /><br /><code>/BIP100/B8/EB2.123456/</code><br /><br /> which indicates a vote for 8M maximum block size, and an enforced <code>hardLimit</code> of 2.123456 megabytes for the block containing the coinbase string.
58+
59+==Deployment==
60+
61+This BIP is presumed deployed and activated as of block height 449568 by implementing nodes on the bitcoin mainnet. It has no effect until a raise value different from 1M is observed, which requires at least 1512 of 2016 blocks to vote differently from 1M.

luke-jr commented at 3:38 pm on May 5, 2017:

Note: This is a past block.

Why does the BIP only activate on increase, rather than equally so on a decrease?

dgenr8 commented at 4:48 pm on May 5, 2017:

Using megabyte resolution for votes, with special meaning of “no change” for a 0 vote, means there is no way for hardLimit to go below 1MB.

luke-jr commented at 4:52 pm on May 5, 2017:

That’s a pretty crappy and on-sided change compared to the original BIP 100! The rest of the BIP currently still assumes sub-MB resolution also.

dgenr8 commented at 5:53 pm on May 5, 2017:

The actual limit is byte-valued, votes are in megabytes.

A limit below 1MB is something virtually nobody wants. But aside from that, raising and lowering are handled symmetrically. Even though raise is checked first, there’s no way for a raise and lower to win simultaneously.

luke-jr commented at 10:43 pm on May 5, 2017:

Plenty of people want a limit below 1 MB today. Alienating us is not going to help you achieve consensus for a hardfork.

AllanDoensen commented at 12:18 pm on May 6, 2017:

Removing the 1M lower limit would be a good idea as IMHO: 1) It would lower the complexity of the code. 2) If there is consensus to move the block size below 1M then it would allow that. 3) It removes restrictions and artificial constraints on emergent consensus with respect to block size.

jameshilliard commented at 4:41 pm on May 6, 2017:

There wasn’t a lower limit in the original BIP100 proposal, that’s another reason this should be done as a separate BIP.

luke-jr commented at 3:42 pm on May 5, 2017: member

This is missing an absolute maximum block size limit. Without it, it is impossible to develop software or even select hardware that can meet the requirements. Although this is mitigated to some extent by the limited 5% max per adjustment period growth (1164% per year), this issue should at least be documented in the BIP.

The original draft of BIP 100 had a 32 MB upper limit.

Also, README.mediawiki needs to be updated.

luke-jr commented at 3:43 pm on May 5, 2017: member

@MarcoFalke While this does differ somewhat from the original BIP 100, it does IMO seem to be essentially the same technical idea, and therefore logically a later draft of the same proposal.

zander commented at 6:04 pm on May 5, 2017: contributor

As this is the first BIP 100 that is to be merged into the bips repo, any comparison to earlier drafts is not useful IMOHO.

zander commented at 9:51 am on May 6, 2017: contributor

Please count my vote for accepting this BIP into the repo.

jameshilliard commented at 10:43 am on May 6, 2017: contributor

NAK on merging as BIP100, to avoid confusion this should be assigned a separate BIP as this is substantially different from the original BIP100 proposal.

zander commented at 10:52 am on May 6, 2017: contributor

NAK on merging as BIP100, to avoid confusion this should be assigned a separate BIP as this is substantially different from the original BIP100 proposal.

Where is the original BIP 100 proposal? Is it in this repo?

jameshilliard commented at 11:22 am on May 6, 2017: contributor

Where is the original BIP 100 proposal? Is it in this repo?

I believe it was this version or perhaps a slightly earlier revision that was published when miners first started tagging BIP100 in their coinbase sigs.

AllanDoensen commented at 12:56 pm on May 6, 2017: none

As the blocksize can only move by 5% up or down, I do not see the point of signalling an actual MB amount as a vote. Only an up/down/unchanged indication is needed for voting.

dgenr8 commented at 2:36 pm on May 6, 2017: contributor

@AllanDoensen It will be very helpful for planning to see where the network is going. It also gives a chance to lobby miners if you think they are going somewhere undesirable.

luke-jr commented at 3:48 pm on May 6, 2017: member

BIPs are not voted into the repo or not. They simply need to meet the criteria for inclusion outlined in BIP 2.

TheBlueMatt commented at 4:08 pm on May 6, 2017: contributor

I tend to agree with @jameshilliard. This should probably either have a section describing why it is so different from the original BIP 100 proposal, or should begin life as a new BIP (new ML discussion/announcement, etc). Looks like virtually every parameter has changed significantly since BIP 100.

bip100: Clarifications 7c23bf14ee

dgenr8 commented at 5:08 pm on May 7, 2017: contributor

@TheBlueMatt For the BIP number assigners to have ignored the original document for so long, only to demand an explanation of why it has changed in the interim, wouldn’t leave much room for interpretation as to the reasons, other than that they don’t agree with the content of the BIP. The number assigners could argue that BIP number 100 was self-assigned by @jgarzik. However the number assigners were unresponsive to the proposal in the past, and the current implementation is very well recognized by the BIP100 label, with 7% of blocks referencing it along with independent web sites.

TheBlueMatt commented at 5:10 pm on May 7, 2017: contributor

I’m confused, when did the original authors do the work to write up a more formal spec and submit it as a PR to this repo? It isn’t Luke’s job to follow up with all of the folks who get BIP numbers to make sure they put in the work.

On May 7, 2017 1:08:13 PM EDT, Tom Harding notifications@github.com wrote:

@TheBlueMatt For the BIP number assigners to have ignored the original document for so long, only to demand an explanation of why it has changed in the interim, wouldn’t leave much room for interpretation as to the reasons, other than that they don’t agree with the content of the BIP. The number assigners could argue that BIP number 100 was self-assigned by @jgarzik. However the number assigners were unresponsive to the proposal in the past, and the current implementation is very well recognized by the BIP100 label, with 7% of blocks referencing it along with independent web sites.

luke-jr commented at 6:46 pm on May 7, 2017: member

@dgenr8 How did we “ignore the original document”? It’s the job of the author to properly format and submit it as a PR. @jgarzik dropped the ball on that, not us.

It is my opinion that this current draft is sufficiently equivalent to be the same BIP, but that is based on my technical reading of the two, not a post-hoc justification for any perceived neglect or whatever.

Also, on what basis do you say the popular BIP100 references are in fact referring to this version of the draft, rather than the original one? Seems to me it is more likely the latter, since these references began long before this draft existed…

jgarzik commented at 6:46 pm on May 7, 2017: contributor

This is the first BIP-formal write-up and appropriately documents what eventually emerged in the field.

jameshilliard commented at 0:35 am on May 8, 2017: contributor

@luke-jr My view that a new BIP number should be assigned for substantial changes to this proposal has nothing to do with how long it has been, I brought this issue up back in 2015 and the changes back then were much smaller than those being proposed here.

luke-jr commented at 0:50 am on May 8, 2017: member

@jameshilliard I agree in principle that substantial changes to a long-standing BIP should require a new BIP. However, I do not agree that the changes in this draft vs the earlier one are in fact substantial.

jameshilliard commented at 1:17 am on May 8, 2017: contributor

I would consider the following to be substantial changes:

The removal of the 32MB hard upper limit.
The initial deployment method, the original indicated that users consent to deployment while this assumes itself to be already deployed without user consent.
The original proposal text indicated the importance that nodes run exactly the same consensus rules while this one indicates itself to be compatible with Emergent Consensus where individuals do not run the same rules.

dgenr8 commented at 5:28 am on May 8, 2017: contributor

@luke-jr coin.dance and nodecounter.com added their BIP100 tracking only after blocks with the new coinbase string format began appearing.

jameshilliard commented at 6:02 am on May 8, 2017: contributor

@dgenr8 the BIP100 tag I’m seeing is still the same “/BIP100/” tag used back in 2015, and is still tracked by blocktrail in the same way it was back then, by using the same tag and BIP number for a substantially different proposal there’s no way to tell which BIP100 variant the pool supports

TheBlueMatt commented at 2:58 pm on May 8, 2017: contributor

If we’re adding this for historical reference (which initial comments seemed to indicate), then we should be adding the original proposal. It sounds, however, now, like folks are moving to re-propose this as a consensus change which they believe should be implemented in the future. While the specific broad strokes of this proposal are similar to what they used to be, the actual constants proposed are so drastically different that I’m not sure why we are considering it to be “the same BIP”. After all, consider all of the other BIP 10X proposals which have nearly the same broad strokes but also different selected constants.

ftrader commented at 3:12 pm on May 16, 2017: none

As jgarzik stated that this “is the first BIP-formal write-up and appropriately documents what eventually emerged” I think it can only be considered as the first formal BIP100 submission.

I support entering this as BIP100.

earonesty commented at 3:29 pm on May 16, 2017: none

If users don’t have a “vote” … then this proposal is kindof useless. UTXO-tagging via OP_RETURN with capability bits could be used to divine what active nodes are capable of. (This could lead to privacy loss if someone doesn’t use a common combination of bits.)

jameshilliard commented at 8:17 pm on May 16, 2017: contributor

@ftrader That’s not true, there was a formal write up here way back in 2015, it just hadn’t been submitted to the BIP repo back then. It’s quite clear that this altering of the original BIP100 proposal is an attempt to hijack the original support BIP100 had back in 2015 for a very different proposal. The BIP process is designed to document concrete proposals so that they can be vetted by the community, substantially changing the BIP halfway through this process makes it much more difficult to measure community support since it is no longer clear which proposal a company is supporting if they say they support “BIP100”.

ftrader commented at 9:08 pm on May 16, 2017: none

@jameshilliard :

there was a formal write up [on a personal repository belonging to jgarzik] way back in 2015, it just hadn’t been submitted to the BIP repo […]

It seems we are in agreement then that this is the initial submission to the BIP repo. As there was no appreciable current support for the original draft version, nor discussion by the community, I welcome this attempt by the author to improve on the draft proposal, revive discussion and formally register it as a BIP here. I wish his proposal all the best, and do not see any potential for confusion, especially not if the updated draft is published here.

TheBlueMatt commented at 2:19 am on May 17, 2017: contributor

Absolutely, its great if the author wants to change a previous proposal to get feedback, but the way you do that is go through the submission process again - it’s specifically designed to provide feedback to authors during the process.

The initial opening of this pr talked about how BIP 100 was never added to this repo and it should be added to document a previous proposal, which it seems we’ve strayed from and are now looking at a completely new proposal?

On May 16, 2017 5:09:00 PM EDT, freetrader notifications@github.com wrote:

@jameshilliard :

there was a formal write up [on a personal repository belonging to jgarzik] way back in 2015, it just hadn’t been submitted to the BIP repo […]

It seems we are in agreement then that this is the initial submission to the BIP repo. As there was no appreciable current support for the original draft version, nor discussion by the community, I welcome this attempt by the author to improve on the draft proposal, revive discussion and formally register it as a BIP here. I wish his proposal all the best, and do not see any potential for confusion, especially not if the updated draft is published here.

AllanDoensen commented at 1:01 pm on May 19, 2017: none

I do not understand the reason for travis to fail here. Apologies if it is my naivety, but how can this travis failure be rectified? Could someone clarify?

jgarzik commented at 0:57 am on May 22, 2017: contributor

@luke-jr What are the remaining tweaks needed to get this merged, if any?

luke-jr commented at 3:27 am on May 24, 2017: member

@jgarzik According to travis, the README lines are incorrect:

0+< | Dynamic block size by miner vote
1+< | Jeff Garzik, Tom Harding, Dagur Valberg
2+> | Dynamic maximum block size by miner vote
3+> | Jeff Garzik, Tom Harding, Dagur Valberg Johannsson

ftrader commented at 6:04 pm on May 24, 2017: none

It looks to me like Travis is failing to accept the corrected lines which replace the uncorrected ones. That appears to be a problem with the build script.

At the very least, it is clear to me from inspection of the change that these are minor amendments of the title and author information in line with the original BIP100.

zander commented at 9:11 pm on May 24, 2017: contributor

On Wednesday, 24 May 2017 20:04:38 CEST freetrader wrote:

It looks to me like Travis is failing to accept the corrected lines which replace the uncorrected ones. That appears to be a problem with the build script.

At the very least, it is clear to me from inspection of the change that these are minor amendments of the title and author information in line with the original BIP100.

No Freetrader, the issue is that there have been new lines added in both the README and the BIP. And then changes were made in the BIP that were not duplicated in the README.

The point of the travis bug is that the lines in the README should be identical to the corresponding lines in the BIP. And they are not.

– Tom Zander Blog: https://zander.github.io Vlog: https://vimeo.com/channels/tomscryptochannel

ftrader commented at 9:38 pm on May 24, 2017: none

Thanks for the clarification @zander !

jonathancross commented at 2:40 am on November 7, 2017: contributor

@jgarzik Seems this is just waiting for you to update the README to match the BIP.

luke-jr merged this on Sep 24, 2019

luke-jr closed this on Sep 24, 2019

Add BIP 100, as currently implemented. #498