A written specification for Testnet 4 has been requested in the PR discussion and a BIP seems to be the right place for this.
Given the active discussion in the PR I think it’s best if this is kept open until the PR is merged. I will keep it up-to-date and track the current status of the PR.
51+
52+- For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
53+- Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature.
54+- Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner.
55+- Increase of the delay in the 20 min difficulty exception was suggested but did not receive significant support.
56+- Re-enabling `acceptnonstdtxn` by default on the network was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests.
MAY, rather than SHOULD or MUST), but explaining pros and cons is useful.
53+- Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature.
54+- Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner.
55+- Increase of the delay in the 20 min difficulty exception was suggested but did not receive significant support.
56+- Re-enabling `acceptnonstdtxn` by default on the network was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests.
57+- Motivating miners to re-org min difficulty blocks was suggested but this may still be done after Testnet 4 is deployed, so it can be considered out of scope for this BIP.
58+- Persisting the real difficulty in the version field was suggested to prevent exploits of the 20-min rule in an even more robust way, but did not receive a critical level of support since it would be a more invasive change.
74+
75+The resulting genesis block hash is 00000000da84f2bafbbc53dee25a72ae507ff4914b867c565be350b0da8bf043, and the block hex is XXX.
76+
77+=== Message Start ===
78+
79+The message start is defined as `0x1c163f28`. These four bytes were randomly generated and have no special meaning.
Backticks don’t work in Mediawiki the same way they do in Markdown. You probably want:
0The message start is defined as <code>0x1c163f28</code>. These four bytes were randomly generated and have no special meaning.
80+
81+== Compatibility ==
82+
83+This specification is backward compatible in the sense that existing software can use Testnet 4 out of the box, assuming support for Testnet 3 already exists.
84+
85+Simply by adding the network parameters for Testnet 4 (magic number, etc), a client can connect to and use Testnet 4 without further modifications. The block headers have valid proof of work, so clients can trivially check that blocks are "probably" valid.
0Simply by adding the network parameters for Testnet 4 (magic number, etc.), a client can connect to and use Testnet 4 without further modifications. The block headers have valid proof of work, so clients can trivially check that blocks are "probably" valid.
8+ Status: Draft
9+ Type: Standards Track
10+ Created: 2024-05-27
11+ License: CC0-1.0
12+ Post-History: https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ
13+ https://github.com/bitcoin/bitcoin/pull/29775
37+
38+This means that the existing 20 min difficulty exception in Testnet 3 is explicitly kept in place, meaning that a block with a timestamp 20 minutes further into the future from the current time is allowed to have a minimum difficulty of 1 instead of whatever the actual level currently is.
39+
40+=== Block Storm Fix ===
41+
42+When the next work required is calculated for the first block in a new difficulty period, the last block of the previous difficulty period is only used as the base for this calculation if its difficulty is not 1. If its difficulty is 1, all blocks in the previous difficulty period are checked in reverse order if any of them have a different difficulty than 1. When a different difficulty is encountered, it is assumed to be the actual difficulty of the network and used as the base for the calculation of the new difficulty level. Only if all blocks from the previous difficulty period have had a difficulty of 1, this is also used as the base for the calculation of the next difficulty period.
This seems a bit imprecise. It sounds like the latest block is ignored, but I suspect that the timestamps from first and last block are always used, it is just the difficulty that is taken from the latest block with a difficulty greater than 1.
Perhaps you could also clarify: When every single block in a difficulty period is mined per the 20-minute exception, the difficulty would still reset to 1, right?
This seems a bit imprecise. It sounds like the latest block is ignored, but I suspect that the timestamps from first and last block are always used, it is just the difficulty that is taken from the latest block with a difficulty greater than 1.
Right, good catch. I tried to clarify this in this last edit.
Perhaps you could also clarify: When every single block in a difficulty period is mined per the 20-minute exception, the difficulty would still reset to 1, right?
In theory yeah but that isn’t possible. The first block of the difficulty period does not allow the usage of the 20-min rule. I managed to confuse myself with this again as well so I changed this further and clarified this. The confusing part here is that this is not a change of behavior which is why I failed to emphasize this. The first block of the difficulty period also can not use the 20-min rule in Testnet 3. But there it can get difficulty 1 when the last block of the previous period was mined with the 20-min rule, which then leads to a block storm.
31+
32+Since then the issue with block storms has been further demonstrated on Testnet 3 when three years' worth of blocks were mined in a few weeks while rendering the network practically unusable at the same time.
33+
34+== Specification ==
35+
36+Consensus of Testnet 4 follows the same rules as Testnet 3 with the exception of the two new rules detailed below.
Reads a bit odd to define testnet 4 in this BIP terms of testnet 3, when there is no BIP for testnet 3. Also, the goal of testing should be to be as close to possible to the real production workload as possible. Readers may be wondering where the differences lay. Thus, it would be better to define it in terms of mainnet, no?
Section === Consensus Rules === already does it this way.
76+
77+The resulting genesis block hash is <code>00000000da84f2bafbbc53dee25a72ae507ff4914b867c565be350b0da8bf043</code>, and the block hex is <code>01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff5504ffff001d01044c4c30332f4d61792f323032342030303030303030303030303030303030303030303165626435386332343439373062336161396437383362623030313031316662653865613865393865303065ffffffff0100f2052a010000002321000000000000000000000000000000000000000000000000000000000000000000ac00000000</code>.
78+
79+=== Message Start ===
80+
81+The message start is defined as <code>0x1c163f28</code>. These four bytes were randomly generated and have no special meaning.
nit: Instead of claiming that they are randomly generated, it may be better to generate them deterministically, so that the same approach can be used for testnet5?
E.g. echo -n testnet4 | sha256sum | head -c 8, or similar, if you recall how you generated them?
I called head -c 4 /dev/urandom | xxd -p locally.
Instead of claiming that they are randomly generated, it may be better to generate them randomly, so that the same approach can be used for testnet5?
E.g. echo -n testnet4 | sha256sum | head -c 8, or similar, if you recall how you generated them?
The suggested way to generate them isn’t random but deterministic as far as I can tell and if my definition of random isn’t off. If there is a serious concern with my suggested sequence of bytes we could switch to something deterministic but making it repeatable and setting a rule in stone like suggested means the bytes for all future testnets will be known now and I am not sure this is a good thing.
Yeah, it was just a nit. I guess the exact value doesn’t matter too much, except that it shouldn’t collide with the bytes of the mainnet, or previous, or current testnets (including signets). I suggested a hash function, because the uniform distribution is probably the best bet at achieving the goal. (Edited out the typo in my comment, thanks).
Signet uses the hash of the challenge, so using the hash of the genesis block for testnet seems fine. (I think this is what you were suggesting in your last reply?)
In any case, just a nit and not a blocker.
making it repeatable and setting a rule in stone like suggested means the bytes for all future testnets will be known now
They don’t have to be. For example, in Signet, you have double SHA-256 of the signet challenge. This can work in a similar way. The code for signet message start chars is already there. So, it can be identical to a signet network, where the message from the Genesis Block is set as a signet challenge. In this way, the code for generating new networks will be known upfront, but the hash of the block will remain unknown, until the new Genesis Block for the new testnet will be mined.
it shouldn’t collide with the bytes of the mainnet, or previous, or current testnets (including signets)
You cannot avoid colliding with signets. Every network creator can pick its own signet challenge. And there are only four bytes. Which means, that finding a signet challenge, which will collide with any existing network, takes roughly the same effort, as mining a single block, with the minimal difficulty. Also, because of that, anyone can make a new network, and grind it to reach 0x1c163f28, and then try to connect it with testnet4.
92+
93+Pull request at https://github.com/bitcoin/bitcoin/pull/29775
94+
95+== References ==
96+
97+# Original mailing list thread: https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/kFPaQCzmBwAJ
What about referring to the gnusha archives as well? They seem to be more stable long term.
0https://gnusha.org/pi/bitcoindev/CADL_X_eXjbRFROuJU0b336vPVy5Q2RJvhcx64NSNPH-3fDCUfw@mail.gmail.com/
Looks good so far, considering the the comments from others.
I’m still to play a bit with the specification.
8+ Status: Draft
9+ Type: Standards Track
10+ Created: 2024-05-27
11+ License: CC0-1.0
12+ Post-History: https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ
13+ https://groups.google.com/g/bitcoindev/c/0BYW_diKiVw/m/9ZqIaxb1AQAJ
https://github.com/bitcoin/bitcoin/pull/29775
17+
18+A new test network with the goal to replace Testnet 3. This network comes with small but important improvements of the consensus rules, that should make it impractical to attack the network using only CPU mining.
19+
20+== Motivation ==
21+
22+Quoting the original mailing list post from Jameson Lopp:
19+
20+== Motivation ==
21+
22+Quoting the original mailing list post from Jameson Lopp:
23+
24+"Testnet3 has been running for 13 years. It's on block 2.5 million something and the block reward is down to ~0.014 TBTC, so mining is not doing a great job at distributing testnet coins anymore.
> for this multi paragraph quote.
28+Testnet3 is being actively used for scammy airdrops; those of us who tend to be generous with our testnet coins are getting hounded by non-developers chasing cheap gains.
29+
30+As a result, TBTC is being actively bought and sold; one could argue that the fundamental principle of testnet coins having no value has been broken."
31+
32+Since then the issue with block storms has been further demonstrated on Testnet 3 when three years' worth of blocks were mined in a few weeks while rendering the network practically unusable at the same time.
33+
31+
32+Since then the issue with block storms has been further demonstrated on Testnet 3 when three years' worth of blocks were mined in a few weeks while rendering the network practically unusable at the same time.
33+
34+== Specification ==
35+
36+Consensus of Testnet 4 follows the same rules as mainnet with the exception of the three rules detailed below.
0Additionally all soft forks that are active on mainnet as of May 2024 are enforced from genesis.
(with “May” being the month of the genesis block)
35+
36+Consensus of Testnet 4 follows the same rules as mainnet with the exception of the three rules detailed below.
37+
38+=== 20-min Exception ===
39+
40+A block with a timestamp that is more than 20 minutes past the timestamp of the previous block is allowed to have a minimum difficulty of 1 (the network's minimum difficulty) instead of whatever the actual difficulty level currently is. This applies to all blocks in a difficulty period except for the first block. The blocks that take advantage of this rule must also change their `nBits` field from the actual difficulty level to the minimum difficulty value `0x1d00ffff`.
It’s not “allowed to” (MAY have), it’s required (“MUST have”).
Unfortunately ContextualCheckBlockHeader does block.nBits != GetNextWorkRequired rather than >=. You can still reorg such a block with the “real” nBits value by backdating the timestamp such that it’s less than 20 minutes after the previous block.
37+
38+=== 20-min Exception ===
39+
40+A block with a timestamp that is more than 20 minutes past the timestamp of the previous block is allowed to have a minimum difficulty of 1 (the network's minimum difficulty) instead of whatever the actual difficulty level currently is. This applies to all blocks in a difficulty period except for the first block. The blocks that take advantage of this rule must also change their `nBits` field from the actual difficulty level to the minimum difficulty value `0x1d00ffff`.
41+
42+Please note: This rule was already previously implemented and active in Testnet 3 but there is no BIP specifying its behavior. This rule also led to the block storms which the following rule seeks to fix.
Let’s move “This rule was already previously implemented and active in Testnet 3” to the top, leaving out “Please note”.
You can probably also drop “there is no BIP specifying its behavior”, since the above “follows the same rules as mainnet with the exception of” already makes it clear why this rule is specified here.
The last sentence “This rule also led to …” can stay where it is.
Would be nice to link to some earlier discussion where this rule was introduced.
41+
42+Please note: This rule was already previously implemented and active in Testnet 3 but there is no BIP specifying its behavior. This rule also led to the block storms which the following rule seeks to fix.
43+
44+=== Block Storm Fix ===
45+
46+When the next work required is calculated for the first block in a new difficulty period, the difficulty of the last block of the previous difficulty period is only used as the base for this calculation if its difficulty is not 1. If its difficulty is 1, all blocks in the previous difficulty period are checked in reverse order if any of them have a different difficulty than 1. When a different difficulty is encountered, it is assumed to be the actual difficulty of the network and used as the base for the calculation of the new difficulty level. Note that the first block in new difficulty period does not allow usage of the 20-min rule (this is prior behavior). This means that in each difficulty period the first block should always have the actual difficulty even if all other blocks were mined with the 20-min rule.
Let’s explain the problem first:
The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). This factor is applied to the nBits value of the last block.
Block storms are generated by causing the nBits value of the last block to be 1, using the rule from the previous section. The multiplication factor then results in a new difficulty between 1 (since it can’t be less) and 4. An arbitrarily large number of blocks can then be generated at very low difficulty. The block storm is bound only by miner hash rate, the need for every last block in the difficulty adjustment period to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule and the requirement that blocks can’t be more than 2 hours in the future.
A block storm does not require a time warp attack, but one can be used to amplify it.
The mitigation consists of adding a new rule, to no longer apply the adjustment factor to the last block of the previous difficulty period when its difficulty is 1. In that case blocks are traversed in reverse order until a block is found with a higher difficulty. If the first block of the previous period also has difficulty 1 than this is used.
The previous block that is found in this manner is assumed to be the actual difficulty of the network and used as the base for the calculation of the new difficulty level. Note that the first block in new difficulty period does not allow usage of the 20-min rule (this is prior behavior). This means that in each difficulty period the first block should always have the actual difficulty even if all other blocks were mined with the 20-min rule.
43+
44+=== Block Storm Fix ===
45+
46+When the next work required is calculated for the first block in a new difficulty period, the difficulty of the last block of the previous difficulty period is only used as the base for this calculation if its difficulty is not 1. If its difficulty is 1, all blocks in the previous difficulty period are checked in reverse order if any of them have a different difficulty than 1. When a different difficulty is encountered, it is assumed to be the actual difficulty of the network and used as the base for the calculation of the new difficulty level. Note that the first block in new difficulty period does not allow usage of the 20-min rule (this is prior behavior). This means that in each difficulty period the first block should always have the actual difficulty even if all other blocks were mined with the 20-min rule.
47+
48+For the avoidance of doubt, no matter which block in the previous difficulty period provides the actual difficulty used as the basis for the calculation, the timestamp of the last block is always the one that is used as the end time of the difficulty period.
One known downside of this approach is that if the difficulty is gradually raised by a miner with significant hash rate, and this miner disappears, then each difficulty adjustment period requires one block at the original difficulty.
This would cause the network to stall once per difficulty adjustment period until the real difficulty is adjusted downwards enough for the remaining hash rate to find this block in reasonable time.
47+
48+For the avoidance of doubt, no matter which block in the previous difficulty period provides the actual difficulty used as the basis for the calculation, the timestamp of the last block is always the one that is used as the end time of the difficulty period.
49+
50+=== Time Warp Fix ===
51+
52+To protect against the time warp attack, the following rule proposed as part of The Great Consensus Cleanup is enforced: "The nTime field of each block whose height, mod 2016, is 0 must be greater than or equal to the nTime field of the immediately prior block minus 600. For the avoidance of doubt, such blocks must still comply with existing Median-Time-Past nTime restrictions."
54+== Rationale ==
55+
56+The applied changes were the result of discussions on the mailing list and the PR. The selected changes try to strike a balance between minimal changes to the network (keeping it as close to mainnet as possible) while making it more robust against attackers that try to disrupt the network. Several alternative designs were considered:
57+
58+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
59+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU.
55+
56+The applied changes were the result of discussions on the mailing list and the PR. The selected changes try to strike a balance between minimal changes to the network (keeping it as close to mainnet as possible) while making it more robust against attackers that try to disrupt the network. Several alternative designs were considered:
57+
58+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
59+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU.
60+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner.
57+
58+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
59+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU.
60+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner.
61+* Increase of the delay in the 20 min difficulty exception was suggested but did not receive significant support.
62+* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests and expected it to behave similar to mainnet.
acceptnonstdtxn is on by default in Bitcoin Core, is not part of the design of testnet4. But it’s fine to just leave it here.
58+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
59+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU.
60+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner.
61+* Increase of the delay in the 20 min difficulty exception was suggested but did not receive significant support.
62+* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests and expected it to behave similar to mainnet.
63+* Motivating miners to re-org min difficulty blocks was suggested but this may still be done after Testnet 4 is deployed, so it can be considered out of scope for this BIP.
Concept ACK aad54df7b3c3db49f05e6c354c2066c2e4911ba7.
I left feedback on the explanation, but the proposed rules themselves look good to me.
I’m still open to someone writing a script to produce useful test vectors, and resetting the genesis block to include those earlier.
Other than that, it seems we ran out of steam coming up with ideas to further improve testnet4 resistance against shenanigans. There’s always testnet5 :-)
I’m still open to someone writing a script to produce useful test vectors, and resetting the genesis block to include those earlier.
I am happy to add this but writing that script is the hard part. I am working on something but there are a lot of ideas, contributors welcome!
40+
41+=== 20-min Exception ===
42+
43+This rule was already previously implemented and active in Testnet 3<ref>https://github.com/bitcoin/bitcoin/pull/686</ref>.
44+
45+A block with a timestamp that is more than 20 minutes past the timestamp of the previous block must have a minimum difficulty of 1 (the network's minimum difficulty) instead of whatever the actual difficulty level currently is. This applies to all blocks in a difficulty period except for the first block. This means the blocks must change their <code>nBits</code> field from the actual difficulty level to the minimum difficulty value <code>0x1d00ffff</code>.
46+
47+This rule also led to the block storms<ref>https://blog.lopp.net/the-block-storms-of-bitcoins-testnet/</ref> which the following rule seeks to fix.
48+
49+=== Block Storm Fix ===
50+
51+The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). This factor is applied to the nBits value of the last block.
Here in the Specification section, this could be understood to be describing the fix. Perhaps it should be clarified that the latter does not apply to Testnet 4.
0The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). On Mainnet and Testnet 3, this factor is applied to the nBits value of the last block.
48+
49+=== Block Storm Fix ===
50+
51+The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). This factor is applied to the nBits value of the last block.
52+
53+Block storms are generated by causing the nBits value of the last block to be 1, using the rule from the previous section. The multiplication factor then results in a new difficulty between 1 (since it can't be less) and 4. An arbitrarily large number of blocks can then be generated at very low difficulty. The block storm is bound only by miner hash rate, the need for every last block in the difficulty adjustment period to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule and the requirement that blocks can't be more than 2 hours in the future.
Perhaps mention that block storms were happening organically before someone figured out that they could create a perpetual storm:
0Block storms happen organically whenever the 20-minute exception is applied to a difficulty period’s last block, setting its nBits value to 1. The difficulty adjustment rules then limit the subsequent difficulty to a value between 1 (the minimum) and 4. Blocks will be generated rapidly in the subsequent low-difficulty periods while the difficulty climbs back to an adequate range. An arbitrarily large number of blocks can be generated quickly by repeatedly using the 20-minute exception on every last block of difficulty periods. The block storm is then bounded only by miner hash rate, the need for last blocks to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule, and the requirement that blocks can't be more than 2 hours in the future. Overall a sustained would eventually be limited to a maximum cadence of one block per second.
50+
51+The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). This factor is applied to the nBits value of the last block.
52+
53+Block storms are generated by causing the nBits value of the last block to be 1, using the rule from the previous section. The multiplication factor then results in a new difficulty between 1 (since it can't be less) and 4. An arbitrarily large number of blocks can then be generated at very low difficulty. The block storm is bound only by miner hash rate, the need for every last block in the difficulty adjustment period to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule and the requirement that blocks can't be more than 2 hours in the future.
54+
55+A block storm does not require a time warp attack, but one can be used to amplify it.
It was not clear to me at first how a time warp attack could be used to exacerbate the perpetual block storm attack, but this is how I explained it to myself then:
0A block storm does not require a time warp attack, but one can be used to amplify<ref>A perpetual block storm attack with entire difficulty period being authored in less than 3.5 days that resets the difficulty to the minimum in the last block of every difficulty period would adjust to a new actual difficulty of 4 every period. An attacker that additionally leverages a time warp attack would start their attack by holding back timestamps until the latest block’s timestamp is at least two weeks in the past, and then limiting their block rate to six blocks per second, incrementing the timestamp on every sixth block. Only on the last block they would use the current time, which both resets the difficulty to one per the 20-minute exception and would result in a difficulty adjustment keeping the difficulty at the minimum due to the elapsed time exceeding the target.</ref> it.
I have added this but maybe @Sjors can chime in if he had a different style of attack in mind.
I think what’s also true and could be added here as well is that the 20-minute exception rule makes it possible to attack Testnet 4 with lower hash power than usual. I realize now that is the inverse of what is written here right now but it’s been in my mind mostly when thinking of why we don’t want time warp attacks.
For example, to ensure that the attacker gets to set the timestamp on the first block (or any particular block) of the difficulty period they could use the 20-min exception to reorg other miners out. They could mine in the pattern of one block with a timestamp as low as the MTP permits, then a few 20-min exception blocks on top of that to reorg anyone else who has found blocks in the meantime, then again another real one with as low as MTP permits. That should allow them to reliably keep MTP low even with low mining power compared to the rest of the network. I guess 10% of the network hashrate might be enough to exploit the time warp on Testnet 4 without timewarp protection, since you need to be able to find at least one real block within the MTP window somewhat reliably. The good thing is of course that each 20-min exception also potentially drags MTP forward so that makes the attack a bit trickier.
In general I think we should see average blocktimes faster than 10 min if the 20-min exception is taken advantage of regularly already even without doing anything to the timestamps.
I need to think a bit more about this but I can try to draft something if you think this belongs in here as well.
Oh, I thought that blocks would count towards the total work of the chain according to their nBits and that would mean that even several 20-minute exception blocks would contribute less to the total work than one block mined at the actual difficulty.
I asked a clarifying question here: https://bitcoin.stackexchange.com/q/123703/5406
Edit: It sounds like you would have to mine a large number of 20-minute exception blocks to oust an actual difficulty block ;)
Was just thinking about this again. I am not quite sure I follow your scenario. If you were suggesting that two 20-minute exception blocks can displace one actual difficulty block, I don’t think that’s the case. The total work added by the two 20-minute blocks would only amount to difficulty 2, whereas the actual difficulty block would probably have a difficulty of more than 2.
However, if there are two actual difficulty blocks competing, one could easily decide the tie by mining a 20-minute exception block as diff_act + 1 would beat diff_act.
52+
53+Block storms are generated by causing the nBits value of the last block to be 1, using the rule from the previous section. The multiplication factor then results in a new difficulty between 1 (since it can't be less) and 4. An arbitrarily large number of blocks can then be generated at very low difficulty. The block storm is bound only by miner hash rate, the need for every last block in the difficulty adjustment period to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule and the requirement that blocks can't be more than 2 hours in the future.
54+
55+A block storm does not require a time warp attack, but one can be used to amplify it.
56+
57+The mitigation consists of adding a new rule, to no longer apply the adjustment factor to the last block of the previous difficulty period when its difficulty is 1. In that case blocks are traversed in reverse order until a block is found with a higher difficulty. If the first block of the previous period also has difficulty 1 than this is used.
Why do we need to traverse in reverse order at all? Wouldn’t it be easier to always use the first block’s nBits to calculate the difficulty adjustment? It seems to me that it would even be compatible with mainnet.
0The mitigation consists of adding a new rule, to no longer apply the adjustment factor to the last block of the previous difficulty period when its difficulty is 1. In that case, blocks are traversed in reverse order until a block is found with a higher difficulty. Only if all blocks of the previous period have a difficulty of 1, including its first block, difficulty 1 is used.
You are right, that makes sense. I think the traversing back just made sense to me initially because that’s what we are also doing when we look for the real difficulty of a normal 20-minute exception block. I will look into simplifying the code in the PR with this in mind and then update here.
Also took the suggested change here.
54+
55+A block storm does not require a time warp attack, but one can be used to amplify it.
56+
57+The mitigation consists of adding a new rule, to no longer apply the adjustment factor to the last block of the previous difficulty period when its difficulty is 1. In that case blocks are traversed in reverse order until a block is found with a higher difficulty. If the first block of the previous period also has difficulty 1 than this is used.
58+
59+The previous block that is found in this manner is assumed to be the actual difficulty of the network and used as the base for the calculation of the new difficulty level. Note that the first block in new difficulty period does not allow usage of the 20-min rule (this is prior behavior). This means that in each difficulty period the first block should always have the actual difficulty even if all other blocks were mined with the 20-min rule.
0The previous block that is found in this manner must contain the actual difficulty of the network and can therefore be used as the base for the calculation of the new difficulty level. Note that the first block in new difficulty period does not allow usage of the 20-min exception (this is prior behavior). This means that in each difficulty period the first block must always have the actual difficulty even if all other blocks were mined with the 20-min exception.
0The previous block that is found in this manner is assumed to be the actual difficulty of the network and used as the base for the calculation of the new difficulty level. Note that the first block in new difficulty period does not allow usage of the 20-minute exception (this is prior behavior). This means that in each difficulty period the first block should always have the actual difficulty even if all other blocks were mined with the 20-minute exception.
58+
59+The previous block that is found in this manner is assumed to be the actual difficulty of the network and used as the base for the calculation of the new difficulty level. Note that the first block in new difficulty period does not allow usage of the 20-min rule (this is prior behavior). This means that in each difficulty period the first block should always have the actual difficulty even if all other blocks were mined with the 20-min rule.
60+
61+=== Time Warp Fix ===
62+
63+To protect against the time warp attack, the following rule proposed as part of The Great Consensus Cleanup<ref>https://github.com/TheBlueMatt/bips/blob/cleanup-softfork/bip-XXXX.mediawiki</ref> is enforced: "The nTime field of each block whose height, mod 2016, is 0 must be greater than or equal to the nTime field of the immediately prior block minus 600. For the avoidance of doubt, such blocks must still comply with existing Median-Time-Past nTime restrictions."
0In addition to a time warp attack potentially exacerbating the perpetual block storm attack, a time warp attack provides an alternative way to increase the block production rate even if the unintended reset of the actual difficulty due to the 20-minute exception was mitigated.
1To protect against the time warp attack, the following rule proposed as part of The Great Consensus Cleanup<ref>https://github.com/TheBlueMatt/bips/blob/cleanup-softfork/bip-XXXX.mediawiki</ref> is enforced: "The nTime field of each block whose height, mod 2016, is 0 must be greater than or equal to the nTime field of the immediately prior block minus 600. For the avoidance of doubt, such blocks must still comply with existing Median-Time-Past nTime restrictions."
65+== Rationale ==
66+
67+The applied changes were the result of discussions on the mailing list and the PR. The selected changes try to strike a balance between minimal changes to the network (keeping it as close to mainnet as possible) while making it more robust against attackers that try to disrupt the network. Several alternative designs were considered:
68+
69+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
70+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. It is also prevents the chain from getting stuck immediately in case a large amount of hash power suddenly leaves the network which allows the chain to recover to a normal difficulty level faster.
Extraneous word in “It is also prevents”, and “It also prevents the chain from getting stuck immediately” bugs me a bit, since that is only true for 2015 blocks out 2016. ;)
Perhaps:
0* Removal of the 20-minute exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. The 20-minute exception also allows CPU users to move the chain forward (except on the first block that needs to be mined at actual difficulty) in case a large amount of hash power suddenly leaves the network. This would allow the chain to recover to a normal difficulty level faster if left stranded at high difficulty.
69+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
70+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. It is also prevents the chain from getting stuck immediately in case a large amount of hash power suddenly leaves the network which allows the chain to recover to a normal difficulty level faster.
71+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner (utilizing the 20-min exception).
72+* Increase of the delay in the 20 min difficulty exception was suggested but did not receive significant support.
73+* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests and expected it to behave similar to mainnet.
74+* Motivating miners to re-org min difficulty blocks was suggested but this may still be done after Testnet 4 is deployed, so it can be considered out of scope for this BIP.
How about:
0* Motivating miners to re-org min difficulty blocks was suggested, but was considered out of scope for this BIP, since adoption of such a mining policy remains available after Testnet 4 is deployed.
70+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. It is also prevents the chain from getting stuck immediately in case a large amount of hash power suddenly leaves the network which allows the chain to recover to a normal difficulty level faster.
71+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner (utilizing the 20-min exception).
72+* Increase of the delay in the 20 min difficulty exception was suggested but did not receive significant support.
73+* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests and expected it to behave similar to mainnet.
74+* Motivating miners to re-org min difficulty blocks was suggested but this may still be done after Testnet 4 is deployed, so it can be considered out of scope for this BIP.
75+* Persisting the real difficulty in the version field was suggested to prevent exploits of the 20-min rule in an even more robust way, but did not receive a critical level of support since it would be a more invasive change.
0* Persisting the real difficulty in the version field was suggested to robustly prevent exploits of the 20-minute exception while allowing it to be used on any block, but did not receive a sufficient level of support to justify the more invasive change.
97+
98+=== Message Start ===
99+
100+The message start is defined as <code>0x1c163f28</code>. These four bytes were randomly generated and have no special meaning.
101+
102+== Compatibility ==
0== Backwards Compatibility ==
103+
104+This specification is backward compatible in the sense that existing software can use Testnet 4 out of the box, assuming support for Testnet 3 already exists.
105+
106+Simply by adding the network parameters for Testnet 4 (magic number, etc.), a client can connect to and use Testnet 4 without further modifications. The block headers have valid proof of work, so clients can trivially check that blocks are "probably" valid.
107+
108+However, without the implementation of the changes detailed in Specifications, a client could follow a chain that does not follow the rules. Any fully validating node should check these rules and reject blocks that fail to follow them. Clients should either validate these rules or connect to trusted peers that do full validation.
It’s not obvious to me why this section speaks about full nodes. Both new rules can be validated on basis of the headers and therefore should even be enforced by light clients.
0The rules used by Testnet 4 are backwards compatible to the rules of Testnet 3. Existing software that implements support for Testnet 3 would only require addition of the network parameters (magic number, genesis block, etc.) to be able to follow Testnet 4.
1
2However, implementations that only implement Testnet 3’s rules would accept a chain that violates Testnet 4’s rules and are therefore susceptible to being forked off. It is recommended that any implementations check blocks in regard to all the new rules of Testnet 4 and reject blocks that fail to comply.
36+
37+== Specification ==
38+
39+Consensus of Testnet 4 follows the same rules as mainnet with the exception of the three rules detailed below. Additionally all soft forks that are active on mainnet as of May 2024 are enforced from genesis.
40+
41+=== 20-min Exception ===
0=== 20-minute Exception ===
66+
67+The applied changes were the result of discussions on the mailing list and the PR. The selected changes try to strike a balance between minimal changes to the network (keeping it as close to mainnet as possible) while making it more robust against attackers that try to disrupt the network. Several alternative designs were considered:
68+
69+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
70+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. It is also prevents the chain from getting stuck immediately in case a large amount of hash power suddenly leaves the network which allows the chain to recover to a normal difficulty level faster.
71+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner (utilizing the 20-min exception).
0* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner (utilizing the 20-minute exception).
67+The applied changes were the result of discussions on the mailing list and the PR. The selected changes try to strike a balance between minimal changes to the network (keeping it as close to mainnet as possible) while making it more robust against attackers that try to disrupt the network. Several alternative designs were considered:
68+
69+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
70+* Removal of the 20-min difficulty exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. It is also prevents the chain from getting stuck immediately in case a large amount of hash power suddenly leaves the network which allows the chain to recover to a normal difficulty level faster.
71+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner (utilizing the 20-min exception).
72+* Increase of the delay in the 20 min difficulty exception was suggested but did not receive significant support.
0* Increase of the delay in the 20-minute exception was suggested but did not receive significant support.
48+
49+=== Block Storm Fix ===
50+
51+The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). On Mainnet and Testnet 3, this factor is applied to the nBits value of the last block.
52+
53+Block storms happen organically whenever the 20-minute exception is applied to a difficulty period’s last block, setting its nBits value to 1. The difficulty adjustment rules then limit the subsequent difficulty to a value between 1 (the minimum) and 4. Blocks will be generated rapidly in the subsequent low-difficulty periods while the difficulty climbs back to an adequate range. An arbitrarily large number of blocks can be generated quickly by repeatedly using the 20-minute exception on every last block of difficulty periods. The block storm is then bounded only by miner hash rate, the need for last blocks to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule, and the requirement that blocks can't be more than 2 hours in the future. Overall a sustained attack would eventually be limited to a maximum cadence of one block per second.
Sorry, I was wrong on this one. The maximum sustainable cadence of blocks is six per second, not one per second.
0Block storms happen organically whenever the 20-minute exception is applied to a difficulty period’s last block, setting its nBits value to 1. The difficulty adjustment rules then limit the subsequent difficulty to a value between 1 (the minimum) and 4. Blocks will be generated rapidly in the subsequent low-difficulty periods while the difficulty climbs back to an adequate range. An arbitrarily large number of blocks can be generated quickly by repeatedly using the 20-minute exception on every last block of difficulty periods. The block storm is then bounded only by miner hash rate, the need for last blocks to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule, and the requirement that blocks can't be more than 2 hours in the future. Overall a sustained attack would eventually be limited to a maximum cadence of six blocks per second.
50+
51+The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). On Mainnet and Testnet 3, this factor is applied to the nBits value of the last block.
52+
53+Block storms happen organically whenever the 20-minute exception is applied to a difficulty period’s last block, setting its nBits value to 1. The difficulty adjustment rules then limit the subsequent difficulty to a value between 1 (the minimum) and 4. Blocks will be generated rapidly in the subsequent low-difficulty periods while the difficulty climbs back to an adequate range. An arbitrarily large number of blocks can be generated quickly by repeatedly using the 20-minute exception on every last block of difficulty periods. The block storm is then bounded only by miner hash rate, the need for last blocks to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule, and the requirement that blocks can't be more than 2 hours in the future. Overall a sustained attack would eventually be limited to a maximum cadence of one block per second.
54+
55+A block storm does not require a time warp attack, but one can be used to amplify<ref>A perpetual block storm attack with entire difficulty period being authored in less than 3.5 days that resets the difficulty to the minimum in the last block of every difficulty period would adjust to a new actual difficulty of 4 every period. An attacker that additionally leverages a time warp attack would start their attack by holding back timestamps until the latest block’s timestamp is at least two weeks in the past, and then limiting their block rate to six blocks per second, incrementing the timestamp on every sixth block. Only on the last block they would use the current time, which both resets the difficulty to one per the 20-minute exception and would result in a difficulty adjustment keeping the difficulty at the minimum due to the elapsed time exceeding the target.</ref> it.
One more sentence to make the amplification obvious.
0A block storm does not require a time warp attack, but one can be used to amplify<ref>A perpetual block storm attack with entire difficulty periods being authored in less than 3.5 days that resets the difficulty to the minimum in the last block of every difficulty period would adjust to a new actual difficulty of 4 every period. An attacker that additionally leverages a time warp attack would start their attack by holding back timestamps until the latest block’s timestamp is at least two weeks in the past, and then limiting their block rate to six blocks per second, incrementing the timestamp on every sixth block. Only on the last block they would use the current time, which both resets the difficulty to one per the 20-minute exception and would result in a difficulty adjustment keeping the difficulty at the minimum due to the elapsed time exceeding the target. This would allow lower the difficulty for all blocks to difficulty 1 instead of difficulty 4.</ref> it.
71+* For the block storm fix an alternative fix could have been to prevent the last block in a difficulty period from applying the existing difficulty exception. Both solutions were deemed acceptable and there was no clear preference among reviewers.
72+* Removal of the 20-minute exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. The 20-minute exception also allows CPU users to move the chain forward (except on the first block that needs to be mined at actual difficulty) in case a large amount of hash power suddenly leaves the network. This would allow the chain to recover to a normal difficulty level faster if left stranded at high difficulty.
73+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner (utilizing the 20-minute exception).
74+* Increase of the delay in the 20-minute exception was suggested but did not receive significant support.
75+* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests and expected it to behave similar to mainnet.
76+* Motivating miners to re-org min difficulty blocks was suggested, but was considered out of scope for this BIP, since adoption of such a mining policy remains available after Testnet 4 is deployed.
The diverging total work contribution of 20-minute exception blocks could be highlighted here:
0* Motivating miners to re-org min difficulty blocks was suggested, but was considered out of scope for this BIP, since adoption of such a mining policy remains available after Testnet 4 is deployed. As 20-minute exception blocks only contribute work corresponding to difficulty one to the chaintip, and actual difficulty blocks should have a difficulty magnitudes higher, a block mined at actual difficulty could easily replace even multiple 20-minute exception blocks.
72+* Removal of the 20-minute exception was discussed but dismissed since several reviewers insisted that it was a useful feature allowing non-standard transactions to be mined with just a CPU. The 20-minute exception also allows CPU users to move the chain forward (except on the first block that needs to be mined at actual difficulty) in case a large amount of hash power suddenly leaves the network. This would allow the chain to recover to a normal difficulty level faster if left stranded at high difficulty.
73+* Increase of minimum difficulty was discussed but dismissed as it would categorically prevent participation in the network using a CPU miner (utilizing the 20-minute exception).
74+* Increase of the delay in the 20-minute exception was suggested but did not receive significant support.
75+* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests and expected it to behave similar to mainnet.
76+* Motivating miners to re-org min difficulty blocks was suggested, but was considered out of scope for this BIP, since adoption of such a mining policy remains available after Testnet 4 is deployed.
77+* Persisting the real difficulty in the version field was suggested to robustly prevent exploits of the 20-minute exception while allowing it to be used on any block, but did not receive a sufficient level of support to justify the more invasive change.
Let’s call this BIP 94.
Could you please add the number to the PR title, put it into the preamble, and insert a table entry in the README for your BIP?
To prevent a premine - take the bits from a hash of a future mainnet block:
psuedo code ’’' if mainnet_blockheight >= 898898 then messageStart = sha256(mainnetblock_hash_898898) … ’''
This method can be reused for all future testnets?
This is both deterministic and verifiable?
To prevent a premine - take the bits from a hash of a future mainnet block:
psuedo code ’’' if mainnet_blockheight >= 898898 then messageStart = sha256(mainnetblock_hash_898898) … ’''
This method can be reused for all future testnets?
This is both deterministic and verifiable?
Couldn’t the mainnet_blockheight be a parameter - enabling devs to spin up their own testnet4? It may be useful to miners for testing? In fact - every future block can seed its own testnet4. :)
Note: I was - in jest - calling this “blobnet” :)
A parameterized testnet4 <mainnet_blockheight> may have the effect of allowing inscribers to use a testnet4 at mainnet_blockheight for their own purposes - mitigating the incentives to pollute mainnet as well as the global testnet4 at blockheight
Part of what makes Testnet still interesting is that there are lots of others using it. So I think this is a different feature that is out of scope for this BIP, requiring seperate conceptual review etc..
0@@ -0,0 +1,120 @@
1+<pre>
2+ BIP: 94
3+ Layer: Applications
4+ Title: Testnet 4
5+ Author: Fabian Jahr <fjahr@protonmail.com>
6+ Comments-Summary: No comments yet.
7+ Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-testnet4
0 Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0094
52+
53+Block storms happen organically whenever the 20-minute exception is applied to a difficulty period’s last block, setting its nBits value to 1. The difficulty adjustment rules then limit the subsequent difficulty to a value between 1 (the minimum) and 4. Blocks will be generated rapidly in the subsequent low-difficulty periods while the difficulty climbs back to an adequate range. An arbitrarily large number of blocks can be generated quickly by repeatedly using the 20-minute exception on every last block of difficulty periods. The block storm is then bounded only by miner hash rate, the need for last blocks to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule, and the requirement that blocks can't be more than 2 hours in the future. Overall a sustained attack would eventually be limited to a maximum cadence of six blocks per second.
54+
55+A block storm does not require a time warp attack, but one can be used to amplify<ref>A perpetual block storm attack with entire difficulty periods being authored in less than 3.5 days that resets the difficulty to the minimum in the last block of every difficulty period would adjust to a new actual difficulty of 4 every period. An attacker that additionally leverages a time warp attack would start their attack by holding back timestamps until the latest block’s timestamp is at least two weeks in the past, and then limiting their block rate to six blocks per second, incrementing the timestamp on every sixth block. Only on the last block they would use the current time, which both resets the difficulty to one per the 20-minute exception and would result in a difficulty adjustment keeping the difficulty at the minimum due to the elapsed time exceeding the target. This would allow lower the difficulty for all blocks to difficulty 1 instead of difficulty 4</ref> it.
56+
57+The mitigation consists of adding a new rule, to no longer apply the adjustment factor to the last block of the previous difficulty period when its difficulty is 1. In that case instead the first block of the difficulty period is used as the base.
46+
47+This rule also led to the block storms<ref>https://blog.lopp.net/the-block-storms-of-bitcoins-testnet/</ref> which the following rule seeks to fix.
48+
49+=== Block Storm Fix ===
50+
51+The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). On Mainnet and Testnet 3, this factor is applied to the nBits value of the last block.
Nit: nBits is the target, and the difficulty is inversely related to the target, so the factor that is being applied to the difficulty is the inverse of the factor being applied to nBits.
I also notice that it is not mentioned that the factor is based on the duration of the prior difficulty period.
48+
49+=== Block Storm Fix ===
50+
51+The work required for a new difficulty period is calculated as multiplication factor to the difficulty of the previous period (but no less than 1/4th and no more than 4x). On Mainnet and Testnet 3, this factor is applied to the nBits value of the last block.
52+
53+Block storms happen organically whenever the 20-minute exception is applied to a difficulty period’s last block, setting its nBits value to 1. The difficulty adjustment rules then limit the subsequent difficulty to a value between 1 (the minimum) and 4. Blocks will be generated rapidly in the subsequent low-difficulty periods while the difficulty climbs back to an adequate range. An arbitrarily large number of blocks can be generated quickly by repeatedly using the 20-minute exception on every last block of difficulty periods. The block storm is then bounded only by miner hash rate, the need for last blocks to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule, and the requirement that blocks can't be more than 2 hours in the future. Overall a sustained attack would eventually be limited to a maximum cadence of six blocks per second.
As mentioned above, the nBits value corresponding to difficulty 1 is 0x1d00ffff. It would be better to clearly distinguish when we are referring to the difficulty and when we are referring to the target—it could be a bit confusing to say that the nBits value is “set to 1”.
0Block storms happen organically whenever the 20-minute exception is applied to a difficulty period’s last block, causing the block to be mined at a difficulty of 1. The difficulty adjustment rules then limit the subsequent period’s difficulty to a value between 1 (the minimum) and 4. Blocks will be generated rapidly in the subsequent low-difficulty periods while the difficulty climbs back to an adequate range. An arbitrarily large number of blocks can be generated quickly by repeatedly using the 20-minute exception on every last block of difficulty periods. The block storm is then bounded only by miner hash rate, the need for last blocks to have a timestamp 20 minutes after the second to last block, the Median-Time-Past nTime rule, and the requirement that blocks can't be more than 2 hours in the future. Overall a sustained attack would eventually be limited to a maximum cadence of six blocks per second.
74+* Increase of the delay in the 20-minute exception was suggested but did not receive significant support.
75+* Re-enabling <code>acceptnonstdtxn</code> in bitcoin core by default was dismissed as it had led to confusion among layer-2s that had used testnet for transaction propagation tests and expected it to behave similar to mainnet.
76+* Motivating miners to re-org min difficulty blocks was suggested, but was considered out of scope for this BIP, since adoption of such a mining policy remains available after Testnet 4 is deployed. As 20-minute exception blocks only contribute work corresponding to difficulty one to the chaintip, and actual difficulty blocks should have a difficulty magnitudes higher, a block mined at actual difficulty could easily replace even multiple 20-minute exception blocks.
77+* Persisting the real difficulty in the version field was suggested to robustly prevent exploits of the 20-minute exception while allowing it to be used on any block, but did not receive a sufficient level of support to justify the more invasive change.
78+
79+One known downside of the chosen approach is that if the difficulty is gradually raised by a miner with significant hash rate, and this miner disappears, then each difficulty adjustment period requires one block at the original difficulty.
Perhaps:
0One known downside of the chosen approach is that if the difficulty is gradually raised by a miner with significant hash rate, and this miner disappears, then each difficulty adjustment period requires one block at the actual difficulty.
76+* Motivating miners to re-org min difficulty blocks was suggested, but was considered out of scope for this BIP, since adoption of such a mining policy remains available after Testnet 4 is deployed. As 20-minute exception blocks only contribute work corresponding to difficulty one to the chaintip, and actual difficulty blocks should have a difficulty magnitudes higher, a block mined at actual difficulty could easily replace even multiple 20-minute exception blocks.
77+* Persisting the real difficulty in the version field was suggested to robustly prevent exploits of the 20-minute exception while allowing it to be used on any block, but did not receive a sufficient level of support to justify the more invasive change.
78+
79+One known downside of the chosen approach is that if the difficulty is gradually raised by a miner with significant hash rate, and this miner disappears, then each difficulty adjustment period requires one block at the original difficulty.
80+
81+This would cause the network to stall once per difficulty adjustment period until the real difficulty is adjusted downwards enough for the remaining hash rate to find this block in reasonable time.
I have been taught that in technical writing we should always use the same word for the same abstract concept, even if it is a bit more repetitive than we would write in prose. :nerd_face:
0This would cause the network to stall once per difficulty adjustment period until the actual difficulty is adjusted downwards enough for the remaining hash rate to find this block in reasonable time.
ACK 451f4533fd5fe9a90b1d462f366042b63db50adc
Left a few nits, but it also seems fine to be merged as is. What’s the plan regarding the Genesis Block? I seem to recall that we might reset another time before rolling it out for good.
What’s the plan regarding the Genesis Block? I seem to recall that we might reset another time before rolling it out for good.
I will bring this up in the Bitcoin Core IRC meeting tomorrow. It may not be the perfect venue but I am not sure what could be a better one and I would like to know what we do before v28 feature freeze. Opinions mentioned to me have been split throughout the last months but I think there is more upside to not resetting. I don’t see an issue with the “pre-mine” of 40k blocks since many of these coins seem to be available through free faucets right now which makes it easier for anyone to get some right from the start. There currently is some distribution of these coins among several miners and it’s not clear to me that a more public re-launch next week gives us a fairer result. Maybe someone invests some real hashrate in order to get all of those first 40k blocks. And not resetting allows us to also set a min chain work as @Sjors mentioned on the PR. So I see more upside on not resetting personally but happy to discuss it.
If we had gotten the task done of embedding scripts then we could have done the reset with that but I don’t think I will be able to finish that work by next week and nobody else seems to have stepped up either. I am still going to try to add these scripts in the later parts of the chain and make the tooling available for an eventual Testnet 5.
