DEPENDENT: Chainparams: Introduce N testnet chains to test different block sizes

jtimon commented at 4:29 pm on July 6, 2015: contributor

This creates std::numeric_limits<uint64_t>::max() new testchains, each one with a different maximum block size and genesis block. It would be generally good to have more people collecting data and conduction simulations related to different consensus maximum block sizes. This PR attempts to simplify that work. Even if it may take long until it is merged (because it requires many little steps to be taken first), this branch (or a fork of it) can be used right now for testing purposes.

One can use it, for example, like this: ./src/qt/bitcoin-qt -chain=sizetest -debug -printtoconsole -gen=1 -genproclimit=20 -blocksize=2000000

I will rebase and update the list of dependencies accordingly as things get merged.

Dependencies:

CTestNetParams and CRegTestParams extend directly from CChainParams #6381
Chainparams: Translations: DRY: options and error strings #6235
Qt: BIP70: Chainparams: DRY: Make qt/guiutil.cpp fit BIP70 chain name strings #6908 (trivial review)
Chainparams: Use a regular factory for creating chainparams #6907 (medium review)
Don’t check the genesis block #6597 (medium review)
Blocksize: Turn MAX_BLOCK_SIZE and MAX_BLOCK_SIGOPS into functions #6625
Generic selection with -chain= in addition of -testnet and -regtest [closed] #5229
Qt: Simplify network/chain styles and add a default purple one (trivial review)
Checkpoints: The hash of the genesis block is the genesis checkpoint and chain id [closed] #6230
Chainparams: Adapt qt/paymentserver to support more than 2 chains
Chainparams: Introduce AssignBase58PrefixStyle() static function NACK
Blocksize: Introduce sizetest mode with infinite chains NACK

Similar branches on top of older versions:

https://github.com/jtimon/bitcoin/tree/chainparams-sizetest-0.11

jtimon force-pushed on Jul 6, 2015

jtimon force-pushed on Jul 7, 2015

jtimon renamed this:
~~DEPENDENT: Chainparams: Introduce N testnet chains to test different block sizes~~
WIP: Chainparams: Introduce N testnet chains to test different block sizes
on Jul 7, 2015

jtimon force-pushed on Jul 12, 2015

laanwj added the label Tests on Jul 17, 2015

jtimon force-pushed on Jul 18, 2015

jtimon force-pushed on Jul 21, 2015

jtimon commented at 5:39 pm on July 21, 2015: contributor

Rebased after #6381. Also rebased back in time to 0.11: https://github.com/jtimon/bitcoin/tree/chainparams-sizetest-0.11

jtimon force-pushed on Jul 21, 2015

JornC commented at 3:56 pm on July 30, 2015: none

Spun up a node and block explorer on top of this PR:

http://sizenet.yogh.io/#json:getinfo http://sizenet.yogh.io/#json:getblockchaininfo

Availability subject to change, will jump back to the main chain when no longer needed - if needed at all.

(still synching)

jtimon force-pushed on Aug 21, 2015

jtimon commented at 11:58 pm on August 21, 2015: contributor

Rebased and fixed (see #6077 (comment) ).

jtimon renamed this:
~~WIP: Chainparams: Introduce N testnet chains to test different block sizes~~
DEPENDENT: Chainparams: Introduce N testnet chains to test different block sizes
on Aug 27, 2015

jtimon force-pushed on Sep 3, 2015

jtimon force-pushed on Sep 4, 2015

jtimon commented at 7:18 pm on September 4, 2015: contributor

This wasn’t passing travis because of a small bug in https://github.com/jtimon/bitcoin/commit/989e37459b7c2f6505f7c98d7fe40e7eec5aba99 (#6625). I will wait for travis to approve this to force push #6625.

jtimon force-pushed on Sep 12, 2015

TheBlueMatt commented at 7:38 am on October 2, 2015: member

NACK. I think the code should exist for people to use for testing, but I dont think Bitcoin Core is the place to support a ton of test/benchmark chains. This is different from the regtest/testnet chains, which exist to create a bitcoin-in-a-box for testing, but which try to mimic Bitcoin where convinient for testing, not to test changes to Bitcoin itself.

priestc commented at 3:40 am on October 20, 2015: none

I think it might be a better idea to just make one testnet with unlimited blocksize, rather than multiple networks with all different size limits. What is the 20MB test chain going to teach us that the 50MB chain will? When testing scalability, it makes the most sense to have as many people working together than to have multiple people working on multiple chains. Take a look at the bip that I recently wrote that deals with the same subject: https://github.com/bitcoin/bips/pull/222

jtimon commented at 11:16 am on October 20, 2015: contributor

I think it might be a better idea to just make one testnet with unlimited blocksize.

You can effectively do that as well (just set -blocksize to std::numeric_limits<uint64_t>::max()). But this also allows you to simulate networks with smaller sizes and to test schism forks scenarios. @TheBlueMatt what about the individual steps (to make things like this more maintainable on top of master)? What about #6235 and #6597, for example?

jtimon force-pushed on Oct 21, 2015

jtimon commented at 11:38 am on October 21, 2015: contributor

Rebased

laanwj commented at 8:08 am on October 27, 2015: member

NACK. I think the code should exist for people to use for testing, but I dont think Bitcoin Core is the place to support a ton of test/benchmark chain

I tend to agree with @bluematt here. Let’s not overcomplicate bitcoind with all kinds of specific purpose test code.

jtimon commented at 9:39 pm on October 29, 2015: contributor

Precisely this PR makes creating a new testchain easier by removing specific purpose code for different chains that is not always properly encapsulated. Then it prepares code for a future change in block size and then it adds a new testchain to test different block sizes. The last part only makes sense if it is a single extremely simple commit like https://github.com/jtimon/bitcoin/commit/b7bce5c5571200361e3b74406d0f9bc986960491 . But even if that change is never accepted, I will maintain it here because that’s the way I test “is it easy to create a new testchain?” every time I rebase. I have marked that step as nacked and solved in the first post. I have also properly marked the PRs that are currently closed and opened #6907 and #6908 which are now much more clear after #6235. People interested in changes to make it easier to create new testchains can follow this PR and I can rebase just one long branch plus only a few little opened PRs at a time. When there’s only nacked commits remaining we can revisit the nacks and merge or close. This was never intended to be merged all at once, that’s why it has the bullet points (which I’m finding very useful). In the meantime, it is more useful to look at this just to “see the big picture”, while we focus review on the few open PRs this depends on.

jtimon force-pushed on Nov 10, 2015

jtimon commented at 4:57 pm on November 10, 2015: contributor

Rebased after #6908 has being merged.

jtimon force-pushed on Nov 11, 2015

jtimon force-pushed on Nov 14, 2015

jtimon force-pushed on Nov 27, 2015

jtimon force-pushed on Dec 1, 2015

Chainparams: Use a regular factory for creating chainparams db11017ef3

Chainparams: Get rid of CChainParams& Params(std::string) d4bb7a8a7d

consensus: don't define MAX_STANDARD_TX_SIGOPS in terms of block size 56735ce533

consensus: teach ExtractMatches to check for an arbitrary max transaction number

This is a no-op change. For now, everything passes MAX_BLOCK_SIZE / 60, so the
result matches what it would've before.

Tests use a number equal to the number of transactions where necessary,
to ensure that they're never rejected when blocksizesize isn't being tested.

809e5def58

consensus: teach CheckTransaction to check for an arbitrary max tx size

This is a no-op change.

Tests use a value of std::numeric_limits<uint64_t>::max() where necessary, to ensure that they're never
rejected when size isn't being tested.

b165fc806c

consensus: Move consensus constants into Consensus::Params and consensus.h (as functions)

The following are now tied to a chain rather than being defined as global
constants. Their values have not changed.

nMinTxSize
nMaxBlockSize
nMaxTxSize
nMaxBlockSigops
nCoinbaseMaturity

Also, for free (diff-wise):

Blocksize: Turn MAX_BLOCK_SIZE (nMaxBlockSize) and MAX_BLOCK_SIGOPS (nMaxBlockSigops) into functions

...which take Consensus::Params as parameter
This will be convenient to reduce the diff of any proposal that changes the blocksize as a hardfork

969ad5a3d8

Testchains: Don't check the genesis block 059480afcd

Chainparams: Generic selection with -chain=<chainString> in addition of -testnet and -regtest 98923c381e

Chainparams: Qt: Simplify network/chain styles and add a default purple one 7ce714b60e

Checkpoints: The hash of the genesis block it's the genesis checkpoint and chain id 160d874dfd

Chainparams: Adapt qt/paymentserver to support more than 2 chains 28d961ff93

Blocksize: Introduce sizetest mode with std::numeric_limits<uint64_t>::max() chains bc87f51723

??? Blocksize: Chainparams: Add to supportedChains dynamically d631bf6a1f

Chainparams: Macros for chain names a99cc57dbc

fixup? 2e7cb318d9

jtimon force-pushed on Dec 3, 2015

jtimon commented at 8:12 pm on January 13, 2016: contributor

Closing. It will be eventually replaced with a similar one that doesn’t use block size as an example.

jtimon closed this on Jan 13, 2016

jtimon referenced this in commit 1cdb9c8b5f on Apr 6, 2016

jtimon referenced this in commit a6bea40841 on Apr 6, 2016

jtimon referenced this in commit 9b36f62e6b on Mar 24, 2017

MarcoFalke locked this on Sep 8, 2021

DEPENDENT: Chainparams: Introduce N testnet chains to test different block sizes #6382