What is this?

This adds a feature to set a delay between the blocks at the initial download.

Why?

A few weeks ago I was downloading the blockchain on my systems drive (which is an SSD). I usually download the blockchain on an external drive which is an HDD. During the sync on my SSD my Computer was completely unusable because there were too many read and write operations. This PR adds a feature to throttle the block downlad to make the Computer usable during that time.

Testing

Just add a -blockpause=1000 to add a pause of ten seconds. The default is 0 miliseconds

Notes

This is being ignored if the regtest chain is in use

Log

Running ./bitcoind --blockpause=10000

2019-08-29T19:42:43Z UpdateTip: new best=00000000faac558a7a5266c3c678e53b53b88a619b00dd825395b8e4ca44cdd9 height=9511 version=0x00000001 log2_work=45.215555 tx=9602 date='2009-04-02T02:23:37Z' progress=0.000022 cache=0.0MiB(1txo)
2019-08-29T19:42:53Z UpdateTip: new best=000000009be075c2742955e8cad5aa09390e2e7feffdfecc371bdcae0fb60951 height=9512 version=0x00000001 log2_work=45.215707 tx=9603 date='2009-04-02T02:27:04Z' progress=0.000022 cache=0.0MiB(2txo)
2019-08-29T19:43:03Z UpdateTip: new best=00000000c60596cdb860839242211d9a17ba734e397636135e7744303d189a0b height=9513 version=0x00000001 log2_work=45.215858 tx=9604 date='2009-04-02T01:41:48Z' progress=0.000022 cache=0.0MiB(3txo)
2019-08-29T19:43:13Z UpdateTip: new best=0000000052ab68c20deabfd537a4f40ce88bbd29bdc91b86fb791a701a7af723 height=9514 version=0x00000001 log2_work=45.21601 tx=9605 date='2009-04-02T01:59:40Z' progress=0.000022 cache=0.0MiB(4txo)

See timestamps

Doesn't it help to set a lower par= value if the sync goes too fast? Or what about picking up one of the earlier PRs that reduced I/O priority? This seems kind of a kludge, tbh.

Concept ACK on a throttle function (unsure about pausing versus disable paralelism in verification).

Relevant: #12965 (same goal, different approach)

@laanwj IMO

       Set the number of script verification threads (-8 to 16, 0 = auto, <0 =
       leave that many cores free, default: 0)

doesn't really indicate an I/O throttle if this is what you mean

doesn't really indicate an I/O throttle if this is what you mean

It's not, but by using less CPU power for verification, the consequence is that it'll go slower and automatically do less I/O as well.

What is if you still have a powerful CPU (even with 1 thread) and a fast internet connection? Also I don't think -par was made to reduce the I/O speed, this is more a side-effect

Neither is adding sleep between blocks! So, that's why I mentioned I/O throttling. Your OS has ways to do this that are actually throttling I/O.

See e.g. #9245

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #16324 (Get cs_main out of the critical path in ProcessMessages by TheBlueMatt)
• #16323 (Call ProcessNewBlock() asynchronously by TheBlueMatt)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

I definitely need to read myself into I/O throttling but as long as a thread sleeps, no I/O stuff is being done as well...

Not sure how the code works, but wouldn't it be better for the program to ignore that line of code when blockpause = 0? I'm thinking this should help, unless the program has to go through this extra if statement every time and not just once.

    if (Params().NetworkIDString() != "regtest" and blockpause != 0)
    {
        std::this_thread::sleep_for(std::chrono::milliseconds(blockpause));
    }

Approach NAK, cs_main is held while sleeping.

@GChuf I also thought about it but I don’t think it actually makes a difference. Take a look at the output log for example

@emilengler still, if we have the option to not include that operation every time a block is found, I think it's worth it, no matter if the difference is perceivable

Concept NACK, the problem you're describing is more an OS resource monitoring issue than a problem in Core. IMO complicating critical code paths with options should be really worth it.

If you are using a Linux, have a look on man 7 cgroups, it provides fine-grained throttling policy to specify upper I/O rate limits on a device.

@GChuf I still think checking for something every block takes longer than just executing a function

@emilengler so it checks for regtest for every block as well? is there a way to check only once at the beginning?

@GChuf Yes, in terms of not calling the function everytime

Done, forget to do the commit yesterday

Before going forward please reply/address to the above NACKS.

Approach NACK for many of the reasons given above -- this is a kludge and not the best way to achieve the desired goal; we should have a high bar for complicating consensus code, and I think adding a spurious sleep to intentionally degrade performance does not meet that bar; and we should not expose confusing command line options to users.

I haven't reviewed #9245 in any depth but the approach there seems much more reasonable for achieving the desired effect. Please consider picking that PR up if you'd like to improve the behavior here.

Closed for the above reasons