[tests] make pruning.py faster #10591

pull jnewbery wants to merge 2 commits into bitcoin:master from jnewbery:fastprune changing 12 files +253 −150

jnewbery commented at 8:43 PM on June 14, 2017: member
Headline: Brings time for pruning.py down to ~4 minutes on my VM when run without other tests in parallel.

A couple of modifications to make pruning.py much faster:
- pruning.py doesn't care what kind of transactions make up the big blocks that are pruned in the test. Instead of making blocks with several large, expensive to construct and validate transactions, instead make the large blocks contain a single coinbase transaction with a huge OP_RETURN txout.
- avoid stop-starting nodes where possible.
Builds on top of #10555 and #10556

My VM has slow disk i/o. This would possibly be faster with faster disk i/o. Unfortunately, pruning.py still isn't able to run on Travis. It appears that disk access is very slow for in the Travis environment, and this test still times out/fails. Unfortunately, it's not possible to run this test without very heavy disk access - the whole point of pruning is to remove block files when disk usage goes over a certain threshold.

I'd be interested to know how this affects runtime for others. @sdaftuar
fanquake added the label Tests on Jun 15, 2017
jnewbery force-pushed on Jun 16, 2017
jnewbery force-pushed on Jun 16, 2017
MarcoFalke commented at 12:24 PM on June 18, 2017: member

Concept ACK. Needs rebase
jnewbery commented at 4:09 PM on June 18, 2017: member

I'll rebase if/when #10556 is merged, since I expect that may require further feedback and iteration. There's no rush for this PR since pruning.py isn't run by many people. Currently just looking for feedback on the concept and whether this helps reduce runtime for other people.
MarcoFalke commented at 1:45 PM on June 30, 2017: member

Since this runs a lot faster, can we also enable it for the travis cron job?
jnewbery commented at 1:47 PM on June 30, 2017: member

Since this runs a lot faster, can we also enable it for the travis cron job?

I don't think so. See the original description:

Unfortunately, pruning.py still isn't able to run on Travis. It appears that disk access is very slow for in the Travis environment, and this test still times out/fails. Unfortunately, it's not possible to run this test without very heavy disk access - the whole point of pruning is to remove block files when disk usage goes over a certain threshold.

If you can think of a cunning way to get this to run on Travis, that'd be great. I haven't been able to come up with anything.
MarcoFalke commented at 3:27 PM on July 2, 2017: member

Needs rebase

[tests] Introduce TestNode

TestNode is a class responsible for all state related to a bitcoind node
under test. It stores local state, is responsible for tracking the
bitcoind process and delegates unrecognised messages to the RPC
connection.

This commit changes start_nodes and stop_nodes to start and stop the
bitcoind nodes in parallel, making test setup and teardown much faster.

91b0979d2d

[tests] make pruning test faster

This commit makes the pruning.py much faster.

Key insights to do this:

- pruning.py doesn't care what kind of transactions make up the big
blocks that are pruned in the test. Instead of making blocks with
several large, expensive to construct and validate transactions,
instead make the large blocks contain a single coinbase transaction with
a huge OP_RETURN txout.
- avoid stop-starting nodes where possible.

This test could probably be made even faster by using the P2P interface
for submitting blocks instead of the submitblock RPC.

806bf4839d

jnewbery force-pushed on Jul 2, 2017
jnewbery commented at 9:06 PM on July 2, 2017: member

rebased
MarcoFalke commented at 2:47 PM on July 3, 2017: member

Does this depend on the TestNode commit?

fanquake commented at 1:14 AM on July 6, 2017: member

Testing this:

2017-07-06 01:08:57.788000 TestFramework (INFO): New best height: 1032
2017-07-06 01:08:57.788000 TestFramework (INFO): Generating new longer chain of 300 more blocks
2017-07-06 01:08:59.991000 TestFramework (INFO): Verify height on node 2: 1320
2017-07-06 01:08:59.992000 TestFramework (INFO): Usage possibly still high because of stale blocks in block files: 512
2017-07-06 01:08:59.992000 TestFramework (INFO): Mine 220 more large blocks so we have requisite history
2017-07-06 01:11:04.998000 TestFramework (INFO): Usage should be below target: 512
2017-07-06 01:11:04.999000 TestFramework (INFO): Test that we can rerequest a block we previously pruned if needed for a reorg
2017-07-06 01:11:06.476000 TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/Users/x/Github/bitcoin/test/functional/test_framework/test_framework.py", line 143, in main
    self.run_test()
  File "test/functional/pruning.py", line 436, in run_test
    self.reorg_back()
  File "test/functional/pruning.py", line 211, in reorg_back
    assert_raises_jsonrpc(-1, "Block not available (pruned data)", self.nodes[2].getblock, self.forkhash)
  File "/Users/x/Github/bitcoin/test/functional/test_framework/util.py", line 87, in assert_raises_jsonrpc
    raise AssertionError("No exception raised")
AssertionError: No exception raised
2017-07-06 01:11:06.477000 TestFramework (INFO): Stopping nodes

MarcoFalke commented at 10:14 AM on August 30, 2017: member

@jnewbery Needs rebase
jnewbery commented at 4:46 PM on August 31, 2017: member

closing this for now. It seems to work most of the time for me, but intermittently fails for the same reason that @fanquake gives above. There's refactoring work going on in the test_framework right now, so this is likely to require more rebases and it doesn't seem worth the effort given that very few people run this test.
jnewbery closed this on Aug 31, 2017
jtimon commented at 12:38 AM on September 1, 2017: contributor

Fwiw, perhaps more people would run this test if it was faster. Personally I avoid it precisely because of how slow it is.
jnewbery added the label Up for grabs on Mar 27, 2019
adamjonas commented at 8:22 PM on October 18, 2019: member

Can remove the up for grabs label with #15686 making this obsolete.
fanquake removed the label Up for grabs on Oct 18, 2019
jnewbery deleted the branch on Oct 18, 2019
MarcoFalke commented at 8:39 PM on October 18, 2019: member

Thanks @adamjonas
DrahtBot locked this on Dec 16, 2021

Contributors

Labels

Linked (view graph)

#10555 [tests] various improvements to zmq_test.py #10556 Move stop/start functions from utils.py into BitcoinTestFramework