test: speed up functional tests #19384

pull S3RK wants to merge 1 commits into bitcoin:master from S3RK:test_speed_up changing 2 files +12 −5
  1. S3RK commented at 10:20 AM on June 26, 2020: member

    Rationale: faster functional test shutdown

    We don't need to wait for clean nodes shutdown during test shutdown, so just send kill signal and move on with the next test. Because we already allow to run tests in parallel there is no possible problems due to resource contention.

    I did a simple measurement of shutdown time. Waiting for a clean shutdown takes on average about 0.4-0.6s on my system depending on amount of nodes. Just sending a SIGKILL is <1ms.

    For the whole test suite this gives ~80s speed up, which is about 2% improvement. I understand that is not much, but the change is also very small and all future tests will automatically benefit from this change.

    This is how a measured nodes shutdown time:

    diff --git a/test/functional/test_framework/test_framework.py b/test/functional/test_framework/test_framework.py
    index 9d9e06515..09721ee34 100755
    --- a/test/functional/test_framework/test_framework.py
    +++ b/test/functional/test_framework/test_framework.py
    @@ -260,8 +260,11 @@ class BitcoinTestFramework(metaclass=BitcoinTestMetaClass):
             self.network_thread.close()
             if not self.options.noshutdown:
                 self.log.info("Stopping nodes")
    +            start = time.time()
                 if self.nodes:
                     self.stop_nodes()
    +            end = time.time()
    +            print("Stop time: %f" % (end - start))
             else:
                 for node in self.nodes:
                     node.cleanup_on_exit = False
    
  2. test: faster functional test shutdown
    We don't need to wait for clean nodes shutdown during test shutdown,
    so just send kill signal and move on with the next test. Because we
    already allow to run tests in parallel there is no possible problems
    due to resource contention.
    3c2e9c854e
  3. promag commented at 10:26 AM on June 26, 2020: member

    IIUC this doesn't let us detect problems with clean shutdowns?

  4. S3RK commented at 10:39 AM on June 26, 2020: member

    IIUC this doesn't let us detect problems with clean shutdowns? @promag Thanks for the question, I forgot to address this point indeed. There are many tests that stop/start nodes cleanly during the execution. If we want we can even create a dedicated test to verify proper node shutdown. This change only affects the procedure when we shutdown test environment after the test itself is performed. So from what I see the change shouldn't decrease the scope of detectable problems.

  5. DrahtBot added the label Tests on Jun 26, 2020
  6. MarcoFalke commented at 11:25 AM on June 26, 2020: member

    How does this interact with sanitizers that check for problems during shutdown?

  7. promag commented at 11:35 AM on June 26, 2020: member

    There are many tests that stop/start nodes cleanly during the execution.

    True. Anyway, I'd rather see improvements on clean shutdown, ~2% "speedup" is not worth IMO.

  8. instagibbs commented at 3:21 PM on June 30, 2020: member

    -0, if the gains aren't huge I don't think it's worth the squeeze. Although I agree that we shouldn't be counting on test ends to be testing clean shutdown behavior.

  9. S3RK closed this on Aug 17, 2020

  10. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-02 03:14 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me