1. 
   JeremyRubin commented at 3:22 am on August 31, 2016: contributor

   The prevector tests are really slow when run under wine. This needs to be fixed because it can cause failures during development if one adds new tests timeouts can be hit in travis. This PR adds 3 worker threads + one master to mitigate performance, and removes unnecessary full testing setup. This PR makes the test time go from 205 seconds to 3.7 seconds.

   I selected the magic number 4 because that is what I had read in the travis docs as a good number. I also switch from TestingSetup to BasicTestingSetup because TestingSetup spawns unused threads

   Prevector tests should be sped up by some other means too, but that doesn’t preclude parallelization as done in this PR.

   Benchmarks

   On a 64-bit windows build off of master, I see: real 3m24.907s user 3m24.705s sys 0m0.038s

   After parallelization: real 0m4.170s user 0m4.492s sys 0m9.319s

   After parallelization+switch to BasicTestingSetup: real 0m3.782s user 0m3.972s sys 0m6.615s

   Analysis

   I’m pretty surprised at the speedup; I did test before PR’ing that:

   1. The for loop iterates the correct number of times
   2. Replacing my macros with function-versions had same speedup
   3. Adding a pBOOST_CHECK(false); in the for loop causes a failure

   So I feel pretty confident in the correctness that all checks are run and errors reported, but perhaps there is something I’ve missed. My theory is the speedup has to do with the wine runtime having high overhead with the main thread.

   Testing with a single worker thread doing all the checks gives:

   real 0m4.682s user 0m4.444s sys 0m0.037s

   Which seems to confirm my theory (I’ve left the 4 threaded version because it’s marginally faster).

2. fanquake added the label Tests on Aug 31, 2016
3. 
   sipa commented at 9:05 am on August 31, 2016: member

   What is the explanation for why these tests are so slow under Wine, and why do these changes speed it up so tremendously?
4. 
   laanwj commented at 1:49 pm on September 1, 2016: member

   Indeed, the speed-up is much, much more than you’d expect from going to four threads.

   What is the most impactful change here? Going from a full testing setup to basic testing setup?

5. 
   JeremyRubin commented at 5:38 pm on September 1, 2016: contributor

   Looking at it with fresh eyes I realized what was making it so slow:

   BOOST_CHECK does some internal logging stuff which is costly, which my code guts out (but still logs something). So single threaded should be plenty fast now.

   real 0m1.237s user 0m1.013s sys 0m0.041s

   I didn’t notice it initially because I had to replace BOOST_CHECK before parallelizing as it isn’t thread safe.

   Sorry for the premature parallelization!

   Should I push a squashed version?

   (Also it may be worth doing this on the whole test suite; which I can do as another patch)

6. 
   JeremyRubin commented at 5:39 pm on September 1, 2016: contributor

   Ignore that last patch for a second.
7. 
   JeremyRubin commented at 5:43 pm on September 1, 2016: contributor

   Should be fine now; was just concerned that I had accidentally shadowed i and that would have an impact; it doesn’t.
8. 
   JeremyRubin commented at 5:50 pm on September 1, 2016: contributor

   Found someone else documenting also having BOOST_CHECK being slow http://lists.boost.org/Archives/boost/2008/07/139575.php
9. 
   sipa commented at 5:55 pm on September 1, 2016: member

   Please squash :)
10. JeremyRubin force-pushed on Sep 1, 2016
11. Make prevector_tests much faster by replacing BOOST_CHECK with a simpler version. dbb13311da
12. JeremyRubin force-pushed on Sep 1, 2016
13. 
    JeremyRubin commented at 6:18 pm on September 1, 2016: contributor

    squashed here.
14. 
    JeremyRubin commented at 5:04 am on September 2, 2016: contributor

    See #8650 for a more general version of this.
15. JeremyRubin closed this on Sep 6, 2016

    ---

16. MarcoFalke locked this on Sep 8, 2021

Contributors
JeremyRubin sipa laanwj

Labels
Tests