Travis: run unit tests in different orders #5855

pull gavinandresen wants to merge 1 commits into bitcoin:master from gavinandresen:test_independent_unit_tests changing 1 files +1 −0

gavinandresen commented at 4:52 PM on March 5, 2015: contributor

Set the BOOST_TEST_RANDOM environment variable, to run unit tests in different orders for each test in the test matrix that runs tests.
gavinandresen force-pushed the base branch on Mar 5, 2015
theuni commented at 8:14 PM on March 5, 2015: member

Looks fine to me (assuming the merge is removed). If you'd like, though, I think you should be able to just set BOOST_TEST_RANDOM=$TRAVIS_BUILD_ID in the global section to get the same effect.

BUILD_ID should be the same for each builder on each run.
gavinandresen commented at 8:51 PM on March 5, 2015: contributor

@theuni : good idea, but it'll have to be $(( TRAVIS_BUILD_ID + 1 )) ... because we don't want -random=1 (that means really and truly random). Can the travis variable assignments be arbitrary bash expressions?

Also: why isn't Travis building commit 6cfd49f ? (which I added just to make sure failures actually fail)
theuni commented at 9:17 PM on March 5, 2015: member

@gavinandresen Any reason not to just concat strings and avoid expressions? Something like BOOST_TEST_RANDOM=1$TRAVIS_BUILD_ID ?

It looks like Travis had some downtime earlier today, it may just still be processing its backlog.
gavinandresen force-pushed on Mar 5, 2015
gavinandresen commented at 9:25 PM on March 5, 2015: contributor

Good idea using string concat (that makes the first random seed eleven!).
laanwj commented at 4:26 PM on March 6, 2015: member

ACK (after removing REMOVEME commit)

Run unit tests in different orders

Set the BOOST_TEST_RANDOM environment variable, to
run unit tests in different orders for each test in the
test matrix that runs tests.

69866436be

gavinandresen force-pushed on Mar 6, 2015
gavinandresen force-pushed on Mar 6, 2015
gavinandresen commented at 6:30 PM on March 6, 2015: contributor

Removed the REMOVEME commit after verifying that Travis failed in exactly the right way. Ready for merge.
theuni commented at 7:29 PM on March 6, 2015: member

ACK
sipa commented at 1:31 PM on March 7, 2015: member

Untested ACK.
fanquake commented at 2:01 PM on March 7, 2015: member

utACK
laanwj merged this on Mar 9, 2015
laanwj closed this on Mar 9, 2015
laanwj referenced this in commit d7989c04f6 on Mar 9, 2015
gavinandresen deleted the branch on Mar 9, 2015
MarcoFalke locked this on Sep 8, 2021

Contributors