This hack edits the test-driver to output the test suite’s stdout (tee’ing it to the file rather than redirecting) and passes -l test_suite so that stuck tests can be seen.
Some hack like this is needed in order for test suites that take a long time, travis will assume they are dead and fail after 10 minutes. (This also helps debug which unittests may be slow). There may be a cleaner way, but as far as I can tell, this is the simplest.
@theuni try with #8632; the problem (for me) was super slow prevector_tests.
I think that I prefer my hack given the output is “-l test_suite” which shows you the names of the tests as they go by, not just an arbitrary progress meter.
(I prefer how you get the progress bar to my hack. My make-fu was not strong enough to figure that out. Can you get the loglevel set to -l test_suite and tee it?)
I think I found a nice compromise:
https://github.com/bitcoin/bitcoin/compare/master...JeremyRubin:test-driver-hack2
This version only runs the sed command in the travis build script.
If this is preferred I can PR it.
Travis will time out after 10 min and if the unit tests take longer than that, we have other issues.
Apart from that, I think it is useful to see how long each test section takes.
@MarcoFalke Well, sometimes it can be hard to reproduce why a test is timing out locally & eat a lot of dev time to build windows or whatever.
This also means that if tests ever grow to take more than 10 minutes (just because there are a lot of them) it will keep working so long as no individual test is that long. (If only we had so many tests).
I’d prefer that test_bitcoin does this by default. (I know there is test_bitcoin -p, but it does not show the current test and the time it took for each test)
This is probably what you meant with
There may be a cleaner way, but as far as I can tell, this is the simplest.
I am fine with this being merged “temporarily”.
I’d prefer that test_bitcoin does this by default. (I know there is test_bitcoin -p, but it does not show the current test and the time it took for each test)
I’m not so sure. Tests that pass ideally should generate no output, to make it more efficient to find what went wrong. It’s frustrating to have to browse through thousands of debug messages to find out that error that caused travis to fail.
For temporary debugging it’s ok of course.
Ok, then this seems too controversial…
If the only goal is to prevent travis from timing out, there is travis_wait as an alternative https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
Labels
Tests