Better output in failed tests #645

1. 
   real-or-random commented at 11:48 pm on July 2, 2019: contributor

   When a test fails, it currently outputs only the failed assertion (with file and line) to stderr. The seed is output to stdout only. Moreover, neither stdout nor stderr are output on make check. Instead, the stderr is written to ./test-suite.log

   As a consequence, a non-reproducible failure on Travis is somewhat useless to us because we just see the FAIL but we have no idea what failed because we don’t see the contents of the log file. (This just happened downstream in secp256k1-zkp). A similar issue was the failed PR build on a Fedora build server, which contained only stderr #558 (comment) and thus we didn’t get the seed.

   We should probably

   • output the seed to stderr, at least if the test fails.
   • output the log file in travis using after_script or after_failure, see https://docs.travis-ci.com/user/job-lifecycle/#the-job-lifecycle
2. jonasnick cross-referenced this on Nov 2, 2019 from issue Fix issue where travis does not show the ./tests seed… by jonasnick
3. real-or-random closed this on Nov 25, 2019

   ---

Contributors
real-or-random