test: Make summary row bold-red if any test failed and show failed tests at end of table #12811

pull laanwj wants to merge 2 commits into bitcoin:master from laanwj:2018_03_tests_summaryrow changing 1 files +13 −1

laanwj commented at 9:22 AM on March 28, 2018: member

Make the summary row of the test runner bold red if any test fails. This helps visibility if something fails. (yesteryday I had a snafu where I missed that feature_blocksdir.py had failed because it's one of the earlier tests in the list, this intends to avoid that in the future)

Before:

After:

If tests pass it still looks the same:

test: Make summary row bold-red if any test failed

Make the summary row of the test runner bold red if any test fails.
This helps visibility if something fails.

f92541f7ea

laanwj added the label Tests on Mar 28, 2018
laanwj commented at 10:07 AM on March 28, 2018: member

Added a commit (by @ajtowns) that moves the failed tests to the bottom of the list, to avoid needing to scroll to find out what failed.
laanwj renamed this:
~~test: Make summary row bold-red if any test failed~~
test: Make summary row bold-red if any test failed and move failed tests to end of table
on Mar 28, 2018
laanwj renamed this:
~~test: Make summary row bold-red if any test failed and move failed tests to end of table~~
test: Make summary row bold-red if any test failed and show failed tests at end of table
on Mar 28, 2018
MarcoFalke commented at 12:54 PM on March 28, 2018: member

utACK d2c6a371a036941bf9c4d58a612ced0970b9b056
in test/functional/test_runner.py:468 in d2c6a371a0 outdated
```
 463 | +        if self.status == "Passed":
 464 | +            return 1, self.name.lower()
 465 | +        elif self.status == "Failed":
 466 | +            return 2, self.name.lower()
 467 | +        elif self.status == "Skipped":
 468 | +            return 0, self.name.lower()
```
jnewbery commented at 1:39 PM on March 28, 2018:

My preference would be to place skipped tests below passed tests for visibility, so the order would be:

passed tests

skipped tests

failed tests

SUMMARY
laanwj commented at 1:45 PM on March 28, 2018:

Fair point, I'll swap the passed and skipped
jnewbery commented at 1:39 PM on March 28, 2018: member

Tested ACK d2c6a371a036941bf9c4d58a612ced0970b9b056 with one minor suggested change

test: List any failed tests at the end of test_runner output

Change sorting output to put failed tests at the end of test_runner
output.

ffb033a6d5

laanwj force-pushed on Mar 28, 2018
laanwj commented at 1:51 PM on March 28, 2018: member

Swapped sort order of skipped and passed tests per @jnewbery d2c6a371a036941bf9c4d58a612ced0970b9b056 → f92541f7ea66b81343426db0b40f7ba4062df342
jnewbery commented at 1:56 PM on March 28, 2018: member

Tested ACK ffb033a6d5d846b19a2b595f7d522d5b73cfca6f
laanwj merged this on Mar 28, 2018
laanwj closed this on Mar 28, 2018
laanwj referenced this in commit 0d8fc8de07 on Mar 28, 2018
MarcoFalke deleted a comment on Mar 29, 2018
Fabcien referenced this in commit d5489985e5 on Aug 30, 2019
PastaPastaPasta referenced this in commit f1165e1aa8 on Sep 27, 2020
PastaPastaPasta referenced this in commit be21a9a685 on Sep 27, 2020
PastaPastaPasta referenced this in commit 79639195ee on Oct 14, 2020
MarcoFalke locked this on Sep 8, 2021

Contributors

Labels