test: Generating lcov line coverage does not fail when tests fail #15648

issue MarcoFalke opened this issue on March 22, 2019

MarcoFalke commented at 8:18 PM on March 22, 2019: member
Generating coverage where the functional tests fail does not fail the coverage. The excerpt:
```
117/118 - feature_shutdown.py passed, Duration: 3 s
make: [functional_test.info] Error 1 (ignored)
```
See: https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#compiling-for-test-coverage
MarcoFalke added the label Build system on Mar 22, 2019
MarcoFalke added the label Tests on Mar 22, 2019
MarcoFalke added the label good first issue on Mar 22, 2019
crackercracked commented at 3:20 AM on April 8, 2019: none

Can somebody elaborate on how to produce the error. It seems the coverage report is still generated fine even given individual functional tests failed
MarcoFalke commented at 3:08 PM on April 8, 2019: member

It seems the coverage report is still generated fine even given individual functional tests failed

The file is generated, but is obviously missing the coverage of the remainder of the test that failed. So it should have never been generated in the first place.
asood123 commented at 7:22 PM on June 12, 2019: contributor

I'd like to help with this issue. Fwiw, I'm a newbie to Bitcoin codebase. Was this fix #15812 incorrect or didn't get rebased in time?
MarcoFalke renamed this:
~~test: Generating coverage does not fail when tests fail~~
test: Generating lcov line coverage does not fail when tests fail
on Jun 12, 2019
MarcoFalke commented at 7:40 PM on June 12, 2019: member

Yeah, it was incorrect. That pull modified rpc coverage. This issue is about lcov, see https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#compiling-for-test-coverage
asood123 commented at 4:12 PM on June 13, 2019: contributor

I'm new to makefiles and build systems, so appreciate pointers if this isn't correct.
MarcoFalke removed the label good first issue on Jun 13, 2019
MarcoFalke referenced this in commit b4ad4c0de3 on Jun 20, 2019
MarcoFalke closed this on Jun 20, 2019
sidhujag referenced this in commit 9a5c4cb2d0 on Jun 20, 2019
Munkybooty referenced this in commit 4128f3020c on Oct 30, 2021
Munkybooty referenced this in commit 87831d3fff on Oct 30, 2021
Munkybooty referenced this in commit 65024e47bd on Nov 2, 2021
Munkybooty referenced this in commit 81d6a880bb on Nov 2, 2021
Munkybooty referenced this in commit 01a6da1eca on Nov 4, 2021
Munkybooty referenced this in commit dda9c6b2e3 on Nov 16, 2021
Munkybooty referenced this in commit feafa02be2 on Nov 18, 2021
MarcoFalke locked this on Dec 16, 2021

Contributors

Labels

Linked (view graph)

#15812 Do not generate coverage report on test failures #16207 test: stop generating lcov coverage when functional tests fail