test: Generate coverage report without running tests #28772

1. 
   achow101 commented at 10:17 pm on November 1, 2023: member

   When generating a coverage report, separate the testing from the generation of the coverage report. This is useful when checking the coverage of a small set of tests.
2. test: Generate coverage report without running tests

   When generating a coverage report, separate the testing from the
   generation of the coverage report. This is useful when checking the
   coverage of a small set of tests.

   29a207820d
3. 
   DrahtBot commented at 10:17 pm on November 1, 2023: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage

   For detailed information about the code coverage, see the test coverage report.

   Reviews

   See the guideline for information on the review process. A summary of reviews will appear here.

4. DrahtBot added the label Tests on Nov 1, 2023
5. 
   maflcko commented at 8:49 am on November 2, 2023: member

   Is it still accurate to call the coverage report test_bitcoin when it may cover something else?

   Do the docs need an update? https://github.com/bitcoin/bitcoin/blob/eca2e430acf50f11da2220f56d13e20073a57c9b/doc/developer-notes.md?plain=1#L478 etc …

6. 
   achow101 commented at 0:20 am on November 14, 2023: member

   Is it still accurate to call the coverage report test_bitcoin when it may cover something else?

   Any suggestions?

   Do the docs need an update?

   I don’t think so? (besides renaming, if we do that). The way I read the docs was that make check had to be run first, and separately. I hadn’t expected make cov to run them for me.

7. 
   dergoegge commented at 10:22 am on November 14, 2023: member

   Should we do the same for make cov_fuzz? i.e. separate running the fuzz test runner from generating the coverage report.
8. 
   maflcko commented at 10:41 am on November 14, 2023: member

   I don’t think so? (besides renaming, if we do that). The way I read the docs was that make check had to be run first, and separately. I hadn’t expected make cov to run them for me.

   The docs don’t mention make check and the snippet similar to the docs may also used by external scripts. For example, https://github.com/maflcko/b-c-cov/blob/83bc0912e2a33cfac2195645ab3a12d092fa7ba3/.cirrus.yml#L46-L47

9. 
   achow101 commented at 5:07 pm on November 14, 2023: member

   The docs don’t mention make check

   Hmm? It’s right there in the first sentence of that section:

   LCOV can be used to generate a test coverage report based upon make check execution.

10. 
    dergoegge commented at 5:13 pm on November 14, 2023: member

    Hmm? It’s right there in the first sentence of that section:

    LCOV can be used to generate a test coverage report based upon make check execution.

    Isn’t that just saying that the generated coverage report covers the code executed in make check, not that you have to invoke make check prior to make cov? (that’s at least how i read it and how make cov currently works). Since there are multiple interpretations, it probably makes sense to clarify.

11. 
    maflcko commented at 8:28 pm on November 14, 2023: member

    Right. I am saying this is certainly a behavior change. Previously one could just type ./configure ... && make && make cov to get the both reports. Now, something else is needed, likely a longer command, so it would be good to mention this in the documentation, or pull request description, or anywhere. Otherwise, everyone is left on their own to figure out the change in behavior.
12. DrahtBot added the label CI failed on Jan 17, 2024
13. achow101 closed this on Jan 25, 2024

    ---

14. bitcoin locked this on Jan 24, 2025

Contributors
achow101 DrahtBot maflcko dergoegge

Labels
Tests CI failed