Code with no test coverage sorted by priority? #16450

issue davereikher opened this issue on July 24, 2019
  1. davereikher commented at 2:20 PM on July 24, 2019: contributor

    Running the test coverage analysis, currently (as of commit 67923d6b3c1e4413570ef34a53a5b1b441814bc6) 42% of functions are not covered. As a first issue for me, I'd like to start with writing tests and, naturally, I'd like to focus on code for which they're really needed - on the most critical code that lacks coverage. Is there perhaps available a list of source files (or higher granularity units of code, e.g. classes, functions) lacking coverage and sorted by how critical it is to have it?

    Of course, being more familiar with the code base would answer my question, but in order to be more familiar, I feel like I need to write tests for existing code, so it's the chicken and the egg...

    Also, as something that I believe could contribute to increasing the coverage, what do you think of making the lcov results accessible online and have it updated on every build of the master branch? That can be later tweaked to display the code that lack coverage sorted by priority, so that striving contributors could have a meaningful starting point.

  2. MarcoFalke commented at 5:09 PM on July 24, 2019: member

    As a first issue for me, I'd like to start with writing tests

    Excellent first issue! You can contact me any time on GitHub, per mail or on IRC if you run into questions.

    Is there perhaps available a list of source files (or higher granularity units of code, e.g. classes, functions) lacking coverage and sorted by how critical it is to have it?

    I'd start with src/consensus, src/script, src/validation, src/net_processing, src/mempool, then src/wallet and other stuff you might find interesting.

    lcov results accessible online

    See here: https://marcofalke.github.io/btc_cov/total.coverage/index.html Missing coverage is shown in red.

  3. MarcoFalke commented at 9:21 PM on July 24, 2019: member

    test_bitcoin is just the unit test, total includes the functional tests:

    https://github.com/bitcoin/bitcoin/blob/d960d5ca99b7ba12f9c1e9042b5e9ac3fcf86473/Makefile.am#L209-L212

  4. jonatack commented at 9:35 PM on July 24, 2019: member

    Thank you :+1:

  5. fanquake added the label Tests on Jul 24, 2019
  6. davereikher commented at 4:56 AM on July 25, 2019: contributor

    Thanks! :) The coverage results on https://marcofalke.github.io/btc_cov/test_bitcoin.coverage/index.html are a bit different than what I got. I'm guessing it is related to #14343? Also, could you please say how to generate the full coverage reports? Following the developer notes I only got the test_bitcoin coverage.

  7. MarcoFalke commented at 11:28 AM on July 25, 2019: member
  8. MarcoFalke added the label Brainstorming on Jul 25, 2019
  9. MarcoFalke added the label good first issue on Jul 25, 2019
  10. jonatack commented at 11:41 AM on July 25, 2019: member

    git grep "TODO\|FIXME" in the test directory (or from root) also turns up things to do and fix.

  11. davereikher commented at 6:57 AM on July 26, 2019: contributor

    git grep "TODO\|FIXME" in the test directory (or from root) also turns up things to do and fix.

    This is great! Thanks for the suggestion :)

  12. MarcoFalke commented at 5:06 PM on July 30, 2019: member

    Some of those TODOs no longer apply and can be removed, though.

  13. ch4ot1c commented at 3:36 AM on September 19, 2019: contributor

    I found that ./configure --enable-lcov needs to have been run before make cov will, ofc.

    On MacOS, make check of src/secp256k1 is where make cov fails on my env -

    ➜  secp256k1 git:(cd737214c) ✗ make check
    make  check-TESTS
    make[1]: Entering directory '/Users/Jon/work/bitcoin/src/secp256k1'
    make[2]: Entering directory '/Users/Jon/work/bitcoin/src/secp256k1'
    ./build-aux/test-driver: line 107: 63277 Segmentation fault: 11  "$@" > $log_file 2>&1
    FAIL: tests
    PASS: exhaustive_tests
    ============================================================================
    Testsuite summary for libsecp256k1 0.1
    ============================================================================
    # TOTAL: 2
    # PASS:  1
    # SKIP:  0
    # XFAIL: 0
    # FAIL:  1
    # XPASS: 0
    # ERROR: 0
    ============================================================================
    See ./test-suite.log
    ============================================================================
    make[2]: *** [Makefile:1261: test-suite.log] Error 1
    make[2]: Leaving directory '/Users/Jon/work/bitcoin/src/secp256k1'
    make[1]: *** [Makefile:1369: check-TESTS] Error 2
    make[1]: Leaving directory '/Users/Jon/work/bitcoin/src/secp256k1'
    make: *** [Makefile:1582: check-am] Error 2
    

    Possibly related to #12602.

  14. ch4ot1c commented at 3:37 AM on September 19, 2019: contributor

    And on Windows -

    test/util_tests.cpp(1294): Entering test case "test_FormatParagraph"
    test/util_tests.cpp(1294): Leaving test case "test_FormatParagraph"; testing time: 15625us
    test/util_tests.cpp(1321): Entering test case "test_FormatSubVersion"
    test/util_tests.cpp(1321): Leaving test case "test_FormatSubVersion"
    test/util_tests.cpp(1333): Entering test case "test_ParseFixedPoint"
    test/util_tests.cpp(1333): Leaving test case "test_ParseFixedPoint"
    test/util_tests.cpp(1436): Entering test case "test_LockDirectory"
    test/util_tests.cpp(1482): error: in "util_tests/test_LockDirectory": check (bool)ch == false has failed [true != false]
    test/util_tests.cpp(1495): error: in "util_tests/test_LockDirectory": check LockDirectory(dirname, lockname, true) == false has failed [true != false]
    test/util_tests.cpp(1436): Leaving test case "test_LockDirectory"; testing time: 15625us
    test/util_tests.cpp(1524): Entering test case "test_DirIsWritable"
    test/util_tests.cpp(1524): Leaving test case "test_DirIsWritable"; testing time: 15625us
    test/util_tests.cpp(1540): Entering test case "test_ToLower"
    test/util_tests.cpp(1540): Leaving test case "test_ToLower"
    test/util_tests.cpp(1554): Entering test case "test_ToUpper"
    test/util_tests.cpp(1554): Leaving test case "test_ToUpper"; testing time: 15625us
    test/util_tests.cpp(1568): Entering test case "test_Capitalize"
    test/util_tests.cpp(1568): Leaving test case "test_Capitalize"
    test/util_tests.cpp(28): Leaving test suite "util_tests"; testing time: 1765625us
    Leaving test module "Bitcoin Core Test Suite"; testing time: 1765625us
    
    *** 2 failures are detected in the test module "Bitcoin Core Test Suite"
    Makefile:13803: recipe for target 'test/util_tests.cpp.test' failed
    make[3]: *** [test/util_tests.cpp.test] Error 1
    make[3]: Leaving directory '/home/jon/bitcoin/src'
    Makefile:13541: recipe for target 'check-am' failed
    make[2]: *** [check-am] Error 2
    make[2]: Leaving directory '/home/jon/bitcoin/src'
    Makefile:13229: recipe for target 'check-recursive' failed
    make[1]: *** [check-recursive] Error 1
    make[1]: Leaving directory '/home/jon/bitcoin/src'
    Makefile:1302: recipe for target 'test_bitcoin.info' failed
    make: *** [test_bitcoin.info] Error 2
    rm baseline_filtered.info baseline.info
    
  15. givanse commented at 9:25 PM on February 20, 2022: contributor

    The questions by OP were answered, so I think this issue can be closed.

  16. MarcoFalke closed this on Feb 21, 2022

  17. DrahtBot locked this on Feb 21, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-13 21:14 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me