build: Add support for llvm-cov

hebasto commented at 9:02 pm on August 8, 2020: member

With this PR it is possible to use lcov with clang:

0$ ./autogen.sh
1$ ./configure --enable-lcov --enable-fuzz --with-sanitizers=fuzzer CC=clang CXX=clang++
2$ make
3$ make cov_fuzz

NOTE: Unfortunately, on my system (clang version 10.0.0-4ubuntu1) due to unknown for me reasons make cov never finishes, trying to Processing src/test/test_bitcoin-util_tests.gcda forever (stopped waiting).

Closes #12602

build, test: Add support for llvm-cov c71bdf93d7

build: Add missed fuzz_filtered.info to COVERAGE_INFO 8ebc0505e9

build: Add missed fuzz.coverage/ directory to .gitignore 75f9659d7a

hebasto commented at 9:03 pm on August 8, 2020: member

friendly ping @practicalswift @Crypt-iQ @darosior

DrahtBot added the label Build system on Aug 8, 2020

fjahr commented at 10:02 pm on August 9, 2020: member

NOTE: Unfortunately, on my system (clang version 10.0.0-4ubuntu1) due to unknown for me reasons make cov never finishes, trying to Processing src/test/test_bitcoin-util_tests.gcda forever (stopped waiting).

I have observed the same issue with 10.0.0_3 on macOS.

fanquake renamed this:
~~build, test: Add support for llvm-cov~~
build: Add support for llvm-cov
on Aug 10, 2020

Crypt-iQ commented at 8:54 pm on August 10, 2020: contributor

Tested ACK 75f9659d7a497ae1f86d905029f83c0193ecfeb2

make cov still hanging after ~11+ hours on test_bitcoin-util_tests.gcda make cov_fuzz output: https://crypt-iq.github.io/fuzz.coverage/

Running clang-10 and ubuntu

vasild approved

vasild commented at 10:47 am on August 11, 2020: member

ACK 75f9659d7

Btw, LLVM provides alternative tools to lcov and genhtml. Maybe remove the dependency on those if --enable-lcov is used and the compiler is Clang (then COV_TOOL_WRAPPER would not be needed) and rename that ./configure option to --enable-coverage.

Crypt-iQ commented at 4:29 pm on August 12, 2020: contributor

Might be a bit unrelated, but I had to move qa-assets from /root to /root/bitcoin for make cov_fuzz to work. Would be nice to be able to specify a directory, but probably out of scope for this PR.

in Makefile.am:196 in 75f9659d7a

191@@ -192,7 +192,11 @@ LCOV_FILTER_PATTERN = \
192 	-p "src/secp256k1" \
193 	-p "depends"
194 
195-baseline.info:
196+$(COV_TOOL_WRAPPER):
197+	@echo 'exec $(COV_TOOL) "$$@"' > $(COV_TOOL_WRAPPER)

MarcoFalke commented at 8:19 pm on August 12, 2020:

Could add a comment that this is needed because of a potential dash in the filename of COV_TOOL? (At least I think that was the reason a wrapper is required)

vasild commented at 8:48 am on August 13, 2020:

My understanding is that the wrapper is needed because we run lcov --gcov-tool /path/to/gcov and our gcov is llvm-cov gcov, so lcov --gcov-tool "llvm-cov gcov" other arguments would not work.

It is also possible to generate the coverage without using gcov/lcov:

 0... compile with clang++ -fprofile-instr-generate -fcoverage-mapping ...
 1export LLVM_PROFILE_FILE="/tmp/coverage/%m-%p.profraw"
 2... run unit + functional tests ...
 3llvm-profdata merge /tmp/coverage/*.profraw -o all.profdata
 4
 5# Generate lcov-like html report:
 6llvm-cov export -instr-profile=all.profdata -format=lcov src/test/test_bitcoin -object=src/bitcoind > coverage_lcov.info
 7genhtml --output-directory /tmp/coverage/html --legend --branch-coverage coverage_lcov.info
 8
 9# Or generate llvm html report (better templates visualization!):
10llvm-cov show -instr-profile=all.profdata -format=html --output-dir=/tmp/coverage/html src/test/test_bitcoin -object=src/bitcoind

MarcoFalke commented at 8:20 pm on August 12, 2020: member

Concept ACK.

Seems good to merge with two tested ACKs

MarcoFalke merged this on Aug 12, 2020

MarcoFalke closed this on Aug 12, 2020

hebasto deleted the branch on Aug 13, 2020

hebasto commented at 11:04 am on August 13, 2020: member

make cov still hanging after ~11+ hours on test_bitcoin-util_tests.gcda

I’ve narrowed this problem down to https://github.com/bitcoin/bitcoin/blob/6757b3ac8f670cbc188fc7531394e713975c9351/src/test/util_tests.cpp#L505

hebasto commented at 1:32 pm on August 13, 2020: member

@Crypt-iQ

make cov still hanging after ~11+ hours on test_bitcoin-util_tests.gcda

Fixed in #19709.

sidhujag referenced this in commit fa44ec8d07 on Aug 14, 2020

MarcoFalke referenced this in commit 1a43cd3f74 on Aug 14, 2020

DrahtBot locked this on Feb 15, 2022

build: Add support for llvm-cov #19688