Document how to run bitcoin unittests + functional tests with sanitizers

elichai commented at 10:49 am on December 30, 2019: contributor

A small guide from ground up on how to run sanitizers(asan/ubsan/msan) on core would be awesome :) @practicalswift ;)

elichai added the label Feature on Dec 30, 2019

elichai commented at 10:50 am on December 30, 2019: contributor

argh forgot it matters what kind of issue I click on (for labeling)

maflcko added the label Docs on Dec 30, 2019

fanquake added the label Tests on Dec 31, 2019

practicalswift commented at 8:31 am on March 12, 2020: contributor

MSan setup is documented by the config in #18288 (“build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory”). @elichai @jonasnick Would you mind reviewing and/or Concept ACK:ing #18288? :)

practicalswift commented at 3:14 pm on May 6, 2020: contributor

A friendly reminder to anyone interested in this issue to review #18288 (a concept ACK would help!) which at least partly solves the issue at hand: it shows which commands to run to setup MSAN for the project :)

MSAN is by far the trickiest of the sanitizers to set-up, so that is a good start :)

IMPranshu commented at 3:26 pm on August 29, 2022: none

Yeah. It will also help beginners like me to understand the approach to writing new unit tests.

maflcko commented at 3:47 pm on August 29, 2022: member

This is pretty simple:

MSan: Compiling with MSan is quite involved. I’d suggest to just compile normally and use valgrind ./src/test/test_bitcoin or ./test/functional/test_runner.py --valgrind instead. Alternatively you can use the ./ci/ system to run any sanitizer task.
Other sanitizer: Just append --with-sanitizers=undefined,integer CC=clang CXX=clang++ to your ./configure call. (Or --with-sanitizers=address, …). Then, run the tests normally, using the env vars as needed:

0export LSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/lsan"
1export TSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/tsan:halt_on_error=1:second_deadlock_stack=1"
2export UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1"

jonatack referenced this in commit 658a6fbaf7 on Aug 30, 2022

jonatack referenced this in commit 3181b105eb on Aug 30, 2022

jonatack referenced this in commit 4bd514521f on Aug 30, 2022

jonatack commented at 12:09 pm on August 30, 2022: contributor

Thanks @MarcoFalke! Updated https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests with that information, along with a link to https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#sanitizers.

jonatack referenced this in commit eac971272b on Aug 30, 2022

jonatack referenced this in commit a33001281e on Aug 30, 2022

jonatack referenced this in commit f2600ab644 on Aug 31, 2022

Sjors commented at 8:05 pm on February 23, 2023: member

Anything left to do here?

Seems mostly a matter of ./configure with-sanitizers=… and then the test suite runs as normal, albeit slower and more memory hungry.

fanquake referenced this in commit 84ca5b349e on Feb 24, 2023

Sjors commented at 12:47 pm on February 24, 2023: member

#27154 adds documentation about suppressions, which is useful.

Running the functional tests with valgrind - and otherwise normal compilation, though with--enable-debug, causes a bunch of failures for me. Compiling without legacy wallet helps, so we could document that. For the other failures I still need to check if they’re deterministic.

maflcko commented at 1:05 pm on February 24, 2023: member

You’ll also need to increase the timeout factor.

./test/functional/test_runner.py --help | grep timeout-factor

glozow closed this on Feb 24, 2023

glozow referenced this in commit be2e748f37 on Feb 24, 2023

sidhujag referenced this in commit d62f99d8ed on Feb 25, 2023

bitcoin locked this on Feb 24, 2024

Document how to run bitcoin unittests + functional tests with sanitizers #17834