[WIP] ci: test valgrind build from source #136

No description provided.

Looks like this is running faster. i.e, comparing a recent PR run:

Run base_encode_decode with args ['valgrind', '--quiet', '--error-exitcode=1', '/tmp/cirrus-build/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', PosixPath('/tmp/cirrus-build/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/base_encode_decode')]INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3272178655
INFO: Loaded 1 modules   (248538 inline 8-bit counters): 248538 [0x27fd278, 0x2839d52), 
INFO: Loaded 1 PC tables (248538 PCs): 248538 [0x2839d58,0x2c04af8), 
INFO:     7233 files found in /tmp/cirrus-build/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/base_encode_decode
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 7233 min: 1b max: 1048576b total: 83751368b rss: 180Mb
[#128](/bitcoin-core-qa-assets/128/)	pulse  cov: 815 ft: 1562 corp: 56/240b exec/s: 64 rss: 183Mb
[#256](/bitcoin-core-qa-assets/256/)	pulse  cov: 1016 ft: 1941 corp: 96/697b exec/s: 64 rss: 206Mb
[#512](/bitcoin-core-qa-assets/512/)	pulse  cov: 1411 ft: 2848 corp: 137/1524b exec/s: 73 rss: 218Mb
[#1024](/bitcoin-core-qa-assets/1024/)	pulse  cov: 2596 ft: 7733 corp: 274/10217b exec/s: 37 rss: 237Mb
[#2048](/bitcoin-core-qa-assets/2048/)	pulse  cov: 4184 ft: 13749 corp: 519/51Kb exec/s: 26 rss: 247Mb
[#4096](/bitcoin-core-qa-assets/4096/)	pulse  cov: 4641 ft: 19775 corp: 906/201Kb exec/s: 20 rss: 253Mb
Slowest unit: 13 s:
artifact_prefix='./'; Test unit written to ./slow-unit-4bbf474818229be7c3eaa56e66d8a671f61a8c4b
Slowest unit: 15 s:
artifact_prefix='./'; Test unit written to ./slow-unit-434da0af3356a442e2e6ea3bcef590755c6574e0
[#7234](/bitcoin-core-qa-assets/7234/)	INITED cov: 4730 ft: 25892 corp: 1581/12441Kb exec/s: 5 rss: 295Mb
[#7234](/bitcoin-core-qa-assets/7234/)	DONE   cov: 4730 ft: 25892 corp: 1581/12441Kb lim: 1048576 exec/s: 5 rss: 295Mb
Done 7234 runs in 1246 second(s)

to this branch:

Run base_encode_decode with args ['valgrind', '--quiet', '--error-exitcode=1', '/tmp/cirrus-build-1174734651/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', PosixPath('/tmp/cirrus-build-1174734651/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/base_encode_decode')]INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3639341585
INFO: Loaded 1 modules   (366489 inline 8-bit counters): 366489 [0x1c106d0, 0x1c69e69), 
INFO: Loaded 1 PC tables (366489 PCs): 366489 [0x1c69e70,0x2201800), 
INFO:     7233 files found in /tmp/cirrus-build-1174734651/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/base_encode_decode
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 7233 min: 1b max: 1048576b total: 83751368b rss: 212Mb
[#256](/bitcoin-core-qa-assets/256/)	pulse  cov: 407 ft: 683 corp: 105/715b exec/s: 128 rss: 236Mb
[#512](/bitcoin-core-qa-assets/512/)	pulse  cov: 545 ft: 935 corp: 155/1712b exec/s: 128 rss: 245Mb
[#1024](/bitcoin-core-qa-assets/1024/)	pulse  cov: 1182 ft: 2772 corp: 321/12232b exec/s: 102 rss: 256Mb
[#2048](/bitcoin-core-qa-assets/2048/)	pulse  cov: 2032 ft: 5181 corp: 640/66Kb exec/s: 93 rss: 268Mb
[#4096](/bitcoin-core-qa-assets/4096/)	pulse  cov: 2481 ft: 8460 corp: 1223/292Kb exec/s: 81 rss: 278Mb
[#7234](/bitcoin-core-qa-assets/7234/)	INITED cov: 2588 ft: 11977 corp: 2183/10809Kb exec/s: 19 rss: 312Mb
[#7234](/bitcoin-core-qa-assets/7234/)	DONE   cov: 2588 ft: 11977 corp: 2183/10809Kb lim: 1042864 exec/s: 19 rss: 312Mb
Done 7234 runs in 373 second(s)

Again, are you sure you aren't missing build options?

Again, are you sure you aren't missing build options?

The only option I can see passed by debian is --enable-tls, which is only used in valgrinds unit tests.

They may also pass some CFLAGS etc?

Also in the commit description you mention:

* No need to "bless" a specific compiler for use with Valgrind, (current
  discussion includes switch from Clang to GCC)

Which seems unrelated, no? Or is compiling from source also going to fix that valgrind gcc false positive?

They may also pass some CFLAGS etc?

They may do, and that could be the reason their binaries are so slow (i.e hardening flags). Unless you can point to something, I very much doubt that any additional cflags would be controlling valgrind features.

Which seems unrelated, no?

No. It's related given we are/were considering dropping support for Clang and using GCC? Hence ending up with a "blessed" compiler.

Looking at https://buildd.debian.org/status/fetch.php?pkg=valgrind&arch=amd64&ver=1%3A3.19.0-1&stamp=1661686809&raw=0, which should be a fairly recent Debian valgrind build, the relevant ./configure / flags are:

dh_auto_configure -- --enable-tls CFLAGS="-g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 "-I/usr/include/x86_64-linux-gnu"   " LDFLAGS="-Wl,-z,relro"
	./configure --build=x86_64-linux-gnu --prefix=/usr --includedir=\${prefix}/include --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=\${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --disable-maintainer-mode --disable-dependency-tracking --enable-tls "CFLAGS=-g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include/x86_64-linux-gnu   " LDFLAGS=-Wl,-z,relro

Other than --enable-tls, these are all standard autoconf flags, and I can't see any special c/ldflags etc.

Will close for now. This has completed in 5 and a half hours, compared to a recent 34 hour runtime.