Pruning large/slow inputs? #118

Some targets currently run for > half an hour, i.e banman or addrman_deserialize, with their slowest inputs running for a minute or more. Is it worth trying to prune these slow inputs, assuming no, or a negligable decrease in coverage, if runtime can be improved by a decent percentage?

https://cirrus-ci.com/task/6079733960540160?logs=ci#L1068:

Run addrman_deserialize with args ['valgrind', '--quiet', '--error-exitcode=1', '/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', '/tmp/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/addrman_deserialize']INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3456924522
INFO: Loaded 1 modules   (243081 inline 8-bit counters): 243081 [0x2736df8, 0x2772381), 
INFO: Loaded 1 PC tables (243081 PCs): 243081 [0x2772388,0x2b27c18), 
INFO:     3096 files found in /tmp/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/addrman_deserialize
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 3096 min: 1b max: 1048576b total: 128207363b rss: 251Mb
[#64](/bitcoin-core-qa-assets/64/)	pulse  cov: 1321 ft: 1729 corp: 31/980b exec/s: 21 rss: 272Mb
[#128](/bitcoin-core-qa-assets/128/)	pulse  cov: 1323 ft: 1754 corp: 33/1080b exec/s: 18 rss: 273Mb
[#256](/bitcoin-core-qa-assets/256/)	pulse  cov: 1324 ft: 1755 corp: 34/1133b exec/s: 15 rss: 273Mb
[#512](/bitcoin-core-qa-assets/512/)	pulse  cov: 1493 ft: 2023 corp: 68/3165b exec/s: 21 rss: 274Mb
[#1024](/bitcoin-core-qa-assets/1024/)	pulse  cov: 2713 ft: 4888 corp: 135/9095b exec/s: 35 rss: 282Mb
[#2048](/bitcoin-core-qa-assets/2048/)	pulse  cov: 2941 ft: 12633 corp: 547/156Kb exec/s: 23 rss: 284Mb
Slowest unit: 12 s:
artifact_prefix='./'; Test unit written to ./slow-unit-fa63ff08d2d73478c8e4dfb1f6d10c314c51bda2
Slowest unit: 20 s:
artifact_prefix='./'; Test unit written to ./slow-unit-2cf6dcdc90d27082d6f06c1bbcf2c9fb617c4665
Slowest unit: 29 s:
artifact_prefix='./'; Test unit written to ./slow-unit-c7043fea9ac0a00b8421f0c66b91141a47e2ce4a
Slowest unit: 77 s:
artifact_prefix='./'; Test unit written to ./slow-unit-c2c4bf148a02aebee1179dbb17b0f1c4ce69d949
[#3097](/bitcoin-core-qa-assets/3097/)	INITED cov: 2960 ft: 15207 corp: 945/20Mb exec/s: 1 rss: 416Mb
[#3097](/bitcoin-core-qa-assets/3097/)	DONE   cov: 2960 ft: 15207 corp: 945/20Mb lim: 760806 exec/s: 1 rss: 416Mb
Done 3097 runs in 2437 second(s)

https://cirrus-ci.com/task/6079733960540160?logs=ci#L943:

Run banman with args ['valgrind', '--quiet', '--error-exitcode=1', '/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', '/tmp/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/banman']INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3396028099
INFO: Loaded 1 modules   (243081 inline 8-bit counters): 243081 [0x2736df8, 0x2772381), 
INFO: Loaded 1 PC tables (243081 PCs): 243081 [0x2772388,0x2b27c18), 
INFO:     4783 files found in /tmp/bitcoin-core/ci/scratch/qa-assets/fuzz_seed_corpus/banman
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 4783 min: 1b max: 4194310b total: 221535216b rss: 254Mb
[#128](/bitcoin-core-qa-assets/128/)	pulse  cov: 1334 ft: 2275 corp: 46/332b exec/s: 64 rss: 276Mb
[#256](/bitcoin-core-qa-assets/256/)	pulse  cov: 1409 ft: 2700 corp: 84/735b exec/s: 64 rss: 276Mb
[#512](/bitcoin-core-qa-assets/512/)	pulse  cov: 2378 ft: 4873 corp: 157/1826b exec/s: 56 rss: 279Mb
[#1024](/bitcoin-core-qa-assets/1024/)	pulse  cov: 2631 ft: 7553 corp: 306/5510b exec/s: 56 rss: 280Mb
[#2048](/bitcoin-core-qa-assets/2048/)	pulse  cov: 2844 ft: 10686 corp: 585/21Kb exec/s: 51 rss: 280Mb
Slowest unit: 10 s:
artifact_prefix='./'; Test unit written to ./slow-unit-f80fda481ebe048a46dc7d56cac745553ac86462
Slowest unit: 12 s:
artifact_prefix='./'; Test unit written to ./slow-unit-3866f35faef1aa8c88ce6a52ba599c96b9773452
Slowest unit: 15 s:
artifact_prefix='./'; Test unit written to ./slow-unit-b4b646db3c8b4309d520a929dd9969cd3d266026
Slowest unit: 17 s:
artifact_prefix='./'; Test unit written to ./slow-unit-92ff80cf45bd551bcace289e5fa514f411b614cf
Slowest unit: 20 s:
artifact_prefix='./'; Test unit written to ./slow-unit-efd82da2184653f7522296ad76c71eb62d226205
Slowest unit: 26 s:
artifact_prefix='./'; Test unit written to ./slow-unit-94cebd976b987de8e532cc562064831761aa3d8d
[#4096](/bitcoin-core-qa-assets/4096/)	pulse  cov: 2896 ft: 17823 corp: 1307/792Kb exec/s: 4 rss: 309Mb
Slowest unit: 32 s:
artifact_prefix='./'; Test unit written to ./slow-unit-e0627c9a132a8619dd91150e67aa5142ef7f1db0
Slowest unit: 43 s:
artifact_prefix='./'; Test unit written to ./slow-unit-79d9d700db64f0f307875e04b72d9c71e3fce554
Slowest unit: 59 s:
artifact_prefix='./'; Test unit written to ./slow-unit-329c339c4dbe81a59cd0167e0ed96c579a71144d
[#4784](/bitcoin-core-qa-assets/4784/)	INITED cov: 2896 ft: 17955 corp: 1346/2709Kb exec/s: 2 rss: 392Mb
[#4784](/bitcoin-core-qa-assets/4784/)	DONE   cov: 2896 ft: 17955 corp: 1346/2709Kb lim: 621890 exec/s: 2 rss: 392Mb
Done 4784 runs in 2100 second(s)

Yes, this is done at or around the branch-off point for every release. See pulls

(The reason this is done at around the branch-off point is that the then resulting fuzz inputs will be "valid" for the longest duration for the latest release branch)

@dergoegge Is someone working on this? If not, I'll do it.

@MarcoFalke Not working on this currently. I've also never done this before so you doing it is probably best (unless there is an easy command/script to run?).

I've been meaning to write some docs for the usual processes in this repo (merging, pruning, maybe linking to the Bitcoin Core fuzzing docs?). Will follow up with that separately.

Yeah, it is really simple. See #97

Make sure to set up a fresh Jammy VM for this, as documented in the bash script.

Ok, I'll run the script and create the pull today. Anyone is then invited to review, if they want.

See #119, I think this can be closed now?

Think we can add some additional notes/here (@dergoegge may follow up), but I'll close this for now.