Delete nonreduced fuzz inputs #204

As per the usual process to avoid wasted CI resources and timeouts when it runs on large and presumed irrelevant inputs.

Previous: #177

To "reproduce"

Install a fresh VM, as explained in the bash script's doc, and run it:

apt update && apt install curl -y
curl -L -O https://raw.githubusercontent.com/bitcoin-core/bitcoin-maintainer-tools/main/delete_nonreduced_fuzz_inputs.sh
bash delete_nonreduced_fuzz_inputs.sh

To "test"

• Keep an eye on coverage stats, to ensure it doesn't drop
• Re-run the script, to ensure it is "reproducible" to some extent
• Anything else you think is important to test or review

CI

CI should pass, except for a lint failure, which should light up on any changes like this pull request, which delete fuzz inputs.

:warning: This was run with the AFL-patched script from https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/169#issuecomment-2318553524, not from the main branch.

Storage device usage (du -sh ./fuzz_seed_corpus/)

4.0G -> 1.7G

Determinism

• ~100k files deleted

git diff --stat origin/main..b306615230c6e4a4ffc82cac1f8882d259e097de | tail -1
 107411 files changed, 632551 deletions(-)

• Cross diff with the non-afl script result of ~40k files

git -c diff.renameLimit=25218 diff --stat HEAD..b306615230c6e4a4ffc82cac1f8882d259e097de | tail -1
 39980 files changed, 15224 insertions(+), 71963 deletions(-)

• Cross-diff with a second run of the afl script of ~7k files

git -c diff.renameLimit=4159 diff --stat HEAD..b306615230c6e4a4ffc82cac1f8882d259e097de | tail -1 
 7678 files changed, 2408 insertions(+), 581 deletions(-)

Coverage

• main: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/ffdc3d6060f6e65e/65ac5a6b0b3f11ad/fuzz.coverage/index.html
• pull: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/ffdc3d6060f6e65e/b306615230c6e4a4/fuzz.coverage/index.html

msan CI fail is unrelated (https://github.com/bitcoin/bitcoin/issues/30760)

Are our logs going missing more quickly recently? Tried to look at the msan fuzz run here, but it seems to be gone: https://api.cirrus-ci.com/v1/task/5379875672948736/logs/ci.log ? The snippet still availalble is:

  Misses:           91 / 447 (20.36%)
+ du -sh /ci_container_base/depends/SDKs/ /ci_container_base/depends/builders/ /ci_container_base/depends/built/ /ci_container_base/depends/hosts/ /ci_container_base/depends/packages/ /ci_container_base/depends/patches/ /ci_container_base/depends/sdk-sources/ /ci_container_base/depends/sources/ /ci_container_base/depends/x86_64-pc-linux-gnu/
4.0K	/ci_container_base/depends/SDKs/
28K	/ci_container_base/depends/builders/
393M	/ci_container_base/depends/built/
32K	/ci_container_base/depends/hosts/
128K	/ci_container_base/depends/packages/
204K	/ci_container_base/depends/patches/
4.0K	/ci_container_base/depends/sdk-sources/
262M	/ci_container_base/depends/sources/
216M	/ci_container_base/depends/x86_64-pc-linux-gnu/
+ du -sh /ci_container_base/prev_releases
4.0K	/ci_container_base/prev_releases
+ [[ x86_64-pc-linux-gnu = *-mingw32 ]]
+ '[' -n '' ']'
+ '[' false = true ']'
+ '[' '' = true ']'
+ '[' false = true ']'
+ '[' false = true ']'
+ '[' true = true ']'
+ LD_LIBRARY_PATH=/ci_container_base/depends/x86_64-pc-linux-gnu/lib
+ test/fuzz/test_runner.py -j6 -l DEBUG /ci_container_base/ci/scratch/qa-assets/fuzz_seed_corpus/ --empty_min_time=60
==4331==WARNING: MemorySanitizer: use-of-uninitialized-value
    [#0](/bitcoin-core-qa-assets/0/) 0x562684dc42db in SetArgs(int, char**) ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:50:5
    [#1](/bitcoin-core-qa-assets/1/) 0x562684dc42db in LLVMFuzzerInitialize ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:216:5
    [#2](/bitcoin-core-qa-assets/2/) 0x562684113cb8 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /msan/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:650:5
    [#3](/bitcoin-core-qa-assets/3/) 0x562684141062 in main /msan/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    [#4](/bitcoin-core-qa-assets/4/) 0x7f36a60831c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
    [#5](/bitcoin-core-qa-assets/5/) 0x7f36a608328a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
    [#6](/bitcoin-core-qa-assets/6/) 0x562684108364 in _start (/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/fuzz+0x892364)

  Member fields were destroyed
    [#0](/bitcoin-core-qa-assets/0/) 0x5626841d3c4d in __sanitizer_dtor_callback_fields /msan/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1048:5
    [#1](/bitcoin-core-qa-assets/1/) 0x562684107222 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /msan/cxx_build/include/c++/v1/string:840:44
    [#2](/bitcoin-core-qa-assets/2/) 0x562684107222 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /msan/cxx_build/include/c++/v1/string:1106:3
    [#3](/bitcoin-core-qa-assets/3/) 0x562684107222 in std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, BCLog::LogFlags>::~pair() /msan/cxx_build/include/c++/v1/__utility/pair.h:80:29
    [#4](/bitcoin-core-qa-assets/4/) 0x562684107222 in __cxx_global_var_init ci/scratch/build-x86_64-pc-linux-gnu/src/util/./src/logging.cpp:170:66
    [#5](/bitcoin-core-qa-assets/5/) 0x562684107222 in _GLOBAL__sub_I_logging.cpp ci/scratch/build-x86_64-pc-linux-gnu/src/util/./src/logging.cpp
    [#6](/bitcoin-core-qa-assets/6/) 0x7f36a6083303 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a303) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
    [#7](/bitcoin-core-qa-assets/7/) 0x562684108364 in _start (/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/fuzz+0x892364)

SUMMARY: MemorySanitizer: use-of-uninitialized-value ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:50:5 in SetArgs(int, char**)
Exiting
Traceback (most recent call last):
  File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 411, in <module>
    main()
  File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 115, in main
    test_list_all = parse_test_list(
                    ^^^^^^^^^^^^^^^^
  File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 397, in parse_test_list
    test_list_all = subprocess.run(
                    ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/fuzz' returned non-zero exit status 1.
��������

Determinism

• ~100k deletions

git diff --stat origin/main..b306615230c6e4a4ffc82cac1f8882d259e097de | tail -1
 107411 files changed, 632551 deletions(-)

Did you mean 632k deletions instead of 100k deletions?

I noticed that the Branch Coverage went down minusculely in src/policy, src/rpc, src/script, and src (circled in image). That seems like an acceptable tradeoff for reducing the corpora by over 630,000 fuzz inputs.

Did you mean 632k deletions instead of 100k deletions?

No, with deletions in this context I mean the number of fuzz input files that were deleted. I think the git "human readable" estimate of how many lines of "code" were deleted isn't useful in this context. I guess it is counting the number of deleted newline characters (or so), which may be skewed. I've renamed "deletions" to "files deleted".

I noticed that the Branch Coverage went down minusculely in src/policy, src/rpc, src/script, and src (circled in image). That seems like an acceptable tradeoff for reducing the corpora by over 630,000 fuzz inputs.

Correct. I think this is due to some leftover non-stability or non-determinism.

Other than that the cross-diff with a second run is the lowest ever recorded (just ~7k files). I presume it is either due to the afl addition, or due to the randomness changes in the master branch.

Kicked the fuzz with msan build, now that https://github.com/bitcoin/bitcoin/pull/30778 has been merged.

Is this acceptable to merge, or is more review needed?