Delete nonreduced fuzz inputs #292

As per the usual process to avoid wasted CI resources and timeouts when CI runs on large and presumed irrelevant inputs.

Normally, deletion of non-reduced fuzz inputs should happen after feature-freeze on the master Bitcoin Core branch, but before branch-off, so that the latest release branch retains mostly valid fuzz inputs.

Previous: #263

To "reproduce"

Install a fresh VM, as explained in the script's doc, and run it:

apt update && apt install -y cargo git && git clone --depth 1 --filter=blob:none --sparse https://github.com/bitcoin-core/qa-assets ./fuzz-cleaner && cd ./fuzz-cleaner && git sparse-checkout set delete-nonreduced-fuzz-inputs && cd ./delete-nonreduced-fuzz-inputs
cargo run -- --extra-ref=31.x --extra-ref=30.x

To "test"

• Keep an eye on coverage stats, to ensure it doesn't drop
• Re-run the script, to ensure it is "reproducible" to some extent
• Anything else you think is important to test or review

CI

CI should pass, except for a lint failure, which should light up on any changes like this pull request, which delete fuzz inputs.

Storage device usage (du -sh ./fuzz_corpora/)

5.6G -> 3.2G

(for reference I also did a run without any --extra-ref=s and it gave 2.7G)

Determinism

About 130k files were deleted:

git diff origin/main 0e2a774fef40403216568f0cd2e9c77b19e1f084 --stat | tail -1
 130848 files changed, 243676 deletions(-)

A cross-diff with a second run shows ~3k fuzz input files differ in the two sets:

git diff --no-renames --stat HEAD 0e2a774fef40403216568f0cd2e9c77b19e1f084 | tail -1 
 6579 files changed, 18032 insertions(+), 1245 deletions(-)

Coverage

main: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/05e49b342faa1412/012fb25a9f760779/fuzz.coverage/index.html pull: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/05e49b342faa1412/0e2a774fef404032/fuzz.coverage/index.html

Looks like coverage is mostly preserved, except for one line:

LogDebug(BCLog::MEMPOOL, "not keeping orphan with rejected parents %s (wtxid=%s)\n",

Which I guess is a https://github.com/bitcoin/bitcoin/issues/29018 bug :(

Actually, this is covered by fuzz input 9de6ba8a790e33081e10d1f3e94ead2917233db8. Maybe this is a gcov bug :( (Ref: https://github.com/bitcoin/bitcoin/issues/31047)

Thanks for doing this. This reduction is with the new script that retains any inputs that are relevant to any of the three most recent major branches, right?

Yes, this is basically doing the same that was done before every branch off, plus re-adding fuzz inputs for the prior release branches as well.

I'd say this is probably rfm. If someone (or their LLM bot) is bored, they can try to investigate the gcov bug for fun, but I wouldn't see it as a blocker.

cc @marcofleon maybe you want to double check this with llvm coverage, but not too important.

maybe you want to double check this with llvm coverage

I'll give it a look, won't take me long.

main: https://marcofleon.github.io/coverage/original_corpora/ pull: https://marcofleon.github.io/coverage/reduced_corpora/

Line hit count differences are in rpc/blockchain.cpp, rpc/mempool.cpp, and secp256k1/src/secp256k1.c. Looks fine to me on first glance.

Looks like there are also llvm-cov bugs, where an assertion failure branch is claimed to be hit?

https://marcofleon.github.io/coverage/original_corpora/coverage/root/bitcoin/src/compressor.cpp.html#L130

Hm yeah good catch. The machine I use is on an old clang and llvm toolchain. Might help if I update.

Either way, I think this is good to go? Only real coverage loss I can see is one successful input for the rpc getmempoolcluster() and an input that hits the false branch when loading a pubkey.

Only real coverage loss I can see is one successful input for the rpc getmempoolcluster()

I think this is an llvm cov bug as well. The coverage exists neither before, nor after. The secp one is likely a cov bug as well, but I haven't tried it myself.

Seems fine to me. The missing coverage should be added back after the next upstream anyway, as at least currently the non-reduced inputs are still part of my active collection.

To clarify, I ran with:

diff --git a/src/rpc/mempool.cpp b/src/rpc/mempool.cpp
index 5a0dc2c20a..28aae4855e 100644
--- a/src/rpc/mempool.cpp
+++ b/src/rpc/mempool.cpp
@@ -889,2 +889,4 @@ static RPCMethod getmempoolcluster()
 
+    Assert(false);
+
     auto cluster = mempool.GetCluster(txid);

and FUZZ=rpc LIMIT_TO_RPC=getmempoolcluster ./bld-cmake/bin/fuzz -runs=1 ../b-c-qa-assets/fuzz_corpora/rpc/ passed before and after this pull request.

Didn't know these coverage tools were so buggy...

I guess the two llvm-cov ones you pointed out look like underflow and off by one somehow.