Add Murch’s inputs 2024-12 #214

Usual procedure of moving aside the upstream directory, creating new merge_set, and restoring the upstream. Did a bunch of targeted fuzzing on the clusterlin targets and new fuzz targets.

This seems to have too many files. [o.0] Will investigate in a few hours.

Number of files seems fine, but the size is doubling the repo?

du -sh ./fuzz_corpora/
4.5G	./fuzz_corpora/

Could be valid, but to double-check, what were your commands? For my own pulls, I add the commands to the description, like: https://github.com/bitcoin-core/qa-assets/pull/202

Mh, I’m running another merge. If I get a similar result, I’ll be mollified.

I use the same commands as you:

1. Pull the latest Bitcoin Core

   cd ~/Workspace/qa-merge
   git pull upstream/main
   git reset --hard upstream/main
   

2. Build the merge setup

   cd ~/Workspace/qa-merge
   cmake --build build_fuzz -j 20
   

3. Enable suppressions

   cd ~/Workspace/qa-merge
   export UBSAN_OPTIONS=suppressions=test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1
   

4. Check out new branch in other window

   cd ~/Workspace/qa-assets
   git checkout -b "202y-mm-murch-inputs"
   

5. Move upstream fuzz inputs aside

   cd ~/Workspace/qa-assets
   mv fuzz_corpora upstream_corpora
   

6. Merge the active fuzzing corpora and old corpora into new corpora

   cd ~/Workspace/qa-merge
   build_fuzz/test/fuzz/test_runner.py -l DEBUG --par 3 --m_dir ../qa-assets-active-fuzzing/fuzz_corpora/ --m_dir ../qa-assets/upstream_corpora/ ../qa-assets/fuzz_corpora/
   

7. Restore the upstream inputs

   cd ~/Workspace/qa-assets
   git restore -- ./fuzz_corpora
   

8. Commit and push

   cd ~/Workspace/qa-assets
   git commit -m "Add Murch’s inputs MONTH YYYY"
   

Mh, I’m running another merge. If I get a similar result, I’ll be mollified.

Ok, let us know how it went. In any case, I think you can drop the wallet_notification fuzz inputs, as this is the reason for the CI timeout.

Did another merge as described with the above instructions. Dropped my additions to wallet_notifications. I got 54k new inputs, vs 57k before, so that roughly checks out.

This seems odd, because there is almost no change in coverage data:

• main: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/62bd61de110b057c/86e21ec4af9cb67d/fuzz.coverage/index.html
• pull: https://drahtbot.space/host_reports/DrahtBot/reports/coverage_fuzz/monotree/62bd61de110b057c/20dd55d302a0ade8/fuzz.coverage/index.html

I'll try to reproduce.

I guess it is plausible that for almost every fuzz input you found a slightly smaller one, which is then selected for inclusion

Locally I am also getting a smaller result, when merging your branch into a new folder. However, that could just be instrumentation differences.

Happy to merge this, but out of interest, I wonder how many files you had before the merge.

I deleted some of the older folders already, but the remainder of my active fuzzing corpora has over 2.2 million inputs. I fuzz with 28 processes in parallel, of which I have limited the max_len in seven of them:

    # Run some threads without sanitizers and limited length to foster reduced length seeds
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -use_value_profile=1 -reload=1 -max_total_time=3600 -max_len=32 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -use_value_profile=1 -reload=1 -max_total_time=3600 -max_len=96 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -reload=1 -max_total_time=3600 -max_len=96 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -use_value_profile=1 -reload=1 -max_total_time=3600 -max_len=288 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -reload=1 -max_total_time=3600 -max_len=288 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -use_value_profile=1 -reload=1 -max_total_time=3600 -max_len=864 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -reload=1 -max_total_time=3600 -max_len=864 $i & \
    # Run most threads without sanitizers
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -use_value_profile=1 -reload=1 -max_total_time=3600 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=18 -reload=1 -max_total_time=3600 $i & \
    # Run a thread each with and without use_value_profile and all sanitizers enabled
    cd $HOME/Workspace/qa-merge && \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -use_value_profile=1 -reload=1 -max_total_time=3600 $i & \
    FUZZ=$(basename $i) build_fuzz/src/test/fuzz/fuzz -fork=1 -reload=1 -max_total_time=3600 $i & \
    wait; done; notify-send -u critical 'FINISHED FUZZING 10 QA-ASSETS for 28×1h each'

thx, looks good