fuzz: Merge with -set_cover_merge=1 #28650

This should be less controversial than commit 151a2b189c3561dda2bb7de809306c1cfeb40e23. The overall size of the qa-assets repo is reduced further from 1.9GB to 1.6GB. Also, the runtime to iterate on the resulting folder is reduced further from ~1699s to ~1149s (N=1).

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage

For detailed information about the code coverage, see the test coverage report.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process.

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

ACK fa858d63a0a5d794aab38c26f60c593513fe08de

Further reading: https://github.com/llvm/llvm-project/commit/e6597dbae84034ce8ef97802584039b723adb526

Would be nice if one (1) person (or more) could re-check the runtime estimate on their machine. The steps to reproduce were:

• Clone the current qa-assets repo
• Call the python script from this pull (and from master) via ./test/fuzz/test_runner.py -j $( nproc ) -l DEBUG --m_dir ../qa-assets/fuzz_seed_corpus/ temp_new_folder_number_00
• Get the runtime (twice) via /usr/bin/time -f '%M KB, %S + %U' ./test/fuzz/test_runner.py -l DEBUG --par $( nproc ) ./temp_folder...

crACK fa858d63a0a5d794aab38c26f60c593513fe08de

I’m attempting to run the requested verification, but utxo_total_supply is absurdly slow on the merge. Will report back when it finally gets done. Definitely reduced the set on a prior merge I did, though.

So much ACK

Running main unmerged:

1. 622632 KB, 161.30 + 6192.01

Merging main to a fresh directory with the code changes in #28650 (IIRC: src/test/fuzz/fuzz -set_cover_merge=1 -shuffle=0 -prefer_small=1 -use_value_profile=0):

1. 460324 KB, 47.46 + 1466.82
2. 459752 KB, 48.92 + 1469.17

BTW, on the second run here, utxo_total_supply took 734 s for 422 seeds, on the main branch it took 2464 s for 2985 runs.

I think you forgot to -merge=1 on the master branch commit, for a fair comparison?

Sorry, I misinterpreted your instructions, now running the merge per the old style to get the execution time on that

After merging main to a fresh directory with src/test/fuzz/fuzz -merge=1 -shuffle=0 -prefer_small=1 -use_value_profile=1 running all seeds took: 592620 KB, 126.62 + 4689.42

After merging main to a fresh directory with src/test/fuzz/fuzz -merge=1 -shuffle=0 -prefer_small=1 -use_value_profile=0 it took: 560756 KB, 73.80 + 2374.22

I think you compared -use_value_profile=, but this pull request is about -set_cover_merge=.

I did four different things:

1. Run the seeds on qa-assets/main:

   622632 KB, 161.30 + 6192.01

2. Merge main to a fresh directory with -set_cover_merge=1 -use_value_profile=0, then run the seeds in the fresh directory:

   460324 KB, 47.46 + 1466.82 459752 KB, 48.92 + 1469.17

3. Merge main to a fresh directory with -merge=1 -use_value_profile=1, then run the resulting seeds:

   592620 KB, 126.62 + 4689.42

4. Merge main to a fresh directory with -merge=1 -use_value_profile=0, then run those seeds:

   560756 KB, 73.80 + 2374.22

So if you’re interested in the comparison of -set_cover_merge and -merge, please refer to 2 and 4, if you are interested in the overall speed-up of both changes (-merge=1 -use_value_profile=1 ↦ -set_cover_merge=1 -use_value_profile=0), please refer to 2 and 3. If you’re interested in how much we could reduce the runtime for the qa-assets/main branch, please refer to 1 and 2. I figured running the main branch qa-assets may also provide a baseline for how fast/slow this system is.