Address feedback on new delete-nonreduced-fuzz-inputs script in rust #270

ekzyis commented at 1:48 PM on March 30, 2026: contributor

Addressed feedback:

use relative link in README
refactor argument parsing to be easier to extend
- drops 'too many arguments' error message
Use ?; consistently
avoid temporary copy when building arguments for Command::new
replace wget with curl
also use --depth=1 when cloning AFLplusplus
use git add + git commit consistently instead of git commit -a
don't require -Znext-lockfile-bump with cargo

Additionally:

fix git_clone error message did not use format! macro
add .gitignore to ignore cargo target/

All in one commit because I think this is easy enough to review, and not "mission-critical code" anyway. Lmk if you would prefer it differently.

This is currently running, but lgtm.

Btw, I'm considering improving the UX/DX: We could pipe the current output to a file and show a progress bar on stdout (without adding a dependency). The progress bar would indicate how many fuzz targets afl-cmin has already processed, at which sanitizer step we are, and maybe there's more useful progress information we can get by looking at the current output. This would help with knowing how long it might still take. I also always want to know how long it took to know what to expect next time, but currently, I have to remember to run the script with time or look at the commit timestamps to get an idea.

With #265, the script will take even longer; making improvements to its long-running nature more important.

in delete-nonreduced-fuzz-inputs/src/main.rs:78 in 3f5ac2d3dd

  83 |      install_deps()?;
  84 |      clone_and_configure_repositories()?;
  85 |  
  86 |      let all_inputs_dir = move_fuzz_inputs()?;
  87 | -    git_commit_all(QA_ASSETS_PATH, "Delete fuzz inputs")?;
  88 | +    // git commit -a does not add untracked files so we use git add manually

maflcko commented at 2:01 PM on March 30, 2026:

    // git commit does not commit untracked removed files

I think this can be removed, or reworded?

<!-- Also, I'd presume the ` git_add(QA_ASSETS_PATH, FUZZ_CORPORA_DIR)?;` to fail, because `FUZZ_CORPORA_DIR` does not exist at this point?

ekzyis commented at 2:27 PM on March 30, 2026:

Isn’t it important to specifically mention that this is about the staging behavior of -a? To me, this reads confusingly, since git commit does not commit anything unless you stage it beforehand.

    // git commit -a does not commit untracked removed files

edit: did this in b89fa6499d

in delete-nonreduced-fuzz-inputs/src/main.rs:70 in 3f5ac2d3dd

  73 | -            }
  74 | -            Err(help("help requested"))?;
  75 | +    for arg in env::args().skip(1) {
  76 | +        match arg.as_str() {
  77 | +            "--help" | "-h" => return Err(help("help requested")),
  78 | +            a => return Err(format!("Unexpected argument: {a}")),

maflcko commented at 2:01 PM on March 30, 2026:

should go through help, no?

ekzyis commented at 2:28 PM on March 30, 2026:

Ahh, yes, thanks, done in b89fa6499d

maflcko approved

maflcko commented at 2:04 PM on March 30, 2026: contributor

Btw, I'm considering improving the UX/DX: We could pipe the current output to a file and show a progress bar on stdout (without adding a dependency). The progress bar would indicate how many fuzz targets afl-cmin has already processed, at which sanitizer step we are, and maybe there's more useful progress information we can get by looking at the current output. This would help with knowing how long it might still take. I also always want to know how long it took to know what to expect next time, but currently, I have to remember to run the script with time or look at the commit timestamps to get an idea.

Yeah, it is a bit long running. Though, the steps may not take equal time, as libfuzzer+sanitizers or AFL may take a longer time than the parallelized libfuzzer run without sanitizers. I usually also take a look at the current git commit (or the output of (h)top to guess the progress), or let it run over night and then check the next morning.

Address feedback on new delete-nonreduced-fuzz-inputs script in rust

Addressed feedback:

* use relative link in README
* refactor argument parsing to be easier to extend
  * drops 'too many arguments' error message
* Use ?; consistently
* avoid temporary copy when building arguments for Command::new
* replace wget with curl
* also use --depth=1 when cloning AFLplusplus
* use git add + git commit consistently instead of git commit -a
* don't require -Znext-lockfile-bump with cargo

Additionally:

* fix git clone error message did not use format! macro
* add .gitignore to ignore cargo target/

b89fa6499d

ekzyis force-pushed on Mar 30, 2026

maflcko merged this on Mar 30, 2026

maflcko closed this on Mar 30, 2026

ekzyis commented at 2:43 PM on March 30, 2026: contributor

Yeah, it is a bit long running. Though, the steps may not take equal time, as libfuzzer+sanitizers or AFL may take a longer time than the parallelized libfuzzer run without sanitizers. I usually also take a look at the current git commit (or the output of (h)top to guess the progress), or let it run over night and then check the next morning.

I will run the script many more times when working on #265. I will then look into a progress bar more, and see if it's worth it

This is currently running, but lgtm.

The script just finished. New data points wrt time and determinism:

commit c763081b3c52080340e24c2b901a57c826ae1126
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Mon Mar 30 14:36:46 2026 +0000

    Reduced inputs for fuzzer,address,undefined,integer

commit b15f92121d032712a10ddc8bfd43ea4327c8c90e
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Mon Mar 30 13:58:47 2026 +0000

    Reduced inputs for fuzzer

commit f9c63da036af837dfd3f389312dca28560561020
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Mon Mar 30 13:44:29 2026 +0000

    Reduced inputs for afl-cmin

commit f52507dac479618168862d4d4a85c9e3328b711b
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Mon Mar 30 13:29:40 2026 +0000

    Delete fuzz inputs

commit 26fe402afc38f1d4b8d61c6ff3a5849221017f1e
Author: maflcko <6399679+maflcko@users.noreply.github.com>
Date:   Mon Mar 30 10:45:44 2026 +0200

    Merge pull request [#269](/bitcoin-core-qa-assets/269/) from ekzyis/rust-delete-nonreduced-fuzz-inputs

    Convert delete_nonreduced_fuzz_inputs.sh to rust

$ find all_inputs -type f | wc -l
122591
$ find qa-assets/fuzz_corpora -type f | wc -l
116405

Previously in #269 (comment):

$ find all_inputs -type f | wc -l
122591
$ find qa-assets/fuzz_corpora -type f | wc -l
116419

Only 14 files less now!

ekzyis deleted the branch on Mar 30, 2026