Convert delete_nonreduced_fuzz_inputs.sh to rust #269

ekzyis commented at 3:51 PM on March 29, 2026: contributor

This converts the existing bash script to rust as suggested in #268 (comment).

Also updated README and added this to the documentation in the script:

Also, it's recommended to run the script twice to ensure that the results are "somewhat" reproducible.

How I tested this:

Setup ubuntu 24.04 vm with virt-manager and 8GiB allocated memory on NixOS
Run these commands:

$ git clone --branch rust-delete-nonreduced-fuzz-inputs https://github.com/ekzyis/qa-assets
$ cp -r qa-assets/delete-nonreduced-fuzz-inputs .
$ cd delete-nonreduced-fuzz-inputs
$ sudo apt install cargo
$ sudo cargo run -Znext-lockfile-bump

Listen to computer sounds for over 1 hour
Check if expected git commits have been created

ekzyis marked this as a draft on Mar 29, 2026

ekzyis force-pushed on Mar 30, 2026

Convert delete_nonreduced_fuzz_inputs.sh to rust fd99a286da

ekzyis force-pushed on Mar 30, 2026

ekzyis commented at 1:31 AM on March 30, 2026: contributor

fd99a286da First successful complete run deleted 6k files and took 1h10min:

$ find all_inputs -type f | wc -l
122591
$ find qa-assets/fuzz_corpora -type f | wc -l
116419

commit aa0c651d8ae39db2e71c0f8a7c7d2af4b7ef648c
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Mon Mar 30 00:44:05 2026 +0000

    Reduced inputs for fuzzer,address,undefined,integer

commit 4b90c7119a1d6eb8b8a300db1c95464d8c02abb8
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Mon Mar 30 00:06:47 2026 +0000

    Reduced inputs for fuzzer

commit 9ddd4d8a1f44e47e6c03c24e29587d82ab73fd59
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Sun Mar 29 23:53:35 2026 +0000

    Reduced inputs for afl-cmin

commit e73b464051a4b1c1a92eb550518da72cdb08ea8d
Author: delete_nonreduced_inputs script <noreply@noreply.noreply>
Date:   Sun Mar 29 23:36:52 2026 +0000

    Delete fuzz inputs

commit bdc226d2037713aac729e047c4e3521f56f45b98
Author: Niklas Gögge <n.goeggi@gmail.com>
Date:   Fri Mar 13 16:44:07 2026 +0100

    Merge pull request [#263](/bitcoin-core-qa-assets/263/) from maflcko/main
    
     Delete nonreduced fuzz inputs

Will run again tomorrow, but I think this is ready for review now

ekzyis marked this as ready for review on Mar 30, 2026

in README.md:30 in fd99a286da

  26 | @@ -27,7 +27,7 @@ runner:
  27 |  
  28 |  To avoid corpora bloat, stale inputs and potential CI timeouts, we usually
  29 |  prune/minimize our corpora around the branch-off point using the
  30 | -[`delete_nonreduced_fuzz_inputs.sh`](https://raw.githubusercontent.com/bitcoin-core/bitcoin-maintainer-tools/main/delete_nonreduced_fuzz_inputs.sh)
  31 | +[`delete-nonreduced-fuzz-inputs`](https://github.com/bitcoin-core/qa-assets/tree/main/delete-nonreduced-fuzz-inputs)

maflcko commented at 7:29 AM on March 30, 2026:

[`delete-nonreduced-fuzz-inputs`](/delete-nonreduced-fuzz-inputs)

in delete-nonreduced-fuzz-inputs/src/main.rs:77 in fd99a286da

  72 | +                Err(help("Too many arguments"))?;
  73 | +            }
  74 | +            Err(help("help requested"))?;
  75 | +        }
  76 | +        Some(a) => Err(help(&format!("Unexpected argument: {a}")))?,
  77 | +    }

maflcko commented at 7:47 AM on March 30, 2026:

    for arg in args {
        if let Some(s) = arg.strip_prefix("-h") {
            Err(help("Help requested"))?;
        } else {
            Err(help(&format!(
                "Too many args, or unknown named arg: {arg}"
            )))?;
        }
     }

nit: Maybe use a pattern that is easier to extend in the future?

in delete-nonreduced-fuzz-inputs/src/main.rs:117 in fd99a286da

 112 | +}
 113 | +
 114 | +fn install_deps() -> AppResult {
 115 | +    install_apt_deps()?;
 116 | +    install_llvm()?;
 117 | +    install_aflpp()

maflcko commented at 7:50 AM on March 30, 2026:

    install_aflpp()?;
    Ok())

nit: I prefer to use ? consistently, so that it is easier to add a single line at the end, without having to modify the earlier line.

in delete-nonreduced-fuzz-inputs/src/main.rs:135 in fd99a286da

 130 | +    }
 131 | +
 132 | +    let mut install_args: Vec<&str> = vec!["install", "-y"];
 133 | +    install_args.extend_from_slice(APT_PACKAGES);
 134 | +    if !Command::new("apt")
 135 | +        .args(&install_args)

maflcko commented at 7:52 AM on March 30, 2026:

nit: No need for a temporary copy, I think you can just pass the two slices as-is:

.args(["install", "-y"])
.args(APT_PACKAGES)

in delete-nonreduced-fuzz-inputs/src/main.rs:149 in fd99a286da

 144 | +}
 145 | +
 146 | +fn install_llvm() -> AppResult {
 147 | +    env::set_var("LLVM_VERSION", LLVM_VERSION);
 148 | +
 149 | +    if !Command::new("wget")

maflcko commented at 7:56 AM on March 30, 2026:

nit: Obviously unrelated, but it could make sense to switch to curl here:

"curl", "--fail", "--location", "--remote-name",

Otherwise, this may break with wget2, similar to https://github.com/bitcoin-core/qa-assets/pull/201

in delete-nonreduced-fuzz-inputs/src/main.rs:186 in fd99a286da

 181 | +
 182 | +fn install_aflpp() -> AppResult {
 183 | +    let clone_path = "AFLplusplus";
 184 | +    git_clone(
 185 | +        "https://github.com/AFLplusplus/AFLplusplus",
 186 | +        &["--branch=stable"],

maflcko commented at 8:19 AM on March 30, 2026:

nit: --depth=1 should work here as well?

in delete-nonreduced-fuzz-inputs/src/main.rs:292 in fd99a286da

 287 | +}
 288 | +
 289 | +fn git_commit_all<P: AsRef<Path>>(repo_path: P, message: &str) -> AppResult {
 290 | +    if !Command::new("git")
 291 | +        .current_dir(repo_path)
 292 | +        .args(["commit", "-a", "-m", message])

maflcko commented at 8:38 AM on March 30, 2026:

fn git_commit<P: AsRef<Path>>(repo_path: P, message: &str) -> AppResult {
    if !Command::new("git")
        .current_dir(repo_path)
        .args(["commit", "-m", message])

nit: Any reason why the --all is needed? It was not there previously?

ekzyis commented at 12:30 PM on March 30, 2026:

It was there inconsistently, see here. I forgot to reconsider this, it's not needed, thanks

maflcko commented at 12:57 PM on March 30, 2026:

Ok, I see. I guess it can be fixed by adding a git_add(QA_ASSETS_PATH, FUZZ_CORPORA_DIR)?; between the two lines:

    let all_inputs_dir = move_fuzz_inputs()?;
    git_commit_all(QA_ASSETS_PATH, "Delete fuzz inputs")?;

?

ekzyis commented at 1:50 PM on March 30, 2026:

Yes, did this in #270

in delete-nonreduced-fuzz-inputs/Cargo.lock:3 in fd99a286da

   0 | @@ -0,0 +1,7 @@
   1 | +# This file is automatically @generated by Cargo.
   2 | +# It is not intended for manual editing.
   3 | +version = 4

maflcko commented at 8:44 AM on March 30, 2026:

version = 3

nit

maflcko approved

maflcko commented at 8:45 AM on March 30, 2026: contributor

lgtm. Thx

Just left some nits, some of which are unrelated, none of which are a blocker.

Going to merge for now, but the nits can be addressed in a follow-up, if they make sense.

maflcko merged this on Mar 30, 2026

maflcko closed this on Mar 30, 2026

ekzyis deleted the branch on Mar 30, 2026

ekzyis commented at 1:51 PM on March 30, 2026: contributor

Addressed the feedback in #270, thank you for the review and merge!