wip: Split fuzz binary (take 2)

dergoegge commented at 3:16 pm on September 12, 2024: member

In addition to the benefits listed in #28971, this should also enable us to use https://github.com/ossf/fuzz-introspector provided by oss-fuzz. Our current runtime harness selection blocks introspector’s static analysis from working properly (e.g. it can’t statically determine which functions are reachable by a given harness).

This PR uses the approach suggested here: #29010 (comment). The list of available harnesses is determined (prior to compiling) by grepping for harness names in FUZZ_TARGET invocations. When compiling with -DBUILD_INDIVIDUAL_FUZZ_BINARIES=ON, individual binaries for each harness are produced that no longer include the runtime lookup via the FUZZ environment variable.

0cmake -B build_fuzz \
1  -DBUILD_FOR_FUZZING=ON \
2  -DBUILD_INDIVIDUAL_FUZZ_BINARIES=ON \
3  -DSANITIZERS=fuzzer
4cmake --build build_fuzz

build_fuzz/src/test/fuzz will contain the individual binaries, which are prefixed with fuzz_*.

I’m opening this now to get some early feedback, there are still a few things to address:

mention -DBUILD_INDIVIDUAL_FUZZ_BINARIES in the docs
include wallet harnesses
CI job that builds individual binaries (perhaps verify that the list of produced harnesses matches the monolithic binary)

DrahtBot commented at 3:16 pm on September 12, 2024: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30882.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#31333 (fuzz: Implement G_TEST_GET_FULL_NAME by hodlinator)
#28676 ([WIP] Cluster mempool implementation by sdaftuar)
#28584 (Fuzz: extend CConnman tests by vasild)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

in src/test/fuzz/util/CMakeLists.txt:7 in 66c288a4cd outdated

20-
21-if(NOT FUZZ_BINARY_LINKS_WITHOUT_MAIN_FUNCTION)
22-  target_compile_definitions(test_fuzz PRIVATE PROVIDE_FUZZ_MAIN_FUNCTION)
23+if(BUILD_INDIVIDUAL_FUZZ_BINARIES)
24+  # bash command produces list of harnesses: <harness name> <source file>
25+  execute_process(

dergoegge commented at 3:20 pm on September 12, 2024:

Currently an individual test_fuzz_* lib and fuzz_* binary is produced for each harness. It’s kind of ugly to duplicate this loop but I’m not sure to avoid it. Another loop would likely need to be added for the wallet harnesses as well. @hebasto @fanquake @maflcko Any ideas?

dergoegge commented at 3:21 pm on September 12, 2024: member

Trying to produce an introspector report using this branch: https://github.com/dergoegge/oss-fuzz/tree/2024-09-bitcoin-introspector

dergoegge force-pushed on Sep 12, 2024

DrahtBot added the label CI failed on Sep 13, 2024

DrahtBot commented at 5:07 am on September 13, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/30061858064

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

dergoegge force-pushed on Oct 1, 2024

[fuzz] Avoid FUZZ_TARGET invocation in namespace 6f08bc5640

[test] Define G_TRANSLATION_FUN per test binary (fuzz, test_bitcoin, bench) 151a4f585b

wip individual fuzz bins 0cb1f4ca90

dergoegge force-pushed on Oct 1, 2024

DrahtBot added the label Needs rebase on Nov 21, 2024

DrahtBot commented at 4:10 pm on November 21, 2024: contributor

🐙 This pull request conflicts with the target branch and needs rebase.

DrahtBot commented at 0:42 am on February 18, 2025: contributor

⌛ There hasn’t been much activity lately and the patch still needs rebase. What is the status here?

Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.

maflcko commented at 6:37 am on February 18, 2025: member

It is now using FI-light, see https://introspector.oss-fuzz.com/project-profile?project=bitcoin-core and https://github.com/google/oss-fuzz/pull/12983. Though, I am not sure about the takeaways.

dergoegge commented at 2:08 pm on February 18, 2025: member

The full report (https://storage.googleapis.com/oss-fuzz-introspector/bitcoin-core/inspector-report/20250216/fuzz_report.html) still seems blocked on our dynamic harness look up. Maybe I’m looking at the wrong thing.

maflcko commented at 2:13 pm on February 18, 2025: member

No it is the right thing. I just wanted to leave a comment to say that FI will now fallback to the light version by default, so the build is already passing, but the result didn’t look like it could be used.

It could still make sense to try a FI run on this branch to see if it will help.

maflcko commented at 2:16 pm on February 18, 2025: member

The changes here could even be useful for afl++ on their own, but I haven’t tried that either, as I am using libfuzzer.

dergoegge commented at 2:24 pm on February 18, 2025: member

It could still make sense to try a FI run on this branch to see if it will help.

I’ve tried in the past but I kept running out of memory on a 128GB machine (see https://github.com/ossf/fuzz-introspector/issues/1742). I don’t know how much more memory is needed currently. The maintainer indicated that they’d look at it but doesn’t look like they had time for this so far.

The changes here could even be useful for afl++ on their own, but I haven’t tried that either, as I am using libfuzzer.

Yes, I listed some of the benefits in #28971. I do think getting FI to work would be the bigger benefit however.

maflcko added the label Upstream on Feb 18, 2025

wip: Split fuzz binary (take 2) #30882

Code Coverage & Benchmarks

Reviews

Conflicts