ci: Add fuzzbuzz integration configuration file #18634

practicalswift commented at 4:48 PM on April 14, 2020: contributor

Add fuzzbuzz integration.

Just like #15338 enabled optional FreeBSD building via Cirrus CI (.cirrus.yml) this PR adds optional fuzzing via fuzzbuzz (.fuzzbuzz.yml).

Having this merged makes is easier for people to fuzz Bitcoin Core (via their forked repos) using their fuzzbuzz account and then hopefully submit coverage increasing corpus additions upstreams to to https://github.com/bitcoin-core/qa-assets.

Historically it has been mostly been me and MarcoFalke who submit test cases to qa-assets, but with this change hopefully more people will join the hunt for coverage increasing fuzzing inputs :)

ci: Add fuzzbuzz integration 8d306862ef

fanquake added the label Tests on Apr 14, 2020

fanquake requested review from MarcoFalke on Apr 15, 2020

MarcoFalke commented at 12:05 PM on April 15, 2020: member

Their cli is a 10 MB binary blob and their TOS require the user to be "human", so neither I or @DrahtBot will be using it.

Though, I don't mind merging it if someone finds it useful.

in .fuzzbuzz.yml:15 in 8d306862ef

  10 | +  - CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  11 | +  - make
  12 | +  - git clone https://github.com/bitcoin-core/qa-assets
  13 | +auto_targets:
  14 | +  find_targets_command: find src/test/fuzz/ -executable -type f ! -name "*.cpp" ! -name "*.h"
  15 | +  base_corpus_dir: qa-assets/fuzz_seed_corpus/

MarcoFalke commented at 12:07 PM on April 15, 2020:

Is there documentation on any of this? I couldn't find docs for find_targets_command.

practicalswift commented at 1:01 PM on April 15, 2020:

The find_targets_command was actually added to the fuzzbuzz platform after I asked them for a way to auto-detect harnesses and not having to enumerate all 120 Bitcoin Core fuzzing harnesses in the configuration file :)

I assume they just haven't gotten around to documenting it yet. I'll ask them to do that :)

practicalswift commented at 12:53 PM on April 15, 2020: contributor

@MarcoFalke Understand fully: I should have made it clear that I'm not suggesting DrahtBot should use this. This is meant as an "opt-in for interested users" configuration file like .cirrus.yml or .style.yapf - as opposed to .appveyor.yml or .travis.yml which are used also "officially" :)

practicalswift commented at 1:03 PM on April 15, 2020: contributor

@MarcoFalke Having this configuration file should be enough to avoid using any of their binaries, no? Just using their web interface plus pointing it to a repo with this configuration file in should be enough to use their service.

Personally I avoid running binaries provided by others :)

practicalswift commented at 6:09 AM on May 12, 2020: contributor

@MarcoFalke Ready for merge? FWIW, it would simplify my fuzzing workflow greatly and thus my ability to contribute fuzzing increasing inputs to https://github.com/bitcoin-core/qa-assets :)

MarcoFalke merged this on May 16, 2020

MarcoFalke closed this on May 16, 2020

sidhujag referenced this in commit 57550416d6 on May 17, 2020

hebasto commented at 5:59 AM on May 24, 2020: member

@practicalswift

I assume they just haven't gotten around to documenting it yet. I'll ask them to do that :)

Docs haven't being updated yet (

From my today's chat with Andrei from https://app.fuzzbuzz.io:

Hey there - our current platform has been down for a bit as we’re doing some maintenance and preparing to launch the next version of the platform. I don’t have an ETA for you unfortunately, but if you’d like to try out the new platform, please let me know your email and I’ll send you an email when we’re letting users try it out

Ah yes, you’re looking to run the bitcoin fuzzers, I’m assuming? I believe those should be working - but I think Practicalswift has the account and will need to invite you to the org. Let me double check

Did he mean that I should be invited by @practicalswift to the org on https://app.fuzzbuzz.io?

practicalswift commented at 6:24 AM on May 24, 2020: contributor

Did he mean that I should be invited by @practicalswift to the org on https://app.fuzzbuzz.io?

I don't know what he means but I know I'm fuzzing my own clone of the project: not bitcoin/bitcoin - I have no special project access for the repo bitcoin/bitcoin AFAICT :)

hebasto commented at 6:26 AM on May 24, 2020: member

Did he mean that I should be invited by @practicalswift to the org on https://app.fuzzbuzz.io?

I don't know what he means but I known I'm fuzzing my own clone of the project: not bitcoin/bitcoin - I have no special project access :)

That is my intention too :)

practicalswift commented at 6:33 AM on May 24, 2020: contributor

@hebasto

Sounds good! I think it is good to have all interested parties create their own accounts to keep things as decentralised and trustless as possible. AFAICT there is nothing stopping multiple independent accounts from having bitcoin/bitcoin as the upstream fuzzing repo :)

hebasto commented at 6:36 AM on May 24, 2020: member

@practicalswift

But their service is down for now (

... our current platform has been down for a bit as we’re doing some maintenance and preparing to launch the next version of the platform. I don’t have an ETA for you unfortunately...

practicalswift deleted the branch on Apr 10, 2021

PastaPastaPasta referenced this in commit bd06f2305f on Jun 27, 2021

PastaPastaPasta referenced this in commit 89af2c3e3a on Jun 28, 2021

PastaPastaPasta referenced this in commit 1842efe1a1 on Jun 29, 2021

PastaPastaPasta referenced this in commit c7821f4148 on Jul 1, 2021

PastaPastaPasta referenced this in commit 8a08aa2cef on Jul 1, 2021

PastaPastaPasta referenced this in commit 4839664069 on Jul 14, 2021

PastaPastaPasta referenced this in commit 08492610e2 on Jul 15, 2021

DrahtBot locked this on Aug 16, 2022