RFC: Do we want to support fuzzing on MacOS? #33731

1. 
   dergoegge commented at 11:17 am on October 29, 2025: member

   Fuzzing on MacOS (i.e. actual fuzzing not just running the inputs through the fuzz binary) is known to be brittle and we’ve had plenty of issues reported to us showcasing this:

   • #33667
   • #32089
   • #31049
   • #27550
   • #19789

   The solution usually involves something along the lines of waiting for a brew llvm update or adding macOS specific hints to our documentation. These issues can however also depend on specific macOS versions, and our hints might not be accurate for every version or get stale with time.

   I usually don’t chime in on these issues because I don’t have a Mac and afaik, all serious fuzzing (at scale & automated) for us (and most other projects) happens on Linux.

   I think there are two options:

   1. Keep the current approach and fix/document as issues are reported.
   2. Deprecate “official” support for fuzzing on macOS and add a section to the docs about using Linux instead (e.g. to use a VM or VPS). MacOS users will still be able to fuzz on a their Mac but it won’t be on us to triage the issues for their specific setup.

   I’d prefer option 2).

2. fanquake added the label Brainstorming on Oct 29, 2025
3. fanquake added the label Tests on Oct 29, 2025
4. 
   fanquake commented at 11:57 am on October 29, 2025: member

   1. Keep the current approach and fix/document as issues are reported.

   If we do do this, it should be clear what is supported (i.e just the latest version of LLVM/Clang shipped via brew, using lld to link?) , otherwise (as you metion above) we’ll be documenting many different “workarounds” depending on the version of macOS, version of Apple clang/binutils install, if brew LLVM is being used, and it’s version, using ld64 to link, vs lld.

   I’d prefer option 2).

   Concept ACK. Getting fuzzing to work on macOS seems like whack-a-mole, and even if somebody can get it to compile, the amount of useful fuzzing they are likely to do, other than a sanity check that the fuzzer doesn’t immeadiately crash, seems limited.

5. 
   Crypt-iQ commented at 12:13 pm on October 29, 2025: contributor

   I have a mac and prefer option 2 also. The fuzz tests usually break each macOS update and I don’t think it makes sense to maintain documentation for it. I occasionally find it useful just to see that a harness is working, but I can fix issues on my own as they come up.
6. maflcko added the label macOS on Oct 29, 2025
7. 
   marcofleon commented at 12:20 pm on October 29, 2025: contributor

   Concept ACK on option 2. I remember struggling on my mac when I first started fuzzing and the section we have in the docs didn’t really help. Ended up just figuring it out on my own. These days pretty much all of my fuzzing happens on Linux.
8. 
   maflcko commented at 12:24 pm on October 29, 2025: member

   No strong opinion, but I don’t mind having docs that just say they only works with a specific brew llvm version and macOS version, and have them updated when they break.

   Though, saying that libFuzzer is not supported on macOS seems also fine.

9. 
   brunoerg commented at 1:05 pm on October 29, 2025: contributor

   Concept ACK on option 2. I use macOS and I agree this is a pain – The effort of updating the documentation is not worth it. The only downside of it would be not attracting macOS people to fuzzing development (?). But anyway, if someone really wants to get in fuzzing dev would have to have a linux at some point.
10. 
    Crypt-iQ commented at 2:34 pm on October 29, 2025: contributor

    The only downside of it would be not attracting macOS people to fuzzing development (?)

    I was also thinking about this, and it could turn away some people from writing fuzz tests. Maybe if the docs mentioned a specific working version as suggested by @maflcko above, but then somebody would need to test these versions. I’ve had issues where even updating to a new macOS version isn’t possible without some hacks.

11. 
    furszy commented at 2:48 pm on October 29, 2025: member

    It’s definitely painful to run it on Mac. I’m not against officially deprecating it if that is the general consensus but it’s quite useful to be able to troubleshoot individual crashes/timeouts locally. It would be handy to have pinned versions with specific docs on how to make it work (if possible).
12. 
    dergoegge commented at 4:12 pm on October 29, 2025: member

    but it’s quite useful to be able to troubleshoot individual crashes/timeouts locally

    This will still be possible, because you can build the fuzz binary without libFuzzer for reproduction purposes (this is what we do in the macOS fuzz CI job). Unless the bugs only manifest when running the fuzzer due to non-determinism.

13. 
    l0rinc commented at 8:56 pm on October 29, 2025: contributor

    I have also documented some workarounds for mac in #32084 (comment) - they kinda’ work, without sanitizers at least, which cannot handle exceptions for some reason.

    But I’m not sure why we want to give up fuzzing on Macs, everything we do is a whack-a-mole game, if we want people to write fuzz tests, we can’t expect them to run them on a different machine and not even try them locally. We’ll just end up with 100 PR pushes because people will debug these on the CI.

14. 
    maflcko commented at 8:58 am on October 30, 2025: member

    My suggestion would be similar to how valgrind is handled, which doesn’t really work unless the dev picks a tested Linux distro, arch, and compiler, with compiler settings:

    https://github.com/bitcoin/bitcoin/blob/72511fd02e72b74be11273e97bd7911786a82e54/contrib/valgrind.supp#L15-L16

    In that case, I’d expect that the doc/fuzzing.md would only list the latest release of macOS and the exact steps (with compiler version) to reproduce there. If someone is on an older macOS, they can check the older version of the docs on their own.

    Obviously this means it will have to be updated yearly, but this is true for a lot of other docs, such as doc/build-netbsd.md, which list the latest tested version to work.

15. 
    fanquake commented at 10:44 am on October 30, 2025: member

    we can’t expect them to run them on a different machine and not even try them locally.

    They can always use Podman or Docker, and run a Linux VM, on their macOS machine.

16. fanquake added the label Fuzzing on Oct 30, 2025

Contributors
dergoegge fanquake Crypt-iQ marcofleon maflcko brunoerg furszy l0rinc

Labels
Brainstorming macOS Tests Fuzzing