doc: update fuzz instructions when on macOS #31954

pull m3dwards wants to merge 1 commits into bitcoin:master from m3dwards:250225-doc-fuzz-mac changing 1 files +5 −2
  1. m3dwards commented at 8:45 pm on February 25, 2025: contributor

    Fixes: #31049

    Updates the instructions for fuzzing on macOS to use lld instead of ld.

    Tested instructions on M1 Mac running 14.6.1

  2. DrahtBot commented at 8:45 pm on February 25, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31954.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK l0rinc, brunoerg, hebasto
    Concept ACK jonatack
    Stale ACK Prabhat1308

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

  3. DrahtBot added the label Docs on Feb 25, 2025
  4. maflcko added the label macOS on Feb 25, 2025
  5. jonatack commented at 10:57 pm on February 25, 2025: member
    Concept ACK
  6. Prabhat1308 commented at 4:12 am on February 26, 2025: none

    Running the build after these configurations on MacOs 15.3.1 M4 Pro, the fuzz tests are running but I do get these warnings on almost every object being built . Is this expected ?

     04 warnings generated.
 1[ 12%] Linking CXX static library libbitcoin_cli.a
 24 warnings generated.
 3[ 12%] Building CXX object src/test/fuzz/util/CMakeFiles/test_fuzz.dir/net.cpp.o
 4[ 12%] Built target bitcoin_cli
 5[ 12%] Building CXX object src/CMakeFiles/bitcoin_common.dir/base58.cpp.o
 6clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
 7clang++: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
 8warning: unknown warning option '-Wduplicated-branches' [-Wunknown-warning-option]
 9warning: unknown warning option '-Wduplicated-cond' [-Wunknown-warning-option]
10warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
11warning: unknown warning option '-Wbidi-chars=any' [-Wunknown-warning-option]
124 warnings generated.
13clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
14clang++: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
15warning: unknown warning option '-Wduplicated-branches' [-Wunknown-warning-option]
16warning: unknown warning option '-Wduplicated-cond' [-Wunknown-warning-option]
17warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
18warning: unknown warning option '-Wbidi-chars=any' [-Wunknown-warning-option]
19[ 13%] Building CXX object src/CMakeFiles/bitcoin_consensus.dir/primitives/block.cpp.o
20clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
21clang++: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
22warning: unknown warning option '-Wduplicated-branches' [-Wunknown-warning-option]
23warning: unknown warning option '-Wduplicated-cond' [-Wunknown-warning-option]
24warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
25warning: unknown warning option '-Wbidi-chars=any' [-Wunknown-warning-option]
264 warnings generated.
27[ 14%] Linking CXX static library libunivalue.a
284 warnings generated.
29[ 14%] Building CXX object src/CMakeFiles/bitcoin_consensus.dir/primitives/transaction.cpp.o
30clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
31clang++: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
32warning: unknown warning option '-Wduplicated-branches' [-Wunknown-warning-option]
33warning: unknown warning option '-Wduplicated-cond' [-Wunknown-warning-option]
34warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
35warning: unknown warning option '-Wbidi-chars=any' [-Wunknown-warning-option]
36[ 14%] Built target univalue
37[ 14%] Building CXX object src/test/util/CMakeFiles/test_util.dir/setup_common.cpp.o
38clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
39clang++: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
40warning: unknown warning option '-Wduplicated-branches' [-Wunknown-warning-option]
41warning: unknown warning option '-Wduplicated-cond' [-Wunknown-warning-option]
42warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
43warning: unknown warning option '-Wbidi-chars=any' [-Wunknown-warning-option]
444 warnings generated.
45[ 15%] Linking CXX static library libbitcoin_clientversion.a
464 warnings generated.
47[ 15%] Built target bitcoin_clientversion
48[ 15%] Building CXX object src/crypto/CMakeFiles/bitcoin_crypto.dir/ripemd160.cpp.o
49[ 15%] Building CXX object CMakeFiles/leveldb.dir/src/leveldb/db/log_writer.cc.o
50clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
51clang++: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
52warning: unknown warning option '-Wduplicated-branches' [-Wunknown-warning-option]
53warning: unknown warning option '-Wduplicated-cond' [-Wunknown-warning-option]
54warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
55warning: unknown warning option '-Wbidi-chars=any' [-Wunknown-warning-option]
56clang++: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
57clang++: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
58warning: unknown warning option '-Wduplicated-branches' [-Wunknown-warning-option]
59warning: unknown warning option '-Wduplicated-cond' [-Wunknown-warning-option]
60warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option]
61warning: unknown warning option '-Wbidi-chars=any' [-Wunknown-warning-option]
624 warnings generated.
634 warnings generated.
  7. in doc/fuzzing.md:156 in e93c0b0882 outdated 
    152@@ -153,13 +153,18 @@ You may also need to take care of giving the correct path for `clang` and
153 `clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
154 `clang` does not come first in your path.
155 
156+Currently the default linker on macOS `ld` does not work with recent versions of `llvm` so you will need to install llvm's linker `lld` and use it with the CXX_FLAG `-fuse-ld=lld`.
    l0rinc commented at 7:17 pm on February 26, 2025:
    Can you add a tracking issue here to know when we can safely remove it
    m3dwards commented at 9:40 pm on February 26, 2025:
    Not sure I feel this is big enough to add to the issue list and I would be happy for the docs to be updated when someone notices again that it’s not needed any more, especially as this workaround should continue to work in the future even if strictly not needed any more.
    m3dwards commented at 9:40 pm on February 26, 2025:
    But obviously if you feel differently, go ahead and make the issue.
    l0rinc commented at 10:04 pm on February 26, 2025:
    I meant the ld issue - not critical
  8. in doc/fuzzing.md:162 in e93c0b0882 outdated 
    157+
158 Full configuration step that was tested on macOS with `brew` installed `llvm`:
159 
160 ```sh
161+$ brew install llvm
162+$ brew install lld
    l0rinc commented at 7:19 pm on February 26, 2025:

    we can likely do this in a single command:

    0$ brew install llvm lld

    We also likely need to remove the command from https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md?plain=1#L150 now

    m3dwards commented at 9:36 pm on February 26, 2025:
    Taken
  9. in doc/fuzzing.md:158 in e93c0b0882 outdated 
    152@@ -153,13 +153,18 @@ You may also need to take care of giving the correct path for `clang` and
153 `clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
154 `clang` does not come first in your path.
155 
156+Currently the default linker on macOS `ld` does not work with recent versions of `llvm` so you will need to install llvm's linker `lld` and use it with the CXX_FLAG `-fuse-ld=lld`.
157+
158 Full configuration step that was tested on macOS with `brew` installed `llvm`:
    l0rinc commented at 7:21 pm on February 26, 2025:

    This seems to need an update - though I’d just remove the command from the comment:

    0Full configuration steps:
    m3dwards commented at 9:37 pm on February 26, 2025:
    Took something similar
  10. in doc/fuzzing.md:167 in e93c0b0882 outdated 
    163 $ cmake --preset=libfuzzer \
164    -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
165    -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
166-   -DAPPEND_LDFLAGS=-Wl,-no_warn_duplicate_libraries
167+   -DAPPEND_LDFLAGS=-Wl,-no_warn_duplicate_libraries \
168+   -DCMAKE_CXX_FLAGS="-fuse-ld=lld"
    l0rinc commented at 7:23 pm on February 26, 2025:

    I’m getting:

    clang++: warning: argument unused during compilation: ‘-fuse-ld=lld’ [-Wunused-command-line-argument]

    m3dwards commented at 9:37 pm on February 26, 2025:
    Resolved warnings
  11. l0rinc changes_requested
  12. l0rinc commented at 7:35 pm on February 26, 2025: contributor

    Concept ACK

    Finally fuzzing is working with these changes (tested on M4 Max with Clang 19.1.7), but we’re still getting warnings - and a I left a few more suggestions

  13. m3dwards force-pushed on Feb 26, 2025
  14. m3dwards commented at 9:42 pm on February 26, 2025: contributor
    @Prabhat1308 and @l0rinc, I’ve updated the instructions to achieve the same goal but without the warnings.
  15. in doc/fuzzing.md:158 in 725c0fd95b outdated 
    152@@ -153,13 +153,17 @@ You may also need to take care of giving the correct path for `clang` and
153 `clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
154 `clang` does not come first in your path.
155 
156-Full configuration step that was tested on macOS with `brew` installed `llvm`:
157+Currently the default linker on macOS `ld` does not work with recent versions of `llvm` so you will need to install llvm's linker `lld` and use it with the CMAKE_EXE_LINKER_FLAG `-fuse-ld=lld`.
158+
159+Full configuration step for macOS:
    l0rinc commented at 10:05 pm on February 26, 2025:

    It’s not exactly full, we still need:

    0$ cmake --build build_fuzz -j$(nproc)
1$ FUZZ=process_message build_fuzz/src/test/fuzz/fuzz
    m3dwards commented at 10:19 pm on February 26, 2025:
    Technically, it is the full configuration step :)
  16. l0rinc approved
  17. l0rinc commented at 10:10 pm on February 26, 2025: contributor
    ACK 725c0fd95b15157bd0ec141cc24d27818f536656
  18. DrahtBot requested review from jonatack on Feb 26, 2025
  19. Prabhat1308 commented at 10:26 pm on February 26, 2025: none

    tACK 725c0fd

    -Checked by running the fuzz tests on several fuzz targets . No warnings displayed now.

  20. maflcko added this to the milestone 29.0 on Feb 27, 2025
  21. in doc/fuzzing.md:165 in 725c0fd95b outdated 
    162+$ brew install llvm lld
163 $ cmake --preset=libfuzzer \
164    -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
165    -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
166-   -DAPPEND_LDFLAGS=-Wl,-no_warn_duplicate_libraries
167+   -DAPPEND_LDFLAGS=-Wl,-no_warn_duplicate_libraries \
    fanquake commented at 10:14 am on March 4, 2025:
    no_warn_duplicate_libraries isn’t an lld option, so this can be dropped.
    m3dwards commented at 12:21 pm on March 4, 2025:
    Thanks, dropped.
    l0rinc commented at 1:00 pm on March 4, 2025:
    Nice, finally we can get rid of that (tested, seems to be working without the warning, good find)! Should we comment anything on https://github.com/llvm/llvm-project/issues/102630?
  22. in doc/fuzzing.md:156 in 725c0fd95b outdated 
    152@@ -153,13 +153,17 @@ You may also need to take care of giving the correct path for `clang` and
153 `clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
154 `clang` does not come first in your path.
155 
156-Full configuration step that was tested on macOS with `brew` installed `llvm`:
157+Currently the default linker on macOS `ld` does not work with recent versions of `llvm` so you will need to install llvm's linker `lld` and use it with the CMAKE_EXE_LINKER_FLAG `-fuse-ld=lld`.
    fanquake commented at 10:15 am on March 4, 2025:
    Rather than adding text that repeats the code below, we could just say something like: “Using lld is required to due to issues with Apples ld and LLVM”.
    m3dwards commented at 12:21 pm on March 4, 2025:
    Changed to this
  23. m3dwards force-pushed on Mar 4, 2025
  24. in doc/fuzzing.md:156 in 4fae769791 outdated 
    152@@ -153,13 +153,16 @@ You may also need to take care of giving the correct path for `clang` and
153 `clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
154 `clang` does not come first in your path.
155 
156-Full configuration step that was tested on macOS with `brew` installed `llvm`:
157+Using lld is required to due to issues with Apples ld and LLVM.
    l0rinc commented at 12:47 pm on March 4, 2025:

    Typo, maybe something like:

    0Using `lld` is required due to issues with Apple's `ld` and `LLVM`.
    m3dwards commented at 1:16 pm on March 4, 2025:

    gah, taken!

    Thanks for the spot.

  25. m3dwards force-pushed on Mar 4, 2025
  26. doc: update fuzz instructions when on macOS 
    Default linker on macOS does not work with recent versions of LLVM. Updated the instructions for fuzzing to use lld instead.
    75486c8ed8
  27. m3dwards force-pushed on Mar 4, 2025
  28. l0rinc commented at 1:23 pm on March 4, 2025: contributor

    ACK 75486c8ed87a480b9f0c4dc7a10f3cd4eee87b12

    Compared to previous review, APPEND_LDFLAGS was removed and the description simplified to avoid repetition.

  29. brunoerg approved
  30. brunoerg commented at 4:37 pm on March 4, 2025: contributor

    ACK 75486c8ed87a480b9f0c4dc7a10f3cd4eee87b12

    I just tested it on macOS 14.3 and worked fine.

  31. hebasto approved
  32. hebasto commented at 5:58 pm on March 4, 2025: member
    ACK 75486c8ed87a480b9f0c4dc7a10f3cd4eee87b12, tested on macOS 15.3.1 (Apple M1) + Clang 19.1.7.
  33. hebasto requested review from dergoegge on Mar 4, 2025
  34. glozow assigned hebasto on Mar 4, 2025
  35. hebasto merged this on Mar 5, 2025
  36. hebasto closed this on Mar 5, 2025
  37. m3dwards deleted the branch on Mar 5, 2025
  38. pablomartin4btc commented at 5:02 pm on March 5, 2025: member

    tACK 75486c8ed87a480b9f0c4dc7a10f3cd4eee87b12

    Managed to reproduce the issue in master (before merge) and this PR fixes it.

  39. jonatack commented at 10:12 pm on March 7, 2025: member

    Post-merge tested ACK using the following incantation with arm64 macos 15.3.1:

    0$ rm -rf ./build_fuzz/ && cmake -B ./build_fuzz -DAPPEND_CXXFLAGS="-std=c++23" --preset=libfuzzer -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" && cmake --build ./build_fuzz --parallel 11 --
1
2$ FUZZ=process_message ./build_fuzz/src/test/fuzz/fuzz


m3dwards DrahtBot jonatack Prabhat1308 l0rinc fanquake brunoerg hebasto pablomartin4btc


dergoegge

Labels
Docs macOS

Milestone
29.0

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-03-31 09:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me