build: macOS fuzz instructions broken using latest macOS linker

fanquake commented at 1:30 pm on October 7, 2024: member

Testing master at 62e4516722115c2d5aeb6c197abc73ca7c078b23 and the fuzzing.md instructions:

 0cmake --preset=libfuzzer \   
 1   -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
 2   -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
 3   -DAPPEND_LDFLAGS=-Wl,-no_warn_duplicate_libraries
 4cmake --build build_fuzz
 5<snip>
 6[100%] Linking CXX executable fuzz
 7ld: multiple errors: invalid r_symbolnum=1 in '/Users/michael/fanquake-bitcoin/build_fuzz/src/test/fuzz/CMakeFiles/fuzz.dir/addition_overflow.cpp.o'; invalid r_symbolnum=1 in '/Users/michael/fanquake-bitcoin/build_fuzz/src/test/fuzz/CMakeFiles/fuzz.dir/fees.cpp.o'; invalid r_symbolnum=1 in '/Users/michael/fanquake-bitcoin/build_fuzz/src/test/fuzz/CMakeFiles/fuzz.dir/float.cpp.o'; invalid r_symbolnum=1 in '/Users/michael/fanquake-bitcoin/build_fuzz/src/test/fuzz/CMakeFiles/fuzz.dir/multiplication_overflow.cpp.o'; invalid r_symbolnum=1 in '../../libbitcoin_cli.a[2](stdin.cpp.o)'; invalid r_symbolnum=1 in '../../../libcrc32c.a[3](crc32c_portable.cc.o)'; invalid r_symbolnum=1 in '../../../libcrc32c_arm64.a[2](crc32c_arm64.cc.o)'; invalid r_symbolnum=1 in '../../libbitcoin_consensus.a[11](script_error.cpp.o)'; invalid r_symbolnum=1 in '../../crypto/libbitcoin_crypto.a[15](sha3.cpp.o)'; invalid r_symbolnum=5 in '../../util/libbitcoin_util.a[29](randomenv.cpp.o)'; invalid r_symbolnum=1 in '../../crypto/libbitcoin_crypto.a[10](poly1305.cpp.o)'; invalid r_symbolnum=18 in '../../crypto/libbitcoin_crypto_arm_shani.a[2](sha256_arm_shani.cpp.o)'; invalid r_symbolnum=1 in '../../crypto/libbitcoin_crypto.a[5](hex_base.cpp.o)'; invalid r_symbolnum=1 in '../../util/libbitcoin_util.a[27](logging.cpp.o)'; invalid r_symbolnum=1 in '../../util/libbitcoin_util.a[24](threadnames.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_consensus.a[5](hash.cpp.o)'; invalid r_symbolnum=1 in '../../../libleveldb.a[37](logging.cc.o)'; invalid r_symbolnum=1 in '../../../libleveldb.a[35](hash.cc.o)'; invalid r_symbolnum=1 in '../../../libleveldb.a[31](crc32c.cc.o)'; invalid r_symbolnum=1 in '../../../libleveldb.a[27](bloom.cc.o)'; invalid r_symbolnum=1 in '../../util/libbitcoin_util.a[16](serfloat.cpp.o)'; invalid r_symbolnum=1 in '../../util/libbitcoin_util.a[15](readwritefile.cpp.o)'; invalid r_symbolnum=1 in '../../util/libbitcoin_util.a[14](rbf.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[47](parsing.cpp.o)'; invalid r_symbolnum=1 in '../../util/libbitcoin_util.a[9](feefrac.cpp.o)'; invalid r_symbolnum=1 in '../../util/libbitcoin_util.a[6](chaintype.cpp.o)'; invalid r_symbolnum=1 in '../../../libcrc32c.a[2](crc32c.cc.o)'; invalid r_symbolnum=1 in '../../../libleveldb.a[8](filename.cc.o)'; invalid r_symbolnum=1 in '../../../libleveldb.a[2](builder.cc.o)'; invalid r_symbolnum=1 in '../util/libtest_util.a[12](str.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[42](request.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[41](rawtransaction_util.cpp.o)'; invalid r_symbolnum=1 in '../util/libtest_util.a[4](index.cpp.o)'; invalid r_symbolnum=1 in '../util/libtest_util.a[3](coins.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[84](torcontrol.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[79](server_util.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[30](merkleblock.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[29](key_io.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[24](deploymentinfo.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[21](compressor.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[20](url.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_common.a[16](run_command.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[68](pow.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[63](fees_args.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[55](psbt.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[54](peerman_args.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[51](miner.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[50](mempool_persist_args.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[43](database_args.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[41](connection_types.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[40](coins_view_args.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[39](coin.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[36](caches.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[34](blockmanager_args.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[31](net_processing.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[28](mempool_removal_reason.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[23](checks.cpp.o)'; invalid r_symbolnum=1 in '../../libbitcoin_node.a[22](chain.cpp.o)'
 8clang++: error: linker command failed with exit code 1 (use -v to see invocation)
 9gmake[2]: *** [src/test/fuzz/CMakeFiles/fuzz.dir/build.make:2186: src/test/fuzz/fuzz] Error 1
10gmake[1]: *** [CMakeFiles/Makefile2:1722: src/test/fuzz/CMakeFiles/fuzz.dir/all] Error 2

I’m not sure when this broke, but my assumption is that it’s an issue with a newer version of ld. Used here is:

0ld -v
1@(#)PROGRAM:ld PROJECT:ld-1115.7.3
2BUILD 13:29:00 Aug  9 2024
3configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
4will use ld-classic for: armv6 armv7 armv7s i386 armv6m armv7k armv7m armv7em
5LTO support using: LLVM version 16.0.0 (static support for 29, runtime is 29)
6TAPI support using: Apple TAPI version 16.0.0 (tapi-1600.0.11.8)

I tried compiling with -Wl,-ld_classic, which should use the older verison of the linker, but it hit an assertion:

 0[100%] Linking CXX executable fuzz
 1ld: warning: -ld_classic is deprecated and will be removed in a future release
 20  0x100483ee4  __assert_rtn + 160
 31  0x100485804  ld::tool::LinkEditAtom::size() const (.cold.1) + 0
 42  0x10035c200  ld::tool::OutputFile::addressOf(ld::Internal const&, ld::Fixup const*, ld::Atom const**) + 244
 53  0x10036b58c  ld::tool::OutputFile::buildChainedFixupInfo(ld::Internal&) + 1196
 64  0x1003702f4  ___ZN2ld4tool10OutputFile20buildLINKEDITContentERNS_8InternalE_block_invoke.408 + 28
 75  0x191fc28f8  _dispatch_call_block_and_release + 32
 86  0x191fc4658  _dispatch_client_callout + 20
 97  0x191fd6570  _dispatch_root_queue_drain + 996
108  0x191fd6b20  _dispatch_worker_thread2 + 156
119  0x19217339c  _pthread_wqthread + 228
12A linker snapshot was created at:
13	/tmp/fuzz-2024-10-07-142750.ld-snapshot
14ld: Assertion failed: (_mode == modeFinalAddress), function finalAddress, file ld.hpp, line 1462.
15clang++: error: linker command failed with exit code 1 (use -v to see invocation)

fanquake added the label macOS on Oct 7, 2024

fanquake renamed this:
~~build: macOS fuzz instructions broken on 15.x~~
build: macOS fuzz instructions broken using latest macOS linker
on Oct 7, 2024

maflcko commented at 2:25 pm on October 7, 2024: member

Does it also happen with llvm@16, instead of the current version?

maflcko added the label Build system on Oct 7, 2024

maflcko added the label Tests on Oct 7, 2024

fanquake commented at 2:39 pm on October 7, 2024: member

It works with llvm@18, so it looks like this could be an incompatibility with LLVM 19 (19.1.1) and Apples ld.

build: macOS fuzz instructions broken using latest macOS linker #31049