Build broken when enabling fuzzing on Apple M1 hw using homebrew llvm. #27550

Is there an existing issue for this?

• I have searched the existing issues

Current behaviour

I got same error in #24386

Expected behaviour

shoule be able to compolice fuzzer

Steps to reproduce

./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined --disable-asm CC=/opt/homebrew/opt/llvm/bin/clang CXX=/opt/homebrew/opt/llvm/bin/clang++

make
Making all in src
  CXXLD    test/fuzz/fuzz
Undefined symbols for architecture arm64:
  "crc32c::ExtendArm64(unsigned int, unsigned char const*, unsigned long)", referenced from:
      crc32c::Extend(unsigned int, unsigned char const*, unsigned long) in libcrc32c.a(libcrc32c_a-crc32c.o)
ld: symbol(s) not found for architecture arm64
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [test/fuzz/fuzz] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

/opt/homebrew/opt/llvm/bin/clang --version
Homebrew clang version 16.0.2
Target: arm64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin

Relevant log output

No response

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

v24.0.1

Operating system and version

MacOS: 13.3.1 (22E261)

Machine specifications

Chip Apple M1 Pro, 16GB memory

For testing and fuzzing I recommend master. Does it happen there as well?

This works for me on master. It's not clear how you're hitting that issue, because passing --disable-asm will skip the usage of the crc32c::ExtendArm64 code entirely.

Also, make sure to type make clean before make

Thanks, after I switch to master and run make clean I got configure: error: cannot find required auxiliary files: ar-lib not sure what is ar-lib, how can I install it? I googled but not much help

What is the config.log, or output?

What is the config.log, or output?

./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined --disable-asm CC=/opt/homebrew/opt/llvm/bin/clang CXX=/opt/homebrew/opt/llvm/bin/clang++
checking for pkg-config... /opt/homebrew/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking build system type... aarch64-apple-darwin22.4.0
checking host system type... aarch64-apple-darwin22.4.0
checking for a BSD-compatible install... /opt/homebrew/bin/ginstall -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /opt/homebrew/bin/gmkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... yes
checking whether make supports nested variables... (cached) yes
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C++... yes
checking whether /opt/homebrew/opt/llvm/bin/clang++ accepts -g... yes
checking for /opt/homebrew/opt/llvm/bin/clang++ option to enable C++11 features... none needed
checking whether make supports the include directive... yes (GNU style)
checking dependency style of /opt/homebrew/opt/llvm/bin/clang++... gcc3
checking whether /opt/homebrew/opt/llvm/bin/clang++ supports C++17 features with -std=c++17... yes
checking whether std::filesystem can be used without link library... yes
checking whether the compiler supports GNU Objective C++... yes
checking whether /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 accepts -g... yes
checking dependency style of /opt/homebrew/opt/llvm/bin/clang++ -std=c++17... gcc3
checking how to print strings... printf
checking for gcc... /opt/homebrew/opt/llvm/bin/clang
checking whether the compiler supports GNU C... yes
checking whether /opt/homebrew/opt/llvm/bin/clang accepts -g... yes
checking for /opt/homebrew/opt/llvm/bin/clang option to enable C11 features... none needed
checking whether /opt/homebrew/opt/llvm/bin/clang understands -c and -o together... yes
checking dependency style of /opt/homebrew/opt/llvm/bin/clang... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by /opt/homebrew/opt/llvm/bin/clang... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 786432
checking how to convert aarch64-apple-darwin22.4.0 file names to aarch64-apple-darwin22.4.0 format... func_convert_file_noop
checking how to convert aarch64-apple-darwin22.4.0 file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for file... file
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver [@FILE](/bitcoin-bitcoin/contributor/file/) support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from /opt/homebrew/opt/llvm/bin/clang object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if /opt/homebrew/opt/llvm/bin/clang supports -fno-rtti -fno-exceptions... yes
checking for /opt/homebrew/opt/llvm/bin/clang option to produce PIC... -fno-common -DPIC
checking if /opt/homebrew/opt/llvm/bin/clang PIC flag -fno-common -DPIC works... yes
checking if /opt/homebrew/opt/llvm/bin/clang static flag -static works... no
checking if /opt/homebrew/opt/llvm/bin/clang supports -c -o file.o... yes
checking if /opt/homebrew/opt/llvm/bin/clang supports -c -o file.o... (cached) yes
checking whether the /opt/homebrew/opt/llvm/bin/clang linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin22.4.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 -E
checking for ld used by /opt/homebrew/opt/llvm/bin/clang++ -std=c++17... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... no
checking whether the /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 linker (/usr/bin/ld) supports shared libraries... yes
checking for /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 option to produce PIC... -fno-common -DPIC
checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 PIC flag -fno-common -DPIC works... yes
checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 static flag -static works... no
checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 supports -c -o file.o... yes
checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 supports -c -o file.o... (cached) yes
checking whether the /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin22.4.0 dyld
checking how to hardcode library paths into programs... immediate
checking for ar... /usr/bin/ar
checking for gcov... /usr/bin/gcov
checking for llvm-cov... no
checking for lcov... no
checking for python3.8... no
checking for python3.9... /opt/homebrew/bin/python3.9
checking for genhtml... no
checking for git... /usr/bin/git
checking for ccache... no
checking for xgettext... /opt/homebrew/bin/xgettext
checking for hexdump... /usr/bin/hexdump
checking for objcopy... no
checking for doxygen... no
checking whether C++ compiler accepts -Werror... yes
checking whether the linker accepts -Wl,-fatal_warnings... yes
checking whether C++ compiler accepts -fsanitize=fuzzer,address,undefined... yes
checking whether the linker accepts -fsanitize=fuzzer,address,undefined... yes
checking whether C++ compiler accepts -Wall... yes
checking whether C++ compiler accepts -Wextra... yes
checking whether C++ compiler accepts -Wgnu... yes
checking whether C++ compiler accepts -Wformat -Wformat-security... yes
checking whether C++ compiler accepts -Wvla... yes
checking whether C++ compiler accepts -Wshadow-field... yes
checking whether C++ compiler accepts -Wthread-safety... yes
checking whether C++ compiler accepts -Wloop-analysis... yes
checking whether C++ compiler accepts -Wredundant-decls... yes
checking whether C++ compiler accepts -Wunused-member-function... yes
checking whether C++ compiler accepts -Wdate-time... yes
checking whether C++ compiler accepts -Wconditional-uninitialized... yes
checking whether C++ compiler accepts -Wduplicated-branches... no
checking whether C++ compiler accepts -Wduplicated-cond... no
checking whether C++ compiler accepts -Wlogical-op... no
checking whether C++ compiler accepts -Woverloaded-virtual... yes
checking whether C++ compiler accepts -Wsuggest-override... yes
checking whether C++ compiler accepts -Wunreachable-code-loop-increment... yes
checking whether C++ compiler accepts -Wimplicit-fallthrough... yes
checking whether C++ compiler accepts -Wunused-parameter... yes
checking whether C++ compiler accepts -Wself-assign... yes
checking whether C++ compiler accepts -Wdeprecated-copy... yes
checking whether C++ compiler accepts -fno-extended-identifiers... no
checking for brew... brew
checking whether the linker accepts -Wl,-headerpad_max_install_names... yes
checking whether byte ordering is bigendian... no
checking how to run the C preprocessor... /opt/homebrew/opt/llvm/bin/clang -E
checking whether /opt/homebrew/opt/llvm/bin/clang is Clang... yes
checking whether pthreads work with "-pthread" and "-lpthread"... yes
checking whether Clang needs flag to prevent "argument unused" warning when linking with -pthread... no
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking whether more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
checking whether std::atomic can be used without link library... yes
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 options needed to detect all undeclared functions... none needed
checking whether strerror_r is declared... yes
checking whether strerror_r returns char *... no
checking for library containing clock_gettime... none required
checking whether C++ compiler accepts -fPIC... yes
checking whether C++ compiler accepts -fstack-reuse=none... no
checking whether C++ compiler accepts -Wstack-protector... yes
checking whether C++ compiler accepts -fstack-protector-all... yes
checking whether C++ compiler accepts -fcf-protection=full... no
checking whether C++ compiler accepts -fstack-clash-protection... no
checking whether C++ preprocessor accepts -D_FORTIFY_SOURCE=3... yes
checking whether C++ preprocessor accepts -U_FORTIFY_SOURCE... yes
checking whether the linker accepts -Wl,--enable-reloc-section... no
checking whether the linker accepts -Wl,--dynamicbase... no
checking whether the linker accepts -Wl,--nxcompat... no
checking whether the linker accepts -Wl,--high-entropy-va... no
checking whether the linker accepts -Wl,-z,relro... no
checking whether the linker accepts -Wl,-z,now... no
checking whether the linker accepts -Wl,-z,separate-code... no
checking whether the linker accepts -fPIE -pie... no
checking whether the linker accepts -Wl,-dead_strip... yes
checking whether the linker accepts -Wl,-dead_strip_dylibs... yes
checking whether the linker accepts -Wl,-bind_at_load... yes
checking for endian.h... no
checking for sys/endian.h... no
checking for byteswap.h... no
checking for sys/select.h... yes
checking for sys/prctl.h... no
checking for sys/sysctl.h... yes
checking for vm/vm_param.h... no
checking for sys/vmmeter.h... yes
checking for sys/resources.h... no
checking whether getifaddrs is declared... yes
checking whether ifaddrs funcs can be used without link library... yes
checking whether freeifaddrs is declared... yes
checking whether ifaddrs funcs can be used without link library... yes
checking whether fork is declared... yes
checking whether setsid is declared... yes
checking whether pipe2 is declared... no
checking for timingsafe_bcmp... yes
checking whether le16toh is declared... no
checking whether le32toh is declared... no
checking whether le64toh is declared... no
checking whether htole16 is declared... no
checking whether htole32 is declared... no
checking whether htole64 is declared... no
checking whether be16toh is declared... no
checking whether be32toh is declared... no
checking whether be64toh is declared... no
checking whether htobe16 is declared... no
checking whether htobe32 is declared... no
checking whether htobe64 is declared... no
checking whether bswap_16 is declared... no
checking whether bswap_32 is declared... no
checking whether bswap_64 is declared... no
checking for __builtin_clzl... yes
checking for __builtin_clzll... yes
checking for getmemoryinfo... no
checking for mallopt M_ARENA_MAX... no
checking for posix_fallocate... no
checking for default visibility attribute... yes
checking for dllexport attribute... no
checking for thread_local support... yes
checking for gmtime_r... yes
checking for Linux getrandom syscall... no
checking for getentropy via random.h... yes
checking for sysctl... yes
checking for sysctl KERN_ARND... no
checking for if type char equals int8_t... no
checking for fdatasync... no
checking for F_FULLFSYNC... yes
checking for O_CLOEXEC... yes
checking for __builtin_prefetch... yes
checking for _mm_prefetch... no
checking for strong getauxval support in the system headers... no
checking for std::system... yes
checking for ::_wsystem... no
configure: WARNING: enable-fuzz will disable all other targets and force --enable-fuzz-binary=yes
checking whether C++ preprocessor accepts -DABORT_ON_FAILED_ASSUME... yes
checking whether main function is needed for fuzz binary... checking whether the linker accepts ... yes
no
checking for __builtin_mul_overflow... yes
checking for sqlite3 >= 3.7.17... yes
checking whether to build wallet with support for sqlite... yes
checking whether Userspace, Statically Defined Tracing tracepoints are supported... no
checking for Boost headers >= 1.64.0 (106400)... yes
checking whether C++ preprocessor accepts -DBOOST_NO_CXX98_FUNCTION_BASE... no
configure: WARNING: Specifying --with-sanitizers forces --without-seccomp since the sanitizers introduce use of syscalls not allowed by the bitcoind syscall sandbox (-sandbox=<mode>).
checking for libevent >= 2.1.8... yes
checking for libevent_pthreads >= 2.1.8... yes
checking if evhttp_connection_get_peer expects const char**... no
checking for libmultiprocess... no
checking whether to build bitcoind... no
checking whether to build bitcoin-cli... no
checking whether to build bitcoin-tx... no
checking whether to build bitcoin-wallet... no
checking whether to build bitcoin-util... no
checking whether to build experimental bitcoin-chainstate... no
checking whether to build libraries... no
checking if ccache should be used... no
checking if wallet should be enabled... yes
checking whether to build with support for UPnP... no
checking whether to build with support for NAT-PMP... no
checking whether to build test_bitcoin... no
checking whether to reduce exports... no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating doc/man/Makefile
config.status: creating share/setup.nsi
config.status: creating share/qt/Info.plist
config.status: creating test/config.ini
config.status: creating contrib/devtools/split-debug.sh
config.status: creating src/config/bitcoin-config.h
config.status: src/config/bitcoin-config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
=== configuring in src/secp256k1 (/Users/steven/workspace/liuyangc3/bitcoin/src/secp256k1)
configure: running /bin/sh ./configure --disable-option-checking '--prefix=/usr/local'  '--enable-fuzz' '--with-sanitizers=fuzzer,address,undefined' '--disable-asm' 'CC=/opt/homebrew/opt/llvm/bin/clang' 'CXX=/opt/homebrew/opt/llvm/bin/clang++' '--disable-shared' '--with-pic' '--enable-benchmark=no' '--enable-module-recovery' '--disable-module-ecdh' --cache-file=/dev/null --srcdir=.
configure: error: cannot find required auxiliary files: ar-lib
configure: error: ./configure failed for src/secp256k1

not sure what is ar-lib, how can I install it? I googled but not much help

This is definitely a local issue. Make sure your tree is entirely clean. Could try a git clean -fxd, and then ./autogen.sh, ./configure ... etc.

Yeah thanks for the advice, I will try more in local before asking help here. maybe is not the bitcoin code issue. I will get back here later and thanks again 🙏

Ok. We'll close this for now. Comment back when you've had a chance to test, and we can reopen if need be.

I'm trying to run a build on Fedora 36 and I'm seeing the same error. When I run ./configure I see the following:

configure: WARNING: Bitcoin Core requires this library for BDB (legacy) wallet support configure: WARNING: Passing --without-bdb will suppress this warning checking for SQLITE... yes checking whether to build wallet with support for sqlite... yes checking whether Userspace, Statically Defined Tracing tracepoints are supported... no checking for natpmp.h... yes checking for initnatpmp in -lnatpmp... yes checking for boostlib >= 1.64.0 (106400)... yes checking whether Boost.Process can be used... yes checking for seccomp-bpf (Linux x86-64)... yes checking for EVENT... yes checking for EVENT_PTHREADS... yes checking if evhttp_connection_get_peer expects const char**... no checking for ZMQ... yes checking for LIBMULTIPROCESS... no checking whether to build bitcoind... yes checking whether to build bitcoin-cli... yes checking whether to build bitcoin-tx... yes checking whether to build bitcoin-wallet... yes checking whether to build bitcoin-util... yes checking whether to build experimental bitcoin-chainstate... no checking whether to build libraries... yes checking if ccache should be used... yes checking whether C compiler accepts -fdebug-prefix-map=A=B... yes checking whether C preprocessor accepts -fmacro-prefix-map=A=B... yes checking if wallet should be enabled... yes checking whether to build with support for UPnP... no checking whether to build with support for NAT-PMP... yes checking whether to build with NAT-PMP enabled by default... no checking whether to build test_bitcoin... yes checking whether to reduce exports... no checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating libbitcoinconsensus.pc config.status: creating Makefile config.status: creating src/Makefile config.status: creating doc/man/Makefile config.status: creating share/setup.nsi config.status: creating share/qt/Info.plist config.status: creating test/config.ini config.status: creating contrib/devtools/split-debug.sh config.status: creating src/config/bitcoin-config.h config.status: src/config/bitcoin-config.h is unchanged config.status: executing depfiles commands config.status: executing libtool commands === configuring in src/secp256k1 (/home/kau/Code/Bitcoin/bitcoin/src/secp256k1) configure: running /bin/sh ./configure --disable-option-checking '--prefix=/usr/local' '--without-miniupnpc' '--disable-bench' '--without-gui' '--disable-shared' '--with-pic' '--enable-benchmark=no' '--enable-module-recovery' '--enable-module-schnorrsig' --cache-file=/dev/null --srcdir=. configure: error: cannot find required auxiliary files: compile missing configure: error: ./configure failed for src/secp256k1

Would be appreciate if you can guide if this error is because of a recent change.