Build broken when enabling fuzzing on Apple M1 hw using homebrew llvm.

liuyangc3 commented at 9:29 am on May 2, 2023: none

Is there an existing issue for this?

I have searched the existing issues

Current behaviour

I got same error in #24386

Expected behaviour

shoule be able to compolice fuzzer

Steps to reproduce

 0./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined --disable-asm CC=/opt/homebrew/opt/llvm/bin/clang CXX=/opt/homebrew/opt/llvm/bin/clang++
 1
 2make
 3Making all in src
 4  CXXLD    test/fuzz/fuzz
 5Undefined symbols for architecture arm64:
 6  "crc32c::ExtendArm64(unsigned int, unsigned char const*, unsigned long)", referenced from:
 7      crc32c::Extend(unsigned int, unsigned char const*, unsigned long) in libcrc32c.a(libcrc32c_a-crc32c.o)
 8ld: symbol(s) not found for architecture arm64
 9clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
10make[2]: *** [test/fuzz/fuzz] Error 1
11make[1]: *** [all-recursive] Error 1
12make: *** [all-recursive] Error 1

0/opt/homebrew/opt/llvm/bin/clang --version
1Homebrew clang version 16.0.2
2Target: arm64-apple-darwin22.4.0
3Thread model: posix
4InstalledDir: /opt/homebrew/opt/llvm/bin

Relevant log output

No response

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

v24.0.1

Operating system and version

MacOS: 13.3.1 (22E261)

Machine specifications

Chip Apple M1 Pro, 16GB memory

maflcko added the label macOS on May 2, 2023

maflcko added the label Build system on May 2, 2023

maflcko added the label Tests on May 2, 2023

maflcko commented at 10:25 am on May 2, 2023: member

For testing and fuzzing I recommend master. Does it happen there as well?

fanquake commented at 11:22 am on May 2, 2023: member

This works for me on master. It’s not clear how you’re hitting that issue, because passing --disable-asm will skip the usage of the crc32c::ExtendArm64 code entirely.

maflcko commented at 11:29 am on May 2, 2023: member

Also, make sure to type make clean before make

maflcko added the label Questions and Help on May 2, 2023

liuyangc3 commented at 11:42 am on May 2, 2023: none

Thanks, after I switch to master and run make clean I got configure: error: cannot find required auxiliary files: ar-lib not sure what is ar-lib, how can I install it? I googled but not much help

maflcko commented at 11:49 am on May 2, 2023: member

What is the config.log, or output?

liuyangc3 commented at 11:52 am on May 2, 2023: none

What is the config.log, or output?

  0./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined --disable-asm CC=/opt/homebrew/opt/llvm/bin/clang CXX=/opt/homebrew/opt/llvm/bin/clang++
  1checking for pkg-config... /opt/homebrew/bin/pkg-config
  2checking pkg-config is at least version 0.9.0... yes
  3checking build system type... aarch64-apple-darwin22.4.0
  4checking host system type... aarch64-apple-darwin22.4.0
  5checking for a BSD-compatible install... /opt/homebrew/bin/ginstall -c
  6checking whether build environment is sane... yes
  7checking for a race-free mkdir -p... /opt/homebrew/bin/gmkdir -p
  8checking for gawk... gawk
  9checking whether make sets $(MAKE)... yes
 10checking whether make supports nested variables... yes
 11checking whether to enable maintainer-specific portions of Makefiles... yes
 12checking whether make supports nested variables... (cached) yes
 13checking whether the C++ compiler works... yes
 14checking for C++ compiler default output file name... a.out
 15checking for suffix of executables...
 16checking whether we are cross compiling... no
 17checking for suffix of object files... o
 18checking whether the compiler supports GNU C++... yes
 19checking whether /opt/homebrew/opt/llvm/bin/clang++ accepts -g... yes
 20checking for /opt/homebrew/opt/llvm/bin/clang++ option to enable C++11 features... none needed
 21checking whether make supports the include directive... yes (GNU style)
 22checking dependency style of /opt/homebrew/opt/llvm/bin/clang++... gcc3
 23checking whether /opt/homebrew/opt/llvm/bin/clang++ supports C++17 features with -std=c++17... yes
 24checking whether std::filesystem can be used without link library... yes
 25checking whether the compiler supports GNU Objective C++... yes
 26checking whether /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 accepts -g... yes
 27checking dependency style of /opt/homebrew/opt/llvm/bin/clang++ -std=c++17... gcc3
 28checking how to print strings... printf
 29checking for gcc... /opt/homebrew/opt/llvm/bin/clang
 30checking whether the compiler supports GNU C... yes
 31checking whether /opt/homebrew/opt/llvm/bin/clang accepts -g... yes
 32checking for /opt/homebrew/opt/llvm/bin/clang option to enable C11 features... none needed
 33checking whether /opt/homebrew/opt/llvm/bin/clang understands -c and -o together... yes
 34checking dependency style of /opt/homebrew/opt/llvm/bin/clang... gcc3
 35checking for a sed that does not truncate output... /usr/bin/sed
 36checking for grep that handles long lines and -e... /usr/bin/grep
 37checking for egrep... /usr/bin/grep -E
 38checking for fgrep... /usr/bin/grep -F
 39checking for ld used by /opt/homebrew/opt/llvm/bin/clang... /usr/bin/ld
 40checking if the linker (/usr/bin/ld) is GNU ld... no
 41checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
 42checking the name lister (/usr/bin/nm -B) interface... BSD nm
 43checking whether ln -s works... yes
 44checking the maximum length of command line arguments... 786432
 45checking how to convert aarch64-apple-darwin22.4.0 file names to aarch64-apple-darwin22.4.0 format... func_convert_file_noop
 46checking how to convert aarch64-apple-darwin22.4.0 file names to toolchain format... func_convert_file_noop
 47checking for /usr/bin/ld option to reload object files... -r
 48checking for file... file
 49checking for objdump... objdump
 50checking how to recognize dependent libraries... pass_all
 51checking for dlltool... no
 52checking how to associate runtime and link libraries... printf %s\n
 53checking for ar... ar
 54checking for archiver [@FILE](/bitcoin-bitcoin/contributor/file/) support... no
 55checking for strip... strip
 56checking for ranlib... ranlib
 57checking command to parse /usr/bin/nm -B output from /opt/homebrew/opt/llvm/bin/clang object... ok
 58checking for sysroot... no
 59checking for a working dd... /bin/dd
 60checking how to truncate binary pipes... /bin/dd bs=4096 count=1
 61checking for mt... no
 62checking if : is a manifest tool... no
 63checking for dsymutil... dsymutil
 64checking for nmedit... nmedit
 65checking for lipo... lipo
 66checking for otool... otool
 67checking for otool64... no
 68checking for -single_module linker flag... yes
 69checking for -exported_symbols_list linker flag... yes
 70checking for -force_load linker flag... yes
 71checking for stdio.h... yes
 72checking for stdlib.h... yes
 73checking for string.h... yes
 74checking for inttypes.h... yes
 75checking for stdint.h... yes
 76checking for strings.h... yes
 77checking for sys/stat.h... yes
 78checking for sys/types.h... yes
 79checking for unistd.h... yes
 80checking for dlfcn.h... yes
 81checking for objdir... .libs
 82checking if /opt/homebrew/opt/llvm/bin/clang supports -fno-rtti -fno-exceptions... yes
 83checking for /opt/homebrew/opt/llvm/bin/clang option to produce PIC... -fno-common -DPIC
 84checking if /opt/homebrew/opt/llvm/bin/clang PIC flag -fno-common -DPIC works... yes
 85checking if /opt/homebrew/opt/llvm/bin/clang static flag -static works... no
 86checking if /opt/homebrew/opt/llvm/bin/clang supports -c -o file.o... yes
 87checking if /opt/homebrew/opt/llvm/bin/clang supports -c -o file.o... (cached) yes
 88checking whether the /opt/homebrew/opt/llvm/bin/clang linker (/usr/bin/ld) supports shared libraries... yes
 89checking dynamic linker characteristics... darwin22.4.0 dyld
 90checking how to hardcode library paths into programs... immediate
 91checking whether stripping libraries is possible... yes
 92checking if libtool supports shared libraries... yes
 93checking whether to build shared libraries... yes
 94checking whether to build static libraries... yes
 95checking how to run the C++ preprocessor... /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 -E
 96checking for ld used by /opt/homebrew/opt/llvm/bin/clang++ -std=c++17... /usr/bin/ld
 97checking if the linker (/usr/bin/ld) is GNU ld... no
 98checking whether the /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 linker (/usr/bin/ld) supports shared libraries... yes
 99checking for /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 option to produce PIC... -fno-common -DPIC
100checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 PIC flag -fno-common -DPIC works... yes
101checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 static flag -static works... no
102checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 supports -c -o file.o... yes
103checking if /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 supports -c -o file.o... (cached) yes
104checking whether the /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 linker (/usr/bin/ld) supports shared libraries... yes
105checking dynamic linker characteristics... darwin22.4.0 dyld
106checking how to hardcode library paths into programs... immediate
107checking for ar... /usr/bin/ar
108checking for gcov... /usr/bin/gcov
109checking for llvm-cov... no
110checking for lcov... no
111checking for python3.8... no
112checking for python3.9... /opt/homebrew/bin/python3.9
113checking for genhtml... no
114checking for git... /usr/bin/git
115checking for ccache... no
116checking for xgettext... /opt/homebrew/bin/xgettext
117checking for hexdump... /usr/bin/hexdump
118checking for objcopy... no
119checking for doxygen... no
120checking whether C++ compiler accepts -Werror... yes
121checking whether the linker accepts -Wl,-fatal_warnings... yes
122checking whether C++ compiler accepts -fsanitize=fuzzer,address,undefined... yes
123checking whether the linker accepts -fsanitize=fuzzer,address,undefined... yes
124checking whether C++ compiler accepts -Wall... yes
125checking whether C++ compiler accepts -Wextra... yes
126checking whether C++ compiler accepts -Wgnu... yes
127checking whether C++ compiler accepts -Wformat -Wformat-security... yes
128checking whether C++ compiler accepts -Wvla... yes
129checking whether C++ compiler accepts -Wshadow-field... yes
130checking whether C++ compiler accepts -Wthread-safety... yes
131checking whether C++ compiler accepts -Wloop-analysis... yes
132checking whether C++ compiler accepts -Wredundant-decls... yes
133checking whether C++ compiler accepts -Wunused-member-function... yes
134checking whether C++ compiler accepts -Wdate-time... yes
135checking whether C++ compiler accepts -Wconditional-uninitialized... yes
136checking whether C++ compiler accepts -Wduplicated-branches... no
137checking whether C++ compiler accepts -Wduplicated-cond... no
138checking whether C++ compiler accepts -Wlogical-op... no
139checking whether C++ compiler accepts -Woverloaded-virtual... yes
140checking whether C++ compiler accepts -Wsuggest-override... yes
141checking whether C++ compiler accepts -Wunreachable-code-loop-increment... yes
142checking whether C++ compiler accepts -Wimplicit-fallthrough... yes
143checking whether C++ compiler accepts -Wunused-parameter... yes
144checking whether C++ compiler accepts -Wself-assign... yes
145checking whether C++ compiler accepts -Wdeprecated-copy... yes
146checking whether C++ compiler accepts -fno-extended-identifiers... no
147checking for brew... brew
148checking whether the linker accepts -Wl,-headerpad_max_install_names... yes
149checking whether byte ordering is bigendian... no
150checking how to run the C preprocessor... /opt/homebrew/opt/llvm/bin/clang -E
151checking whether /opt/homebrew/opt/llvm/bin/clang is Clang... yes
152checking whether pthreads work with "-pthread" and "-lpthread"... yes
153checking whether Clang needs flag to prevent "argument unused" warning when linking with -pthread... no
154checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
155checking whether more special flags are required for pthreads... no
156checking for PTHREAD_PRIO_INHERIT... yes
157checking whether std::atomic can be used without link library... yes
158checking for special C compiler options needed for large files... no
159checking for _FILE_OFFSET_BITS value needed for large files... no
160checking for /opt/homebrew/opt/llvm/bin/clang++ -std=c++17 options needed to detect all undeclared functions... none needed
161checking whether strerror_r is declared... yes
162checking whether strerror_r returns char *... no
163checking for library containing clock_gettime... none required
164checking whether C++ compiler accepts -fPIC... yes
165checking whether C++ compiler accepts -fstack-reuse=none... no
166checking whether C++ compiler accepts -Wstack-protector... yes
167checking whether C++ compiler accepts -fstack-protector-all... yes
168checking whether C++ compiler accepts -fcf-protection=full... no
169checking whether C++ compiler accepts -fstack-clash-protection... no
170checking whether C++ preprocessor accepts -D_FORTIFY_SOURCE=3... yes
171checking whether C++ preprocessor accepts -U_FORTIFY_SOURCE... yes
172checking whether the linker accepts -Wl,--enable-reloc-section... no
173checking whether the linker accepts -Wl,--dynamicbase... no
174checking whether the linker accepts -Wl,--nxcompat... no
175checking whether the linker accepts -Wl,--high-entropy-va... no
176checking whether the linker accepts -Wl,-z,relro... no
177checking whether the linker accepts -Wl,-z,now... no
178checking whether the linker accepts -Wl,-z,separate-code... no
179checking whether the linker accepts -fPIE -pie... no
180checking whether the linker accepts -Wl,-dead_strip... yes
181checking whether the linker accepts -Wl,-dead_strip_dylibs... yes
182checking whether the linker accepts -Wl,-bind_at_load... yes
183checking for endian.h... no
184checking for sys/endian.h... no
185checking for byteswap.h... no
186checking for sys/select.h... yes
187checking for sys/prctl.h... no
188checking for sys/sysctl.h... yes
189checking for vm/vm_param.h... no
190checking for sys/vmmeter.h... yes
191checking for sys/resources.h... no
192checking whether getifaddrs is declared... yes
193checking whether ifaddrs funcs can be used without link library... yes
194checking whether freeifaddrs is declared... yes
195checking whether ifaddrs funcs can be used without link library... yes
196checking whether fork is declared... yes
197checking whether setsid is declared... yes
198checking whether pipe2 is declared... no
199checking for timingsafe_bcmp... yes
200checking whether le16toh is declared... no
201checking whether le32toh is declared... no
202checking whether le64toh is declared... no
203checking whether htole16 is declared... no
204checking whether htole32 is declared... no
205checking whether htole64 is declared... no
206checking whether be16toh is declared... no
207checking whether be32toh is declared... no
208checking whether be64toh is declared... no
209checking whether htobe16 is declared... no
210checking whether htobe32 is declared... no
211checking whether htobe64 is declared... no
212checking whether bswap_16 is declared... no
213checking whether bswap_32 is declared... no
214checking whether bswap_64 is declared... no
215checking for __builtin_clzl... yes
216checking for __builtin_clzll... yes
217checking for getmemoryinfo... no
218checking for mallopt M_ARENA_MAX... no
219checking for posix_fallocate... no
220checking for default visibility attribute... yes
221checking for dllexport attribute... no
222checking for thread_local support... yes
223checking for gmtime_r... yes
224checking for Linux getrandom syscall... no
225checking for getentropy via random.h... yes
226checking for sysctl... yes
227checking for sysctl KERN_ARND... no
228checking for if type char equals int8_t... no
229checking for fdatasync... no
230checking for F_FULLFSYNC... yes
231checking for O_CLOEXEC... yes
232checking for __builtin_prefetch... yes
233checking for _mm_prefetch... no
234checking for strong getauxval support in the system headers... no
235checking for std::system... yes
236checking for ::_wsystem... no
237configure: WARNING: enable-fuzz will disable all other targets and force --enable-fuzz-binary=yes
238checking whether C++ preprocessor accepts -DABORT_ON_FAILED_ASSUME... yes
239checking whether main function is needed for fuzz binary... checking whether the linker accepts ... yes
240no
241checking for __builtin_mul_overflow... yes
242checking for sqlite3 >= 3.7.17... yes
243checking whether to build wallet with support for sqlite... yes
244checking whether Userspace, Statically Defined Tracing tracepoints are supported... no
245checking for Boost headers >= 1.64.0 (106400)... yes
246checking whether C++ preprocessor accepts -DBOOST_NO_CXX98_FUNCTION_BASE... no
247configure: WARNING: Specifying --with-sanitizers forces --without-seccomp since the sanitizers introduce use of syscalls not allowed by the bitcoind syscall sandbox (-sandbox=<mode>).
248checking for libevent >= 2.1.8... yes
249checking for libevent_pthreads >= 2.1.8... yes
250checking if evhttp_connection_get_peer expects const char**... no
251checking for libmultiprocess... no
252checking whether to build bitcoind... no
253checking whether to build bitcoin-cli... no
254checking whether to build bitcoin-tx... no
255checking whether to build bitcoin-wallet... no
256checking whether to build bitcoin-util... no
257checking whether to build experimental bitcoin-chainstate... no
258checking whether to build libraries... no
259checking if ccache should be used... no
260checking if wallet should be enabled... yes
261checking whether to build with support for UPnP... no
262checking whether to build with support for NAT-PMP... no
263checking whether to build test_bitcoin... no
264checking whether to reduce exports... no
265checking that generated files are newer than configure... done
266configure: creating ./config.status
267config.status: creating Makefile
268config.status: creating src/Makefile
269config.status: creating doc/man/Makefile
270config.status: creating share/setup.nsi
271config.status: creating share/qt/Info.plist
272config.status: creating test/config.ini
273config.status: creating contrib/devtools/split-debug.sh
274config.status: creating src/config/bitcoin-config.h
275config.status: src/config/bitcoin-config.h is unchanged
276config.status: executing depfiles commands
277config.status: executing libtool commands
278=== configuring in src/secp256k1 (/Users/steven/workspace/liuyangc3/bitcoin/src/secp256k1)
279configure: running /bin/sh ./configure --disable-option-checking '--prefix=/usr/local'  '--enable-fuzz' '--with-sanitizers=fuzzer,address,undefined' '--disable-asm' 'CC=/opt/homebrew/opt/llvm/bin/clang' 'CXX=/opt/homebrew/opt/llvm/bin/clang++' '--disable-shared' '--with-pic' '--enable-benchmark=no' '--enable-module-recovery' '--disable-module-ecdh' --cache-file=/dev/null --srcdir=.
280configure: error: cannot find required auxiliary files: ar-lib
281configure: error: ./configure failed for src/secp256k1

fanquake commented at 12:21 pm on May 2, 2023: member

not sure what is ar-lib, how can I install it? I googled but not much help

This is definitely a local issue. Make sure your tree is entirely clean. Could try a git clean -fxd, and then ./autogen.sh, ./configure ... etc.

liuyangc3 commented at 1:18 pm on May 2, 2023: none

Yeah thanks for the advice, I will try more in local before asking help here. maybe is not the bitcoin code issue. I will get back here later and thanks again 🙏

fanquake commented at 1:19 pm on May 2, 2023: member

Ok. We’ll close this for now. Comment back when you’ve had a chance to test, and we can reopen if need be.

fanquake closed this on May 2, 2023

desibitcoiner commented at 5:57 am on June 3, 2023: none

I’m trying to run a build on Fedora 36 and I’m seeing the same error. When I run ./configure I see the following:

configure: WARNING: Bitcoin Core requires this library for BDB (legacy) wallet support configure: WARNING: Passing –without-bdb will suppress this warning checking for SQLITE… yes checking whether to build wallet with support for sqlite… yes checking whether Userspace, Statically Defined Tracing tracepoints are supported… no checking for natpmp.h… yes checking for initnatpmp in -lnatpmp… yes checking for boostlib >= 1.64.0 (106400)… yes checking whether Boost.Process can be used… yes checking for seccomp-bpf (Linux x86-64)… yes checking for EVENT… yes checking for EVENT_PTHREADS… yes checking if evhttp_connection_get_peer expects const char**… no checking for ZMQ… yes checking for LIBMULTIPROCESS… no checking whether to build bitcoind… yes checking whether to build bitcoin-cli… yes checking whether to build bitcoin-tx… yes checking whether to build bitcoin-wallet… yes checking whether to build bitcoin-util… yes checking whether to build experimental bitcoin-chainstate… no checking whether to build libraries… yes checking if ccache should be used… yes checking whether C compiler accepts -fdebug-prefix-map=A=B… yes checking whether C preprocessor accepts -fmacro-prefix-map=A=B… yes checking if wallet should be enabled… yes checking whether to build with support for UPnP… no checking whether to build with support for NAT-PMP… yes checking whether to build with NAT-PMP enabled by default… no checking whether to build test_bitcoin… yes checking whether to reduce exports… no checking that generated files are newer than configure… done configure: creating ./config.status config.status: creating libbitcoinconsensus.pc config.status: creating Makefile config.status: creating src/Makefile config.status: creating doc/man/Makefile config.status: creating share/setup.nsi config.status: creating share/qt/Info.plist config.status: creating test/config.ini config.status: creating contrib/devtools/split-debug.sh config.status: creating src/config/bitcoin-config.h config.status: src/config/bitcoin-config.h is unchanged config.status: executing depfiles commands config.status: executing libtool commands === configuring in src/secp256k1 (/home/kau/Code/Bitcoin/bitcoin/src/secp256k1) configure: running /bin/sh ./configure –disable-option-checking ‘–prefix=/usr/local’ ‘–without-miniupnpc’ ‘–disable-bench’ ‘–without-gui’ ‘–disable-shared’ ‘–with-pic’ ‘–enable-benchmark=no’ ‘–enable-module-recovery’ ‘–enable-module-schnorrsig’ –cache-file=/dev/null –srcdir=. configure: error: cannot find required auxiliary files: compile missing configure: error: ./configure failed for src/secp256k1

Would be appreciate if you can guide if this error is because of a recent change.

bitcoin locked this on Jun 2, 2024

Build broken when enabling fuzzing on Apple M1 hw using homebrew llvm. #27550