depends: llvm-ranlib (etc): "No such file or directory" on Intel macOS 15.0

Sjors commented at 12:52 pm on September 26, 2024: member

Tried on Intel macOS 15.0 (Xcode 16.0) and 13.7 (Xcode 15.2):

0$ cd depends
1$ make
2/bin/sh: command -v llvm-ranlib: No such file or directory
3/bin/sh: command -v llvm-strip: No such file or directory
4/bin/sh: command -v llvm-nm: No such file or directory
5/bin/sh: command -v llvm-objdump: No such file or directory
6/bin/sh: command -v dsymutil: No such file or directory
7Fetching libevent-2.1.12-stable.tar.gz from ...

It does seem to finish building dependencies, but I haven’t tested if they actually work.

I do have /usr/bin/{ranlib,strip,nm,objdump,dsymutil}.

itornaza commented at 4:46 pm on September 26, 2024: contributor

I replicated the same on Apple silicon (Apple M2 Max) and the same setup (macos 15.0 and Xcode 16.0). Also checked that /usr/bin/{ranlib,strip,nm,objdump,dsymutil} are present on my system. The compilation process successfully completes.

Also, when I try make download-osx, I get the following related ouput

0/bin/sh: command -v llvm-ranlib: No such file or directory
1/bin/sh: command -v llvm-strip: No such file or directory
2/bin/sh: command -v llvm-nm: No such file or directory
3/bin/sh: command -v llvm-objdump: No such file or directory
4/bin/sh: command -v dsymutil: No such file or directory
5/bin/sh: command -v llvm-ranlib: No such file or directory
6/bin/sh: command -v llvm-strip: No such file or directory
7/bin/sh: command -v llvm-nm: No such file or directory
8/bin/sh: command -v llvm-objdump: No such file or directory
9/bin/sh: command -v dsymutil: No such file or directory

itornaza commented at 5:51 pm on September 27, 2024: contributor

Following the macOS Build Guide I am using the llvm installed from brew and in my .zshrc I have an export to override the default macos llvm export PATH="$(brew --prefix)/opt/llvm/bin:$PATH".

Using the which command with the missing tools, I get the corresponding tool paths that are of course not the ones in the usr/bin/ directory.

 0% which llvm-ranlib
 1/opt/homebrew/opt/llvm/bin/llvm-ranlib
 2
 3% which llvm-strip
 4/opt/homebrew/opt/llvm/bin/llvm-strip
 5
 6% which llvm-nm
 7/opt/homebrew/opt/llvm/bin/llvm-nm
 8
 9% which llvm-objdump
10/opt/homebrew/opt/llvm/bin/llvm-objdump
11
12% which dsymutil
13/opt/homebrew/opt/llvm/bin/dsymutil

According to this observation I could think of the following solutions that all of them do the compilation and do not display the error messages.

Solution 1

Running make with the tools defined inline.

RANLIB=llvm-ranlib STRIP=llvm-strip NM=llvm-nm OBJDUMP=llvm-objdump DSYMUTIL=dsymutil make

Solution 2

Define the tools in the ~/.zshrc file:

0export RANLIB=llvm-ranlib
1export STRIP=llvm-strip
2export NM=llvm-nm
3export OBJDUMP=llvm-objdump
4export DSYMUTIL=dsymutil

Solution 3

Define them in our ./depends/Makefile:

0RANLIB := llvm-ranlib
1STRIP := llvm-strip
2NM := llvm-nm
3OBJDUMP := llvm-objdump
4DSYMUTIL := dsymutil

fanquake commented at 9:41 am on October 2, 2024: member

Tried on Intel macOS 15.0 (Xcode 16.0) and 13.7 (Xcode 15.2):

Tried which code? If this was a clean build, according to the log, it’s also skipped building Boost for some reason?

This looks odd, because you seem to be hitting code paths for macOS cross-compilation, when building natively. i.e when building on macOS, the tooling should be found by calling xcrun:

https://github.com/bitcoin/bitcoin/blob/fc642c33ef28829eda0119a0fe39fd9bc4b84051/depends/builders/darwin.mk#L4-L8

I got hold of a macOS machine running 15.0, and wasn’t able to recreate at fc642c33ef28829eda0119a0fe39fd9bc4b84051.

Sjors commented at 10:52 am on October 2, 2024: member

I get it every time on master and any branch I test on.

Here’s the initial log from a clean run on fc642c33ef28829eda0119a0fe39fd9bc4b84051 on Intel macOS 15.0:

 0git clean -dfx
 1cd depends
 2make
 3/bin/sh: command -v llvm-ranlib: No such file or directory
 4/bin/sh: command -v llvm-strip: No such file or directory
 5/bin/sh: command -v llvm-nm: No such file or directory
 6/bin/sh: command -v llvm-objdump: No such file or directory
 7/bin/sh: command -v dsymutil: No such file or directory
 8Fetching boost_1_81_0.tar.gz from https://archives.boost.io/release/1.81.0/source/
 9  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
10                                 Dload  Upload   Total   Spent    Left  Speed
11 35  133M   35 47.4M    0     0  12.6M      0  0:00:10  0:00:03  0:00:07 12.6M^Cmake: *** Deleting file `/Users/sjors/dev/bitcoin/depends/sources/download-stamps/.stamp_fetched-boost-boost_1_81_0.tar.gz.hash'
12make: *** [/Users/sjors/dev/bitcoin/depends/sources/download-stamps/.stamp_fetched-boost-boost_1_81_0.tar.gz.hash] Interrupt: 2

the tooling should be found by calling xcrun

Indeed it finds those:

0% xcrun -f ranlib
1/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib

Same on my (Intel) iMac running macOS 13.7 (with Xcode 15.2 manually installed):

0% xcrun -f ranlib
1/Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib

Sjors commented at 10:55 am on October 2, 2024: member

Mmm, maybe it’s because my macOS 15.0 machine has the full Xcode install? (not needed for Bitcoin Core)

Sjors commented at 10:59 am on October 2, 2024: member

Deleting Xcode (on the macOS 15.0 machine) doesn’t fix the error.

It’s possible that the installation made permanent changes, but this does point to a new place:

0 % xcrun -f ranlib          
1/Library/Developer/CommandLineTools/usr/bin/ranlib

I’ll try a reboot.

Reboot didn’t help. I also tried removing Homebrew llvm (19.1), just in case that was interfering, but it made no difference.

I also tried using env -i zsh -f to get rid of potential interference from PyEnv and similar tools, but that didn’t help.

I wrote in the description:

but I haven’t tested if they actually work.

I have since, and afaik everything works fine despite these errors.

itornaza commented at 9:21 am on October 4, 2024: contributor

Same error messages as @Sjors on a clean build as well.

0ioannis@ergot_a depends % make
1/bin/sh: command -v llvm-ranlib: No such file or directory
2/bin/sh: command -v llvm-strip: No such file or directory
3/bin/sh: command -v llvm-nm: No such file or directory
4/bin/sh: command -v llvm-objdump: No such file or directory
5/bin/sh: command -v dsymutil: No such file or directory
6Fetching boost_1_81_0.tar.gz from https://archives.boost.io/release/1.81.0/source/
7  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
8                                 Dload  Upload   Total   Spent    Left  Speed
9  8  133M    8 12.0M    0     0   672k      0  0:03:23  0:00:18  0:03:05  734k

Noticing that in the error messages display llvm-ranlib as opposed to ranlib ? On my system they live in different locations because llvm-ranlib is under the brew installation.

0ioannis@ergot_a bitcoin % which ranlib; which llvm-ranlib; xcrun -f ranlib
1/usr/bin/ranlib
2/opt/homebrew/opt/llvm/bin/llvm-ranlib
3/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib

I hope this helps a little.

willcl-ark commented at 1:41 pm on October 4, 2024: member

I also see this issue on MacOS 15.0.

This looks odd, because you seem to be hitting code paths for macOS cross-compilation, when building natively. i.e when building on macOS, the tooling should be found by calling xcrun:

I don’t think this is correct? These checks are coming from hosts/darwin.mk which IIUC is the correct host file to be usedwhen building on a darwin host?:

https://github.com/bitcoin/bitcoin/blob/51c698161b54402f9a8302c9a36f28a4db01fc4f/depends/hosts/darwin.mk#L19-L24

I think this still works correctly, despite the error, is because the darwin builder overrides the host config:

https://github.com/bitcoin/bitcoin/blob/51c698161b54402f9a8302c9a36f28a4db01fc4f/depends/builders/darwin.mk#L12-L20

itornaza commented at 4:33 pm on October 5, 2024: contributor

To get a better understanding I run some experiments on the depeds/hosts/darwin.mk file.

When either:

Comment out the lines with the definitions of darwin_RANLIB, darwin _STRIP, darwin_OBJDUMP, darwin_NM, and darwin_DSYMUTIL, or
Set the values there as

0RANLIB := llvm-ranlib
1STRIP := llvm-strip
2NM := llvm-nm
3OBJDUMP := llvm-objdump
4DSYMUTIL := dsymutil

instead of

0darwin_DSYMUTIL=$(shell $(SHELL) $(.SHELLFLAGS) "command -v dsymutil")
1darwin_NM=$(shell $(SHELL) $(.SHELLFLAGS) "command -v llvm-nm")
2darwin_OBJDUMP=$(shell $(SHELL) $(.SHELLFLAGS) "command -v llvm-objdump")
3darwin_RANLIB=$(shell $(SHELL) $(.SHELLFLAGS) "command -v llvm-ranlib")
4darwin_STRIP=$(shell $(SHELL) $(.SHELLFLAGS) "command -v llvm-strip")

the errors are not present. However, I do not at present understand the implications of this change in other OS configurations.

It seems that during these $(shell...) assignments the shell is invoked and produces that error but then as @willcl-ark points out, the job is correctly handled by the depends/builders/darwin.mk

maflcko added the label macOS on Oct 24, 2024

maflcko added the label Build system on Oct 24, 2024

depends: llvm-ranlib (etc): “No such file or directory” on Intel macOS 15.0 #30978

Solution 1

Solution 2

Solution 3