CMake 4.1.0+ fails to configure cross-build for macOS with Ninja generator #34513

When cross-compiling for macOS on Linux, the configuration step fails when using the Ninja generator:

$ gmake -j $(nproc) -C depends HOST=x86_64-apple-darwin NO_QT=1
$ cmake -B build -G Ninja --toolchain depends/x86_64-apple-darwin/toolchain.cmake
<snip>
CMake Error at src/CMakeLists.txt:301 (add_executable):
  The install of the bitcoind target requires changing an RPATH from the
  build tree, but this is not supported with the Ninja generator unless on an
  ELF-based or XCOFF-based platform.  The CMAKE_BUILD_WITH_INSTALL_RPATH
  variable may be set to avoid this relinking step.


<snip>

This behavior is observed with CMake starting from version 4.1.0.

This issue does not occur when building with depends on macOS.

cc @purpleKarrot

What is there for us to do here that isn't explained in the error output?

The output message hints that there are two options to avoid the limitation:

1. Use a different generator, like "Unix Makefiles", to make the build slow.
2. Set the CMAKE_BUILD_WITH_INSTALL_RPATH variable to avoid the relinking step but potentially break unit tests because objects are no longer found.

But there is a third option: Remove the need for changing the RPATH in the first place (by not setting CMAKE_INSTALL_RPATH). What is this needed for? It was introduced in #20046, maybe @ryanofsky can explain what this setting was intended to fix (the issue mentions it as a fix for "not working").

Looks like our upstream question never got any reply: https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892.

The upstream issue aside, we should just make sure that a workaround for making mpgen usable at build time does not affect the RPATH in binaries that are shipped to clients, such as bitcoind.

Our release binaries are built with CMAKE_SKIP_RPATH, which should prevent any RPATH information from being added (we also check that the binaries do not have any rpath or runpath information).

Remove the need for changing the RPATH in the first place

I'm not sure the reason cmake wants to relink mpgen before installing it, so I would expect CMAKE_BUILD_WITH_INSTALL_RPATH to work and not break anything, but I could be missing something.

For background on the current code: mpgen has the INSTALL_RPATH_USE_LINK_PATH property enabled because it depends on cap'nproto shared libraries, so needs the rpath to find them if they aren't on the normal runtime search path.

https://github.com/bitcoin/bitcoin/blob/eb97250421d35f8c172d43b69965af6273025702/src/ipc/libmultiprocess/CMakeLists.txt#L206-L207

Additionally, since #20046, the depends system has also been setting CMAKE_INSTALL_RPATH in native package builds to work around a (possibly buggy) behavior in cmake where setting the LDFLAGS environment variable caused INSTALL_RPATH_USE_LINK_PATH to be ignored. This is the problem reported https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892.

https://github.com/bitcoin/bitcoin/blob/eb97250421d35f8c172d43b69965af6273025702/depends/funcs.mk#L230-L232

It's possible the latter issue no longer exists and we could drop the CMAKE_INSTALL_RPATH setting from depends. Or maybe depends should be setting CMAKE_BUILD_WITH_INSTALL_RPATH alongside CMAKE_INSTALL_RPATH. Unclear if those changes would fix this issue though because the relinking step here seems to be happening in the bitcoin build, not the depends build. I also don't understand what exactly causes cmake to decide it needs a relinking step, or if a reiinking step is actually needed here.

The upstream issue aside, we should just make sure that a workaround for making mpgen usable at build time does not affect the RPATH in binaries that are shipped to clients, such as bitcoind.

In addition to what fanquake wrote, this should also not be an issue becuase INSTALL_RPATH_USE_LINK_PATH is only set on the mpgen target which is not distributed, and CMAKE_INSTALL_RPATH is only set by depends on native packages providing build-time tools, not on host packages providing runtime dependencies.

Thinking about this more, I think reported issue is probably unrelated to use of INSTALL_RPATH_USE_LINK_PATH / CMAKE_INSTALL_RPATH settings. Former is only set on mpgen target so I don't think it should trigger the reported error about bitcoind. Latter is only set inside depends cmake package builds for native executable tools, so should not affect the top-level bitcoin build.

So I think fanquake's initial reaction that there probably isn't anything for us to do here seems right. If this issue started happening with cmake 4.1.0, maybe it is a new bug or limitation of that cmake version. It is unclear why cmake thinks "The install of the bitcoind target requires changing an RPATH" when it doesn't seem like this should be true.

If this issue started happening with cmake 4.1.0, maybe it is a new bug or limitation of that cmake version.

Yea. I'd think the first thing to look at would be the release notes for that version, as I'd a assume a change in behaviour like this would be documented. If not, maybe another issue can be opened upstream.

I also cannot recreate this issue using the steps provided, and CMake 4.2.3:

gmake -j15 -C depends HOST=x86_64-apple-darwin NO_QT=1
cmake -B build -G Ninja --toolchain depends/x86_64-apple-darwin/toolchain.cmake
cmake --build build -j15

Works fine, building on arm64 macOS.

I also cannot recreate this issue using the steps provided, and CMake 4.2.3:

gmake -j15 -C depends HOST=x86_64-apple-darwin NO_QT=1 cmake -B build -G Ninja --toolchain depends/x86_64-apple-darwin/toolchain.cmake cmake --build build -j15

Works fine, building on arm64 macOS.

Right, the issue only manifests when building on Linux.

only manifests when building on Linux.

Can you put all of the relevant information, and required steps into the issue description. As well as test with the latest available version of CMake?

only manifests when building on Linux.

Can you put all of the relevant information, and required steps into the issue description. As well as test with the latest available version of CMake?

Updated.

The issue is caused by this code:https://github.com/bitcoin/bitcoin/blob/fd625d84ae9e73cb2ce98cceca199ed831d8b0f4/CMakeLists.txt#L72-L76

The issue is caused by this code:

That code has existed since we migrated to CMake, so it looks like you'll need to investigate what has changed upstream, to now cause this change in behaviour.

The issue is caused by this code:

That code has existed since we migrated to CMake, so it looks like you'll need to investigate what has changed upstream, to now cause this change in behaviour.

Bisecting pointed at commit eb8facd39606e9472f9ff6de40d7caf76767d0da from MR [#10721](/bitcoin-bitcoin/10721/).

cc @purpleKarrot

From what I can tell, this doesn't effect 30.x?

From what I can tell, this doesn't effect 30.x?

True.