build: add root dir to CMAKE_PREFIX

willcl-ark commented at 2:22 pm on June 23, 2025: member

Nix patches cmake to remove the root directory / from CMAKE_PREFIX_PATH: https://github.com/NixOS/nixpkgs/blob/428b49b28ebc8938a6d9f6c540d32d7a06713972/pkgs/by-name/cm/cmake/001-search-path.diff#L10

Without this, and when using the toolchain for depends builds, cmake’s find_path() and find_package() do not know where to find dependencies, causing issues like as seen in #32428

Adding this path back is harmless on other systems, and fixes the toolchain for Nix users.

As described in #32428 (comment) I think this can be taken as a temporary fix whilst a longer-term solution is worked on.

DrahtBot added the label Build system on Jun 23, 2025

DrahtBot commented at 2:23 pm on June 23, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32798.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	hebasto, josibake, janb84

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

willcl-ark commented at 2:24 pm on June 23, 2025: member

As this is not exercised in our CI currently, a build using Nix and this branch can be found here: https://github.com/bitcoin-dev-tools/bix/actions/runs/15823397050?pr=29

cc @josibake

in depends/toolchain.cmake.in:109 in 629f00e0b4 outdated

105+#
106+# Make sure we only append once, as this file may be called repeatedly.
107+list(FIND CMAKE_SYSTEM_PREFIX_PATH "/" _nix_idx)
108+if(_nix_idx EQUAL -1)
109+  list(APPEND CMAKE_SYSTEM_PREFIX_PATH "/")
110+endif()

hebasto commented at 2:43 pm on June 23, 2025:

0if(NOT "/" IN_LIST CMAKE_SYSTEM_PREFIX_PATH)
1  list(APPEND CMAKE_SYSTEM_PREFIX_PATH "/")
2endif()

willcl-ark commented at 4:08 pm on June 23, 2025:

That’s cleaner, taken in a010bc2dbb701a2c536c8b343891635737f97ecb

bitcoin deleted a comment on Jun 23, 2025

willcl-ark force-pushed on Jun 23, 2025

in depends/toolchain.cmake.in:92 in a010bc2dbb outdated

91@@ -92,6 +92,22 @@ set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
92 set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

josibake commented at 5:18 pm on June 23, 2025:

micro-nit: co-author lines need to be at the bottom of the commit message

willcl-ark commented at 1:04 pm on June 24, 2025:

done in e27a94596f2a1f5e04722a16165717cc6e891d36

janb84 commented at 5:18 pm on June 23, 2025: contributor

ACK a010bc2dbb701a2c536c8b343891635737f97ecb

Can reproduce error on master:

 0bitcoin]$ cmake -B build --toolchain depends/aarch64-apple-darwin24.5.0/toolchain.cmake
 1-- The CXX compiler identification is Clang 19.1.7
 2-- Detecting CXX compiler ABI info
 3-- Detecting CXX compiler ABI info - done
 4-- Check for working CXX compiler: /nix/store/cvhbqa01i5yy7xmmpqp1hbnvq7kpvgsx-clang-wrapper-19.1.7/bin/clang++ - skipped
 5-- Detecting CXX compile features
 6-- Detecting CXX compile features - done
 7-- Setting build type to "RelWithDebInfo" as none was specified
 8-- Performing Test CXX_SUPPORTS__WERROR
 9-- Performing Test CXX_SUPPORTS__WERROR - Success
10-- Performing Test CXX_SUPPORTS__G3
11-- Performing Test CXX_SUPPORTS__G3 - Success
12-- Performing Test LINKER_SUPPORTS__G3
13-- Performing Test LINKER_SUPPORTS__G3 - Success
14-- Performing Test CXX_SUPPORTS__FTRAPV
15-- Performing Test CXX_SUPPORTS__FTRAPV - Success
16-- Performing Test LINKER_SUPPORTS__FTRAPV
17-- Performing Test LINKER_SUPPORTS__FTRAPV - Success
18CMake Error at /nix/store/446sv4sjjz03j7zw44icjbd81mfl8inx-cmake-3.31.6/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
19  Could NOT find SQLite3 (missing: SQLite3_INCLUDE_DIR SQLite3_LIBRARY)
20  (Required is at least version "3.7.17")
21Call Stack (most recent call first):
22  /nix/store/446sv4sjjz03j7zw44icjbd81mfl8inx-cmake-3.31.6/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
23  /nix/store/446sv4sjjz03j7zw44icjbd81mfl8inx-cmake-3.31.6/share/cmake-3.31/Modules/FindSQLite3.cmake:68 (find_package_handle_standard_args)
24  CMakeLists.txt:115 (find_package)

Works with this patch. (tested on MacOS (aarch64) with Nix )

 0NOTE: The summary above may not exactly match the final applied build flags
 1      if any additional CMAKE_* or environment variables have been modified.
 2      To see the exact flags applied, build with the --verbose option.
 3
 4Treat compiler warnings as errors ..... OFF
 5Use ccache for compiling .............. ON
 6
 7
 8-- Configuring done (24.3s)
 9-- Generating done (0.2s)
10-- Build files have been written to:  <<snip>>

josibake approved

josibake commented at 5:20 pm on June 23, 2025: member

ACK https://github.com/bitcoin/bitcoin/pull/32798/commits/a010bc2dbb701a2c536c8b343891635737f97ecb

Thanks for opening! Tested with a modified version of https://github.com/bitcoin-dev-tools/bix, where I removed all of the build/runtime dependencies and only included the dependencies needed for the depends system. Was able to build via depends without any errors:

0cd bitcoin
1nix develop ../bix # local copy of bix with only the depends dependencies in the shell
2make -C depends -j$(nproc)
3cmake -B build --toolchain /home/josie/bitcoin/depends/x86_64-pc-linux-gnu/toolchain.cmake

Note: Just wanted to mention/document if Qt is installed in the nix environment, the build phase for Qt will fail. This is due to Qt find_package preferring the system installed Qt over depends when it is doing the plugin configuration phase. I was able to fix this with a depends patch in a previous PR with:

 0--- a/depends/packages/qt.mk
 1+++ b/depends/packages/qt.mk
 2@@ -158,6 +158,11 @@ $(package)_config_env_darwin := OBJC="$$($(package)_cc)"
 3 $(package)_config_env_darwin += OBJCXX="$$($(package)_cxx)"
 4
 5 $(package)_cmake_opts := -DCMAKE_PREFIX_PATH=$(host_prefix)
 6+$(package)_cmake_opts += -DCMAKE_FIND_ROOT_PATH=$(host_prefix)
 7+$(package)_cmake_opts += -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY
 8+$(package)_cmake_opts += -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY
 9+$(package)_cmake_opts += -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY
10+$(package)_cmake_opts += -DCMAKE_SYSTEM_PREFIX_PATH=$(host_prefix)
11 $(package)_cmake_opts += -DQT_FEATURE_cxx20=ON
12 $(package)_cmake_opts += -DQT_ENABLE_CXX_EXTENSIONS=OFF
13 ifneq ($(V),)

.. but I think this is PR fine as is without my patch. The whole purpose of this PR is to make it so we can do a depends in nix, which to me implies we wouldn’t be including any of the system packages as inputs to our nix environment in the first place. Longer term, I plan on revisiting the dependency provider approach to see if I can get something where we can remove any nix specific workarounds and ensure that we have full control over where CMake sources dependencies from, when building in a depends context.

hebasto commented at 10:57 am on June 24, 2025: member

cc @theuni

hebasto commented at 11:57 am on June 24, 2025: member

Considering the commit history of the NixOS patch, it seems it was required for some packages, but perhaps it’s no longer needed.

On the other hand, the CMake documentation states:

CMAKE_SYSTEM_PREFIX_PATH is not intended to be modified by the project; use CMAKE_PREFIX_PATH for this.

Why not follow that?

build: add root dir to CMAKE_PREFIX_PATH

Nix patches cmake to remove the root directory `/` from
`CMAKE_SYSTEM_PREFIX_PATH`:
https://github.com/NixOS/nixpkgs/blob/428b49b28ebc8938a6d9f6c540d32d7a06713972/pkgs/by-name/cm/cmake/001-search-path.diff#L10

Without this, and when using the toolchain for depends builds, cmake's
`find_path()` and `find_package()` do not know where to find
dependencies, causing issues like:
https://github.com/bitcoin/bitcoin/issues/32428

Adding this path back via CMAKE_PREFIX_PATH is harmless on other
systems, and fixes the toolchain for Nix users.

We append the `/` dir a maximum of once, as the toolchain may be called
repeatedly during builds.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: josibake <josibake@protonmail.com>

e27a94596f

willcl-ark force-pushed on Jun 24, 2025

willcl-ark commented at 1:04 pm on June 24, 2025: member

CMAKE_SYSTEM_PREFIX_PATH is not intended to be modified by the project; use CMAKE_PREFIX_PATH for this.

Why not follow that?

The initial approach was to add it back where it was removed. But I have tested CMAKE_PREFIX_PATH locally and this works too. Happy to stick to cmake guidelines and so have switched to that now in e27a94596f2a1f5e04722a16165717cc6e891d36

in depends/toolchain.cmake.in:100 in e27a94596f

 91@@ -92,6 +92,22 @@ set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
 92 set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
 93 set(QT_TRANSLATIONS_DIR "${CMAKE_CURRENT_LIST_DIR}/translations")
 94 
 95+# The following is only necessary when using cmake from Nix or NixOS, because
 96+# Nix patches cmake to remove the root directory `/` from
 97+# CMAKE_SYSTEM_PREFIX_PATH. Adding it back is harmless on other platforms and
 98+# necessary on Nix because without it cmake find_path, find_package, etc
 99+# functions do not know where to look in CMAKE_FIND_ROOT_PATH for dependencies
100+# (https://github.com/bitcoin/bitcoin/issues/32428).

hebasto commented at 1:23 pm on June 24, 2025:

To be precise, “Adding it back … on other platforms” does not occur on every other platform that supports our depends build subsystem. According to the CMake source code, Platform/UnixPaths is processed on all platforms: Linux, Darwin, and all *BSD.

hebasto approved

hebasto commented at 1:24 pm on June 24, 2025: member

ACK e27a94596f2a1f5e04722a16165717cc6e891d36, I have reviewed the code and it looks OK.

DrahtBot requested review from janb84 on Jun 24, 2025

DrahtBot requested review from josibake on Jun 24, 2025

willcl-ark renamed this:
~~build: add root dir to CMAKE_SYSTEM_PREFIX_PATH in toolchain~~
build: add root dir to CMAKE_PREFIX_PATH in toolchain
on Jun 24, 2025

josibake commented at 2:47 pm on June 24, 2025: member

reACK https://github.com/bitcoin/bitcoin/commit/e27a94596f2a1f5e04722a16165717cc6e891d36

The only change is to use CMAKE_PREFIX_PATH over CMAKE_SYSTEM_PREFIX_PATH, which is more in-line with CMake’s documentation as mentioned in #32798#pullrequestreview-2953470821

janb84 commented at 3:09 pm on June 24, 2025: contributor

reACK e27a94596f2a1f5e04722a16165717cc6e891d36

Changes sinds last ACK:

CMAKE_SYSTEM_PREFIX_PATH -> CMAKE_PREFIX_PATH

Validated that change still solves problem ✅

fanquake added the label DrahtBot Guix build requested on Jun 24, 2025

fanquake added the label Needs backport (29.x) on Jun 24, 2025

fanquake commented at 4:34 pm on June 24, 2025: member

If we ship this, it should also be backported to 29.x.

DrahtBot commented at 3:31 am on June 25, 2025: contributor

Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]

File	commit ad654a4807cd584be9ffcd8640f628ab40cb5170(master)	commit 43ceb0f1af456500ecfb10c7a4fc84c55f67a45d(pull/32798/merge)
*-aarch64-linux-gnu-debug.tar.gz	`43c86144894c61de...`	`25091af6c7609851...`
*-aarch64-linux-gnu.tar.gz	`f80def4d4f154d4e...`	`9546fbd8976990d0...`
*-arm-linux-gnueabihf-debug.tar.gz	`145dd69c0e36134c...`	`dde5b97b3d662cfd...`
*-arm-linux-gnueabihf.tar.gz	`a5f6156ff046e395...`	`7e09cfd196d22bcd...`
*-arm64-apple-darwin-codesigning.tar.gz	`a65a241805a86cd1...`	`952abedd48b42104...`
*-arm64-apple-darwin-unsigned.tar.gz	`d3d8861a15a6f387...`	`f10fc172b57db5d4...`
*-arm64-apple-darwin-unsigned.zip	`4f4be2b13f68cabc...`	`ef5fc86c39816690...`
*-powerpc64-linux-gnu-debug.tar.gz	`52338a03b8183fbe...`	`04c5396ef265003e...`
*-powerpc64-linux-gnu.tar.gz	`85fe3660c0fcc88d...`	`397facffc92fd53b...`
*-riscv64-linux-gnu-debug.tar.gz	`d0561cd84bafbcd4...`	`81876562bdb121a6...`
*-riscv64-linux-gnu.tar.gz	`80a2df40da7503c7...`	`e941e714f0de797e...`
*-x86_64-apple-darwin-codesigning.tar.gz	`7289352f5a15d88b...`	`a9cfdbdd41ec0157...`
*-x86_64-apple-darwin-unsigned.tar.gz	`cdc38e7d97422861...`	`11efe5d49c13b315...`
*-x86_64-apple-darwin-unsigned.zip	`0d15b411149f7736...`	`d6201fbeb768c575...`
*-x86_64-linux-gnu-debug.tar.gz	`ae8bda8f113e9c84...`	`e089bf3179e249a3...`
*-x86_64-linux-gnu.tar.gz	`0b3dd2187070e6d3...`	`d71c19347aa9df59...`
*.tar.gz	`8031730d67ad8158...`	`fe1edf9149b3b877...`
SHA256SUMS.part	`baaf02f0409fbf0c...`	`123339d417ef7b00...`
guix_build.log	`f7a88ef4982941d3...`	`959b2f0bb1ce6e1c...`
guix_build.log.diff		`b54c018d33bd7e90...`

DrahtBot removed the label DrahtBot Guix build requested on Jun 25, 2025

fanquake merged this on Jun 25, 2025

fanquake closed this on Jun 25, 2025

fanquake referenced this in commit e37a70bf71 on Jun 25, 2025

fanquake removed the label Needs backport (29.x) on Jun 25, 2025

fanquake commented at 10:41 am on June 25, 2025: member

Backported to 29.x in #32810.

build: add root dir to CMAKE_PREFIX_PATH in toolchain #32798

Code Coverage & Benchmarks

Reviews

Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]