macOS 13.7 depends build can't find qt (symlink) #31050

Is there an existing issue for this?

• I have searched the existing issues

Current behaviour

cd depends
make NO_BDB=1
...
copying packages: boost libevent qt qrencode sqlite miniupnpc zeromq
to: /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0

cd ..
cmake -B build --toolchain depends/x86_64-apple-darwin22.6.0/toolchain.cmake

This fails, see below

(building BDB hasn't work for me for a while, but that's going away anyway)

Expected behaviour

To build

Steps to reproduce

.

Relevant log output

xcode-select -p                                                             
/Applications/Xcode15.2.app/Contents/Developer

-- The CXX compiler identification is AppleClang 15.0.0.15000100
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found SQLite3: /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/include (found suitable version "3.38.5", minimum required is "3.7.17")
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.29.2")
-- Found MiniUPnPc: /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/libminiupnpc.a
-- Checking for module 'libzmq>=4'
--   Found libzmq, version 4.3.5
-- Checking for module 'libqrencode'
--   Found libqrencode, version 4.1.1
-- Could NOT find Qt5Core (missing: Qt5Core_DIR)
-- Could NOT find Qt5Gui (missing: Qt5Gui_DIR)
-- Could NOT find Qt5Widgets (missing: Qt5Widgets_DIR)
-- Could NOT find Qt5LinguistTools (missing: Qt5LinguistTools_DIR)
-- Could NOT find Qt5Network (missing: Qt5Network_DIR)
-- Could NOT find Qt5Test (missing: Qt5Test_DIR)
CMake Error at depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5/Qt5Config.cmake:51 (_qt5_Core_check_file_exists):
  Unknown CMake command "_qt5_Core_check_file_exists".
Call Stack (most recent call first):
  cmake/module/FindQt.cmake:43 (find_package)
  CMakeLists.txt:180 (find_package)


CMake Warning at cmake/module/FindQt.cmake:43 (find_package):
  Found package configuration file:

    /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5/Qt5Config.cmake

  but it set Qt5_FOUND to FALSE so package "Qt5" is considered to be NOT
  FOUND.  Reason given by package:

  Failed to find Qt5 component "Core" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Core/Qt5CoreConfig.cmake"


  Failed to find Qt5 component "Gui" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake"


  Failed to find Qt5 component "Widgets" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake"


  Failed to find Qt5 component "LinguistTools" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5LinguistTools/Qt5LinguistToolsConfig.cmake"


  Failed to find Qt5 component "Network" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Network/Qt5NetworkConfig.cmake"


  Failed to find Qt5 component "Test" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Test/Qt5TestConfig.cmake"




Call Stack (most recent call first):
  CMakeLists.txt:180 (find_package)

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

PR 31048 which builds on master @ 05d25304bc4e0c3058c8ee8a89448ce63ac77304

Operating system and version

macOS 13.7

Machine specifications

No response

Will investigate and provide more details later.

Here's my depends/x86_64-apple-darwin22.6.0/toolchain.cmake

# Copyright (c) 2023-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or https://opensource.org/license/mit/.

# This file is expected to be highly volatile and may still change substantially.

# If CMAKE_SYSTEM_NAME is set within a toolchain file, CMake will also
# set CMAKE_CROSSCOMPILING to TRUE, even if CMAKE_SYSTEM_NAME matches
# CMAKE_HOST_SYSTEM_NAME. To avoid potential misconfiguration of CMake,
# it is best not to touch CMAKE_SYSTEM_NAME unless cross-compiling is
# intended.
if(FALSE)
  set(CMAKE_SYSTEM_NAME Darwin)
  set(CMAKE_SYSTEM_VERSION 20.1)
  set(CMAKE_SYSTEM_PROCESSOR x86_64)
endif()

if(NOT DEFINED CMAKE_C_FLAGS_INIT)
  set(CMAKE_C_FLAGS_INIT "-pipe -std=c11 -mmacos-version-min=12.0 -arch x86_64")
endif()
if(NOT DEFINED CMAKE_C_FLAGS_RELWITHDEBINFO_INIT)
  set(CMAKE_C_FLAGS_RELWITHDEBINFO_INIT "-O2")
endif()
if(NOT DEFINED CMAKE_C_FLAGS_DEBUG_INIT)
  set(CMAKE_C_FLAGS_DEBUG_INIT "")
endif()

if(NOT DEFINED CMAKE_C_COMPILER)
  set(CMAKE_C_COMPILER /Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -isysroot/Applications/Xcode15.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk)
endif()

if(NOT DEFINED CMAKE_CXX_FLAGS_INIT)
  set(CMAKE_CXX_FLAGS_INIT "-pipe -std=c++20 -mmacos-version-min=12.0 -arch x86_64")
  set(CMAKE_OBJCXX_FLAGS_INIT "-pipe -std=c++20 -mmacos-version-min=12.0 -arch x86_64")
endif()
if(NOT DEFINED CMAKE_CXX_FLAGS_RELWITHDEBINFO_INIT)
  set(CMAKE_CXX_FLAGS_RELWITHDEBINFO_INIT "-O2")
  set(CMAKE_OBJCXX_FLAGS_RELWITHDEBINFO_INIT "-O2")
endif()
if(NOT DEFINED CMAKE_CXX_FLAGS_DEBUG_INIT)
  set(CMAKE_CXX_FLAGS_DEBUG_INIT "")
  set(CMAKE_OBJCXX_FLAGS_DEBUG_INIT "")
endif()

if(NOT DEFINED CMAKE_CXX_COMPILER)
  set(CMAKE_CXX_COMPILER /Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -stdlib=libc++ -isysroot/Applications/Xcode15.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk)
  set(CMAKE_OBJCXX_COMPILER ${CMAKE_CXX_COMPILER})
endif()

# The DEPENDS_COMPILE_DEFINITIONS* variables are to be treated as lists.
set(DEPENDS_COMPILE_DEFINITIONS )
set(DEPENDS_COMPILE_DEFINITIONS_RELWITHDEBINFO )
set(DEPENDS_COMPILE_DEFINITIONS_DEBUG )

if(NOT DEFINED CMAKE_EXE_LINKER_FLAGS_INIT)
  set(CMAKE_EXE_LINKER_FLAGS_INIT "-Wl,-platform_version,macos,12.0,14.0")
endif()
if(NOT DEFINED CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO_INIT)
  set(CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO_INIT "")
endif()
if(NOT DEFINED CMAKE_EXE_LINKER_FLAGS_DEBUG_INIT)
  set(CMAKE_EXE_LINKER_FLAGS_DEBUG_INIT "")
endif()

set(CMAKE_AR "/Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar")
set(CMAKE_RANLIB "/Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib")
set(CMAKE_STRIP "/Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip")
set(CMAKE_OBJCOPY "objcopy")
set(CMAKE_OBJDUMP "/Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump")

# Using our own built dependencies should not be
# affected by a potentially random environment.
set(CMAKE_FIND_USE_CMAKE_ENVIRONMENT_PATH OFF)

set(CMAKE_FIND_ROOT_PATH "/Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
set(QT_TRANSLATIONS_DIR "/Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/translations")

if(CMAKE_SYSTEM_NAME STREQUAL "Darwin" AND NOT CMAKE_HOST_APPLE)
  # The find_package(Qt ...) function internally uses find_library()
  # calls for all dependencies to ensure their availability.
  # In turn, the find_library() inspects the well-known locations
  # on the file system; therefore, a hint is required.
  set(CMAKE_FRAMEWORK_PATH "/Volumes/SSD/Dev/bitcoin/depends/SDKs/Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers/System/Library/Frameworks")
endif()


# Customize pkg-config behaviour.
cmake_path(APPEND CMAKE_FIND_ROOT_PATH "lib" "pkgconfig" OUTPUT_VARIABLE pkg_config_path)
set(ENV{PKG_CONFIG_PATH} ${pkg_config_path})
set(ENV{PKG_CONFIG_LIBDIR} ${pkg_config_path})
unset(pkg_config_path)
set(PKG_CONFIG_ARGN --static)


# Set configuration options for the main build system.
set(qt_packages  qt  qrencode)
if("${qt_packages}" STREQUAL "")
  set(BUILD_GUI OFF CACHE BOOL "")
else()
  set(BUILD_GUI ON CACHE BOOL "")
endif()

set(qrencode_packages qrencode)
if("${qrencode_packages}" STREQUAL "")
  set(WITH_QRENCODE OFF CACHE BOOL "")
else()
  set(WITH_QRENCODE ON CACHE BOOL "")
endif()

set(zmq_packages zeromq)
if("${zmq_packages}" STREQUAL "")
  set(WITH_ZMQ OFF CACHE BOOL "")
else()
  set(WITH_ZMQ ON CACHE BOOL "")
endif()

set(wallet_packages  sqlite)
if("${wallet_packages}" STREQUAL "")
  set(ENABLE_WALLET OFF CACHE BOOL "")
else()
  set(ENABLE_WALLET ON CACHE BOOL "")
endif()

set(bdb_packages )
if("${wallet_packages}" STREQUAL "" OR "${bdb_packages}" STREQUAL "")
  set(WITH_BDB OFF CACHE BOOL "")
else()
  set(WITH_BDB ON CACHE BOOL "")
endif()

set(sqlite_packages sqlite)
if("${wallet_packages}" STREQUAL "" OR "${sqlite_packages}" STREQUAL "")
  set(WITH_SQLITE OFF CACHE BOOL "")
else()
  set(WITH_SQLITE ON CACHE BOOL "")
endif()

set(upnp_packages miniupnpc)
if("${upnp_packages}" STREQUAL "")
  set(WITH_MINIUPNPC OFF CACHE BOOL "")
else()
  set(WITH_MINIUPNPC ON CACHE BOOL "")
endif()

set(usdt_packages )
if("${usdt_packages}" STREQUAL "")
  set(WITH_USDT OFF CACHE BOOL "")
else()
  set(WITH_USDT ON CACHE BOOL "")
endif()

if("")
  set(ENABLE_HARDENING OFF CACHE BOOL "")
else()
  set(ENABLE_HARDENING ON CACHE BOOL "")
endif()

if("" STREQUAL "1")
  set(WITH_MULTIPROCESS ON CACHE BOOL "")
  set(Libmultiprocess_ROOT "${CMAKE_CURRENT_LIST_DIR}" CACHE PATH "")
  set(LibmultiprocessNative_ROOT "${CMAKE_CURRENT_LIST_DIR}/native" CACHE PATH "")
else()
  set(WITH_MULTIPROCESS OFF CACHE BOOL "")
endif()

Tried again on master @ 62e4516722115c2d5aeb6c197abc73ca7c078b23 with the same result. Since it takes about an hour to build Qt on that machine it's a bit difficult to bisect.

I also still get #30978, in case that's related.

Ditto on d71ac768424333b65a6d88c9752cc9c7fdb276f3.

Could you please test aba02f0c06fdcf28fe76de4cf00dc73c3aee6208 from this branch?

I have no access to macOS 13.7 (Intel), so not able to test the patch myself.

Tried https://github.com/bitcoin/bitcoin/commit/aba02f0c06fdcf28fe76de4cf00dc73c3aee6208 with make NO_WALLET=1 NO_UPNP=1 NO_ZMQ=1 NO_BDB=1 so save some build time:

...
copying packages: boost libevent qt qrencode
to: /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0
To build Bitcoin Core with these packages, pass '--toolchain /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/toolchain.cmake' to the first CMake invocation.
sjors@Sjorss-iMac depends % cd .. 
sjors@Sjorss-iMac bitcoin % cmake -B build --toolchain depends/x86_64-apple-darwin22.6.0/toolchain.cmake
-- The CXX compiler identification is AppleClang 15.0.0.15000100
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.29.2")
-- Checking for module 'libqrencode'
--   Found libqrencode, version 4.1.1
-- Could NOT find Qt5Core (missing: Qt5Core_DIR)
-- Could NOT find Qt5Gui (missing: Qt5Gui_DIR)
-- Could NOT find Qt5Widgets (missing: Qt5Widgets_DIR)
-- Could NOT find Qt5LinguistTools (missing: Qt5LinguistTools_DIR)
-- Could NOT find Qt5Test (missing: Qt5Test_DIR)
CMake Error at depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5/Qt5Config.cmake:51 (_qt5_Core_check_file_exists):
  Unknown CMake command "_qt5_Core_check_file_exists".
Call Stack (most recent call first):
  cmake/module/FindQt.cmake:43 (find_package)
  CMakeLists.txt:180 (find_package)


CMake Warning at cmake/module/FindQt.cmake:43 (find_package):
  Found package configuration file:

    /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5/Qt5Config.cmake

  but it set Qt5_FOUND to FALSE so package "Qt5" is considered to be NOT
  FOUND.  Reason given by package:

  Failed to find Qt5 component "Core" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Core/Qt5CoreConfig.cmake"


  Failed to find Qt5 component "Gui" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake"


  Failed to find Qt5 component "Widgets" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake"


  Failed to find Qt5 component "LinguistTools" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5LinguistTools/Qt5LinguistToolsConfig.cmake"


  Failed to find Qt5 component "Test" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Test/Qt5TestConfig.cmake"




Call Stack (most recent call first):
  CMakeLists.txt:180 (find_package)

Is /Volumes/SSD/Dev/ and /Users/sjors/dev/ the same path? I don't know why, but it seems the two are mixed up. My recommendation would be to put the source code and the depends dir on the same path (one of the two) and try again.

/Users/sjors/dev is a symlink to /Volumes/SSD/Dev/. I'll try a build without using a symlink. (will be a few weeks before I can access this machine)

symlink

See also https://github.com/bitcoin/bitcoin/issues/31145

Yep. That's most likely the root cause.

That seems plausible. I'll keep an eye on that issue. Once that's resolved and there's still a macOS specific problem remaining, I'll reopen.

I tried #31361 and it makes no difference.

% cmake -B build --toolchain /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/toolchain.cmake
-- The CXX compiler identification is AppleClang 15.0.0.15000100
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found QRencode: /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/libqrencode.a (found version "4.1.1")
-- Could NOT find Qt5Core (missing: Qt5Core_DIR)
-- Could NOT find Qt5Gui (missing: Qt5Gui_DIR)
-- Could NOT find Qt5Widgets (missing: Qt5Widgets_DIR)
-- Could NOT find Qt5LinguistTools (missing: Qt5LinguistTools_DIR)
-- Could NOT find Qt5Test (missing: Qt5Test_DIR)
CMake Error at depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5/Qt5Config.cmake:51 (_qt5_Core_check_file_exists):
  Unknown CMake command "_qt5_Core_check_file_exists".
Call Stack (most recent call first):
  cmake/module/FindQt.cmake:43 (find_package)
  CMakeLists.txt:165 (find_package)


CMake Warning at cmake/module/FindQt.cmake:43 (find_package):
  Found package configuration file:

    /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5/Qt5Config.cmake

  but it set Qt5_FOUND to FALSE so package "Qt5" is considered to be NOT
  FOUND.  Reason given by package:

  Failed to find Qt5 component "Core" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Core/Qt5CoreConfig.cmake"


  Failed to find Qt5 component "Gui" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake"


  Failed to find Qt5 component "Widgets" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake"


  Failed to find Qt5 component "LinguistTools" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5LinguistTools/Qt5LinguistToolsConfig.cmake"


  Failed to find Qt5 component "Test" config file at
  "/Users/sjors/dev/bitcoin/depends/x86_64-apple-darwin22.6.0/lib/cmake/Qt5Test/Qt5TestConfig.cmake"




Call Stack (most recent call first):
  CMakeLists.txt:165 (find_package)


-- Configuring incomplete, errors occurred!

Will try doing a build without a symlink to see if that has anything to do with it at all.

It does indeed seem related to symlinks though, maybe something slightly different than what #31361 tries to fix. If I go from the symlinked to ~/dev to its target (cd /Volumes/SSD/Dev/bitcoin), and then do the above cmake -B ... it works fine.

Yes, this is a different problem. #31361 had to do with generated (moc) files, not finding Qt itself.

cd depends
make NO_BDB=1
...
copying packages: boost libevent qt qrencode sqlite miniupnpc zeromq
to: /Volumes/SSD/Dev/bitcoin/depends/x86_64-apple-darwin22.6.0

/Users/sjors/dev is a symlink to /Volumes/SSD/Dev/. I'll try a build without using a symlink. (will be a few weeks before I can access this machine)

On macOS Sequoia 15.1, when I build depends in a symlinked directory, the resulted toolchain file still refers to the real directory. @Sjors

Could you please provide all steps to reproduce the issue without involving an external drive?

What make are you using?

@Sjors

I've managed to reproduce the issue.

I believe #31358 should resolve it. Could you please verify the fix on your system?

Unfortunately I misconfigured wake on lan for this machine, so it'll be a few weeks before I can test it. A made a reminder to do so.

Update: it does! #31358 (comment)