depends: Override host compilers for FreeBSD and OpenBSD #32716

pull hebasto wants to merge 1 commits into bitcoin:master from hebasto:250610-depends-bsd changing 2 files +8 −0
  1. hebasto commented at 2:02 pm on June 10, 2025: member

    When building depends on FreeBSD/OpenBSD aarch64, the host compilers default to default_host_{CC,CXX}, which resolves to gcc/g++. This is incorrect on these systems, where Clang is the default system compiler.

    To ensure proper compiler selection, this PR adopts the same approach used for darwin:https://github.com/bitcoin/bitcoin/blob/c1d4253d316ea627f46b26e395d7b48842258381/depends/builders/darwin.mk#L12-L14

    Fixes #32691.

  2. depends: Override host compilers for FreeBSD and OpenBSD
    When building depends on FreeBSD/OpenBSD `aarch64`, the host compilers
    default to `default_host_{CC,CXX}`, which resolves to `gcc`/`g++`. This
    is incorrect on these systems, where Clang is the default system
    compiler.
    4f10a57671
  3. hebasto added the label Build system on Jun 10, 2025
  4. DrahtBot commented at 2:02 pm on June 10, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32716.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK Sjors

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

  5. Sjors commented at 3:35 pm on June 10, 2025: member

    ACK 4f10a57671c19cacca630b2401e42a213aacff1b

    I tested that this fixes #32691 on an aarch64 OpenBSD VM, and doesn’t break anything on an x86 VM.

    I didn’t test FreeBSD.

    While testing this I noticed the depends build doesn’t bother creating qt. It’s not obvious to me why that is and doesn’t seem to be documented. The non-depends instructions for *BSD do include qt6.

  6. hebasto commented at 4:08 pm on June 10, 2025: member
    cc @vasild
  7. hebasto commented at 2:44 pm on June 11, 2025: member
  8. theuni commented at 3:06 pm on June 11, 2025: member

    Since gcc (and thus $(host_toolchain)gcc) is basically never the right option for these, I’m wondering if we should just do something like:

     0diff --git a/depends/hosts/freebsd.mk b/depends/hosts/freebsd.mk
     1index 009d215f82f..0a9d8bb423d 100644
     2--- a/depends/hosts/freebsd.mk
     3+++ b/depends/hosts/freebsd.mk
     4@@ -7,25 +7,24 @@ freebsd_release_CXXFLAGS=$(freebsd_release_CFLAGS)
     5 freebsd_debug_CFLAGS=-O1 -g
     6 freebsd_debug_CXXFLAGS=$(freebsd_debug_CFLAGS)
     7
     8+freebsd_CC=clang
     9+freebsd_CXX=clang++
    10+
    11+ifneq ($(host),$(build))
    12+freebsd_CC+=-target $(canonical_host)
    13+freebsd_CXX+=-target $(canonical_host)
    14+endif
    15+
    16 ifeq (86,$(findstring 86,$(build_arch)))
    17-i686_freebsd_CC=clang -m32
    18-i686_freebsd_CXX=clang++ -m32
    19 i686_freebsd_AR=ar
    20 i686_freebsd_RANLIB=ranlib
    21 i686_freebsd_NM=nm
    22 i686_freebsd_STRIP=strip
    23
    24-x86_64_freebsd_CC=clang -m64
    25-x86_64_freebsd_CXX=clang++ -m64
    26 x86_64_freebsd_AR=ar
    27 x86_64_freebsd_RANLIB=ranlib
    28 x86_64_freebsd_NM=nm
    29 x86_64_freebsd_STRIP=strip
    30-else
    31-i686_freebsd_CC=$(default_host_CC) -m32
    32-i686_freebsd_CXX=$(default_host_CXX) -m32
    33-x86_64_freebsd_CC=$(default_host_CC) -m64
    34-x86_64_freebsd_CXX=$(default_host_CXX) -m64
    35 endif
    36
    37 freebsd_cmake_system_name=FreeBSD
    
  9. hebasto commented at 3:18 pm on June 11, 2025: member

    Since gcc (and thus $(host_toolchain)gcc) is basically never the right option for these, I’m wondering if we should just do something like…

    I’d considered this option and another one. I chose the current approach for two reasons: (a) it results in a minimal diff, and (b) it avoids changing compiler flags on the x86_64 and i686 build platforms.

  10. vasild commented at 4:57 am on June 13, 2025: contributor

    When building depends on FreeBSD/OpenBSD aarch64, the host compilers default to default_host_{CC,CXX}, which resolves to gcc/g++

    This is the root of the problem ^^^

    Since gcc (and thus $(host_toolchain)gcc) is basically never the right option for these

    Exactly.

    Is it not possible to use cc/c++ by default?

     0--- i/depends/hosts/default.mk
     1+++ w/depends/hosts/default.mk
     2@@ -3,6 +3,6 @@ host_toolchain:=$(host)-
     3 endif
     4 
     5-default_host_CC = $(host_toolchain)gcc
     6-default_host_CXX = $(host_toolchain)g++
     7+default_host_CC = $(host_toolchain)cc
     8+default_host_CXX = $(host_toolchain)c++
     9 default_host_AR = $(host_toolchain)ar
    10 default_host_RANLIB = $(host_toolchain)ranlib
    

    That will address the root of the problem.

    On Linux:

    0$ cc --version
    1cc (GCC) 13.2.1 20230801
    2
    3$ c++ --version
    4c++ (GCC) 13.2.1 20230801
    

    on FreeBSD:

    0$ cc --version
    1FreeBSD clang version 19.1.7
    2
    3$ c++ --version
    4FreeBSD clang version 19.1.7
    
  11. Sjors commented at 8:14 am on June 13, 2025: member
    I tried @vasild’s patch instead of the commit here on an aarch64 and x86 OpenBSD VM (on top of #31802), and it seems to work fine.
  12. fanquake commented at 9:31 am on June 13, 2025: member

    On Linux: c++ (GCC) 13.2.1 20230801

    Just note that you can’t assume c++ is GCC on Linux. There are Linux distros built on LLVM/Clang, like Chimera, but using c++ could be a better default.

  13. hebasto commented at 11:02 am on June 13, 2025: member

    Is it not possible to use cc/c++ by default?

    There is a risk of mixing C and C++ compilers from different toolchains:

     0$ guix shell --pure --container clang-toolchain gcc-toolchain
     1$ cc --version 
     2clang version 20.1.5
     3Target: aarch64-unknown-linux-gnu
     4Thread model: posix
     5InstalledDir: /gnu/store/kjlzha1hhsdzawcnq2xig5cz7kpfwnna-clang-20.1.5/bin
     6$ c++ --version
     7c++ (GCC) 15.1.0
     8Copyright (C) 2025 Free Software Foundation, Inc.
     9This is free software; see the source for copying conditions.  There is NO
    10warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
  14. vasild commented at 12:15 pm on June 13, 2025: contributor
    Huh! An environment where cc is clang and c++ is gcc is broken, IMO. CMake will also fail to build in such an environment, I guess.
  15. theuni commented at 2:31 pm on June 13, 2025: member

    Huh! An environment where cc is clang and c++ is gcc is broken, IMO. CMake will also fail to build in such an environment, I guess.

    Agreed, I wouldn’t worry about that config.

    I agree with using cc/c++ for the build == host case as that’s what the docs for both suggest. But I suggested the clang -target=foo above for the cross case because $(host)-cc doesn’t make sense.

  16. hebasto commented at 2:37 pm on June 13, 2025: member

    Huh! An environment where cc is clang and c++ is gcc is broken, IMO. CMake will also fail to build in such an environment, I guess.

    Agreed, I wouldn’t worry about that config.

    Isn’t this the case for our own Guix environment when building for macOS?

  17. vasild commented at 11:10 am on June 16, 2025: contributor

    Isn’t this the case for our own Guix environment when building for macOS?

    No idea, but if it is, then can it be fixed? If it is and cannot be fixed or is too hard to fix it, then maybe it would be better to have the depends build system support overriding the compiler. Then this overridability can be used from weird environments. That seems to be the natural way gmake, bsd make, autotools, CMake and possibly others work - to honor the CC and CXX variables from the environment where they are started. This would mean that instead of

    0default_host_CC = $(host_toolchain)cc
    

    we can use

    0default_host_CC = $(host_toolchain)$(CC)
    

    The CC variable is set by default in make to cc. Then in the weird environment:

    0cd depends
    1CC=whatever make
    
  18. josibake commented at 4:53 pm on June 24, 2025: member

    At the risk of further derailing this thread from FreeBSD and OpenBSD (sorry!), I also ran into a similar problem with not being able to override the compiler in depends. TLDR; built a nix dev environment that includes all llvm tooling (no gcc, g++) and was trying to do a depends build. This fails like so:

     0make -C depends NO_QT=1 MULTIPROCESS=1
     1make: Entering directory '/home/josie/bitcoin/depends'
     2Extracting native_capnp...
     3/home/josie/bitcoin/depends/sources/capnproto-cxx-1.2.0.tar.gz: OK
     4Preprocessing native_capnp...
     5Configuring native_capnp...
     6CMake Error at /nix/store/wphngc22a7aphbp5pi5jqmqlsqmisgn5-cmake-3.31.6/share/cmake-3.31/Modules/CMakeDetermineCXXCompiler.cmake:48 (message):
     7  Could not find compiler set in environment variable CXX:
     8
     9  g++.
    10
    11Call Stack (most recent call first):
    12  CMakeLists.txt:2 (project)
    

    .. because g++ is being set as the default but doesn’t exist in my toolchain. I was able to get around this with the following snippet (h/t @fanquake ):

    0make -C depends/ MULTIPROCESS=1 capnp default_host_CC=clang default_host_CXX=clang++
    

    .. which did work, but I couldn’t figure out how to do this override globally, e.g., something like:

    0make -C depends default_host_CC=clang default_host_CXX=clang++ -j$(nproc)
    

    After reading through this thread, I think something like @vasild ’s suggestion in #32716 (comment) is what I’m looking for. For my usecase currrently, I’m just symlinking gcc/g++ to clang/clang++ (which is basically what macOS does, iiuc?).

  19. maflcko commented at 9:09 am on June 25, 2025: member

    but I couldn’t figure out how to do this override globally

    shouldn’t this just be CC=... CXX=..., like in CI (https://github.com/bitcoin/bitcoin/blob/ad654a4807cd584be9ffcd8640f628ab40cb5170/ci/test/00_setup_env_native_nowallet_libbitcoinkernel.sh#L13), or is this issue MULTIPROCESS=1 capnp specific?

  20. willcl-ark commented at 9:30 am on June 25, 2025: member

    but I couldn’t figure out how to do this override globally

    shouldn’t this just be CC=... CXX=..., like in CI (

    https://github.com/bitcoin/bitcoin/blob/ad654a4807cd584be9ffcd8640f628ab40cb5170/ci/test/00_setup_env_native_nowallet_libbitcoinkernel.sh#L13 ), or is this issue MULTIPROCESS=1 capnp specific?

    Seems to be MULTIPROCESS=1-specific. e.g.:

    0docker run -it nixos/nix
    1
    2# Start a shell in container with no `gcc`/`g++` present
    3nix-shell --pure -E 'with import <nixpkgs> {}; mkShellNoCC { buildInputs = [ cmake curl gnumake patch git cacert which clang coreutils binutils ]; }'
    4
    5# In the nix shell
    6git clone --depth=1 https://github.com/bitcoin/bitcoin && cd bitcoin
    7make -C depends -j$(nproc) NO_QT=1 CC=clang CXX=clang++ MULTIPROCESS=1
    
  21. vasild commented at 10:28 am on June 25, 2025: contributor

    shouldn’t this just be CC=... CXX=...

    Yes, I think it should be. But is not because currently depends/hosts/default.mk contains:

    0default_host_CC = $(host_toolchain)gcc
    1default_host_CXX = $(host_toolchain)g++
    

    gcc and g++ are hardcoded there. The following would make it possible to override the C and C++ compiler from the environment:

    0-default_host_CC = $(host_toolchain)gcc
    1-default_host_CXX = $(host_toolchain)g++
    2+default_host_CC = $(host_toolchain)$(CC)
    3+default_host_CXX = $(host_toolchain)$(CXX)
    

    The $(CC) and $(CXX) variables have meaningful defaults set by make itself, so we do not need to do something like “if CC is not set in the environment, then set it to whatever”.

  22. josibake commented at 11:44 am on June 25, 2025: member

    or is this issue MULTIPROCESS=1 capnp specific?

    I don’t think the issue is specific to multiprocess, but rather I’m running into the issue when trying to build multiprocess. I think the root cause is as @vasild mentions, i.e., hardcoding gcc/g++. Admittedly, it is weird that I’m hitting this with multiprocess but its because I am specifically trying to build in an environment without gcc/g++.

    I was surprised that make -C depends -j$(nproc) CC=clang CXX=clang++ MULTIPROCESS=1 didn’t work.

  23. hebasto commented at 3:40 pm on July 1, 2025: member
  24. hebasto commented at 3:49 pm on July 1, 2025: member

    I was surprised that make -C depends -j$(nproc) CC=clang CXX=clang++ MULTIPROCESS=1 didn’t work.

    FWIW, on some systems, such as NetBSD, it may be necessary to be more specific about the compilers used to build native packages:

    0gmake -C depends MULTIPROCESS=1 build_CC=/usr/pkg/gcc14/bin/gcc build_CXX=/usr/pkg/gcc14/bin/g++ CC=/usr/pkg/gcc14/bin/gcc CXX=/usr/pkg/gcc14/bin/g++
    
  25. fanquake commented at 9:49 am on July 3, 2025: member

    This also is related: https://codeberg.org/guix/guix/issues/556.

    Reading the discussion there, it seems like the behaviour is intentional.

  26. fanquake commented at 9:51 am on July 3, 2025: member
    There are more improvements we could make here, but we can take, and backport, the (simpler) current change as-is. Anything more will at least require some Guix changes.
  27. fanquake merged this on Jul 3, 2025
  28. fanquake closed this on Jul 3, 2025

  29. hebasto deleted the branch on Jul 3, 2025
  30. fanquake referenced this in commit 8a4a938db5 on Jul 3, 2025
  31. fanquake commented at 11:16 am on July 3, 2025: member
    Backported to 29.x in #32863.

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-07-06 06:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me