depends: bump boost to 1.86.0 and use new CMake buildsystem

theuni commented at 6:18 pm on July 11, 2024: member

Marked as draft because this is much more relevant after we’ve switched to CMake.

This has a few advantages over the old method of simply copying headers:

Installs proper cmake files which can be picked up by our buildsystem
Only installs necessary headers, not all of boost

The only drawback is that it builds a few libs that we end up throwing away. date_time and test can both be optionally used header-only (which we do), but boost’s CMake buildsystem doesn’t expose an option to skip building them.

DrahtBot commented at 6:18 pm on July 11, 2024: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30434.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

DrahtBot added the label Build system on Jul 11, 2024

theuni commented at 6:27 pm on July 11, 2024: member

Ping @hebasto. Tested on cmake-staging with Boost_NO_BOOST_CMAKE removed. Seems to work as expected. Though ofc we wouldn’t be able to remove that until we require boost 1.82 for non-depends builds.

DrahtBot added the label CI failed on Jul 12, 2024

maflcko commented at 9:14 am on July 12, 2024: member

The CI failure https://cirrus-ci.com/task/4950868422819840:

 0Extracting boost...
 1/ci_container_base/depends/sources/boost-1.85.0-cmake.tar.gz: OK
 2Preprocessing boost...
 3Configuring boost...
 4-- The CXX compiler identification is Clang 18.1.3
 5-- Detecting CXX compiler ABI info
 6-- Detecting CXX compiler ABI info - done
 7-- Check for working CXX compiler: /usr/bin/env - skipped
 8-- Detecting CXX compile features
 9-- Detecting CXX compile features - done
10-- Boost: using system layout: include, bin, lib, lib/cmake
11-- Boost: Release build, static libraries, MPI OFF, Python OFF, testing OFF
12-- Boost: libraries included: date_time;multi_index;signals2;test
13-- The C compiler identification is Clang 18.1.3
14CMake Error at /usr/share/cmake-3.28/Modules/CMakeFindBinUtils.cmake:251 (message):
15  Could not find install_name_tool, please check your installation.
16Call Stack (most recent call first):
17  /usr/share/cmake-3.28/Modules/CMakeDetermineCCompiler.cmake:196 (include)
18  libs/container/CMakeLists.txt:7 (project)
19
20
21-- Configuring incomplete, errors occurred!
22make: *** [funcs.mk:302: /ci_container_base/depends/x86_64-apple-darwin/.boost_stamp_configured] Error 1
23make: Leaving directory '/ci_container_base/depends'
24
25Exit status: 2

hebasto commented at 10:39 am on July 12, 2024: member

The CI failure https://cirrus-ci.com/task/4950868422819840:

Maybe revert 3bee51427a05075150721f0a05ead8f92e1ba019?

FWIW, in the CMake staging branch, set(CMAKE_PLATFORM_HAS_INSTALLNAME FALSE) is used in between the project() and enable_language(CXX) calls to workaround it:

0#=============================
1# Language setup
2#=============================
3if(CMAKE_SYSTEM_NAME STREQUAL "Darwin" AND NOT CMAKE_HOST_APPLE)
4  # We do not use the install_name_tool when cross-compiling for macOS.
5  # So disable this tool check in further enable_language() commands.
6  set(CMAKE_PLATFORM_HAS_INSTALLNAME FALSE)
7endif()
8enable_language(CXX)

in depends/packages/boost.mk:9 in a7b3547b42 outdated

 7+$(package)_download_path=https://github.com/boostorg/boost/releases/download/boost-$($(package)_version)
 8+$(package)_file_name=boost-$($(package)_version)-cmake.tar.gz
 9+$(package)_sha256_hash=ab9c9c4797384b0949dd676cf86b4f99553f8c148d767485aaac412af25183e6
10+$(package)_build_subdir=build
11+
12+# This compiles a few libs unnecessarily because date_time and test don't have

fanquake commented at 10:41 am on July 12, 2024:

Are we going to upstream this? Suprised there’s not already an option to do this for at least date_time, given it’s a stub library that only exists for back compat. Wonder if we can (easily) patch it out in the interim.

theuni commented at 1:23 pm on July 12, 2024:

Blah. I started working on it, but there’s no precedent for a cmake -DHEADER_ONLY in the boost libs and their CMake files are auto-generated. So I assumed there’d be resistance to a patch like this.

But now that we have a reason, sure, I’ll patch them and PR the patches and see how it goes.

fanquake commented at 1:56 pm on July 26, 2024:

So I assumed there’d be resistance to a patch like this.

Given some recent happenings, that may no-longer be the case. Seems like there’s quite a substantial effort to newly “modularize” Boost. See mailing list discussion here: https://lists.boost.org/Archives/boost/2024/01/255704.php and [WIP] overview here: https://github.com/grafikrobot/boost-b2-modular/blob/b2-modular/README.adoc.

fanquake commented at 10:42 am on July 12, 2024: member

Maybe revert https://github.com/bitcoin/bitcoin/commit/3bee51427a05075150721f0a05ead8f92e1ba019?

A better solution would be to fix the CMake build systems, so we don’t have to pointlessly compile stub libraries, that we ultimately don’t even use.

theuni commented at 1:31 pm on July 12, 2024: member

Pushed a workaround hack in the meantime.

in depends/packages/boost.mk:14 in e0d041d88f outdated

 7@@ -8,8 +8,10 @@ $(package)_build_subdir=build
 8 # This compiles a few libs unnecessarily because date_time and test don't have
 9 # header-only build/install options
10 
11+#install_name_tool is unused, so set it to the `true` binary so that CMake thinks it exists
12 define $(package)_set_vars
13   $(package)_config_opts=-DBOOST_INCLUDE_LIBRARIES="date_time;multi_index;signals2;test" -DBOOST_INSTALL_LAYOUT=system
14+  $(package)_config_opts_darwin=-DCMAKE_INSTALL_NAME_TOOL=true

hebasto commented at 1:33 pm on July 12, 2024:

Do we want to avoid it when building depends on macOS?

fanquake commented at 1:36 pm on July 12, 2024:

None of the compiled libraries are used, so if this is scoped to Boost, why would it make any difference?

DrahtBot removed the label CI failed on Jul 12, 2024

fanquake commented at 3:39 pm on August 29, 2024: member

@theuni want to rebase this, (and even kick it along to 1.86.0?), to see what works, what breaks etc.

theuni force-pushed on Aug 29, 2024

depends: bump boost to 1.86.0 and use new CMake buildsystem

This has a few advantages over the old method of simply copying headers:
- Installs proper cmake files which can be picked up by our buildsystem
- Only installs necessary headers, not all of boost

The only drawback is that it builds a few libs that we end up throwing away.
date_time and test can both be optionally used header-only (which we do), but
boost's CMake buildsystem doesn't expose an option to skip building them.

43bd397ea6

theuni force-pushed on Aug 29, 2024

theuni renamed this:
~~depends: bump boost to 1.85.0 and use new CMake buildsystem~~
depends: bump boost to 1.86.0 and use new CMake buildsystem
on Aug 29, 2024

DrahtBot added the label CI failed on Aug 30, 2024

in depends/packages/boost.mk:14 in 43bd397ea6

13+# This compiles a few libs unnecessarily because date_time and test don't have
14+# header-only build/install options
15+
16+define $(package)_set_vars
17+  $(package)_config_opts=-DBOOST_INCLUDE_LIBRARIES="date_time;multi_index;signals2;test" -DBOOST_INSTALL_LAYOUT=system
18+  $(package)_config_opts_darwin=-DCMAKE_PLATFORM_HAS_INSTALLNAME="FALSE"

hebasto commented at 1:33 pm on October 22, 2024:

It seems that the previous variant is correct:

0  $(package)_config_opts_darwin=-DCMAKE_INSTALL_NAME_TOOL=true

hebasto commented at 2:56 pm on November 1, 2024: member

FWIW, Debian provides boost_headers-config.cmake:

depends: bump boost to 1.86.0 and use new CMake buildsystem #30434

Code Coverage & Benchmarks

Reviews