cmake: Increase cmake policy version

ryanofsky commented at 6:40 pm on September 9, 2025: collaborator

Increase cmake policy version from 3.12 to 4.1 to stop using old and deprecated CMake policies in standalone builds.

Also stop overriding policy version if a cmake parent project has already set one, so parent projects are able to control which policies are enabled by default based on their own needs. Specifically, in the Bitcoin Core subtree, this change causes the libmultiprocess cmake policy version to increase from 3.12 to 3.22, which is the version Bitcoin Core uses.

This PR also adds a new newdeps CI job to test CMake 4.1 and the master branch of Cap’n Proto. This PR doesn’t change the minimum version of cmake required to build the project, which is still 3.12.

This PR is an alternative to #163 which increases the policy version to 3.31 but doesn’t include fixes for CI jobs, and doesn’t allow the bitcoin core build to choose a lower policy version. This PR is also an alternative to #175 which sets the policy version to 3.22 like the bitcoin build, but also causes builds with earlier versions of cmake to fail.

ci: add newdeps job testing newest versions of cmake and capnproto 0ffdb140bf

DrahtBot commented at 6:40 pm on September 9, 2025: none

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#175 (Set cmake_minimum_required(VERSION 3.22) by maflcko)
#163 (build: set cmake policy version to 3.31 by purpleKarrot)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

ryanofsky force-pushed on Sep 9, 2025

ryanofsky commented at 7:23 pm on September 9, 2025: collaborator

Updated 6e29c02ec5afdb86874e83aa46c93b0d88c27968 -> 233b394123fcb96c5164dc1f4b1876b8d0ac6514 (pr/policy.1 -> pr/policy.2, compare) with a few fixes for CI errors: openbsd and freebsd git log errors and newdeps git clone TLS certificate error

Updated 233b394123fcb96c5164dc1f4b1876b8d0ac6514 -> 31206fc95a80c818c61b95e548f362dc9a10c7c8 (pr/policy.2 -> pr/policy.3, compare) with more CI fixes: openbsd and freebsd FindThreads errors and newdeps capnproto build openssl include error

Updated 31206fc95a80c818c61b95e548f362dc9a10c7c8 -> e6fd980b6c6960ea3bbda9194de1d1583bdef702 (pr/policy.3 -> pr/policy.4, compare) to fix more CI errors: openbsd and freebsd FindThreads errors and new llvm build FindThreads error.

Updated e6fd980b6c6960ea3bbda9194de1d1583bdef702 -> c33b81291896a4e1c82625f7c3bc423675899304 (pr/policy.4 -> pr/policy.5, compare) to fix CI errors: openbsd and freebsd “/bin/sh: CMAKE_CXX_COMPILER_CLANG_SCAN_DEPS-NOTFOUND: not found” error, and default and llvm IWYU errors

Updated c33b81291896a4e1c82625f7c3bc423675899304 -> 1c2958bfffd60c8cf9ec9715699c1496f6377922 (pr/policy.5 -> pr/policy.6, compare) with better debug output and better commit message after figuring out root cause of the find_package threads failure

ryanofsky force-pushed on Sep 9, 2025

ci: add cmake debug output and --parallel build option 6ba1050bef

cmake: fix find_package(Threads REQUIRED) error with new cmake policies

With latest cmake policies, find_package(Threads REQUIRED) fails in freebsd and
openbsd builds. The specific failures in these jobs happens due to the CMP0155
policy turning CMAKE_CXX_SCAN_FOR_MODULES on, which is turned off in the next
commit. But there are other things that could cause the threads package not to
be found, and there are platforms where it may not be required, so it is better
to make it an optional instead of required dependency.

Technically this change is not needed to make all CI jobs pass as long as the
next commit is present. But since this find_package error obscures other errors
that would be clearer, and since there are other conditions that could trigger
it, it is worth fixing generally. For example, this same failure also seems to
happen in the llvm job as well when CMP0137 is disabled or
CMAKE_TRY_COMPILE_NO_PLATFORM_VARIABLES is set to true.

Error looks like:

 + cmake /home/runner/work/libmultiprocess/libmultiprocess -G Ninja
-- The CXX compiler identification is Clang 16.0.6
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
CMake Error at /usr/local/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /usr/local/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake/Modules/FindThreads.cmake:226 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  CMakeLists.txt:41 (find_package)

and inside the CMakeConfigureLog.yaml file there are "/bin/sh:
CMAKE_CXX_COMPILER_CLANG_SCAN_DEPS-NOTFOUND: not found" errors.

9441e5a8f3

in CMakeLists.txt:30 in 31206fc95a outdated

26+  # the version in ci/configs/olddeps.bash should be changed to match.
27+  #
28+  # The purpose of the policy version is to opt out of policies introduced in
29+  # newer versions of CMake until they have been tested. If this number is
30+  # changed, the version in ci/configs/newdeps.bash should be changed to match.
31+  cmake_minimum_required(VERSION 3.12...4.1 FATAL_ERROR)

hebasto commented at 8:31 pm on September 9, 2025:

FATAL_ERROR will probably be ignored.

ryanofsky commented at 11:12 am on September 10, 2025:

re: #209 (review)

FATAL_ERROR will probably be ignored.

My understanding from that page is that FATAL_ERROR is a nice thing to have for very old versions of cmake that treat it like a warning, and as self-documentation since the error is just always fatal in new version of cmakes. If there is a problem or I got something wrong can drop

ryanofsky force-pushed on Sep 10, 2025

cmake: fix CMAKE_CXX_COMPILER_CLANG_SCAN_DEPS-NOTFOUND error with new cmake policies

With latest cmake policies, openbsd and freebsd CI jobs fail due to a lack of a
clang-scan-deps tool. The tool could potentially be installed on these
platforms but it is unclear how to do that and the project isn't using modules
anyway, so just disable them here. Errors look like:

+ cmake --build . --parallel -t all tests mpexamples -- -k 0
[1/114] Scanning /home/runner/work/libmultiprocess/libmultiprocess/src/mp/util.cpp for CXX dependencies
FAILED: CMakeFiles/mputil.dir/src/mp/util.cpp.o.ddi
"CMAKE_CXX_COMPILER_CLANG_SCAN_DEPS-NOTFOUND" -format=p1689 -- /usr/bin/c++  -I/home/runner/work/libmultiprocess/libmultiprocess/include -I/home/runner/work/libmultiprocess/libmultiprocess/build-openbsd/include -isystem /usr/local/include -Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter -std=gnu++20 -x c++ /home/runner/work/libmultiprocess/libmultiprocess/src/mp/util.cpp -c -o CMakeFiles/mputil.dir/src/mp/util.cpp.o -resource-dir "/usr/lib/clang/16" -MT CMakeFiles/mputil.dir/src/mp/util.cpp.o.ddi -MD -MF CMakeFiles/mputil.dir/src/mp/util.cpp.o.ddi.d > CMakeFiles/mputil.dir/src/mp/util.cpp.o.ddi.tmp && mv CMakeFiles/mputil.dir/src/mp/util.cpp.o.ddi.tmp CMakeFiles/mputil.dir/src/mp/util.cpp.o.ddi
/bin/sh: CMAKE_CXX_COMPILER_CLANG_SCAN_DEPS-NOTFOUND: not found

e416d45528

cmake: Increase cmake policy version

Increase cmake policy version from 3.12 to 4.1 in standalone builds to stop
using very old and deprecated CMake policies in standalone builds.

Also stop overriding policy version if a parent project has already set one, so
parent projects are able to control which policies are used by default based on
their own practices and needs.

In the Bitcoin Core subtree, this change causes the libmultiprocess policy
version to increase from 3.12 to 3.22, which is the version Bitcoin Core sets.

This commit only changes the policy version, it does not change the specified
minimum version of cmake required to build the project, which is 3.12.

1c2958bfff

ryanofsky force-pushed on Sep 10, 2025

in ci/scripts/ci.sh:60 in 1c2958bfff

58 else
59   # Older versions of cmake can only build one target at a time with --target,
60   # and do not support -t shortcut
61   for t in "${BUILD_TARGETS[@]}"; do
62-    cmake --build . --target "$t" -- "${BUILD_ARGS[@]+"${BUILD_ARGS[@]}"}"
63+    cmake --build . --parallel --target "$t" -- "${BUILD_ARGS[@]+"${BUILD_ARGS[@]}"}"

hebasto commented at 12:34 pm on September 15, 2025:

6ba1050bef0b5ea5df35d3623c9e101cceb82677

If the --parallel option is used without an argument:

the native build tool’s default number is used.

This means that the build will be efficiently parallelized with the “Ninja” generator, but not with the “Unix Makefiles” generator. However, not all scripts in ci/configs/ pass -G Ninja.

ryanofsky commented at 9:44 am on September 17, 2025:

re: #209 (review)

This means that the build will be efficiently parallelized with the “Ninja” generator, but not with the “Unix Makefiles” generator. However, not all scripts in ci/configs/ pass -G Ninja.

On my system I see it passing -j to make which seems to work well in practice because this is a small build. Would happily accept a PR or patch to do something different, but I think this change is an improvement.

in ci/scripts/ci.sh:50 in 1c2958bfff

46+if ! cmake "$src_dir" "${cmake_args[@]}"; then
47+  # If cmake failed, try it again with debug options.
48+  # Could add --trace / --trace-expand here too but they are very verbose.
49+  cmake_args+=(--debug-find --debug-output --debug-trycompile --log-level=DEBUG)
50+  cmake "$src_dir" "${cmake_args[@]}" || : "cmake exited with $?"
51+  cat CMakeFiles/CMakeConfigureLog.yaml || true

hebasto commented at 12:36 pm on September 15, 2025:

6ba1050bef0b5ea5df35d3623c9e101cceb82677

This log file name has been used only since CMake 3.26.

ryanofsky commented at 9:44 am on September 17, 2025:

re: #209 (review)

6ba1050

This log file name has been used only since CMake 3.26.

Yes that’s part of the reason for adding || true here. I think this code could potentially print log files used by older versions of cmake too, and originally this change tried that, but since all CI jobs except one are using new enough versions of cmake it didn’t seem worth complexity. Would happily accept a PR to patch to improve this though.

purpleKarrot commented at 1:28 pm on September 15, 2025: contributor

I am not happy with this. The title of the topic says “Increase cmake policy version”, which I would expect to be a one line change. Instead, it has +95/-17 changes. The changes themselves are questionable:

the length of the bash buildscript is doubled. Why isn’t this written in a portable cmake script? Why is CMake preferred over handwritten Makefiles when the actual CI logic is written in bash?
The ``cmake_minimum_required()` call has a 25 line comment to explain a rationale. If you have to explain what you are doing, you are doing it wrong.
Putting Threads::Threads behind a non-namespaced abstraction mpdeps makes the build less robust.

Please don’t proceed in this direction. Make sure the CMakeLists.txt files are declarative and follow best practices instead of inventing new approaches and deviations.

hebasto commented at 1:30 pm on September 15, 2025: member

I’m struggling to create a MRE for the issue with the FindThreads module on FreeBSD. The following works just fine:

0cmake_minimum_required(VERSION 3.12...4.1)
1project(freebsd-threads CXX)
2set(CMAKE_CXX_STANDARD 20)
3set(CMAKE_CXX_STANDARD_REQUIRED YES)
4find_package(Threads REQUIRED)

What am I missing?

ryanofsky commented at 2:51 pm on September 15, 2025: collaborator

re: #209 (comment)

I am not happy with this. The title of the topic says “Increase cmake policy version”, which I would expect to be a one line change. Instead, it has +95/-17 changes.

Happy to change the title if you have a suggestion. This PR is only increasing the policy version and improving the CI, and I did not feel the CI changes were were mentioning since they are supporting changes, not the goal of the PR.

* the  length of the bash buildscript is doubled. Why isn't this written in a portable cmake script? Why is CMake preferred over handwritten Makefiles when the actual CI logic is written in bash?

I think the language question should be raised new issue if you want to discuss it. (But just to provide a little context here if it can avoid an argument about the language of a ~70 line script, I just personally find bash a lot nicer and more useful than cmake and am much familiar with it. More generally, bash is commonly used in CI, there many more developers who know bash than cmake.)

The script does increase from 37 to 67 lines here to do two things:

To support cloning and building unreleased versions of cap’n proto, so we can fix or report compatibility issues upstream cap’n proto earlier, and not be blindsided by new changes.
To provide better output when find_package fails, because by default the information cmake provides is not useful for debugging.

IMO both changes are worth making and do not have any real downsides.

If you have to explain what you are doing, you are doing it wrong.

Interesting point. It might be helpful if you could explain practical downsides of the change. Unless you also believe that if you have to explain why a change is wrong, then it must be right?

Putting Threads::Threads behind a non-namespaced abstraction mpdeps makes the build less robust.

Can you explain this a little more as well? Is there a practical downside here or a different approach you would recommend?

ryanofsky commented at 3:11 pm on September 15, 2025: collaborator

re: #209 (comment)

I’m struggling to create a MRE for the issue with the FindThreads module on FreeBSD. The following works just fine:

The error happens when you don’t have clang-scan-deps installed on the build machine, as seems to be the case currently with our freebsd and openbs vms. This issue is not specifically related to the find_package(Threads REQUIRED) call but the find_package(Threads REQUIRED) call makes it harder to understand what is wrong because it obscures the real error. Making the threads package optional produces a clearer error later, and I think it’s just more correct anyway since modern platforms don’t use separate threading libraries.

If it helps there is a failing CI run with more complete output:

https://github.com/ryanofsky/libmultiprocess/actions/runs/17622518177/job/50071319558

And the relevant cmake policy is https://cmake.org/cmake/help/latest/policy/CMP0155.html

ryanofsky commented at 3:28 pm on September 15, 2025: collaborator

Make sure the CMakeLists.txt files are declarative and follow best practices instead of inventing new approaches and deviations.

To be sure, I agree with you it would be better if Bitcoin Core followed cmake’s recommended best practices and used new policies instead of setting an really old policy version.

But bitcoin core isn’t doing that, and other bitcoin core developers have given reasons why it shouldn’t do that, and while I don’t agree with these reasons, I think it is a reasonable judgement call, and they are just choosing a different tradeoffs than I would. This PR makes libmultiprocess compatible with either approach for choosing a policy version and stops using really old policies (older than the ones bitcoin core chooses) in any case.

hebasto commented at 3:37 pm on September 15, 2025: member

The error happens when you don’t have clang-scan-deps installed on the build machine, as seems to be the case currently with our freebsd and openbs vms.

Here is a build log from GHA using FreeBSD:

 0-- The CXX compiler identification is Clang 19.1.7
 1-- Detecting CXX compiler ABI info
 2-- Detecting CXX compiler ABI info - done
 3-- Check for working CXX compiler: /usr/bin/c++ - skipped
 4-- Detecting CXX compile features
 5-- Detecting CXX compile features - done
 6-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
 7-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
 8-- Looking for pthread_create in pthreads
 9-- Looking for pthread_create in pthreads - not found
10-- Looking for pthread_create in pthread
11-- Looking for pthread_create in pthread - found
12-- Found Threads: TRUE
13-- Configuring done (1.2s)
14-- Generating done (0.0s)
15-- Build files have been written to: /home/runner/work/github-actions/github-actions/build

ryanofsky commented at 4:04 pm on September 15, 2025: collaborator

Here is a build log from GHA using FreeBSD:

Is that link right? It seems to be a 404 for me. I also don’t think I understand what it indicates if find_package(Threads REQUIRED) works in another build. We already know it works in many builds. You can see some builds where it is failing in #209 (comment) and https://github.com/ryanofsky/libmultiprocess/actions/runs/17622518177/job/50071319558. Making the threads library optional improves error output, works well and doesn’t have other downsides, and makes sense conceptually on modern platforms that do not use separate libraries for threading.

hebasto commented at 4:13 pm on September 15, 2025: member

Here is a build log from GHA using FreeBSD:

Is that link right? It seems to be a 404 for me.

Sorry. I’ve made that repo public. The link should work now.

I also don’t think I understand what it indicates if find_package(Threads REQUIRED) works in another build. We already know it works in many builds. You can see some builds where it is failing in #209 (comment) and https://github.com/ryanofsky/libmultiprocess/actions/runs/17622518177/job/50071319558. Making the threads library optional improves error output, works well and doesn’t have other downsides, and makes sense conceptually on modern platforms that do not use separate libraries for threading.

I want to report the issue upstream and I need an MRE for it.

ryanofsky commented at 4:41 pm on September 15, 2025: collaborator

I want to report the issue upstream and I need an MRE for it.

Oh, thanks! It seems like your branch is https://github.com/hebasto/github-actions/compare/250915-freebsd-cmake-threads and I agree I don’t see an obvious reason why that CI succeeds while my CI runs such as https://github.com/ryanofsky/libmultiprocess/actions/runs/17622518177/job/50071319558 with branch https://github.com/ryanofsky/libmultiprocess/commits/citest/policy at 3b8a0e6e4590ae50cb681de6034fdf547cdb8cf3 failed. Some things I might suggest:

Starting from my failing commit and deleting unnecessary steps to find a minimal failing case
Printing yaml build log unconditionally instead of only on failure in your current branch
Callling clang-scan-deps –version or seeing if it is available

ryanofsky commented at 4:44 pm on September 15, 2025: collaborator

I want to report the issue upstream and I need an MRE for it.

Also, I don’t think there is necessarily there is necessarily an upstream bug here, since we are asking cmake to look for a package and it fails and reports the failure. It just doesn’t seem to do a good job reporting the cause of the failure.

hebasto commented at 10:05 pm on September 15, 2025: member

@ryanofsky

Some things I might suggest:

Starting from my failing commit and deleting unnecessary steps to find a minimal failing case

Thank you for the suggestion.

The culprit turned out to be the “Ninja” generator, which was quite surprising to me.

ryanofsky commented at 11:08 pm on September 15, 2025: collaborator

The culprit turned out to be the “Ninja” generator, which was quite surprising to me.

Nice! And that is really surprising. I would expect the generator not to matter much during configuration.

ryanofsky commented at 9:56 am on September 17, 2025: collaborator

Thanks for the review! Note: first two commits are now part of #212

cmake: Increase cmake policy version #209

Reviews

Conflicts