cmake: Emulate GNU Autotools ‘make check -jN’ #1294

1. 
   hebasto commented at 1:42 pm on May 4, 2023: member

   It seems parallelism is requested for make check target when building with CMake and the “Unix Makefiles” generator which is the default on Linux and macOS. See:

   • https://gnusha.org/secp256k1/2023-03-08.log:

   13:53 < sipa> make -j check doesn’t actually parallellize in the cmake build, which is a bit unfortunate

   • https://github.com/hebasto/bitcoin/pull/15#issuecomment-1531155633:

   Is make check being run with multiple jobs? I assume not

   With this PR, it is possible to build like that:

   0cmake -S . -B  build
   1make -C build -j $(nproc)
   2make check -C build -j $(nproc)
   

   Making this PR a draft as it seems low priority and the diff seems a bit hackish.

   My personal preference is to use the CMake’s native ctest command :)

   Anyway, this PR fixes an item in #1235.

   ---

   UPD. Here is a couple of alternative approaches:

   • set the CTEST_PARALLEL_LEVEL environment variable:

   0export CTEST_PARALLEL_LEVEL=$(nproc)
   

   • use the test target and (undocumented) ARGS variable:

   0make test -C build ARGS=-j$(nproc)
   

2. cmake: Emulate GNU Autotools 'make check -jN' 3d9479a1b3
3. hebasto cross-referenced this on May 4, 2023 from issue build: Meta-issue for follow-ups to initial CMake merge (#1113) by hebasto
4. in CMakeLists.txt:240 in 3d9479a1b3

   239   enable_testing()
   240+  if(CMAKE_GENERATOR STREQUAL "Unix Makefiles")
   241+    # Emulate GNU Autotools 'make check -jN'.
   242+    add_custom_target(check
   243+      COMMAND @$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan \"Running tests...\"
   244+      COMMAND ${CMAKE_CTEST_COMMAND} --force-new-ctest-process -$(MAKEFLAGS)
   

   ---

   ---

   

   real-or-random commented at 8:29 am on May 5, 2023:

   Passing $MAKEFLAGS to ctest is indeed pretty hackish (and ctest doesn’t like -j without an integer and some users have more MAKEFLAGS set in their env…)
5. 
   real-or-random commented at 8:30 am on May 5, 2023: contributor

   My personal preference is to use the CMake’s native ctest command :)

   Yep, we should just update the CMake build docs in the README. We could even remove the “make check” emulation entirely if this is more CMake-ish. What do you think?

6. 
   hebasto commented at 9:10 am on May 5, 2023: member

   My personal preference is to use the CMake’s native ctest command :)

   Yep, we should just update the CMake build docs in the README. We could even remove the “make check” emulation entirely if this is more CMake-ish. What do you think?

   As we discussed at CoreDev, it would be better for the common good to use a unified approach for both projects, Bitcoin Core and libsecp256k1. A similar discussion is currently taking place in Bitcoin Core (here, here and here). Regarding your question, I think it would be advisable to ask for the opinions of developers who expressed concerns about make check, namely @sipa and @fanquake.

7. 
   hebasto commented at 9:20 am on May 5, 2023: member

   I’ve updated the PR description with a couple of alternatives.
8. 
   real-or-random commented at 12:32 pm on January 17, 2024: contributor

   As we discussed at CoreDev, it would be better for the common good to use a unified approach for both projects, Bitcoin Core and libsecp256k1 @hebasto Has been any progress on this point in Core?

9. real-or-random added the label build on Jan 17, 2024
10. 
    hebasto commented at 12:50 pm on January 17, 2024: member

    As we discussed at CoreDev, it would be better for the common good to use a unified approach for both projects, Bitcoin Core and libsecp256k1

    @hebasto Has been any progress on this point in Core?

    In the https://github.com/hebasto/bitcoin/tree/cmake-staging, we lean to use the native CMake’s invocations in all scripts (CI, Guix etc). This approach is agnostic to the underlying build system, and make check parallelism issue is no longer a concern.

    However, it might change in the future :)

11. real-or-random referenced this in commit 3777e3f36a on Jan 17, 2024
12. 
    real-or-random commented at 2:24 pm on January 17, 2024: contributor

    In the hebasto/bitcoin@cmake-staging, we lean to use the native CMake’s invocations in all scripts (CI, Guix etc). This approach is agnostic to the underlying build system, and make check parallelism issue is no longer a concern.

    Let’s stick with this for now. Then people can use ctest -j3 or ctest -j$(nproc) or they can also set CTEST_PARALLEL_LEVEL="$(nproc)" similar to MAKEFLAGS="-j($proc)". Just be aware that ctest -j (without an explicit int) doesn’t have an effect.

13. hebasto closed this on Jan 17, 2024

    ---

14. real-or-random cross-referenced this on Jan 17, 2024 from issue cmake: Recommend native CMake commands in README by real-or-random

Contributors
hebasto real-or-random

Labels
build