build: use -mbig-obj for Windows debug builds #31458

pull fanquake wants to merge 1 commits into bitcoin:master from fanquake:win_debug_O0_big_obj changing 1 files +9 −0
  1. fanquake commented at 2:06 pm on December 10, 2024: member

    Windows cross builds using -O0 (-DCMAKE_BUILD_TYPE=Debug) currently fail to compile, as some objects have too many sections. As a convenience, add -mbig-obj to our compile flags when using the Debug build type, so that if someone tries to build this way, it will work.

    This would also be needed if we switched the depends flags to -O0. (maybe in #29796).

    -mbig-obj

    On PE/COFF target this option forces the use of big object file format, which allows more than 32768 sections.

    Closes #28109. Seems unlikely that we are going to break up the relevant object files, and the main issue is still the inclusion of Boost.

  2. DrahtBot commented at 2:06 pm on December 10, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31458.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK theuni, hebasto
    Concept ACK TheCharlatan
    Stale ACK laanwj

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

  3. DrahtBot added the label Build system on Dec 10, 2024
  4. fanquake added the label DrahtBot Guix build requested on Dec 10, 2024
  5. laanwj commented at 2:20 pm on December 10, 2024: member
    Is there a drawback to enabling this? Does it increase the size of binaries in non-O0 case? Break compatibility with old tools (we don’t care), etc?
  6. TheCharlatan commented at 2:26 pm on December 10, 2024: contributor
    Concept ACK
  7. theuni commented at 8:42 pm on December 10, 2024: member

    Is there a drawback to enabling this? Does it increase the size of binaries in non-O0 case? Break compatibility with old tools (we don’t care), etc?

    This was my concern as well. I’m seeing various answers around the net.

    I guess it’s not the end of the world if we further slow down the debug binaries, but it’d be a shame to pessimize the release ones for an issue that doesn’t affect them. Maybe do this only in debug mode?

  8. fanquake force-pushed on Dec 11, 2024
  9. fanquake commented at 10:19 am on December 11, 2024: member

    I guess it’s not the end of the world if we further slow down the debug binaries

    Slow binaries are likely still better than no binaries, given that they don’t currently compile.

    I did initally (locally) have this scoped to just the debug flags in depends, but dropped that given it wont work all the time, given the CMake flag handling in that regard is still a bit of a mess. I’ve changed the PR so that this is now only applied when the build type is Debug.

    Doing the following (which currently fails on master) now works with this PR:

    0make -C depends/ HOST=x86_64-w64-mingw32
    1cmake -B build --toolchain /root/ci_scratch/depends/x86_64-w64-mingw32/toolchain.cmake -DCMAKE_BUILD_TYPE=Debug
    2cmake --build build
    
  10. laanwj commented at 10:34 am on December 11, 2024: member

    ACK cfdf8d91111838887344b920d6b680a559ca3d2b now that it’s scoped to just Debug

    I guess it’s not the end of the world if we further slow down the debug binaries, but it’d be a shame to pessimize the release ones for an issue that doesn’t affect them. Maybe do this only in debug mode?

    Right. To be clear i don’t care about the Debug binaries either, but yes making the windows release binaries slower/bigger would be bad. Especially as mingw-w64 already isn’t the toolchain producing the most optimal windows binaries. But scoping it to Debug is fine.

  11. DrahtBot requested review from TheCharlatan on Dec 11, 2024
  12. fanquake commented at 1:46 pm on December 11, 2024: member

    Guix Build (aarch64):

     07d96f621b3d764c5699e366079b7d96330b1d8e7358375723d4e4575bdab3dd2  guix-build-cfdf8d911118/output/aarch64-linux-gnu/SHA256SUMS.part
     162707fe70727d75d6b097e70ceaa362bbdfd1e6b1fdbeee41ee38e0a35a04825  guix-build-cfdf8d911118/output/aarch64-linux-gnu/bitcoin-cfdf8d911118-aarch64-linux-gnu-debug.tar.gz
     29902efd7dc5a3d484728ab93ea3aee59e2b61a100c2760fe95cd9c4de08190f5  guix-build-cfdf8d911118/output/aarch64-linux-gnu/bitcoin-cfdf8d911118-aarch64-linux-gnu.tar.gz
     3fb3ef53d43e15a07ce8b64668adcc2626f5d1f984b3198127c3cb18f934b8b07  guix-build-cfdf8d911118/output/arm-linux-gnueabihf/SHA256SUMS.part
     4d4e67ab4961d404c2f05a98e9a98475fc312e63173cc7487d67d9783b141ced7  guix-build-cfdf8d911118/output/arm-linux-gnueabihf/bitcoin-cfdf8d911118-arm-linux-gnueabihf-debug.tar.gz
     5b2656b83bd5c9379daf1a35ac1bd82f7912d32f7cfb83bd586b672c6dca5f78d  guix-build-cfdf8d911118/output/arm-linux-gnueabihf/bitcoin-cfdf8d911118-arm-linux-gnueabihf.tar.gz
     606fb7ef2266323327d31ebaa8c44cd5ff25550721b813b2f3830e5a7bae46779  guix-build-cfdf8d911118/output/arm64-apple-darwin/SHA256SUMS.part
     7607fc3e24c1f02753e1fcb548bd69140b1a0cacf05e533b0cfd0b1000c69d4f1  guix-build-cfdf8d911118/output/arm64-apple-darwin/bitcoin-cfdf8d911118-arm64-apple-darwin-unsigned.tar.gz
     8a0cb8803e05a20d81172cfc9a4c677b84c14f313e3659a33f2cc174430a06c86  guix-build-cfdf8d911118/output/arm64-apple-darwin/bitcoin-cfdf8d911118-arm64-apple-darwin-unsigned.zip
     97c0d840d3fac4caa8b677049e927372ad96166ec3938d4a39465fcfcddbd61cc  guix-build-cfdf8d911118/output/arm64-apple-darwin/bitcoin-cfdf8d911118-arm64-apple-darwin.tar.gz
    10061addd042f8db1abd1f00e13b2eb24184ea2f7731b56b1fcf6f1d306ff3db08  guix-build-cfdf8d911118/output/dist-archive/bitcoin-cfdf8d911118.tar.gz
    111dbd919c512a3269092dbc4f1e5d4973379167b487f2cc41db3689461f1021ab  guix-build-cfdf8d911118/output/powerpc64-linux-gnu/SHA256SUMS.part
    12ab88fb4a1d055c921350e087011bbaf19f60d0f2aa032066b0dce64eaebb715d  guix-build-cfdf8d911118/output/powerpc64-linux-gnu/bitcoin-cfdf8d911118-powerpc64-linux-gnu-debug.tar.gz
    13c8168db8a7d64bccfa27838b33f4300e8efe75e526834948e98a53d03b7b658b  guix-build-cfdf8d911118/output/powerpc64-linux-gnu/bitcoin-cfdf8d911118-powerpc64-linux-gnu.tar.gz
    14bdb2d33d4c7e923289e1a9ad63d61933765a68790bef5cf023679f659bac80cf  guix-build-cfdf8d911118/output/riscv64-linux-gnu/SHA256SUMS.part
    1526e87d7a044cd0c2d6c6ace2f0e37659575ff3dd99799569bd63936bfa974fff  guix-build-cfdf8d911118/output/riscv64-linux-gnu/bitcoin-cfdf8d911118-riscv64-linux-gnu-debug.tar.gz
    16eab9c725295bff3eb084da052b4470f2dfd840392b2b0bce55507299b84e5184  guix-build-cfdf8d911118/output/riscv64-linux-gnu/bitcoin-cfdf8d911118-riscv64-linux-gnu.tar.gz
    1758dead5e434e27d0ef78fbd5217b01f52b8ca0046cfea052cbeafc89a071da28  guix-build-cfdf8d911118/output/x86_64-apple-darwin/SHA256SUMS.part
    1864d7f0a51343a56097d904be862ebef9843071736d6894a26855d202e239d573  guix-build-cfdf8d911118/output/x86_64-apple-darwin/bitcoin-cfdf8d911118-x86_64-apple-darwin-unsigned.tar.gz
    19ef7ff11245ebb35354958d94eb1479247a78016c654854859636930d90671a32  guix-build-cfdf8d911118/output/x86_64-apple-darwin/bitcoin-cfdf8d911118-x86_64-apple-darwin-unsigned.zip
    20cb0d4e21ff86e1ad89798426a76f8ed0c76ca881e90fc4fd0015d22d92d02e1b  guix-build-cfdf8d911118/output/x86_64-apple-darwin/bitcoin-cfdf8d911118-x86_64-apple-darwin.tar.gz
    2149626bb821c455080318b0b5182b2baa5c4c89163d75aab9167c86a6a093142b  guix-build-cfdf8d911118/output/x86_64-linux-gnu/SHA256SUMS.part
    22773b95cecde41bf76665df6727ef5826fd7a0df3862658d4a1cfbf45193b95de  guix-build-cfdf8d911118/output/x86_64-linux-gnu/bitcoin-cfdf8d911118-x86_64-linux-gnu-debug.tar.gz
    237cd05daa1c8ad84f8dc08c29608d6260114a3611654eb678a48d3c646cfeb7b9  guix-build-cfdf8d911118/output/x86_64-linux-gnu/bitcoin-cfdf8d911118-x86_64-linux-gnu.tar.gz
    24166656fc14944fea880552bb94a7e40655650863b825d0eeac349f8925558ea1  guix-build-cfdf8d911118/output/x86_64-w64-mingw32/SHA256SUMS.part
    25e48af1419d64095075b931735f8dd4a0799299296ed3cf8c32f2f6525a123bdf  guix-build-cfdf8d911118/output/x86_64-w64-mingw32/bitcoin-cfdf8d911118-win64-debug.zip
    26f4c71272f482358051ea6856350ed7edea37244ffb91df78bd50057e12d4c80c  guix-build-cfdf8d911118/output/x86_64-w64-mingw32/bitcoin-cfdf8d911118-win64-setup-unsigned.exe
    277b1b7489a43b9ff11784cf2cf752f4a3d19dc2c6300c538f3ec2c78de96f9099  guix-build-cfdf8d911118/output/x86_64-w64-mingw32/bitcoin-cfdf8d911118-win64-unsigned.tar.gz
    284f73e067fd5a70670eb4d48277c3a91cf8035e2dcd640e5adc625941eb57b2ab  guix-build-cfdf8d911118/output/x86_64-w64-mingw32/bitcoin-cfdf8d911118-win64.zip
    
  13. fanquake removed the label DrahtBot Guix build requested on Dec 11, 2024
  14. DrahtBot commented at 5:10 pm on December 11, 2024: contributor

    Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]

    File commit 37e49c2c7ca5969124830d79b2cb31041c570755(master) commit d34bcd8050ae24568baeeddc18f0b3e92b67c8f4(master and this pull)
    SHA256SUMS.part a66e47c4d98e61d1... 89b14b0e2e2ccfbe...
    *-aarch64-linux-gnu-debug.tar.gz d655799ea0297fc1... 7679210c500a860b...
    *-aarch64-linux-gnu.tar.gz 935c1dd1b5bea91a... e63ce82c9fee087d...
    *-arm-linux-gnueabihf-debug.tar.gz 97c0ddfc15e8d6c8... 6ee4f6d265d5d0f4...
    *-arm-linux-gnueabihf.tar.gz 095a52e358438536... 90b9a0fac7fcdedd...
    *-arm64-apple-darwin-unsigned.tar.gz d7588fcee7d76fcc... 7c527da778c95578...
    *-arm64-apple-darwin-unsigned.zip 39c022a19ff84a34... 7c865a4c62ba7cf7...
    *-arm64-apple-darwin.tar.gz 8f61db7b2cf39dcf... bf0342524b2af3f6...
    *-powerpc64-linux-gnu-debug.tar.gz 8130f9196b1047eb... 81939feb1d339571...
    *-powerpc64-linux-gnu.tar.gz 7201b24aa920e6c3... 41dbec8b5b252c2b...
    *-riscv64-linux-gnu-debug.tar.gz 5236af13f5d5aa45... 951ce6c9117d9ddb...
    *-riscv64-linux-gnu.tar.gz 5fc333a6f5bcc630... 631ba70cb3df0b51...
    *-x86_64-apple-darwin-unsigned.tar.gz e99ef6f3757ae8e1... e4e53f7d9643a797...
    *-x86_64-apple-darwin-unsigned.zip 12747d0086d554fa... fbf45cd8b6ca23fe...
    *-x86_64-apple-darwin.tar.gz b8f5fb35cc1779d6... 81b3ca6f267937c5...
    *-x86_64-linux-gnu-debug.tar.gz a2f6ad4833ca2e81... 1d65e270192d4eb0...
    *-x86_64-linux-gnu.tar.gz ba42d29b05c732cc... bb7aeafc309ff344...
    *.tar.gz e5d216290f1860d1... ed88645021a89c50...
    guix_build.log 79785e69b22d1878... 82161da80a6eee9b...
    guix_build.log.diff 0e3f62a2ce360cb5...
  15. theuni commented at 7:08 pm on December 11, 2024: member

    @fanquake I know this is going to make you grumpy but… I think that check a bit too simplistic depending on how robust the fix needs to be. Specifically:

    • CMAKE_BUILD_TYPE is case-sensitive
    • This check won’t work for multi-config builds

    https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#build-configurations

    I think try_append_cxx_flags is just kinda incompatible with multi-config because the solution is to use a generator expression, which is evaluated a build-time.

    Since this isn’t the most important thing, maybe just use the hack in the example at the link above, and add a note that it doesn’t work for multi-config?

    0string(TOLOWER ${CMAKE_BUILD_TYPE} build_type)
    1if (build_type STREQUAL debug)
    2  try_append_cxx_flags("-Wa,-mbig-obj" TARGET core_interface SKIP_LINK)
    3endif()
    
  16. hebasto commented at 1:08 pm on December 13, 2024: member

    Concept ACK.

    The same problem happens when building on Windows with --config Debug. This case should also be fixed.

    I think try_append_cxx_flags is just kinda incompatible with multi-config because the solution is to use a generator expression, which is evaluated a build-time.

    We have core_interface_debug for such a case: https://github.com/bitcoin/bitcoin/blob/78f1bff7099b854bb71e57d0307b8c21a1ac32c5/CMakeLists.txt#L203-L209

  17. fanquake commented at 1:12 pm on December 13, 2024: member

    The same problem happens when building on Windows with –config Debug. This case should also be fixed.

    if you’d like that fixed in this PR, you’ll need to supply a diff, otherwise you can fix it in a followup.

  18. hebasto commented at 2:08 pm on December 13, 2024: member

    The same problem happens when building on Windows with –config Debug. This case should also be fixed.

    if you’d like that fixed in this PR, you’ll need to supply a diff, otherwise you can fix it in a followup.

    Sure :)

    Fill free to grab it: https://github.com/hebasto/bitcoin/commit/c4fa619dc3346677c0b3bb89f0dca8b2a43899a9:

     0--- a/CMakeLists.txt
     1+++ b/CMakeLists.txt
     2@@ -298,6 +298,15 @@ if(WIN32)
     3     try_append_linker_flag("-Wl,--major-subsystem-version,6" TARGET core_interface)
     4     try_append_linker_flag("-Wl,--minor-subsystem-version,2" TARGET core_interface)
     5   endif()
     6+
     7+  # Workaround producing large object files, which cannot be handled by the assembler.
     8+  # More likely to happen with no, or lower levels of optimisation.
     9+  # See discussion in [#28109](/bitcoin-bitcoin/28109/).
    10+  if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
    11+    try_append_cxx_flags("/bigobj" TARGET core_interface_debug SKIP_LINK)
    12+  else()
    13+    try_append_cxx_flags("-Wa,-mbig-obj" TARGET core_interface_debug SKIP_LINK)
    14+  endif()
    15 endif()
    16 
    17 # Use 64-bit off_t on 32-bit Linux.
    
  19. build: use `-mbig-obj` for mingw-w64 Debug builds
    Windows cross builds using `-O0` currently fail to compile, as some
    objects have too many sections. As a convenience, add `-mbig-obj` to
    our compile flags when using the `Debug` build type, so that if someone
    tries to build this way, it will work.
    
    This would also be needed if we switched the depends flags to -O0.
    
    `-mbig-obj`
    
    > On PE/COFF target this option forces the use of big object
    > file format, which allows more than 32768 sections.
    
    Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
    2b9ff4a66d
  20. fanquake force-pushed on Dec 13, 2024
  21. fanquake commented at 3:33 pm on December 13, 2024: member
  22. theuni commented at 4:34 pm on December 13, 2024: member

    We have core_interface_debug for such a case:

    Aha, nice! Happy to be wrong on that one :)

  23. theuni approved
  24. theuni commented at 4:43 pm on December 13, 2024: member
    utACK 2b9ff4a66d31d06fd97be9729b8f71458b5e982e
  25. DrahtBot requested review from laanwj on Dec 13, 2024
  26. DrahtBot requested review from hebasto on Dec 13, 2024
  27. hebasto approved
  28. hebasto commented at 7:11 am on December 14, 2024: member

    ACK 2b9ff4a66d31d06fd97be9729b8f71458b5e982e, tested in the following scenarios:

    • on Windows 11 Pro 24H2 + VS 17.12.3:
    0cmake -B build --preset vs2022-static -DVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON -DVCPKG_MANIFEST_FEATURES="wallet;tests" -DBUILD_GUI=OFF
    1cmake --build build --target bitcoind --config Debug
    
    • on Ubuntu 24.04:
    0make -C depends HOST=x86_64-w64-mingw32 NO_QT=1
    1cmake -B build --toolchain depends/x86_64-w64-mingw32/toolchain.cmake -DCMAKE_BUILD_TYPE=Debug
    2cmake --build build --target bitcoind
    
  29. fanquake renamed this:
    build: use `-mbig-obj` for mingw-w64 builds
    build: use `-mbig-obj` for Windows debug builds
    on Dec 16, 2024
  30. fanquake merged this on Dec 17, 2024
  31. fanquake closed this on Dec 17, 2024

  32. fanquake deleted the branch on Dec 17, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-21 12:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me