build: Fix static Darwin builds #21782

pull hebasto wants to merge 1 commits into bitcoin:master from hebasto:210426-darwin changing 4 files +16 −1
  1. hebasto commented at 9:06 pm on April 26, 2021: member

    From ##bitcoin-core-gui IRC channel:

    <fanquake> hebasto: I think I might have solved the darkmode issue, or at least some of the macOS qt problems. Will PR shortly <hebasto> was I on the right way? <fanquake> Yes I think it’s related to how the SDK is being setup <hebasto> fanquake: expecting of adding -Wl,-sdk_version to linker flags :) <fanquake> hebasto: heh. That may be added at some point, however I think the solution here will be even simpler

    Assuming, that fanquake’s solution does not involve linker flags, I’m suggesting an alternative one that does use them.

    When doing static builds, for some reasons, SDK version is not handled correctly when Darwin Driver passes it to a linker via the -platform_version option.

    With this PR we pass the -platform_version option to a linker explicitly.

    Fix #21771.

    This change causes harmless warnings:

    0ld: warning: passed two min versions (10.14.0, 10.14) for platform macOS. Using 10.14.
    

    Building with depends on macOS Big Sur 11.2.3 (20D91):

    0% ld -v
    1@(#)PROGRAM:ld  PROJECT:ld64-609.8
    2BUILD 15:07:46 Dec 18 2020
    3configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
    4LTO support using: LLVM version 12.0.0, (clang-1200.0.32.29) (static support for 27, runtime is 27)
    5TAPI support using: Apple TAPI version 12.0.0 (tapi-1200.0.23.5)
    
    • on master (683dda2a70e7a210996fa34be23bd0c563971ba9)
    0% otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
    1Load command 9
    2      cmd LC_BUILD_VERSION
    3  cmdsize 32
    4 platform 1
    5    minos 10.14
    6      sdk n/a
    7   ntools 1
    8     tool 3
    9  version 609.8
    
    • with this PR
    0% otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION                       
    1Load command 9
    2      cmd LC_BUILD_VERSION
    3  cmdsize 32
    4 platform 1
    5    minos 10.14
    6      sdk 11.1
    7   ntools 1
    8     tool 3
    9  version 609.8
    
  2. hebasto added the label Bug on Apr 26, 2021
  3. hebasto added the label macOS on Apr 26, 2021
  4. hebasto added the label Build system on Apr 26, 2021
  5. DrahtBot commented at 10:50 pm on April 26, 2021: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #19817 (build: macOS toolchain bump by fanquake)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  6. jarolrod commented at 2:13 am on April 27, 2021: member

    macOS 10.14 SDK doesn’t seem to have -platform_version available. When trying to perform a native depends build on macOS 10.14 I get the following error when building libevent:

    0ld: unknown option: -platform_version
    1clang: error: linker command failed with exit code 1 (use -v to see invocation)
    2configure:4267: $? = 1
    3configure:4307: result: no
    

    The above error would mean this PR wouldn’t work in its current form.

    Besides broken compatibility with 10.14, this PR seems to be a step in the right direction. The following will be based on my testing of the state of dark mode: https://github.com/bitcoin-core/gui/pull/275#pullrequestreview-639461197

    macOS 10.15

    Master PR
    115327729-6e549700-a15d-11eb-8c46-7c830007d71e 10 15
     0otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION  
     1       
     2Load command 9
     3      cmd LC_BUILD_VERSION
     4  cmdsize 32
     5 platform 1
     6    minos 10.14
     7      sdk 10.15.6
     8   ntools 1
     9     tool 3
    10  version 609.8
    

    macOS 11.2.3

    Master PR
    115330418-fccb1780-a161-11eb-84ca-1c87a6b5f448 11 2
    0Load command 9
    1      cmd LC_BUILD_VERSION
    2  cmdsize 32
    3 platform 1
    4    minos 10.14
    5      sdk 11.1
    6   ntools 1
    7     tool 3
    8  version 609.8
    
  7. MarcoFalke added the label Needs gitian build on Apr 27, 2021
  8. MarcoFalke added the label Needs Guix build on Apr 27, 2021
  9. hebasto commented at 10:56 am on April 27, 2021: member

    @RandyMcMillan

    Tested on macOS 10.14.6 (18G103)

    Did you test a static build?

  10. build: Fix static Darwin builds
    When doing static builds, for some reasons, SDK version is not handled
    correctly when Darwin Driver passes it to a linker via the
    -platform_version option.
    
    This change makes passing the -platform_version option to a linker
    explicit.
    d7c83cb067
  11. hebasto force-pushed on Apr 27, 2021
  12. hebasto commented at 11:47 am on April 27, 2021: member

    Updated 15d41b3b0120d9093036dc3d1740df228be25cc7 -> d7c83cb067992f38d0b6f6d3a7896f971718e24b (pr21782.01 -> pr21782.02, diff):

    • fixed compatibility with macOS Mojave
  13. RandyMcMillan commented at 12:04 pm on April 27, 2021: contributor

    For clarity:

    brew install mingw-w64 was required to build depends on Mojave

    Screen Shot 2021-04-27 at 8 01 27 AM Screen Shot 2021-04-27 at 8 02 08 AM

  14. hebasto commented at 12:09 pm on April 27, 2021: member

    @RandyMcMillan

    For clarity:

    brew install mingw-w64 was required to build depends on Mojave

    Sorry, I don’t understand this.

    mingw-w64 is required only when cross-compiling for Windows target.

    This pr deals with macOS target only.

    UPDATE: To make a static build with depends on Mojave 10.14.6:

    0$ make -C depends
    1$ ./autogen.sh
    2$ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin18.7.0/share/config.site ./configure
    3$ make
    
  15. RandyMcMillan commented at 3:41 pm on April 27, 2021: contributor

    Almost got thru it.

    0Darwin ₿ 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64
    

    Screen Shot 2021-04-27 at 11 43 01 AM

    commit d7c83cb067992f38d0b6f6d3a7896f971718e24b

    Used commands:

    0$ make -C depends
    1$ ./autogen.sh
    2$ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin18.7.0/share/config.site ./configure
    3$ make
    

    Screen Shot 2021-04-27 at 11 32 12 AM

  16. hebasto commented at 4:49 pm on April 27, 2021: member
    @RandyMcMillan Could make clean && make help?
  17. RandyMcMillan commented at 9:06 pm on April 27, 2021: contributor

    Error:

    Screen Shot 2021-04-27 at 4 38 09 PM

    Disabled Tests:

    0CONFIG_SITE=$PWD/depends/x86_64-apple-darwin18.7.0/share/config.site ./configure --disable-tests
    1make
    

    Screen Shot 2021-04-27 at 4 55 15 PM

    Besides the test error d7c83cb06 seems good - This may have fixed a dbus issue I was seeing as well.

  18. jarolrod commented at 9:12 pm on April 27, 2021: member
    @RandyMcMillan I’m not sure what’s wrong there, but, d7c83cb builds just fine for me on macOS 10.14
  19. hebasto commented at 9:19 pm on April 27, 2021: member

    @RandyMcMillan

    To figure out problems with this PR, could you confirm a successful build the master branch with depends on macOS 10.14?

  20. fanquake commented at 3:43 am on April 28, 2021: member

    How is this comment:

    macOS 10.14 SDK doesn’t seem to have -platform_version available. When trying to perform a native depends build on macOS 10.14 I get the following error

    solved with the current change? If the linker doesn’t support -platform_version, then building with a depends prefix would still be broken, because -platform_version still gets added to LDFLAGS when configuring. @jarolrod It’s not the 10.14 SDK that doesn’t support -platform_version, it’s the linker. What version of ld are you using where -platform_version isn’t supported?

  21. jarolrod commented at 3:58 am on April 28, 2021: member

    @fanquake on my macOS 10.14.6 partition:

    0ld -v
    1
    2@(#)PROGRAM:ld  PROJECT:ld64-450.3
    3BUILD 18:16:53 Apr  5 2019
    4configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
    5LTO support using: LLVM version 10.0.1, (clang-1001.0.46.4) (static support for 22, runtime is 22)
    6TAPI support using: Apple TAPI version 10.0.1 (tapi-1001.0.4.1)
    

    I failed to build natively on macOS with depends using 15d41b3, but it builds using d7c83cb. Also, I don’t see -platform_version under the LDFLAGS section of my configure output on 10.14.6. I do see it on 10.15 and 11.

  22. fanquake commented at 4:57 am on April 28, 2021: member

    Right, because ld actually throws an error, AX_CHECK_LINK_FLAG wont append the flags. I had been thinking it was just warning, and because the call wasn’t using $LDFLAG_WERROR they would end up appended and then we’d fail to link.

    Regardless, as-is, this change is a bit of an undocumented hack. It need explanation either in depends, or in configure.

    Just to clarify, as I understand it, static builds aren’t “broken”, it’s just that our current macOS toolchain, whether cross-compiling via depends, or building on macOS, doesn’t fill out the sdk field in the LC_BUILD_VERSION command. This is an issue because Qt is now detecting at runtime, via the version in that field, whether or not to enable some features, such as “Dark mode”?

  23. jarolrod commented at 5:50 am on April 28, 2021: member

    @fanquake hopefully this can clarify a bit, going by my “state of dark mode” testing https://github.com/bitcoin-core/gui/pull/275#pullrequestreview-639461197

    From the linked review, we can see that this sdk field issue, seemingly, manifests itself only on native depends builds and with the bumped toolchain cross-compile (did not test bumped toolchain native depends build).

    Linux Cross Compile The sdk field is filled in when cross-compiling for macOS. Albeit, with a (maybe) strange value for sdk. Anyways, Dark Mode is enabled. Below is the output of otool on a bitcoin-qt binary cross-compiled from linux on master:

     0otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
     1
     2Load command 9
     3      cmd LC_BUILD_VERSION
     4  cmdsize 32
     5 platform 1
     6    minos 10.14
     7      sdk 505.0
     8   ntools 1
     9     tool 3
    10  version 530.0
    

    Native macOS Depends Build On a native macOS depends build, the SDK value is not filled in. it registers as N/A using otool. Dark Mode is disabled. The only exception, before this PR, is macOS 10.14 where there is a value for sdk.

    Bumped Toolchain Cross Compile Building a macOS cross-compile using the bumped toolchain (https://github.com/bitcoin/bitcoin/pull/19817). The sdk field is not filled in and Dark Mode is disabled. Here is the output of otool on a bitcoin-qt binary cross-compiled from Linux on the bumped toolchain branch:

     0bitcoin % otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
     1
     2Load command 9
     3      cmd LC_BUILD_VERSION
     4  cmdsize 32
     5 platform 1
     6    minos 10.14
     7      sdk n/a
     8   ntools 1
     9     tool 3
    10  version 609.0
    

    Where this PR fits in From my testing, this PR fills in the sdk field of the affected builds (native depends, and bumped toolchain cross-compile)

    Native depends build have the sdk field filled in and Dark Mode works ✅

    I combined the bumped toolchain and this PR on this branch: https://github.com/jarolrod/bitcoin/tree/ctool-and-static-darwin. Bumped Toolchain Cross Compile otool output with this pr:

     0otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
     1
     2Load command 9
     3      cmd LC_BUILD_VERSION
     4  cmdsize 32
     5 platform 1
     6    minos 10.14
     7      sdk 10.15.6
     8   ntools 1
     9     tool 3
    10  version 609.0
    

    Another (welcome) side-effect of this PR is that the sdk field is properly filled in on a cross-compile from the current master. otool reports back the expected value of 10.15.1 instead of 505.0 that we currently get without this PR.

     0otool -l src/qt/bitcoin-qt | grep -A 7 -B 1 BUILD_VERSION
     1
     2Load command 9
     3      cmd LC_BUILD_VERSION
     4  cmdsize 32
     5 platform 1
     6    minos 10.14
     7      sdk 10.15.1
     8   ntools 1
     9     tool 3
    10  version 530.0
    

    Qt The inference here is that the sdk version has a role in whether or not ‘Dark Mode’ is enabled.

  24. DrahtBot commented at 6:51 am on April 28, 2021: member

    Gitian builds

    File commit f8f55522c4697b77b1f739a2ed0c483396d62154(master) commit ef4c52d980021ee89e4e45e781657ca49b8cc736(master and this pull)
    bitcoin-core-linux-22-res.yml d273ec877cddaf98... 6abb1328d3f9409b...
    bitcoin-core-osx-22-res.yml 08d13a0cb2d974a2... ba7f640bbcd7afed...
    bitcoin-core-win-22-res.yml ede99ec33c5a8d56... 39274666f0a21b1e...
    *-aarch64-linux-gnu-debug.tar.gz 1de746dfaa67f9c4... 1e8f86ad5942a76a...
    *-aarch64-linux-gnu.tar.gz 7b273c1e3aa31429... 25a4bc14c35a10b9...
    *-arm-linux-gnueabihf-debug.tar.gz 18221fa46c7add35... b0a49fd9d8ad6cf3...
    *-arm-linux-gnueabihf.tar.gz d5deb88e186a2364... a43cd5227bec8515...
    *-osx-unsigned.dmg 604184eac9e1a1f3... b9a38e0b009274fe...
    *-osx64.tar.gz 1f00241bdd70b453... 5d0a3c2a7b4a0cb1...
    *-powerpc64-linux-gnu-debug.tar.gz d270a69fd2282a9a... 101c1dd1ccc731c7...
    *-powerpc64-linux-gnu.tar.gz 074b0c3f3b97a9da... 57fbfbaaad1f68c9...
    *-powerpc64le-linux-gnu-debug.tar.gz ec241e33b68b4884... 5dfe0b83166c99bb...
    *-powerpc64le-linux-gnu.tar.gz d4ed9cfa2b07a865... 064cc1c427a1343a...
    *-riscv64-linux-gnu-debug.tar.gz 8cc40fa9d29ff86c... f2669743db7b54cb...
    *-riscv64-linux-gnu.tar.gz 6b7fa831a17028f5... faa34c775c2d75fb...
    *-win64-debug.zip 4164b00781ed2f3c... 4a285a79874fe6af...
    *-win64-setup-unsigned.exe 5bcfe14df146cb9d... af46cf71ea85452a...
    *-win64.zip 18c7aab75e106c71... 87583f4945b08104...
    *-x86_64-linux-gnu-debug.tar.gz 8fcb8a5289c16944... 76320b69e446713e...
    *-x86_64-linux-gnu.tar.gz bd6c67109928d0e8... eeda642239c10d9b...
    *.tar.gz 6e60d9ec8da1a7df... a4a7e6d193d15dda...
    linux-build.log 44ff3907b0e24252... 1e57bea95ea78312...
    osx-build.log 9c4a058a6161e80e... cb4c58d92f0f1ca2...
    win-build.log 722bf1061e446819... db14ca9338d878ed...
    bitcoin-core-linux-22-res.yml.diff 18737c624495b3af...
    bitcoin-core-osx-22-res.yml.diff ad05356a1d0801fe...
    bitcoin-core-win-22-res.yml.diff c5850541825899cc...
    linux-build.log.diff 20453e65173389b4...
    osx-build.log.diff dc8709e5d117fcea...
    win-build.log.diff d4833329b78364f7...
  25. DrahtBot removed the label Needs gitian build on Apr 28, 2021
  26. fanquake commented at 9:31 am on April 28, 2021: member

    From the linked review, we can see that this sdk field issue, seemingly, manifests itself only on native depends builds and with the bumped toolchain cross-compile (did not test bumped toolchain native depends build).

    The SDK field has also been present in the LC_VERSION_MIN_MACOSX command; LC_BUILD_VERSION is actually fairly new. I’m not convinced this approach is the way to solve our issue. I think the more correct fix is #21793.

  27. hebasto commented at 3:23 pm on April 28, 2021: member
    Closing in favor of #21793 which is more correct solution for #21771.
  28. hebasto closed this on Apr 28, 2021

  29. hebasto commented at 3:25 pm on April 28, 2021: member

    @fanquake

    The SDK field has also been present in the LC_VERSION_MIN_MACOSX command; LC_BUILD_VERSION is actually fairly new.

    I cannot see LC_VERSION_MIN_MACOSX in the otool output on macOS.

  30. fanquake commented at 1:16 am on April 29, 2021: member

    I cannot see LC_VERSION_MIN_MACOSX in the otool output on macOS.

    It’s going to depend on which toolchain you’ve used. The most recently released binaries have LC_VERSION_MIN_MACOSX not LC_BUILD_VERSION.

  31. hebasto deleted the branch on Apr 29, 2021
  32. fanquake referenced this in commit bb11a988fa on May 1, 2021
  33. sidhujag referenced this in commit 788d32deca on May 1, 2021
  34. MarcoFalke removed the label Needs Guix build on May 5, 2021
  35. DrahtBot locked this on Aug 18, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-08 22:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me