fanquake
commented at 6:58 am on July 15, 2019:
member
TLDR: This updates our macOS toolchain to use a newer version of Clang, cctools (including new dependency on libtapi), LD64 and the macOS SDK.
I’ve been testing depends builds (HOST=x86_64-apple-darwin16) inside a Debian Buster Docker container, and running the resultant bitcoind and bitcoin-qt binaries on a macOS 10.14.4 system. The .dmg generated by a make deploy also mounts correctly on the same macOS system.
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
fanquake force-pushed
on Jul 15, 2019
in
depends/packages/native_libdmg-hfsplus.mk:3
in
31f7825155outdated
This works, but I get a flood of warnings at the end: log
Will test on macOS 10.15 Catalina later.
We should backport this to 0.18 given #16387 and ideally do a 0.18.2 tag before Catalina is out (September or October). Our 0.19 release is scheduled for November #15940, so otherwise there’s no tag to build from.
Concept ACK on bumping macOS from 10.10 to 10.12, though a small percentage of users would have to stay on the 0.18 branch (the last tag before backport).
The bottleneck for upgrading macOS is hardware, and that’s pretty durable. Here are the minimum specs for 10.10:
Minimum specs for 10.12:
Some refurbished machines for sale:
Some second hand machines at an even lower price point still work:
Some don’t:
fanquake
commented at 0:56 am on July 26, 2019:
member
I still had to do this annoying workaround:
Yep, that isn’t new with this change, but may go away if we can remove any reliance on having the headers in the now deprecated location. See details in the Command Line Tools section of the Xcode 10 release notes.
If I remove protobuf and native_protobuf it does build. I can then compile QT:
Did you try to build depends before installing the new headers? If you make clean in depends, then try rebuilding as usual it should work. Nothing Protobuf related is being changed in this PR.
fanquake force-pushed
on Jul 26, 2019
fanquake removed the label
Needs rebase
on Jul 26, 2019
jonasschnelli
commented at 8:44 am on July 26, 2019:
contributor
Aidium publishes their update stats and it looks like, >10% was using 10.11 this week.
https://www.adium.im/sparkle/
It’d say we wait with this.
Sjors
commented at 12:32 pm on July 26, 2019:
member
Did you try to build depends before installing the new headers? If you make clean in depends
I thought I did, but maybe not. I just tried make clean and now protobuf indeed just buids (on
536034b).
It’d say we wait with this.
If it doesn’t break anything else, maybe we can do this bump when we bump QT to 5.12 (LTS)?
fanquake force-pushed
on Jul 30, 2019
fanquake
commented at 2:05 am on July 30, 2019:
member
Updated to add some documentation (Linux) for the macOS 10.14 SDK extraction. This doesn’t have to be the final docs, but should be useful for someone on a Linux machine wanting to extract the SDK and test this PR.
It’d say we wait with this.
We don’t have to bump the minimum to 10.12 right now. We could do 10.11, or potentially leave it at 10.10 assuming there aren’t any issues using the new SDK. Regardless, I’d like the rest of the changes (or at least the new Clang) to make it into v0.19.0, so that building for macOS isn’t broken on Debian Buster #16052.
Sjors
commented at 5:19 pm on August 1, 2019:
member
From Apple: “All Macintosh computers that can run Mountain Lion, Mavericks, or Yosemite can run El Capitan” (10.11). I suspect many people just don’t realize they can upgrade.
I managed to build depends on macOS 10.14.6 (Mojave) with the minimum version set to 10.10
[…] may go away if we can remove any reliance on having the headers in the now deprecated location
That means this PR is not enough to support Catalina, at least not without some other workaround.
in
depends/hosts/darwin.mk:4
in
7decfd42efoutdated
All of the LD64_VERSION stuff (used by darwin_CC and darwin_CXX) stuff may be unnecessary now. I believe “-mlinker-version” was a hack around a bug that may have now been fixed.
Could you try removing the -mlinker-version=... and seeing if the build still succeeds?
The main reason for adding this dependency here rather than separately would be so that we can use our fresh compiler (clang) to build it. The same way we handle cctools.
Unless I’m misunderstanding, the way this is done currently, the system compiler is used. We’ve headed off a few frustrating incompatibilities by using a fixed compiler for the related toolchain components (cctools) in the past, and I think we should continue that.
Can you move the configure/build around to make that happen? Maybe handle both in _preprocess?
Sadly I can’t remember why the libc++abi.so removal is needed, maybe the SDK ships with the proper one?
As for dsymutil, that hack was added because the previous cctools didn’t ship with one, and some build scripts expect to find it. If the new cctools ships with one, or if whatever required it has been fixed, this can be removed.
We want this built statically if possible, otherwise this introduces a (rather unnecessary) LDPATH runtime requirement. Does build.sh have static lib support?
theuni
commented at 6:32 pm on August 2, 2019:
member
For the most part this looks great. I’ve added a few notes around libtapi which still needs a little work IMO.
I’ve not tried building yet, I’ll work on getting an SDK extracted.
theuni
commented at 8:47 pm on August 2, 2019:
member
I tested a few of my cleanup suggestions above, and they seem to work fine.
Feel free to pull the top two from here: https://github.com/theuni/bitcoin/commits/16392, or adapt them into your commits however’s easiest.
fanquake force-pushed
on Aug 5, 2019
fanquake
commented at 3:50 am on August 5, 2019:
member
@theuni I’ve incorporated some of your suggestions (trimming the LD64 stuff, and the clang hacks) and added you as co-author to the relevant commits.
We are now pointing back to upstream libdmg-hfsplus, with a single patch included in depends.
We are also now pointing back to the upstream cctools-port repo.
libtapi is now being built with our 6.0.1 Clang and I’ve removed one of the intermediate directories I was creating.
I still need to address:
We want this built statically if possible, otherwise this introduces a (rather unnecessary) LDPATH runtime requirement. Does build.sh have static lib support?
Sjors
commented at 9:58 am on August 5, 2019:
member
Building depends on macOS 10.14.6 Mojave still works as of 97b3349.
fanquake force-pushed
on Aug 6, 2019
fanquake
commented at 6:02 am on August 6, 2019:
member
I’ve updated this to pull the macOS DMG notes (with some minor changes) out of build-osx.md and put them in the contrib/macdeploy/README.md with the other deployment notes.
I’ve also added a new section on extracting the new SDK when your using macOS.
BlockMechanic
commented at 5:50 am on August 12, 2019:
contributor
:+1: Concept ACK, this may help alleviate issues i’ve had trying to create build bitcoin-qt on aarch64.
DrahtBot added the label
Needs rebase
on Aug 19, 2019
fanquake force-pushed
on Aug 20, 2019
fanquake removed the label
Needs rebase
on Aug 20, 2019
laanwj added this to the "Chasing Concept ACK" column in a project
laanwj
commented at 11:47 am on September 5, 2019:
member
I think it’s too late to do this for 0.19, might want to bump the milestone to 0.20 and then do it early in the merge window?
jonasschnelli
commented at 1:27 pm on September 5, 2019:
contributor
I also think it is too early for no good reason.
According to statista, in April 2019, 10.10 and 10.11 makes about 16% of mac installations.
fanquake
commented at 0:51 am on September 10, 2019:
member
I think it’s too late to do this for 0.19,
Ok. I’ve punted this to v0.20.0, for an early merge. Will address remaining comments here shortly.
fanquake removed this from the milestone 0.19.0
on Sep 10, 2019
fanquake added this to the milestone 0.20.0
on Sep 10, 2019
DrahtBot added the label
Needs rebase
on Sep 17, 2019
fanquake force-pushed
on Sep 23, 2019
fanquake removed the label
Needs rebase
on Sep 23, 2019
DrahtBot added the label
Needs rebase
on Oct 2, 2019
laanwj
commented at 5:23 pm on October 2, 2019:
member
Ok. I’ve punted this to v0.20.0, for an early merge. Will address remaining comments here shortly.
:smiling_imp:
dongcarl
commented at 7:12 pm on October 8, 2019:
member
Obtained using cd /Applications && tar -C Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.14.sdk.tar.gz MacOSX.sdk after downloading Xcode from App Store
fanquake
commented at 7:19 pm on October 8, 2019:
member
@dongcarl Has it been untarred inside depends/SDKs?
Every time I’ve seen fatal error: 'unistd.h' file not found has been because depends can’t find the macOS SDK.
dongcarl
commented at 7:20 pm on October 8, 2019:
member
@fanquake Here’s what I see inside the docker container:
0root@694ebefb6b63:/home/ubuntu/build/bitcoin/depends/SDKs/MacOSX.sdk# ls
1Entitlements.plist Library SDKSettings.json SDKSettings.plist System usr
dongcarl
commented at 7:22 pm on October 8, 2019:
member
Oh wait… I have too new of an Xcode… It’s actually MacOSX10.15.sdk that I tarred up…
fanquake
commented at 7:24 pm on October 8, 2019:
member
Oh wait… I have too new of an Xcode… It’s actually MacOSX10.15.sdk that I tarred up…
@dongcarl That would likely actually work fine.
The real problem is that you need to rename the directory to MacOSX10.14.sdk rather than MacOSX.sdk. See how the path is built in depends.
dongcarl
commented at 8:30 pm on October 8, 2019:
member
@fanquake Thanks for offline help for the above.
I also ran into another error mentioning libc++abi.so that was solved by readding a previously-removed line:
If @theuni could shed light on the context of f6448f99ef1cb688edffcd4b5934af938b0e228a, we should document it.
laanwj referenced this in commit
d882f63589
on Oct 14, 2019
BlockMechanic
commented at 7:29 pm on October 14, 2019:
contributor
I do not see an actual change to Qt 5.12 in the PR… is this in preparation ?
Fuzzbawls
commented at 11:39 pm on October 14, 2019:
contributor
Regarding the bumping of minimum supported macOS (OS X) version: I would say that 10.11 minimum is fine, but 10.12 requires newer CPU based SSE4.1 support that prevent otherwise perfectly viable CPUs (older Xeon CPUs mainly) from upgrading to 10.12.
sidhujag referenced this in commit
ecad5267fa
on Oct 15, 2019
fanquake force-pushed
on Oct 15, 2019
fanquake removed the label
Needs rebase
on Oct 15, 2019
in
contrib/macdeploy/README.md:9
in
b052792719outdated
8+```bash
9+make deploy
10+```
1112-This script should not be run manually, instead, after building as usual:
13+During the deployment process, the disk image window will pop up briefly
MarcoFalke
commented at 12:33 pm on October 16, 2019:
0During the deployment process, the disk image window will pop up briefly
:eyes:
dongcarl
commented at 2:21 pm on October 16, 2019:
member
fanquake
commented at 3:10 pm on October 16, 2019:
member
@dongcarl pulled your changes in as part of 1052e1a44e93a3a3cf2fe18a17916ea5f2dbe07f.
dongcarl
commented at 3:31 pm on October 16, 2019:
member
@fanquake Also, as mentioned above, you probably also need f6448f99ef1cb688edffcd4b5934af938b0e228a to fix Gitian builds. :grimacing:
MarcoFalke deleted a comment
on Oct 16, 2019
MarcoFalke deleted a comment
on Oct 16, 2019
MarcoFalke deleted a comment
on Oct 16, 2019
fanquake force-pushed
on Oct 16, 2019
fanquake
commented at 3:52 pm on October 16, 2019:
member
@dongcarl I’ve reverted the removal of that line in bf9488b1dd228a011f61ae21a642ee9ce69e03c2 as well updated 9d51eb8cca106ecaf9a05e52e15bfbf93fbde4a8 to point to the latest upstream version of cctools.
dongcarl
commented at 3:46 pm on October 17, 2019:
member
@fanquake Would you say that this is no longer WIP? I’d say it’s ready for merge :grimacing:
MarcoFalke
commented at 3:49 pm on October 17, 2019:
member
Not so fast. Let’s fix the travis and DrahtBot failure first. ;)
MarcoFalke
commented at 3:49 pm on October 17, 2019:
member
MarcoFalke
commented at 3:51 pm on October 17, 2019:
So the minimum macOS version is 10.12, the SDK version is 10.14, and the cross compile target is x86_64-apple-darwin16? This is not confusing at all.
fanquake
commented at 3:56 pm on October 17, 2019:
x86_64-apple-darwin16 maps to macOS 10.12, more info here. We’re building with the 10.14 SDK, however this can be newer, and is the same as we’ve been doing in previous releases. i.e using the 10.11 SDK while building for 10.7.
fanquake renamed this:
WIP build: macOS toolchain update
build: macOS toolchain update
on Oct 17, 2019
fanquake removed the label
Needs Conceptual Review
on Oct 17, 2019
fanquake removed this from the "Chasing Concept ACK" column in a project
fanquake force-pushed
on Oct 17, 2019
in
contrib/gitian-build.py:212
in
35250677b3outdated
207@@ -208,7 +208,7 @@ def main():
208 args.macos = 'm' in args.os
209210 # Disable for MacOS if no SDK found
211- if args.macos and not os.path.isfile('gitian-builder/inputs/MacOSX10.11.sdk.tar.gz'):
212+ if args.macos and not os.path.isfile('gitian-builder/inputs/MacOSX10.14.sdk.tar.gz'):
MarcoFalke
commented at 5:25 pm on October 17, 2019:
Should the gitian-build in master support compiling older releases?
dongcarl
commented at 6:58 pm on October 17, 2019:
Not sure that’s a burden we wanna take on. It’s probably going to be more sane for those with that use-case to check out the older tag.
MarcoFalke
commented at 12:28 pm on October 18, 2019:
Seems fine
in
doc/release-notes.md:55
in
35250677b3outdated
54 as frequently tested on them.
5556-From Bitcoin Core 0.17.0 onwards, macOS versions earlier than 10.10 are no
57-longer supported, as Bitcoin Core is now built using Qt 5.9.x which requires
58-macOS 10.10+. Additionally, Bitcoin Core does not yet change appearance when
59+From 0.19.0 onwards, macOS <10.12 is no longer supported.
MarcoFalke
commented at 1:22 pm on October 18, 2019:
0From 0.20.0 onwards, macOS <10.12 is no longer supported.
?
MarcoFalke added the label
Needs gitian build
on Oct 18, 2019
DrahtBot removed the label
Needs gitian build
on Oct 19, 2019
in
depends/hosts/darwin.mk:2
in
6552fbd779outdated
fanquake
commented at 12:40 pm on October 21, 2019:
Looks like you’re commenting on an outdated diff. This is 10.14 in 1728e348c7678504502fc28db109cc870e286695.
in
doc/build-osx.md:113
in
6552fbd779outdated
109@@ -110,7 +110,7 @@ tail -f $HOME/Library/Application\ Support/Bitcoin/debug.log
110 ```
111112 ## Notes
113-* Tested on OS X 10.10 Yosemite through macOS 10.14 Mojave on 64-bit Intel
114+* Tested on OS X 10.12 Sierra through macOS 10.14 Mojave on 64-bit Intel
In general this section refers to several different versions of Xcode. For SDK 10.14 we need Xcode 10. A more future proof instruction might be :This can be found here: https://developer.apple.com/download/more/ by filtering "xcode 10"
fanquake
commented at 12:37 pm on October 21, 2019:
Why update this URL, it’s meant to be pointing to the previously used SDK? The link to the newer SDK is later in the file.
Ah ok, let’s just drop that altogether then (can be done in a followup, along with bumping xcode 10.2.1 to 10.3 and putting the URL above the place where you need it instead of below).
in
contrib/macdeploy/README.md:17
in
1728e348c7outdated
20-During the process, the disk image window will pop up briefly where the fancy
21-settings are applied. This is normal, please do not interfere.
22+## SDK Extraction
2324-When finished, it will produce `Bitcoin-Qt.dmg`.
25+After Xcode version 7.x, Apple started shipping the `Xcode.app` in a `.xip` archive.
Nit: we can lose the historical reference and just say “Xcode.app is packaged in a .xip archive.
Sjors
commented at 10:49 am on October 21, 2019:
member
Tested 3525067 on macOS 10.15 Catalina. Although QT depends are built and configure log says checking whether to build Bitcoin Core GUI... yes (Qt5), the configure summary drops it: with gui / qt = no. See configure log.
extract-osx-sdk.sh needs to be updated as (though I’ve never used it)
macOS deploy instructions still refer to 10.11
there’s a reference to MAC_OS_X_VERSION_MIN_REQUIRED > 101100 in optionsdialog.cpp which presumably can be dropped?
fanquake force-pushed
on Oct 21, 2019
fanquake
commented at 12:47 pm on October 21, 2019:
member
Rebased on master and pushed some changes to fix the doc suggestions above.
Although QT depends are built and configure log says checking whether to build Bitcoin Core GUI… yes (Qt5), the configure summary drops it: with gui / qt = no. See configure log.
That’s unrelated to this PR. The reason Qt isn’t being picked up is in the log: configure: WARNING: PROTOC not found; bitcoin-qt frontend will not be built.
there’s a reference to MAC_OS_X_VERSION_MIN_REQUIRED > 101100 in optionsdialog.cpp which presumably can be dropped?
There are multiple similar changes that can be made post this PR, but I’m not going to do them here.
fanquake force-pushed
on Oct 21, 2019
Sjors
commented at 4:14 pm on October 21, 2019:
member
ACKdedc8c7 (modulo Travis being able to download the SDK)
Re: WARNING: PROTOC not found: that means it’s not finding protobuf in the depends. I’m also getting this issue on master, so indeed unrelated.
Without BIP70 I can build and do make deploy. Some light testing didn’t reveal any issues.
fanquake
commented at 6:06 pm on October 23, 2019:
member
Me and Cory have been discussing moving straight to Clang 9, instead of 6. However that has turned up some issues related to Boost and Clangs -dumpversion. Cory is going to PR some changes first, then we can circle back around to here.
fanquake force-pushed
on Oct 25, 2019
fanquake
commented at 7:49 pm on October 25, 2019:
member
Pushed up a rebase here as reqeusted by theuni. Going to look at using the 10.15 SDK instead of the 10.14 SDK as well.
MarcoFalke deleted a comment
on Nov 12, 2019
MarcoFalke
commented at 6:46 pm on November 12, 2019:
member
Pushed up a rebase here as reqeusted by theuni. Going to look at using the 10.15 SDK instead of the 10.14 SDK as well.
MarcoFalke
commented at 6:48 pm on November 12, 2019:
member
Anything left to do here except pushing up the new SDK to our depends mirror?
fanquake
commented at 7:26 pm on November 12, 2019:
member
Anything left to do here except pushing up the new SDK to our depends mirror?
I have some cctools-port changes to review and we will likely switch to a newer version of Clang. I will get that done over the next 2 days.
fanquake added the label
Waiting for author
on Nov 12, 2019
fanquake force-pushed
on Nov 22, 2019
fanquake removed the label
Waiting for author
on Nov 22, 2019
fanquake
commented at 2:11 pm on November 22, 2019:
member
Sorry for the delay. I’ve pushed up the discussed changes. This is now based on top of #17550. I’ve also reordered the commits to have the SDK bump (10.14) at the end.
One thing left TODO is diff the the 10.14 & 10.15 headers. As well as the LLVM headers. Still going to do this even though we can’t use the 10.15 SDK (Qt 5.9 or 5.12 wont build with it).
fanquake referenced this in commit
a6f5b6f47d
on Nov 22, 2019
fanquake force-pushed
on Nov 22, 2019
MarcoFalke added the label
Needs gitian build
on Nov 22, 2019
MarcoFalke removed the label
Needs gitian build
on Nov 22, 2019
sidhujag referenced this in commit
016b46aabb
on Nov 22, 2019
MarcoFalke added the label
Needs gitian build
on Nov 22, 2019
DrahtBot removed the label
Needs gitian build
on Nov 23, 2019
MarcoFalke added the label
Needs gitian build
on Dec 10, 2019
DrahtBot
commented at 11:38 pm on December 11, 2019:
member
Sjors
commented at 3:28 pm on January 31, 2020:
member
tACK248526e4f941edea8851a5a72a2a1da0ffdfabcf on macOS 10.15.3 (also tried make deploy), provided you can make Travis happy too. I’m not a build system connaisseur though, so an ACK from @theuni would be great.
We should drop the instructions for obtaining the 10.11 SDK, but that can wait for a followup.
laanwj
commented at 10:23 am on February 3, 2020:
member
hebasto
commented at 9:42 pm on February 3, 2020:
member
Tested ca5055a5aa07aba81a87cf12f6f0526a63c423b5
Cross compilation on Linux Mint 19.3 with depends and MacOSX10.14.sdk was contaminated by system Qt packages (#18042, #18045, #18051):
0$ make HOST=x86_64-apple-darwin16 -C depends
1$ ./autogen.sh
2$ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin16/share/config.site ./configure
3$ make
4...
5 CXX qt/qt_libbitcoinqt_a-peertablemodel.o
6qt/notificator.cpp:50:21: error: allocation of incomplete type 'QDBusInterface'
7 interface = new QDBusInterface("org.freedesktop.Notifications",
8 ^~~~~~~~~~~~~~
9...
The patch from #18051 was applied, then compilation passed successfully with loads of linker warnings like this:
0 AR qt/libbitcoinqt.a
1 CXXLD qt/bitcoin-qt
2 CXXLD qt/test/test_bitcoin-qt
3ld: warning: direct access in function '__ZN5boost10filesystem6detail9canonicalERKNS0_4pathES4_PNS_6system10error_codeE' from file '/home/hebasto/GitHub/bitcoin/depends/x86_64-apple-darwin16/lib/libboost_filesystem-mt-x64.a(operations.o)' to global weak symbol '__ZZN5boost6system16generic_categoryEvE25generic_category_instance' from file 'libbitcoin_server.a(libbitcoin_server_a-init.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
4...
The resulted Bitcoin-Core.dmg was successfully installed on macOS 10.14 then launched .
DrahtBot
commented at 7:02 am on February 4, 2020:
member
Empact
commented at 11:35 am on February 5, 2020:
member
Built depends successfully on 10.14.6
Is there anything more to say re the “obsolete mlinker-version option”? Is it obsolete only in that it is no longer required for build?
laanwj referenced this in commit
c8ce2632eb
on Feb 5, 2020
laanwj merged this
on Feb 5, 2020
laanwj closed this
on Feb 5, 2020
laanwj
commented at 1:30 pm on February 5, 2020:
member
@fanquake I apologize for my lateness, but here’s my outstanding changes so that we can generate and use the SDK with headers as requested: fanquake/bitcoin@macos-toolchain-update…dongcarl:2020-01-macos-sdk-with-headers
Thanks, please add these instructions in a new PR!
dongcarl
commented at 4:35 pm on February 5, 2020:
member
@fanquake I apologize for my lateness, but here’s my outstanding changes so that we can generate and use the SDK with headers as requested: fanquake/bitcoin@macos-toolchain-update…dongcarl:2020-01-macos-sdk-with-headers
Thanks, please add these instructions in a new PR!
This is a metadata mirror of the GitHub repository
bitcoin/bitcoin.
This site is not affiliated with GitHub.
Content is generated from a GitHub metadata backup.
generated: 2025-06-29 15:13 UTC
This site is hosted by @0xB10C More mirrored repositories can be found on mirror.b10c.me
