Osx toolchain update #5582

tl;dr: Update to a newer stable toolchain and SDK for OSX without giving up any backwards compatibility. We can move to clang 3.5 as a next step which allows use to use libc++ and the 10.10 sdk, but we'll need to find a build that works in gitian/travis first.

Switch to a new, better maintained fork of cctools: https://github.com/tpoechtrager/cctools-port

I've forked this and will be working on it some as well: https://github.com/theuni/cctools-port

This brings in: cctools v862 ld64: v241.9

It also fixes 64bit builds, so there's no longer any need to use a 32bit clang. Since clang is no longer tied to an old/crusty 32bit build, clang has been upgraded to 3.3. Unfortunately, there's a bug in 3.4 that breaks builds. 3.5 works fine, but there are no binary builds compatible with precise, which is currently used for gitian and travis. We could always build our own if necessary.

After updating to stable clang/linker/cctools, it's possible to use a more recent SDK. The current SDK (10.7) through the most recent 10.10 have all been built/tested successfully, both with and without 10.6 compatibility. However, 10.10 requires clang 3.5.

SDKs >= 10.9 use libc++ rather than libstdc++. This is verified working as well.

Sorry for the thought-dump, there are lots of pieces here. Despite that, this should be a pretty smooth upgrade. The messier parts (bumping the minimum runtime requirement, switching build SDKs, switching to libc++, rolling our own clang, etc) are left for later. All of those things have been tested and work fine, they just need to be discussed first.

I was a bit foggy last night when writing the description after going in circles with SDKs and toolchains all day, so I'll try to provide a better breakdown here.

XCode

When building in the normal OSX ecosystem (download a monolithic Xcode release and use it without asking questions), the compiler, linker, build tools, and SDKs all come together. However, they retain a relatively high amount of back/forward-compatibility with each-other. Each Xcode release typically comes with ~2 SDKs for use. That’s ignoring iOS, which we’re not interested in.

The current stable release is Xcode 6.1.1.

SDKs

The SDKs contain features available for the respective OS version, but they also remain backwards-compatible depending on a compiler switch that sets the minimum desired runtime environment. A new SDK is typically released just before each major OS upgrade.

Example: It’s possible to build against the 10.10 sdk while using -mmacosx-version-min=10.6. Weak linking is used to ensure that missing symbols on 10.6 won’t cause runtime failure, but if the symbols are detected at runtime on 10.7-10.10, they can be used. That looks something like this: #5477 (comment)

As you can see, 10.10 features can happily be compiled and used, but care needs to be taken to ensure that they’re tested at runtime first. So the only downside in building against the latest/greatest SDK is that someone might have forgotten to add a runtime check for a new feature which would cause a crash in older environments. Using an older SDK in the same scenario would cause a build failure instead.

Toolchains

Each version of XCode ships with a new toolchain, which is comprised of a compiler (clang/clang++), linker (ld64, apple’s own, not gold/bfd), and cctools (binutils in the rest of the unix world: ar, ranlib, nm, etc).

For the most part, the toolchains and SDKs are unrelated. A sufficiently new toolchain can build using any SDK.

Stdlib

Prior to 10.7, OSX shipped with only gnu libstdc++ runtime libs. The 10.7 SDK switched to clang and added the libc++ option, so devs could build using libc++ if they were willing to set a minimum supported runtime of 10.7.

The 10.9 SDK made libc++ the default, however it’s still possible to use -stdlib=libstdc++ to avoid the change. Also, setting -mmacosx-version-min=10.6 will force the use of libstdc++, since it’s not available there.

The main take-away here is that in order to build using libc++, the minimum runtime version will require a bump from 10.6 to 10.7.

Putting it all together

Because we roll our own toolchain and build environments in Linux, we’re free to mix and match the above components however we desire, as long as they all play nicely together.

For the work in this PR, I originally bumped to the following: clang 3.5 (xcode 6.1.1 uses “Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)”) 10.10 SDK (from xcode 6.1.1) ld64 241.9 (same version used in xcode 6.1.1) cctools 862 (same version used in xcode 6.1.1)

That put us 100% up-to-date with apple’s current stable tools. Everything worked well.

However, there’s not an official clang 3.5 binary for Precise, which is used by Gitian and Travis. There is a Trusty binary, but it requires a newer libstdc++ and can’t run.

To get around that, I next tried downgrading clang to 3.4, which is compatible with Precise. Unfortunately, there’s a bug in 3.4 that breaks darwin builds, as discussed briefly with @gmaxwell last night. I’m still working to track down the exact cause.

Finally, I arrived at clang 3.3, which builds and works fine, however it is not capable of building with the 10.10 sdk. It’s fine up to 10.9 though.

Takeaway

So the final result here is that with these changes we're using the latest/greatest apple tools and SDKs and we're in good shape for the foreseeable future, with the exception that we're currently slightly constrained by our build environments.

Before these changes, we were using much older versions of all of these tools, which limited the SDKs we could target. Those versions of cctools/ld64 also only worked when compiled as 32bit. Since ld64 requires the use of clang's libLTO, that meant that we were also stuck using a 32bit clang.

While I was looking at the clang-3.4 problem yesterday, robbak on IRC happened to be wrestling with the same problem on FreeBSD. I tested a vanilla osx build from llvm.org natively on osx, and it had the same problem.

I've managed to track down the problem, and have a confirmed fix. This https://github.com/llvm-mirror/clang/commit/c174fe5f125f749eaf5ea7cc41f3cbf049bc5ada needs to be backported to the 3.4 branch. I'll report it to the LLVM list after the new year and see if they're interested in doing a 3.4.3 release, since the official 3.4.2 binaries are quite broken for (at a minimum) OSX and FreeBSD.

Nice. Tested. This fixes the Qt issue of building on 10.9 with the 10.10 SDK.

Thanks for figuring this all out @theuni! Is this intended for 0.10rc2 or just master?

@laanwj master only. I don't foresee any problems, but it'd be far too risky for 0.10 at the last minute.

First commit dropped, and rebased on top of #5586

utACK