doc: explain why passing -mlinker-version is required when cross-compiling #19407

I have been down a 🐇 hole. Closes #19359.

When Clang is compiled, a check is run to define HOST_LINK_VERSION as the output of $CMAKE_LINKER -v. Note the this is the version of the linker being used to compile Clang itself.. and this check is only run when compiling Clang for macOS.

In the Clang driver, if HOST_LINK_VERSION has been defined, there is some additional runtime functionality. An -mlinker-version argument, with the value of HOST_LINK_VERSION will be added to the linker arguments, if -mlinker-version has not been passed in by the user.

This is a bit weird, as by default, you are setting -mlinker-version to the version of the linker that was used to build the Clang binary, not the linker which will be used when compiling. The commit which introduced the functionality, https://github.com/llvm/llvm-project/commit/628fcf4e3b79ba9f41dd1b49b1aba7a20b68fc0e, described it as a "hack", that should be replaced. However, that was 10 years ago, and the behaviour is still here.

In the Darwin driver, a check is done for the -mlinker-version argument. If there is no argument, the version will default to 0. Given the above, this should never happen when using Clang for macOS. A series of comparisons are then performed, to check whether the linker version is modern enough to enable certain features, like -demangle.

What this means

macOS

A Clang compiled for macOS, i.e clang+llvm-8.0.0-x86_64-apple-darwin, will have HOST_LINKER_VERSION set to the version of the linker used to compile Clang itself.

At runtime, -mlinker-version=HOST_LINKER_VERSION will be added to the linker args, if -mlinker-version wasn't passed in. In the Darwin driver, additional arguments, like -demangle, will be added to the linker arguments, because HOST_LINKER_VERSION was likely some very modern version of lld or ld64.

Linux (cross compilation in depends)

A Clang compiled for Linux, i.e clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-14.04, which we now use for macOS builds in depends, will behave differently. As it's built for Linux, HOST_LINKER_VERSION was not defined at compile time, and there will be no default behaviour of appending -mlinker-version=HOST_LINKER_VERSION to the linker args. Thus, unless you pass in -mlinker-version yourself, when the version checks are done in the Darwin driver, no modern linker features will be enabled, as the version will have defaulted to 0.

Therefore, it's important that we continue to pass -mlinker-version="our LD64 version" as part of our compilation flags, if we want to have "modern" linker features enabled for our macOS builds.

Summary

Clang 8. Building a macOS binary. Link line with path arguments trimmed.

Note: Most links here are pointing to the 8.x branch of LLVM/Clang, as we are using that version in depends.

Note: To add a little more confusion, you wont see -mlinker-version X in your compile flags, you'll see -target-linker-version X.

ACK a8d39b88406e2047746366355666b5f603105a2e

I guess this option isn't mentioned in the MacOS build doc as that doesn't deal with cross-compilation? But, it seems this information would be useful in some documentation. I expect no one to look in that ,mk file unless they've already descended into a another warren of rabbit holes.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #17919 ([DO NOT MERGE] depends: Allow building with system clang by dongcarl)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

I guess this option isn't mentioned in the MacOS build doc as that doesn't deal with cross-compilation?

Correct. When you're building on macOS, for macOS, Apples Clang etc takes care of wiring things up correctly. As I mentioned in my comment here, I think I'll try and combine a more concise version of these notes into some depends documentation.