This is a slightly belated follow up to #17686 and some discussion with Cory. It’s not entirely clear if we should make this change due to the way the macOS dynamic loader appears to work. However I’m opening this for some discussion. Also related to #17768.
Issue:
LD64 doesn’t set the MH_BINDATLOAD bit in the header of MACHO executables, when building with -bind_at_load. This is in contradiction to the documentation:
0-bind_at_load
1 Sets a bit in the mach header of the resulting binary which tells dyld to
2 bind all symbols when the binary is loaded, rather than lazily.
The ld in Apples cctools does set the bit, however the cctools-port that we use for release builds, bundles LD64.
However; even if the linker hasn’t set that bit, the dynamic loader (dyld) doesn’t seem to ever check for it, and from what I understand, it looks at a different part of the header when determining whether to lazily load symbols.
Note that our release binaries are currently working as expected, and no lazy loading occurs.
Example:
Using a small program, we can observe the behaviour of the dynamic loader.
Conducted using:
0clang++ --version
1Apple clang version 11.0.0 (clang-1100.0.33.17)
2Target: x86_64-apple-darwin18.7.0
3
4ld -v
5@(#)PROGRAM:ld PROJECT:ld64-530
6BUILD 18:57:17 Dec 13 2019
7LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)
8TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)
0#include <iostream>
1int main() {
2 std::cout << "Hello World!\n";
3 return 0;
4}
Compile and check the MACHO header:
0clang++ test.cpp -o test
1otool -vh test
2...
3Mach header
4 magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
5MH_MAGIC_64 X86_64 ALL LIB64 EXECUTE 16 1424 NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
6
7# Run and dump dynamic loader bindings:
8DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=no_bind.txt ./test
9Hello World!
Recompile with -bind_at_load. Note still no BINDATLOAD flag:
0clang++ test.cpp -o test -Wl,-bind_at_load
1otool -vh test
2Mach header
3 magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
4MH_MAGIC_64 X86_64 ALL LIB64 EXECUTE 16 1424 NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
5...
6DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=bind.txt ./test
7Hello World!
If we diff the outputs, you can see that dyld doesn’t perform any lazy bindings when the binary is compiled with -bind_at_load, even if the BINDATLOAD flag is not set:
0@@ -1,11 +1,27 @@
1+dyld: bind: test:0x103EDF030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x103EDF030 = 0x7FFF70C9FA58
2+dyld: bind: test:0x103EDF038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x103EDF038 = 0x7FFF70CA12C2
3+dyld: bind: test:0x103EDF068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x103EDF068 = 0x7FFF70CA12B6
4+dyld: bind: test:0x103EDF070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x103EDF070 = 0x7FFF70CA1528
5+dyld: bind: test:0x103EDF080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x103EDF080 = 0x7FFF70C9FAE6
6<trim>
7-dyld: lazy bind: test:0x10D4AC0C8 = libsystem_platform.dylib:_strlen, *0x10D4AC0C8 = 0x7FFF73C5C6E0
8-dyld: lazy bind: test:0x10D4AC068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x10D4AC068 = 0x7FFF70CA12B6
9-dyld: lazy bind: test:0x10D4AC038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x10D4AC038 = 0x7FFF70CA12C2
10-dyld: lazy bind: test:0x10D4AC030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x10D4AC030 = 0x7FFF70C9FA58
11-dyld: lazy bind: test:0x10D4AC080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x10D4AC080 = 0x7FFF70C9FAE6
12-dyld: lazy bind: test:0x10D4AC070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x10D4AC070 = 0x7FFF70CA1528
Note: dyld also has a DYLD_BIND_AT_LAUNCH=1 environment variable, that when set, will force any lazy bindings to be non-lazy:
0dyld: forced lazy bind: test:0x10BEC8068 = libc++.1.dylib:__ZNSt3__113basic_ostream
Thoughts:
After looking at the dyld source, I can’t find any checks for MH_BINDATLOAD. You can see the flags it does check for, such as MH_PIE or MH_BIND_TO_WEAK here.
It seems that the lazy binding of any symbols depends on whether or not lazy_bind_size from the LC_DYLD_INFO_ONLY load command is > 0. Which was mentioned in #17686.
Changes:
This PR is one of Corys commits, that I’ve rebased and modified to make build. I’ve also included an addition to the security-check.py script to check for the flag.
However, given the above, I’m not entirely sure this patch is the correct approach. If the linker no-longer inserts it, and the dynamic loader doesn’t look for it, there might be little benefit to setting it. Or, maybe this is an oversight from Apple and needs some upstream discussion. Looking for some thoughts / Concept ACK/NACK.
One alternate approach we could take is to drop the patch and modify security-check.py to look for lazy_bind_size == 0 in the LC_DYLD_INFO_ONLY load command, using otool -l.
