While testing #19530 I noticed that we couldn’t call dsymutil after LTO:
0../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
This updates configure to call AC_PATH_TOOL so that we end up with the
full path to dsymutil, similar to otool and install_name_tool, ie:
/bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-dsymutil.
While testing #19530 I noticed that we couldn't call dsymutil after LTO:
```bash
../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
```
This updates configure to call `AC_PATH_TOOL` so that we end up with the
full path to dsymutil, similar to `otool` and `install_name_tool`, ie:
`/bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-otool`.
Nice find. Though I don’t understand the interaction with lto or #19530 ?
Because I’m not sure what’s calling it, I’m not sure if we should be concerned about others missing as well.
From my libtool:
0# Tool to manipulate archived DWARF debug symbol files on Mac OS X.
1DSYMUTIL="x86_64-apple-darwin16-dsymutil"
2
3# Tool to change global to local symbols on Mac OS X.
4NMEDIT="x86_64-apple-darwin16-nmedit"
5
6# Tool to manipulate fat objects and archives on Mac OS X.
7LIPO="x86_64-apple-darwin16-lipo"
8
9# ldd/readelf like tool for Mach-O binaries on Mac OS X.
10OTOOL="/home/cory/dev/bitcoin2/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-otool"
Should we add one for nmedit as well? I know we don’t need lipo.
Nice find. Though I don’t understand the interaction with lto or #19530 ?
The interaction is that at least when building a dylib, and using LTO, LLVM adds some actions to the compiler runtime which invoke dsymutil. You can test this by building #19530 + your patch and doing a libconsensus build (I’ve trimmed some other warnings that aren’t relevant to this PR):
0make -C depends/ -j6 HOST=x86_64-apple-darwin16
1./autogen.sh
2./configure --prefix=/bitcoin/depends/x86_64-apple-darwin16 --with-gui=no --disable-wallet --disable-tests --disable-bench --with-utils=no --with-daemon=no --with-libs=yes
3make -j8
4...
5make[3]: Leaving directory '/bitcoin/src/secp256k1'
6 CXXLD libbitcoinconsensus.la
7../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
8make[2]: Leaving directory '/bitcoin/src'
With this patch, dsymutil is found, but unless you’re building for debugging it’s not really interesting, so I also tried that.
When building #19530 + your patch + this PR and --enable-debug I’d end up with warnings and an empty .dSYM:
0./autogen.sh
1./configure --prefix=/bitcoin/depends/x86_64-apple-darwin16 --with-gui=no --disable-wallet --disable-tests --disable-bench --with-utils=no --with-daemon=no --with-libs=yes --enable-debug
2make -j8 V=1
3...
4libtool: link: /bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-dsymutil .libs/libbitcoinconsensus.0.dylib || :
5warning: (x86_64) /tmp/lto.o unable to open object file: No such file or directory
6<snip>
7warning: (x86_64) /tmp/lto.o unable to open object file: No such file or directory
8warning: no debug symbols in executable (-arch x86_64)
9libtool: link: (cd ".libs" && rm -f "libbitcoinconsensus.dylib" && ln -s "libbitcoinconsensus.0.dylib" "libbitcoinconsensus.dylib")
The solution seemed to be to pass -object_path_lto through to ld64. Doing that I end up with a properly generated .dSYM:
0./autogen.sh
1./configure --prefix=/bitcoin/depends/x86_64-apple-darwin16 --with-gui=no --disable-wallet --disable-tests --disable-bench --with-utils=no --with-daemon=no --with-libs=yes --enable-debug LDFLAGS="-Wl,-object_path_lto /temp_dir"
2make -j8
3...
4llvm-dwarfdump-7 src/.libs/libbitcoinconsensus.0.dylib.dSYM
5src/.libs/libbitcoinconsensus.0.dylib.dSYM/Contents/Resources/DWARF/libbitcoinconsensus.0.dylib: file format Mach-O 64-bit x86-64
6
7.debug_info contents:
80x00000000: Compile Unit: length = 0x0000036c version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08 (next unit at 0x00000370)
9
100x0000000b: DW_TAG_compile_unit
11 DW_AT_producer ("clang version 8.0.0 (tags/RELEASE_800/final)")
12 DW_AT_language (DW_LANG_C_plus_plus)
13 DW_AT_name ("support/cleanse.cpp")
14 DW_AT_stmt_list (0x00000000)
15 DW_AT_comp_dir ("/bitcoin/src")
16 DW_AT_GNU_pubnames (true)
17 DW_AT_APPLE_optimized (true)
18 DW_AT_low_pc (0x0000000000000fc0)
19 DW_AT_high_pc (0x0000000000000ff8)
20
210x0000002a: DW_TAG_namespace
22 DW_AT_name ("std")
23
240x0000002f: DW_TAG_namespace
25 DW_AT_name ("__1")
26 DW_AT_export_symbols (true)
27
280x00000034: DW_TAG_imported_declaration
29 DW_AT_decl_file ("/bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/usr/include/c++/v1/cstring")
30 DW_AT_decl_line (69)
31 DW_AT_import (0x00000000000000d7)
32<snip>
330x00000620: DW_TAG_const_type
34 DW_AT_type (0x00000000000005d9 "AES_state")
35
360x00000625: DW_TAG_subprogram
37 DW_AT_linkage_name ("_ZL17KeySetupTransformP9AES_statePKS_")
38 DW_AT_name ("KeySetupTransform")
39 DW_AT_decl_file ("/bitcoin/src/./crypto/ctaes/ctaes.c")
40 DW_AT_decl_line (378)
41 DW_AT_APPLE_optimized (true)
42 DW_AT_inline (DW_INL_inlined)
There’s at least one mention of LTO and dsymutil in the Darwin Driver that would suggest that we shouldn’t have to pass this through explicitly? Although maybe something isn’t hooked up correctly:
// If we are using full LTO, then automatically create a temporary file // path for the linker to use, so that it’s lifetime will extend past a // possible dsymutil step. …. // If we are using thin LTO, then create a directory instead.
In any case that is discussion for the #19530. I’ll make sure I dig around in LLVM some more.
Should we add one for nmedit as well? I know we don’t need lipo.
I haven’t seen any obvious calls / usage of nmedit so far, but if you’d prefer I can add it here anyways.
This is some good digging! I think this change looks good, but I’d like to understand fully as I think we may want to upstream some clang/libtool fixes here.
With this patch,
dsymutilis found, but unless you’re building for debugging it’s not really interesting, so I also tried that.
I don’t understand how this patch actually helps dsymutil to be found, though, if it’s invoked internally by clang? How does it help to give libtool the correct path? I would guess that libtool/Make are exporting DSYMUTIL, but I don’t see clang checking for it in the environment.
From what I can tell, clang should find the correct dsymutil with no further help due to our use of -B.
Rather, it looks like libtool itself is calling it directly:
0../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
Could you paste the relevant part of your generated libtool?
Ok, managed to find it in my local libtool. Not sure how I missed it last time I grepped:
0# Commands used to build a shared archive.
1archive_cmds="\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring \$single_module~\$DSYMUTIL \$lib || :"
So, yes, this is coming from libtool and not from clang (though it’s not clear if it’s actually needed anymore, since clang seems to call it as necessary). Which means libtool needs to know the path.
ACK ef3d4ce4c301caa57946f772f554678cd872fca8.
Also, as far as I can tell, clang should manage to find dsymutil just fine.
I am also seeing these with debugging on:
warning: (x86_64) /tmp/lto.o unable to open object file: No such file or directory warning: (x86_64) /tmp/lto.o unable to open object file: No such file or directory
This one looks like a real clang bug, let’s discuss it in #19530.
Labels
Build system