release: use static libstdc++ and disable reduced exports by default

theuni commented at 2:23 am on February 24, 2015: member

Since the release of 0.10.0, we’ve seen several reports of failed sanity tests due to failing to catch basic exceptions. At the moment, Debian Wheezy fails to run the official Linux binary, and FreeBSD fails to run when compiled natively. I assume that there are several other broken setups as well.

From what I’ve seen, the brokenness is always due to type_info visibility. Building with -fvisibility=hidden can cause problems with old/busted stdlibs. See here for some background.

It’s nearly impossible to tell (without blacklisting std libs like we already do with boost) if libstdc++ is going to play nice with –enable-reduce-exports. Rather than trying, just disable it by default. Packagers can re-enable if it’s known to be safe.

For Linux, when compiling against a new libstdc++ but using an older one at runtime, exceptions can again be missed. Rather than take any chances, use -static-libstdc++. This means we don’t have to worry about trying to maintain a compat wrapper.

Binaries become slightly bigger (about 1M each), but not enough to worry about imo. The final compressed sizes:

0before:
117416   built/linux/bitcoin-0.10.0-linux32.tar.gz
217196   built/linux/bitcoin-0.10.0-linux64.tar.gz

0after:
118920   build/out/bitcoin-0.10.99-linux32.tar.gz
218804   build/out/bitcoin-0.10.99-linux64.tar.gz

Debian Wheezy is verified fixed. @koobs has verified that building without reduced exports fixes freebsd.

build: disable reduced exports by default

This is really a packager's option. While it's helpful to encourage devs to
test this option for daily builds, it's not reliable in several real-world
use-cases. Some older libstdc++ runtimes (freebsd 9, debian wheezy, for
example) fail to properly catch exceptions due to mismatched type_info.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19664 for more info.

3ee028f131

build: remove libstdc++ backwards-compat

Backwards-compatibility for libstdc++ is not limited to straightforward abi
changes. Symbol visibility also needs to be taken into consideration, and
that really can't be addressed simply.

Instead, just static-link libstdc++ for backwards-compat.

aa3673064c

build: change reduce exports/static libstdc++ options for gitian and travis

For Gitian releases:
  - Windows builds remain unchanged. libstdc++ was already linked statically.
  - OSX builds remain unchanged. libstdc++ is tied to the SDK and not worth
    messing with.
  - Linux builds now statically link libstdc++.

For Travis:
  - Match the previous behavior by adding --enable-reduce-exports as
  necessary.
  - Use static libstdc++ for the full Linux build.

06715165f9

gitian: fix x86_64 build with static libstdc++ c95ac83e51

build: fix typo in configure help 3448b132c4

theuni commented at 2:23 am on February 24, 2015: member

ping @gmaxwell, your thoughts would be appreciated since we discussed this a few nights ago.

gmaxwell commented at 3:27 am on February 24, 2015: contributor

Sad to hear that visibility control is being problematic, especially as we move towards making a library out of things. In the long term I don’t think we can have the library that way (or callers will use random internal symbols). In the short term sure.

The size increase is smaller than I expected; 8% is worth a reduction in consensus risk and getting binaties working on debian wheezy esp considering how unstable in the fine details stdc++ has been historically. I’m not aware of any problems with static stdc++ of the sort that libc, and many commercial applications do this already. So indeed, it sounds good to me.

koobs commented at 4:11 am on February 24, 2015: none

@theuni While we’re here, –disable-reduced-exports in the error messaging needs to be changed to –disable-reduce-exports

theuni commented at 4:37 am on February 24, 2015: member

@koobs It’s fixed, see 3448b13. @gmaxwell As far as I’m aware, all trouble is caused by old (gcc 4.2 era) libstdc++. So eventually this won’t be much of a problem (or it will be replaced with newer problems…).

But more to your point, I suppose I’ve explained the changes here poorly. For our official releases, we still build with hidden symbols as before, we just also link libstdc++ statically. The change here in that regard is that –disable-reduced-exports becomes the default in our configure… builders can re-enable if they’d like (as we do). So it should be effectively a no-op as far as official release symbol visibility goes.

gmaxwell commented at 4:55 am on February 24, 2015: contributor

@theuni I followed wrt visibility and releases. I /think/ virtually no one using the software as a library will be using our release binaries, so we can expect the excessive exports being abused. I think thats acceptable for now as none of the library interface stuff is stable to begin with.

koobs commented at 5:01 am on February 24, 2015: none

@theuni Just for additional reference here, your statement about libstdc++ being the issue is confirmed and evidenced in the fact that whether using cc (gcc 4.2.1) or clang (3.4.1) on FreeBSD 9-STABLE, libstdc++ is linked.

clang can use both GNU libstdc++ and libc++, depending on the -stdlib= option. On 9.x it uses the former by default, on 10.x and later the latter.

laanwj commented at 8:17 am on February 24, 2015: member

Good catch. Again shows that shipping portable binaries for Linux is all but impossible. Maybe something like xdg-app sandboxing can change this in the near future.

One question: Won’t statically linking libstdc++ in the gitian build give problems with Qt (as we don’t like that statically, and thus the dynamic qt will imports its own libstdc++)?

uqs referenced this in commit cac0c8bf84 on Feb 24, 2015

koobs referenced this in commit 696fb41b79 on Feb 24, 2015

theuni commented at 6:54 am on February 25, 2015: member

@laanwj I’m really unsure. As an initial data point, it appears to run fine on Debian Wheezy (where it was broken before).

Thinking through it a bit… bitcoin-qt’s own throw/catch usage should be fine because the exception types are defined internally. Qt’s throw/catch usage should be fine because their types are dynamically loaded, and we don’t export them.

I think the worrisome case would be if qt throws us a std::exception. In that case, I’m really not sure what definition is used. Though from a quick grep, it looks like that never happens.

That’s all conjecture though. I’d like to have hard evidence. Beyond expanding our sanity checks to other std exceptions, can you think of other tests we could add here?

laanwj added the label Build system on Feb 25, 2015

laanwj commented at 10:47 am on February 25, 2015: member

@theuni Maybe I’m misunderstanding something. I’m not only worried about exceptions, but as I see it this will result in Qt and bitcoin-qt using different versions of libstdc++. The two versions of libstdc++ may have slightly different interpretations of types. When stdc++ types are passed across the dynamic library barrier (for example for QString::fromStdStding) this could cause potential issues. Another potential cause of issues are global data structures in libstdc++.

I know this is a common issue on Windows, at least. Maybe there is some magic workaround, but it seems brittle to me. A way to invite unknown unknowns. Hence I’m not so much convinced by ‘it happens to work on X’.

The recommendation when statically linking libstdc++ statically is to also link all c++ libraries statically. See e.g. http://www.trilithium.com/johan/2005/06/static-libstdc . Dynamic linking of C libraries is fine.

For Linux, when compiling against a new libstdc++ but using an older one at runtime

The caveman solution that comes to mind is: build against an old libstdc++.
Or should we bite the bullet and link Qt statically. More and more that sounds like the only robust solution (remember the OpenSSL interop issues). Static qt5 sucks on Ubuntu, but that is cosmetic and preferable to these kinds of issues.

gmaxwell commented at 10:49 am on February 25, 2015: contributor

Well for example, GCC 5’s libstdc++ includes an ABI incompatible string type. … but there I expect bitcoin core won’t work at all on those systems unless all C++ parts are statically linked.

laanwj commented at 10:57 am on February 25, 2015: member

@gmaxwell Right. Running the binary on such a system would only work if Qt is statically linked as well. Another point in favor of that.

theuni commented at 6:33 pm on February 25, 2015: member

@laanwj I’m all for static qt if you’re up for it. I’ll run a quick gitian test with the above changes + static qt to see if there are any surprises.

depends: always use static qt5 for linux d23b0a2711

theuni commented at 6:48 pm on February 26, 2015: member

Changed to static qt. Release sizes roughly doubled :(

For testing: https://bitcoincore.org/cfields/static-qt/bitcoin-0.10.99-linux32.tar.gz https://bitcoincore.org/cfields/static-qt/bitcoin-0.10.99-linux64.tar.gz

Works fine on Debian Squeeze 32.

binary depends:

 0 0x0000000000000001 (NEEDED)             Shared library: [libX11-xcb.so.1]
 1 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 2 0x0000000000000001 (NEEDED)             Shared library: [libxcb.so.1]
 3 0x0000000000000001 (NEEDED)             Shared library: [libfontconfig.so.1]
 4 0x0000000000000001 (NEEDED)             Shared library: [libfreetype.so.6]
 5 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 6 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 7 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 8 0x0000000000000001 (NEEDED)             Shared library: [libanl.so.1]
 9 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
10 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
11 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
12 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]

I’m not sure whether we should build qt using -no-xcb-xlib or not. That would drop the libX11-xcb.so.1 and libX11.so.6 deps, but I’m not clear on the implications.

sipa commented at 11:14 am on March 1, 2015: member

No opinion.

laanwj commented at 3:26 pm on March 3, 2015: member

That would drop the libX11-xcb.so.1 and libX11.so.6 deps, but I’m not clear on the implications.

Those are extremely common libraries, and also purely C not C++, so I have no problem with having them as dependency for the GUI.

laanwj merged this on Mar 3, 2015

laanwj closed this on Mar 3, 2015

laanwj referenced this in commit f8e68f7bfb on Mar 3, 2015

laanwj referenced this in commit e5eb8ba564 on Sep 10, 2018

PastaPastaPasta referenced this in commit c6bb3c3559 on Jun 27, 2021

PastaPastaPasta referenced this in commit a2dd4f6559 on Jun 28, 2021

PastaPastaPasta referenced this in commit 420e56c208 on Jun 29, 2021

PastaPastaPasta referenced this in commit 8c6a581920 on Jul 1, 2021

Munkybooty referenced this in commit 20942ef2cd on Jul 1, 2021

DrahtBot locked this on Sep 8, 2021

release: use static libstdc++ and disable reduced exports by default #5819