Linux static qt5 #4284

This bumps linux+qt up to static 5.2.1 to match osx and (soon) windows. It should result in a much more portable release binary, since it does not depend on the user having qt installed. It does away with the current scary combination of qt4.6 headers and qt4.8 libs.

The nasty graphics dependencies (x11, xcb, etc) are kept to an absolute minimum. This build uses qt's built-in xcb libs and dyloaders in order to reduce the runtime dependencies on xcb* and x11. The end-result is that only libxcb is required at runtime. Other dependencies on freetype/fontconfig may be reduced to static libs as well if desired.

From my quick tests, this is fully working. All builds are deterministic, and no new glibc/libstdc++ symbols have been introduced as far as I can tell.

The only thing that I'm aware of which requires more testing is qt's internal translations, which may need to be statically linked as well.

Full dependencies:

Before:

        linux-gate.so.1 =>  (0xf7715000)
        librt.so.1 => /lib/i386-linux-gnu/librt.so.1 (0xf6b3c000)
        libQtGui.so.4 => /usr/lib/i386-linux-gnu/libQtGui.so.4 (0xf606b000)
        libQtNetwork.so.4 => /usr/lib/i386-linux-gnu/libQtNetwork.so.4 (0xf5f27000)
        libQtCore.so.4 => /usr/lib/i386-linux-gnu/libQtCore.so.4 (0xf5c3f000)
        libQtDBus.so.4 => /usr/lib/i386-linux-gnu/libQtDBus.so.4 (0xf5bc1000)
        libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf5ba6000)
        libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf5abd000)
        libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf5a79000)
        libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf5a5c000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf58a8000)
        libfontconfig.so.1 => /usr/lib/i386-linux-gnu/libfontconfig.so.1 (0xf586f000)
        libaudio.so.2 => /usr/lib/i386-linux-gnu/libaudio.so.2 (0xf5856000)
        libglib-2.0.so.0 => /lib/i386-linux-gnu/libglib-2.0.so.0 (0xf5754000)
        libpng12.so.0 => /lib/i386-linux-gnu/libpng12.so.0 (0xf572b000)
        libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xf5712000)
        libfreetype.so.6 => /usr/lib/i386-linux-gnu/libfreetype.so.6 (0xf5677000)
        libgobject-2.0.so.0 => /usr/lib/i386-linux-gnu/libgobject-2.0.so.0 (0xf5627000)
        libSM.so.6 => /usr/lib/i386-linux-gnu/libSM.so.6 (0xf561d000)
        libICE.so.6 => /usr/lib/i386-linux-gnu/libICE.so.6 (0xf5603000)
        libXi.so.6 => /usr/lib/i386-linux-gnu/libXi.so.6 (0xf55f3000)
        libXrender.so.1 => /usr/lib/i386-linux-gnu/libXrender.so.1 (0xf55e9000)
        libXext.so.6 => /usr/lib/i386-linux-gnu/libXext.so.6 (0xf55d7000)
        libX11.so.6 => /usr/lib/i386-linux-gnu/libX11.so.6 (0xf549f000)
        libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf549a000)
        /lib/ld-linux.so.2 (0xf7716000)
        libQtXml.so.4 => /usr/lib/i386-linux-gnu/libQtXml.so.4 (0xf545a000)
        libdbus-1.so.3 => /lib/i386-linux-gnu/libdbus-1.so.3 (0xf5410000)
        libexpat.so.1 => /lib/i386-linux-gnu/libexpat.so.1 (0xf53e8000)
        libXt.so.6 => /usr/lib/i386-linux-gnu/libXt.so.6 (0xf538b000)
        libXau.so.6 => /usr/lib/i386-linux-gnu/libXau.so.6 (0xf5387000)
        libpcre.so.3 => /lib/i386-linux-gnu/libpcre.so.3 (0xf5346000)
        libffi.so.6 => /usr/lib/i386-linux-gnu/libffi.so.6 (0xf533f000)
        libuuid.so.1 => /lib/i386-linux-gnu/libuuid.so.1 (0xf5339000)
        libxcb.so.1 => /usr/lib/i386-linux-gnu/libxcb.so.1 (0xf5316000)
        libXdmcp.so.6 => /usr/lib/i386-linux-gnu/libXdmcp.so.6 (0xf530f000)

After:

        linux-gate.so.1 =>  (0xf7707000)
        librt.so.1 => /lib/i386-linux-gnu/librt.so.1 (0xf5b5a000)
        libxcb.so.1 => /usr/lib/i386-linux-gnu/libxcb.so.1 (0xf5b38000)
        libfontconfig.so.1 => /usr/lib/i386-linux-gnu/libfontconfig.so.1 (0xf5afe000)
        libfreetype.so.6 => /usr/lib/i386-linux-gnu/libfreetype.so.6 (0xf5a63000)
        libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf5a48000)
        libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xf5a2f000)
        libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf5a2a000)
        libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf59e6000)
        libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf58fd000)
        libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf58e0000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf572c000)
        /lib/ld-linux.so.2 (0xf7708000)
        libXau.so.6 => /usr/lib/i386-linux-gnu/libXau.so.6 (0xf5728000)
        libXdmcp.so.6 => /usr/lib/i386-linux-gnu/libXdmcp.so.6 (0xf5720000)
        libexpat.so.1 => /lib/i386-linux-gnu/libexpat.so.1 (0xf56f8000)

Note that libXau.so, libXdmcp.so, and libexpat.so are indirect dependencies.

:+1:

As I've said before many times, I'm not happy about statically linking qt on Linux - and certainly not everything it depends on like font libraries. This should be up to the system. This will mess up appmenu integration in Ubuntu, for example, and other distro-specific customization.

Using an executable linked against 4.6 headers with 4.8 is safe, as Qt is backwards compatible. This has always been the case with the executables built on Ubuntu 10.04!

BTW: if you show dependencies please show the direct dependencies. The current bitcoin-qt only has a few libraries it depends, on, none at all X or font libraries. I suppose it should even work with non-X qt (for example wayland).

objdump -p bin/32/bitcoin-qt |grep NEEDED                                                                                        
  NEEDED               librt.so.1
  NEEDED               libQtGui.so.4
  NEEDED               libQtNetwork.so.4
  NEEDED               libQtCore.so.4
  NEEDED               libpthread.so.0
  NEEDED               libstdc++.so.6
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.6

As you see this is already a bare minimum. You list the recursive dependencies which also include what the dynamic libraries on your system may happen to include.

Which concrete problem does statically linking Qt on linux solve?

I've changed my mind here. Seems Ubuntu 10.04 has an older version of Qt4 (4.6.2) than the one whose headers we link against (4.6.4), causing a runtime error about a missing symbol.

Instead of dropping the version even further, we may as well link Qt5 statically. Better to have no Ubuntu menu integration than to be incompatible. At least if the executables produced by this work on 10.04...

Edit: needs rebase

Ok. I'll summarize my discussion with @laanwj on this PR, since we had it on IRC.

Pros of static link:

• No client Qt dependency
• Not stuck with old Qt
• No build tricks to make it work

Cons:

• Hard dependency on libxcb and libfreetype
• No Ubuntu app-menu integration

For the first con, I believe this is somewhat moot because these are nearly guaranteed to exist on a linux system, and the qt libs almost certainly depend on them anyway.

The second con is what @laanwj addresses above: better to work at all than work without app-menu integration.

At the time, we discussed the fact that Ubuntu has upstreamed their app-menu changes, and that they're included in qt5.3. This pull can be trivially updated to use 5.3 rather than 5.2, but I'm unsure if the theme plugins will load when using a static build. I'm testing that right now.

Rebased.

Unfortunately, the same plugin restrictions apply to themes as well with qt 5.3. Meaning that themes can't be loaded dynamically from a static binary.

I spent a little while investigating the stability of the interfaces for appmenu-qt, and it seems that it would be no problem to link in statically, since there are only 2 functions in the external api, and the rest happens over dbus. Unfortunately though, it means that bitcoin-qt would gain a dependency on libgtk+ and libglib2, so that's a non-starter.

Since there's no obvious benefit in moving to 5.3, I left it at 5.2.1. 5.3 can be dropped in later with no changes necessary, except that this workaround can be removed: https://github.com/theuni/bitcoin/commit/d124f1120d41344555b984d1dbc5840ac25c1073#diff-8fd2f50cc3bac9865790ada2825a3006R131

Also, note that the move of qt's build into the -deps descriptor was not arbitrary. We need the to provide Qt with our static libssl and that seemed like the most straightforward approach.

"Also, note that the move of qt's build into the -deps descriptor was not arbitrary. We need the to provide Qt with our static libssl and that seemed like the most straightforward approach." @theuni In my opinion, this isn't a good thing to do. It might have made sense to have the Qt build in with the deps when it's just the headers etc, but Qt is a huge package to build and it would be best to separate it out. What's wrong with the approach we use for Windows, where the deps are an input for Qt?

I agree with @michagogo here. Qt is by far the biggest thing we build, so having it as an separate dependency makes sense, and there is no need to rebuild it when any of the other dependencies - except for OpenSSL - change.

@Michagogo I was just trying to avoid the chaining complexity, but no real reason. Will split it up.

This was no fun, but it's rebased and split back out into its own descriptor.

While I was hacking around in there, I added support in bitcoin's configure for a static osx qt as well. I realize that's out of the scope of this PR and I really don't like doing that, but I added it here because it touches the same stuff as Linux. It's not useful yet, just groundwork (though I do have local hacks that make it fully work).

This needs to be tested for regressions before switching. There's an issue with the systray icon colors being inverted which I haven't yet poked at, but that's pretty minor.

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/p4284_bfc964de9a661045cba25ade6b7feea89f4eba1a/ for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

I tried building this, and no bitcoin-qt exectuable was created at all (just bitcoind, bitcoin-cli and bitcoin_test). Looks like Qt wasn't detected. Build log: https://download.visucore.com/tmp/build.log.bz2

Hmm, not sure what went wrong there, I probably borked something in a last-minute rebase. I'll take a look in the next day or two. Is there any particular rush on this one? If so, I'll get to it sooner than later.

No particular rush, this change doesn't have a very high priority, but I thought I'd test it a bit as you mentioned visual issues.

Closing this one too. The configure/code changes have already been merged, and the rest is part of a bigger overhaul.