gitian: build against Qt 4.6 #4094

laanwj commented at 7:59 PM on April 25, 2014: member

Should make it possible to run the resulting GUI executable on Linux distributions that use Qt 4.6, such as Debian Wheezy and Tails.

Builds a mini-SDK for building against Qt 4.6. This includes the headers as well as host utilities such as lrelease, qrc and moc.

This speeds up the gitian build a bit - libqt4-dev pulled in a lot of packages, and is no longer needed as this provides a replacement of our own.

Note: This does not replace the Qt build with at static library. After this commit we still build dynamically against the system Qt library. The only difference is that compatibility with an older version is maintained. This loses minor GUI functionality (such as setPlaceholderText) but still allows integration into the theming and window management of the host OS, unlike when statically linking.

Test build: https://download.visucore.com/bitcoin/linux-gitian-1b2454a.tar.gz (sigs: https://download.visucore.com/bitcoin/linux-gitian-1b2454a.tar.gz.sig )

in contrib/gitian-descriptors/gitian-linux.yml:None in 2d48b9628b outdated

  14 | @@ -16,6 +15,11 @@ packages:
  15 |  - "automake"
  16 |  - "faketime"
  17 |  - "bsdmainutils"
  18 | +- "libqt4-core"
  19 | +- "libqt4-gui"
  20 | +- "libqt4-dbus"
  21 | +- "libqt4-network"
  22 | +- "libqt4-test"

wtogami commented at 8:37 PM on April 25, 2014:

Why does it need system qt4?

laanwj commented at 8:49 PM on April 25, 2014:

Maybe uhm... to link against it?

Edit: this only replaces the headers and host utilities, the goal is still to link against the system Qt.

wtogami commented at 12:12 AM on April 26, 2014:

Oops, I see.

Michagogo commented at 8:19 PM on April 26, 2014: contributor

Is this Qt build a full one? I ask because I know that the Qt build for Windows is huge and takes a very long time... and also runs out of disk space on older Gitian VMs.

wtogami commented at 9:44 PM on April 26, 2014: contributor

No, it is only headers to build against. It is quite small and fast.

gmaxwell commented at 10:02 PM on April 26, 2014: contributor

The benefits sound pretty nice to me and the costs don't sound so bad, at least today. Obviously maintaining this compatibility now doesn't mean we have to maintain it forever if it imposes more compromises later.

laanwj commented at 6:48 AM on April 27, 2014: member

@Michagogo It builds all of the Qt developer tools (moc, uic, qrelease,...) needed for the build, as well as the headers, not the whole thing. @gmaxwell It indeed isn't too bad. At some point in the future, when more Linux distributions start to use Qt5 (exclusively) we may want to ship an executable built against that as well. But it's not worth to have a fuss about 4.6 versus 4.8.

laanwj commented at 3:00 PM on April 29, 2014: member

For some reason, the resulting executable crashes on startup in Tails. Too bad the OS has no developer tools, so I cannot get a traceback. Will try in Debian 6 instead, as it's based on that...

Edit: the Debian 6 gdb doesnt support position independent executables, interesting...

Traceback seems to find the error in the OpenSSL usage of Qt. Very strange.

[#0](/bitcoin-bitcoin/0/)  *__GI_strncmp (s1=0xaaef8610 "AES:ALL:!aNULL:!eNULL:+RC4:@STRENGTH",                                                                               │········································
    s2=0x80 <Address 0x80 out of bounds>, n=3) at strncmp.c:66                                                                                         │········································
[#1](/bitcoin-bitcoin/1/)  0xaaeeff44 in ssl_cipher_process_rulestr (rule_str=<value optimized out>,                                                                          │········································
    co_list=<value optimized out>, head_p=0xbfffe7dc, tail_p=0xbfffe7d8,                                                                               │········································
    ca_list=0x9819e20) at ssl_ciph.c:876                                                                                                               │········································
[#2](/bitcoin-bitcoin/2/)  0xaaef06fc in ssl_create_cipher_list (ssl_method=0xaaeff280,                                                                                       │········································
    cipher_list=0x9819544, cipher_list_by_id=0x9819548,                                                                                                │········································
    rule_str=0xaaef8610 "AES:ALL:!aNULL:!eNULL:+RC4:@STRENGTH")                                                                                        │········································
    at ssl_ciph.c:1040                                                                                                                                 │········································
[#3](/bitcoin-bitcoin/3/)  0xaaeea422 in SSL_CTX_new (meth=0xaaeff280) at ssl_lib.c:1529                                                                                      │········································
[#4](/bitcoin-bitcoin/4/)  0xb74c6654 in q_SSL_CTX_new (a=0xaaeff280)                                                                                                         │········································
    at ssl/qsslsocket_openssl_symbols.cpp:170                                                                                                          │········································
[#5](/bitcoin-bitcoin/5/)  0xb74bff51 in QSslSocketPrivate::resetDefaultCiphers ()                                                                                            │········································
    at ssl/qsslsocket_openssl.cpp:458                                                                                                                  │········································
[#6](/bitcoin-bitcoin/6/)  0xb74c02dd in QSslSocketPrivate::ensureInitialized ()                                                                                              │········································
    at ssl/qsslsocket_openssl.cpp:444                                                                                                                  │········································
[#7](/bitcoin-bitcoin/7/)  0xb74b914f in QSslSocket::systemCaCertificates ()                                                                                                  │········································
    at ssl/qsslsocket.cpp:1357                                                                                                                         │········································
[#8](/bitcoin-bitcoin/8/)  0x080d13c8 in PaymentServer::LoadRootCAs(x509_store_st*) ()                                                                                        │········································
[#9](/bitcoin-bitcoin/9/)  0x08088518 in BitcoinApplication::initializeResult(int) ()

Commenting out that call to PaymentServer::LoadRootCAs results in a (seemingly) normal running GUI...

laanwj commented at 8:42 PM on April 29, 2014: member

After much experimentation to find the cause of this crash, I found out the bitcoind/bitcoin-qt executables produced by gitian export the symbols from the included, statically linked, OpenSSL. The (dynamic) Qt library that is linked picks up some of these symbols (instead of those from the older OS's library), causing the crash.

I don't want to export symbols from the executable at all. This wastes space, slows down startup, and probably causes other problems. I think to solve this we need to compile all the dependency libraries with -fvisiblity=hidden.

See also: https://stackoverflow.com/questions/2375209/avoiding-exporting-symbols-from-executables-on-linux : "I'm finding that when I link an executable against a static library (.a), the symbols from the static library end up being exported by the executable file.". That's exactly what we're seeing here.

Edit: building OpenSSL with -fvisibility=hidden indeed makes the problem go away. However, it still leaves a few symbols exported from the executable (I think it's the functions implemented in assembly, which makes sense ... -fvisiblity applies to gcc, not as).

Edit2: a promising solution appears to be to use -Wl,--exclude-libs,ALL on link-time (suggested by fons on stack overflow ).

Edit3: another option is to provide a linker version script with -Wl,--version-script= that marks all symbols as local:

{
  local: *;
};

This avoids exporting any symbols dynamically, not only from third-party libs but even from bitcoin itself, and thus looks like the cleanest option.

laanwj commented at 6:59 AM on April 30, 2014: member

That seems to have fixed the issue. New test build available (see opening post)

laanwj referenced this in commit 1b2454a119 on Apr 30, 2014

laanwj referenced this in commit a2ee959d79 on Apr 30, 2014

gitian: build against Qt 4.6

Should make it possible to run the resulting GUI executable on
Linux distributions that use Qt 4.6, such as Debian Wheezy and Tails.

Builds a mini-SDK for building against Qt 4.6. This includes the headers
as well as host utilities such as `lrelease`, `qrc` and `moc`.

This speeds up the gitian build a bit - libqt4-dev pulled in a lot of packages,
and is no longer needed as this provides a replacement of our own.

Note: This does not replace the Qt build with at static library. After this
commit we still build dynamically against the system Qt library. The only
difference is that compatibility with an older version is maintained. This
loses minor GUI functionality (such as setPlaceholderText) but still
allows integration into the window management of the host OS, unlike
when statically linking.

3ab1664594

gitian: don't export any symbols from executable

This avoids conflicts between the libraries statically linked into bitcoin and any
libraries we may link dynamically (such as Qt and OpenSSL, see issue #4094).
It also avoids start-up overhead to not export any unnecessary symbols.
To do this, build a linker script that marks all symbols as local.

92e3022f88

devtools: have symbol check script check for exported symbols

After last commit, our executables should export no symbols anymore.  To
make sure that this stays the case, verify this in the symbol checker
script.

202c95c216

BitcoinPullTester commented at 7:32 PM on April 30, 2014: none

Automatic sanity-testing: FAILED BUILD/TEST, see http://jenkins.bluematt.me/pull-tester/202c95c216ac193fc8ea0f668f1b26f7ddc2907e for binaries and test log.

This could happen for one of several reasons:

It chanages paths in makefile.linux-mingw or otherwise changes build scripts in a way that made them incompatible with the automated testing scripts (please tweak those patches in qa/pull-tester)
It adds/modifies tests which test network rules (thanks for doing that), which conflicts with a patch applied at test time
It does not build on either Linux i386 or Win32 (via MinGW cross compile)
The test suite fails on either Linux i386 or Win32
The block test-cases failed (lookup the first bNN identifier which failed in https://github.com/TheBlueMatt/test-scripts/blob/master/FullBlockTestGenerator.java)

If you believe this to be in error, please ping BlueMatt on freenode or TheBlueMatt here.

This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

laanwj merged this on May 2, 2014

laanwj closed this on May 2, 2014

laanwj referenced this in commit b397248436 on May 2, 2014

DrahtBot locked this on Sep 8, 2021