gitian: add statically built variant of bitcoind/bitcoin-cli for linux build #3914

See #3803 and #3781.

This increases compatibility with older linux distributions.

  2943584 Mar 20 11:43 32/bitcoin-cli
  4376728 Mar 20 11:43 32/bitcoin-cli.static
  7748512 Mar 20 11:43 32/bitcoind
  9140856 Mar 20 11:43 32/bitcoind.static
 12164000 Mar 20 11:43 32/bitcoin-qt
  9740608 Mar 20 11:43 32/test_bitcoin
  8420256 Mar 20 11:43 32/test_bitcoin-qt
  3103728 Mar 20 11:53 64/bitcoin-cli
  4355112 Mar 20 11:53 64/bitcoin-cli.static
  7773680 Mar 20 11:53 64/bitcoind
  8890088 Mar 20 11:53 64/bitcoind.static
 12226032 Mar 20 11:53 64/bitcoin-qt
  9860528 Mar 20 11:53 64/test_bitcoin
  8457776 Mar 20 11:53 64/test_bitcoin-qt

A warning shown during build is:

 warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

Note I have not explicitly disabled hardening/-pie in this build. This makes that the 32-bit version still has a dependency on dynamic libc.so even though it is statically linked:

$ ldd 32/bitcoind.static
    statically linked
$ readelf -a 32/bitcoind.static |grep interpreter
      [Requesting program interpreter: /usr/lib/libc.so.1]

Not sure how much of an issue this is on true 32-bit systems. It needs to be tested. This is not the case for the 64-bit build:

$ ldd 64/bitcoind.static                                                       
    not a dynamic executable
$ readelf -a 64/bitcoind.static |grep interpreter
(nothing)

Edit: v2: -pie now explicitly disabled for 32-bit build as it was unusable otherwise.

Tested on:

• Debian 7.2 64-bit: OK (tested by me)
• Ubuntu 10.04.4 LTS 64-bit: OK (tested by me)
• Ubuntu 10.04.4 LTS 32-bit: OK (tested by me, v2)
• CentOS 6.5 64bit: OK (tested by alexeft on forums and anton000, v2)
• CentOS 6.5 32bit: OK (tested by anton000, v2)

I've uploaded the result here (temporary download URI only!):

https://download.visucore.com/bitcoin/bitcoin-0.9.0-linux-static-2.tar.gz https://download.visucore.com/bitcoin/bitcoin-0.9.0-linux-static-2.tar.gz.sig

OK, whereas the 64-bit ones seem to work fine, the 32-bit static executables produced this way are absolutely broken. I have added a commit to disable -pie for 32-bit static executables, and updated the above downoad.

I'm still very hesitant on this approach. I'd be much more comfortable with picking a minimum required glibc version and wrapping symbols for newer versions. @laanwj What's the minimum glibc you've got on any of those systems? I'll see if I can work up a POC for the sake of discussion.

Glibc 2.11 for Ubuntu 12.10, 2.12 for RH. But there were also problems with libc++ symbols...

+1 thank you @laanwj for this! Currently running fine as daemon on CentOS 6.5 64bit :) Will try to fire up my 32bit centos box to test

edit: works fine on fully updated CentOS 6.5 32bit

Thanks for testing @anton000

When will this be public? Need it to run 0.9 on debian.

When it's ready.

In the meantime you can use and help testing the executables here or build your own.

already did - runs perfectly on debian wheezy 64 bit.

@laanwj May as well go ahead and use this, since it fixes real-world problems in the short-term. I'll keep looking into options for future releases... nothing I've come up with so far has been any more appealing.

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/ddcd1afc5fdd148cd56f257b40a12f70841bd1b3 for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

Squashed into one commit, removed special-case for 32/64 bit (-static -pie is invalid on all architectures).

@laanwj Please see https://github.com/theuni/bitcoin/tree/libc-compat . That should fix for 2.13. If the approach works as intended, I can move forward with the other symbols as well.

Ok, hold on that. glibc behavior changed at some point, so some of my assumptions were wrong on the above. Looking again.

Ok, fresh new idea is pushed up to https://github.com/theuni/bitcoin/tree/libc-compat

To use: ./configure --with-glibc-back-compat or --with-glibc-back-compat=X where X is glibc version 2.X.

The idea is to eventually be able to only enable the work-arounds needed for a specific version. For now, there's only one enabled for 2.15.

@theuni so I can point my gitian build at that tree and generate testing executables (to try on the various platforms)? Just have to make sure I pass --with-glibc-back-compat ?

@laanwj Yes. Though, the only work-around included for now is for glibc 2.15, so it won't help on anything older than that. If the approach works and seems reasonable, I can begin adding more.

@laanwj Hold on that last one as well. I finally hit on a clean way of handling this, and the memcpy for 2.14 is resolved with it, so now the only concern is libstdc++.

If you're ok with waiting a week or so, I'll quit spamming here and just submit a PR that includes gitian descriptors once it's nice and tidy.

By all means keep 'spamming' to keep us informed as you're working on this, it's nice to know what people are up to.

@theuni Because of the SSL heartbeat vulnerability we'd like to spin a 0.9.1 as soon as possible. It would be useful to have a portable executable in it, so I'd like to pull this now. This solution has been extensively tested. Your symbol-filtering based solution can be used for next release, however it needs testing first.

@laanwj Huh? The heartbleed vuln doesn't affect Bitcoin Core afaik?

@luke-jr The vulnerability does not affect the bitcoin protocol but may affect auxilary usage of TLS/HTTPS in RPC SSL and payment request fetching.

Fetching too? I thought it was just server-side :(

I think it would be good to release a 0.9.1 with the SSL bug patched.