gitian: Build OpenSSL and Boost dependencies for linux #3545

After these changes, openssl and boost are built statically in the linux build and use the same versions as the win32 build.

elfread -d now reports

bitcoind:

0x00000001 (NEEDED)                     Shared library: [librt.so.1]
0x00000001 (NEEDED)                     Shared library: [libstdc++.so.6]
0x00000001 (NEEDED)                     Shared library: [libm.so.6]
0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
0x00000001 (NEEDED)                     Shared library: [libpthread.so.0]
0x00000001 (NEEDED)                     Shared library: [libc.so.6]

bitcoin-qt:

0x00000001 (NEEDED)                     Shared library: [librt.so.1]
0x00000001 (NEEDED)                     Shared library: [libQtGui.so.4]
0x00000001 (NEEDED)                     Shared library: [libQtNetwork.so.4]
0x00000001 (NEEDED)                     Shared library: [libQtCore.so.4]
0x00000001 (NEEDED)                     Shared library: [libQtDBus.so.4]
0x00000001 (NEEDED)                     Shared library: [libpthread.so.0]
0x00000001 (NEEDED)                     Shared library: [libstdc++.so.6]
0x00000001 (NEEDED)                     Shared library: [libm.so.6]
0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
0x00000001 (NEEDED)                     Shared library: [libc.so.6]

Nack on the boost changes.

The user does not control whether we link against shared libs or static, the linker does. The linker can be explicitly told what to do by passing something like "LDFLAGS=-static". Having an option for it does not make sense.

Instead, we should test with -DBOOST_TEST_DYN_LINK, and store the pass/fail result as a way of knowing whether we're shared or static. Basically, if it links with the define, use the define.

As for librt, that should not be hard-coded, because it's implementation-specific and may or may not be in librt. AC_SEARCH_LIBS should be used instead.

I'm not happy with the --with-static-boost either, but for me it was the only way to get it to work. Feel free to improve on it later.

If you're trying to reduce build-time for boost, you should specify on the modules you want, that avoids building/installing the ones we don't use. Something like:

--with-system --with-filesystem --with-program_options --with-thread --with-chrono

I just explained how to make it work. Is there a rush here?

Indeed, specifying the modules is a possibility to make the boost build even faster. I may do that later. We don't use the trigonometry and geometry and such modules and we likely never will for bitcoin :)

No rush but I'm happy to see this working at all. I've been working for two days on gitian building stuff and I'm sick and tired of it.

I suck at autotools and if there was a way to do the -DBOOST_TEST_DYN_LINK/-lrt stuff without touching configure.ac I'd have preferred that, but I didn't find a way. What you propose sounds like the better solution, no argument there...

c665d663e0bae284a23076160e4ffa77450480489199955c0c8b561de577a464 bitcoin-deps-linux32-gitian-r2.tar.gz 91f8c787aeb263f88ac812f61d5585480451121b8b755cb46214735693205299 bitcoin-deps-linux64-gitian-r2.tar.gz

Do your builds match?

1a0100fe7ad0fb77ed51b46dc8c82f25528023e8c2953fbab9b9ff6de54164f7 boost-linux32-1.55.0-gitian-r1.tar.gz e9490de332efe04e4c6b65c5a1d4b6cc4bbfef22ec9351e5bcafdabd3c669d9c boost-linux64-1.55.0-gitian-r1.tar.gz

99% chance tarballs won't match without: #3546 (comment)

@wtogami I haven't focused on determinism of the intermediate outputs yet at all. Let's first make sure that the build works and produces correct executables.

@laanwj Executables produced seem to work as expected on Fedora 20.

Thanks for testing!

Going to rebase and use a .zip for the built boost dependency as well. @theuni I've thought about the static boost link bit:

• The libboost_chrono on -lrt dependency should be handled outside our autoconf, as bitcoin should indeed not care about the dependencies of dependencies.
  • The .la file for libraries was invented for exactly this (https://stackoverflow.com/questions/1238035/what-is-libtools-la-file-for), so I'm going to try an approach based on that.
• As for -DBOOST_TEST_DYN_LINK I'm going to follow your solution and add a test with/without.

Edit: darn, .la files are simply ignored, libtool is not used for the autoconf tests.

Ok I added a check for -DBOOST_TEST_DYN_LINK. It's not being assumed based on the platform anymore.

I've also removed the option --with-static-boost. There is now an undocumented environment variable BOOST_CHRONO_EXTRALIBS to specify extra libraries for Boost Chrono. This is not perfect either, but at least doesn't have as much impact on the configure.ac script. It would be great if other ways to specify implied dependencies between libraries (such as .la) worked.

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/7f51b918109263501de1592e647ebf136e1cd5be for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

2ee4dfb8c7351a95859b6b19f2568841326f1ae7f4b2c74ef7477bd0496261e9 bitcoin-deps-linux32-gitian-r2.zip 82e27e988049a6a9384e377c10cd0997a580dfb44eabdab068f3af1fada22c2c bitcoin-deps-linux64-gitian-r2.zip

0f44759738bff1eaa1284e7f53e51b0679658c0161d9e085b4d5ea4fa70fdab8 boost-linux32-1.55.0-gitian-r1.zip 03c959bbda6d085146b84f75c2973fda66072145eebf2269a099c5030be912f6 boost-linux64-1.55.0-gitian-r1.zip

ACK

@laanwj Any time you link a static lib you have to worry about dependencies of dependencies. It comes with the territory. It's been "solved" many times via libtool, pkgconfig, etc. In this case, as boost doesn't ship a .pc and we're not using libtool, so it's our responsibility to solve it. It's a major strike against boost in my opinion, but one that many projects are forced to deal with.

Either way, the lib containing clock_gettime isn't harmful for any OS, so I really suggest adding that as a general dependency. It's almost guaranteed that we'll need those symbols eventually.