Gitian descriptors overhaul #4727

1. 
   theuni commented at 3:47 AM on August 19, 2014: member

   This teaches Gitian to use the new depends system rather than the old build scripts. See #4592 for the reasoning.

   Important: This relies on a new caching feature in Gitian. When building (for ex) the linux descriptor, the dependencies will be built and cached. On subsequent builds, if the dependencies have not changed, a cached version will be used rather than rebuilding. For that reason, the descriptors can be a clean one-per-OS. Gitian rev e9741525c or newer is required to take advantage of this.

   In order to take advantage of the new cache, sources are no longer listed as inputs. Instead they are handled by the dependency builder. After they've been fetched once, they'll be cached and re-used.

   All target libs are cross-built as necessary now, rather than using apt-get to install. That means that a single OS image (in this case Precise x64) can be used for everything.

2. laanwj added the label Build system on Aug 19, 2014
3. 
   laanwj commented at 9:18 AM on August 19, 2014: member

   How does gitian manage the dependency cache? (ie, what does it look like? is it possible to keep around dependencies for different versions of bitcoin?)

4. 
   theuni commented at 6:01 PM on August 21, 2014: member

   Gitian doesn't manage the dependency cache, it blindly copies the result of the build. So if the build nukes the cache, it's gone (see huge caveat below, this is completely untrue in practice). Currently the dependency builder removes everything but the most recent result for each dependency, so that it can be used by set-and-forget systems without worry of disks filling up.

   Huge caveat: Gitian uses scp to xfer the build cache back to the bare-metal builder. If a file is deleted in the cache, when the dir is scp'd back to the bare-metal, all left-over files still remain there. This would be solved by using something like "rsync --delete" instead of scp, but I'm not sure that is the most reasonable behavior.

   So the actual answer is: all built dependencies are saved until they're manually deleted. The cache can be safely purged at any time. So if you're bouncing back and forth between branches, both cached versions will be preserved.

5. 
   TheBlueMatt commented at 11:30 PM on August 21, 2014: member

   Though it may be a bit too late, I'm not sure about the way this was done. I'd generally much have preferred that gitian was used directly on travis (dunno what they support, but some neutered LXC containerish thing should be possible, or maybe a null-op vm where there is no vm?) and continue using the gitian dependency management stuff instead of a wrapper around our own dependency management system in-tree. I'd generally avoid having people use depends/ in any case where they can avoid it (ie unless they're cross-compiling) and encourage people to use gitian anyway. Maybe I'm just not happy with depends/ being /depends and not /contrib/depends.

6. 
   laanwj commented at 6:46 AM on August 22, 2014: member

   @TheBlueMatt Using gitian in travis would add quite a lot of overhead. Note that using depends/ directly is effectively a null-op VM, as gitian uses depends too. The only advantage of using gitian in the tester would have been that the gcc/binutils/libc version matches what is used to build releases. Which would be nice but not sure it's worth the extra testing time. The rest is built from scratch.

   As for moving depends to contrib/depends, that's a matter of taste, I considered suggesting it too but meh. @TheUni Ok, thanks for explaining.

7. 
   TheBlueMatt commented at 7:32 AM on August 22, 2014: member

   @laanwj a null-op VM for gitian wouldn't add any overhead. But in any case, I would strongly think about making a release-builds/ TLD or so, to put gitian things and depends/, I'm really not a fan of them being in the top-level, as it appears to encourage people to go use those when building Bitcoin themselves, whereas they should just use whats in their OS (which is kinda the point of having autotools to begin with).

8. 
   laanwj commented at 7:37 AM on August 22, 2014: member

   @TheBlueMatt But as I see it in a null-op VM, you'd still have the gcc/libc difference issue. Unless you somehow inject Ubuntu 12.04's compilers and linkers.

   As for 'encouraging people to use depends', that's very UNIX-focused, I think it wouldn't be crazy to encourage using depends on:

   • MacOSX: lots of problem with homebrew and similar package systems needed to build
   • Windows: no package system to speak of to install dependencies, building them manually w/mingw is so tricky that people don't even bother

   So basically on everything but UNIXy OSes.

9. 
   TheBlueMatt commented at 7:39 AM on August 22, 2014: member

   Sure, depends/ makes sense exactly for windows (though I dont have too much experience with OSX). Somehow I would just very strongly like to make that clear to people who download and see "oh hey, this shitty software wants to go use specific dependency versions, these guys cant write (UNIX) software"

10. 
    laanwj commented at 7:42 AM on August 22, 2014: member

    We don't require any specific dependency versions at this point (apart from the soft requirement of BDB4.8). The only mandated versions are for 'official' releases. If you think that's unclear, is that a documentation issue?

11. 
    TheBlueMatt commented at 7:45 AM on August 22, 2014: member

    Yea, I dunno if its a documentation issue or not, personally, I see /depends/ and think "first-class feature they want us to use" not "this is here if you need it". Sure, doc changes would help, but maybe something like /buildutils/ would make it more clear?

12. 
    laanwj commented at 1:51 PM on August 22, 2014: member

    Another question: is it still possible to verify determinism of the intermediate dependencies after this?

13. 
    theuni commented at 3:10 PM on August 22, 2014: member

    @laanwj yep, pretty much the same procedure as before. The output yml shows the hashes of the intermediates. If you want them rebuilt, just nuke em from the cache.

14. 
    theuni commented at 3:14 PM on August 22, 2014: member

    @TheBlueMatt @laanwj answered your concerns pretty much as I would have. As to the file location, I really don't much care either way. Given that they're (soon?) to be used for gitian and pull-tester, and likely to become the easiest and/or primary way of building for win/osx, imo they should be treated as a first-class citizen.

    That said, I do see your point about a Unixy dev possibly thinking that this is how they should build. If it's decided that they should move, I'll move em.

15. 
    theuni commented at 3:23 PM on August 22, 2014: member

    @TheBlueMatt btw, here is a perfect example of why the naive Gitian scripts really are insufficient: #4648 (comment) . Real dependency management ensures that these problems don't arise (even if it was a non-issue this time) because bumping openssl automatically triggers a qt bump.

16. 
    TheBlueMatt commented at 6:00 PM on August 22, 2014: member

    @theuni I'm confused as to how we wont have that same problem with the new dependency tracking system?

17. 
    theuni commented at 7:33 PM on August 22, 2014: member

    @TheBlueMatt Each dependency is aware of its own dependencies. In this case, qt depends on openssl. So when openssl is bumped, it forces a qt bump as well.

18. 
    laanwj commented at 7:10 AM on August 23, 2014: member

    @cfields with the depends mechanism, is there a way to separately download the sources and do the build? ie, something like make fetch. This would be useful to be able to fetch on one machine, upload the sources, then build without further internet connection.

19. 
    theuni commented at 1:34 PM on August 23, 2014: member

    See 'make download' ;)

20. 
    TheBlueMatt commented at 9:51 PM on August 23, 2014: member

    Conceptual ack if we move depends/ to something like buildutils/depends (plus I think it makes sense to move gitian-* to something other than contrib/, which is more for useful scripts these days)

21. 
    laanwj commented at 12:27 PM on August 26, 2014: member

    I've been testing the linux executables. Testcases run OK and cursory runs of the various executables work.

    However there are various problems reported by symbol-check.py

    • For example on bitcoin-cli (64-bit, 32-bit is the same):

    build/out/bin/64/bitcoin-cli: symbol memcpy from unsupported version GLIBC_2.14
    build/out/bin/64/bitcoin-cli: symbol std::out_of_range::~out_of_range() from unsupported version GLIBCXX_3.4.15
    build/out/bin/64/bitcoin-cli: export of symbol typeinfo for boost::asio::detail::typeid_wrapper<boost::asio::ip::resolver_service<boost::asio::ip::tcp> > not allowed
    build/out/bin/64/bitcoin-cli: export of symbol std::basic_string<char, std::char_traits<char>, std::allocator<char> > std::operator+<char, std::char_traits<char>, std::allocator<char> >(char const*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) not allowed
    build/out/bin/64/bitcoin-cli: export of symbol vtable for boost::exception not allowed
    build/out/bin/64/bitcoin-cli: export of symbol vtable for boost::exception_detail::error_info_injector<boost::bad_get> not allowed
    build/out/bin/64/bitcoin-cli: export of symbol typeinfo for boost::asio::detail::typeid_wrapper<boost::asio::deadline_timer_service<boost::posix_time::ptime, boost::asio::time_traits<boost::posix_time::ptime> > > not allowed
    ... (more boost exports follow)
    

    Bitcoind:

    build/out/bin/64/bitcoind: symbol std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) from unsupported version GLIBCXX_3.4.15
    build/out/bin/64/bitcoind: symbol memcpy from unsupported version GLIBC_2.14
    build/out/bin/64/bitcoind: symbol std::length_error::~length_error() from unsupported version GLIBCXX_3.4.15
    build/out/bin/64/bitcoind: symbol __fdelt_chk from unsupported version GLIBC_2.15
    build/out/bin/64/bitcoind: symbol std::out_of_range::~out_of_range() from unsupported version GLIBCXX_3.4.15
    build/out/bin/64/bitcoind: symbol std::__detail::_List_node_base::_M_unhook() from unsupported version GLIBCXX_3.4.15
    build/out/bin/64/bitcoind: export of symbol std::basic_string<char, std::char_traits<char>, std::allocator<char> > std::operator+<char, std::char_traits<char>, std::allocator<char> >(char const*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) not allowed
    ...
    

    • From this you'd expect that the compatibility wrapper is not enabled, and symbol filtering is disabled? Not according to build.log, so I'm not sure where all the boost exports come from:

    checking for working boost reduced exports... yes
    checking whether the linker accepts -Wl,--exclude-libs,ALL... yes
    

    • --enable-glibc-back-compat is not logged, but there is no message about __fdelt_chk being logged. So that is probably disabled.
    • The static Qt has a few issues on my Ubuntu system, as expected: no longer it takes the theme from the system, and the 'tray icon' looks weird and doesn't use the OS's menu anymore.
22. 
    laanwj commented at 1:19 PM on August 26, 2014: member

    The windows executables build and work fine. No problems found. The caching mechanism works great.

23. theuni force-pushed on Aug 26, 2014
24. 
    theuni commented at 5:51 PM on August 26, 2014: member

    Pushed fixes for the visible symbols and glibc.

    The tray icon I'm not sure what to do about. I'm relatively sure it's this upstream bug.

    If their tracker can be trusted, it affects 5.2 and 5.3, and it should be fixed in 5.4. I'm thinking the more realistic option may be to just use an icon with a white background :\

25. 
    theuni commented at 7:06 AM on August 27, 2014: member

    Now that Travis is up and running, I'll write up some docs and remove the WIP tag here. I believe the only technical issue that remains (assuming the results prove to be deterministic) is what to do about the Qt tray icon, which will be worked out in depends. I'm fine with that being a merge-blocker though, if necessary. @TheBlueMatt I really don't see the need for shuffling around. Unless others speak up as well, I'm inclined to keep the paths as they are.

26. 
    laanwj commented at 7:57 AM on August 27, 2014: member

    Well - see for comparison against 0.9.2.1 release. In 0.9.2.1 it picks up my Ubuntu/GTK theme and shows the tray icon in the right place. Now it shows the Qt5 default fusion theme, misplaces the tray icon, and shows a black block around it.

    0.9.2.1 release

    

    gitian w/ this pull

    

    I'm sorry to be so picky about this, but yes I think this can be considered a merge blocker.

27. 
    theuni commented at 10:00 AM on August 27, 2014: member

    Yep, that's very fair. The icon problem we can probably work out one way or another.

    The themes are a different story though. I have a feeling that what you're seeing is about the best we can hope for with static qt5. If that's not going to cut it then our only option is switching back to shared and dealing with the drawbacks of that. Or, I suppose, shipping both ways :\

28. theuni force-pushed on Aug 29, 2014
29. 
    theuni commented at 6:42 AM on August 29, 2014: member

    @laanwj I'm trying a new approach here. I've added a qt 4.8 build to depends and set it as the Linux default. 4.6 is a major pain to build. It's very flaky with newer compilers and requires lots of patching. If 4.6 turns out to be required I can go that route.

    I elected instead to go with 4.8 and hook up a back-compat option for qt, similar to what we have with glibc. For now, I'd like to avoid any bikeshedding on the option itself and how its implemented, and just focus on whether or not it actually works. I haven't tested yet on a distro with an old 4.6 or 4.8.

    If it does work, it would be a big advantage because it wouldn't require gitian builds to test release-version behavior, it would only need a configure switch.

    Edit: I meant we should avoid discussing the semantics of the configure switch itself. Arguing the code implementation is fair game.

30. 
    laanwj commented at 7:57 AM on August 29, 2014: member

    @theuni Why not just use my approach of generating the headers for 4.6 only? We don't actually need the compiled library. We just need the headers!

31. theuni force-pushed on Sep 3, 2014
32. theuni force-pushed on Sep 4, 2014
33. in contrib/gitian-descriptors/README.md:None in 1b46dd3bfc outdated

      26 | @@ -27,7 +27,6 @@ Once you've got the right hardware and software:
      27 |  
      28 |      # Create base images
    

    ---

    ---

    

    Michagogo commented at 7:33 PM on September 4, 2014:

    s/s$//

34. in contrib/gitian-descriptors/gitian-linux.yml:None in 1b46dd3bfc outdated

      72 | +  for prog in ${FAKETIME_PROGS}; do
      73 | +    echo '#!/bin/bash' > ${WRAP_DIR}/${prog}
      74 | +    echo "REAL=\`which -a ${prog} | head -2 | tail -1\`" >> ${WRAP_DIR}/${prog}
      75 | +    echo 'export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1' >> ${WRAP_DIR}/${prog}
      76 | +    echo "export FAKETIME=\"${REFERENCE_DATETIME}\"" >> ${WRAP_DIR}/${prog}
      77 | +    echo "\$REAL \$@" >> $WRAP_DIR/${prog}
    

    ---

    ---

    

    Michagogo commented at 8:21 PM on September 4, 2014:

    Missing braces here?

35. in contrib/gitian-descriptors/gitian-win.yml:None in 1b46dd3bfc outdated

      44 | -  export TZ=UTC
      45 | -  INDIR=$HOME/build
      46 | -  OPTFLAGS='-O2'
      47 | -  TEMPDIR="$HOME/tempdir"
      48 | -  NEEDDIST=1
      49 | -  # Qt: workaround for determinism in resource ordering
    

    ---

    ---

    

    Michagogo commented at 8:40 PM on September 4, 2014:

    Why drop this comment?

36. in contrib/gitian-descriptors/gitian-win.yml:None in 1b46dd3bfc outdated

     166 | -  mkdir -p $TEMPDIR
     167 | -  cd $TEMPDIR
     168 | -  tar -xvf $HOME/build/bitcoin/$DISTNAME | sort | tar --no-recursion -cT /dev/stdin --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 --mtime="$REFERENCE_DATETIME" | gzip -n > $OUTDIR/src/$DISTNAME
     169 | -
     170 | +  mv $DISTNAME $OUTDIR/src
     171 | +  mv $OUTDIR/x86_64-* $OUTDIR/64
    

    ---

    ---

    

    Michagogo commented at 8:45 PM on September 4, 2014:

    Are these the dirs containing the plain binaries? If so, maybe we should put them in a $OUTDIR/bin, so there's src, bin, and setups? shrug

    ---

    

    theuni commented at 10:10 PM on September 4, 2014:

    These are the install prefixes, so it should be that way already. Under there should be bin/, lib/, etc (hah). Looking at it now (don't have a build handy so i can't say for sure), it should look like:

    setup32.exe
    setup64.exe
    src/source.tar.gz
    32/bin/bitcoin_qt (and others)
    32/include/(this will exist when we start shipping libs)
    32/lib/(this will exist when we start shipping libs)
    32/share/(docs and stuff)
    64/(same)
    

    I believe that's the same as how it is now, just that the new stuff will automagically appear when we add libs.

    ---

    

    Michagogo commented at 2:00 PM on September 5, 2014:

    Eh? Look at Linux. It looks to me like Linux puts them in $OUTDIR/bin/{32,64}. I'm suggesting that we do the same here. -$OUTDIR --bin ---64 ---32 --src

    ---

    

    theuni commented at 5:43 PM on September 5, 2014:

    @Michagogo that's fine, doesn't matter to me.

37. in depends/packages/libICE.mk:None in 1b46dd3bfc outdated

       0 | @@ -0,0 +1,23 @@
       1 | +package=libICE
       2 | +$(package)_version=1.0.9
    

    ---

    ---

    

    Michagogo commented at 8:49 PM on September 4, 2014:

    EDIT: Nevermind, this isn't bash.

38. theuni force-pushed on Sep 23, 2014
39. theuni force-pushed on Sep 23, 2014
40. 
    BitcoinPullTester commented at 4:26 PM on September 23, 2014: none

    Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/p4727_964c661e00c3fd741c87c5c3929498dbcbec15e9/ for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

41. theuni force-pushed on Oct 25, 2014
42. 
    theuni commented at 4:04 AM on October 25, 2014: member

    Rebased and updated to cope with a6a3f141b1b62a816510ba78d9e6aca991f4815b . Qt4 indeed broke without sslv2 in openssl. One of the benefits of the nasty qt4 build is that we hit those incompatibilities at build-time.

    Still not merge-ready, I'd like to go through it one more time top-to-bottom first.

43. 
    laanwj commented at 7:14 AM on October 25, 2014: member

    Stupid Qt4. Well if that requirement of sslv2 poses any possible security issue, we should just screw desktop integration and move to static Qt5 then.

44. 
    theuni commented at 9:18 PM on October 26, 2014: member

    Well we have no control over the qt4 that's used, so it will depend on whether sslv2 is patched out of it. Debian and most derivatives do, I'm not sure about others.

    As for our usage, I should think that we should probably be able to specify in code what protocols are allowed?

45. theuni force-pushed on Nov 11, 2014
46. 
    theuni commented at 6:59 AM on November 11, 2014: member

    Rebased and updated. Results are now deterministic (locally, at least). Now waiting on https://github.com/devrandom/gitian-builder/pull/73 and a final docs update. I'll need to emulate a full release and update the docs accordingly. I'll do that tomorrow, then this will be ready for final testing/review.

47. theuni force-pushed on Nov 12, 2014
48. theuni renamed this:
    [WIP] Gitian descriptors overhaul
    Gitian descriptors overhaul
    on Nov 12, 2014
49. 
    theuni commented at 12:43 AM on November 12, 2014: member

    All seems to be in order now. Ready for final review/testing.

    Note that the descriptors now contain a version as part of their name. For ex, "bitcoin-linux-0.10". That doesn't affect any paths relating to building/signing, but it serves to separate build caches. This way, we'll be able to maintain separate caches for each release to avoid having to rebuild everything. Though of course caches can always be blown away at any time if they're no longer needed or to save space.

50. laanwj added this to the milestone 0.10.0 on Nov 18, 2014
51. 
    laanwj commented at 1:07 PM on November 19, 2014: member

    Build was successful using these descriptors. Output (as of commit d7c2739bfd4db69ddb0f2f3bd96cae77073e4314):

    04fd1355e51356c52af9057f7d396ad55e3cea7037c2269b63e2d36ee4efd866  ./linux/32/bin/bitcoind
    23fe28403e118780e470d7cbcfe3972ee8bfa0625bd206de5b72be03eef98c82  ./linux/32/bin/bitcoin-tx
    52499a61a53634ffa47acafe181f0b00d865e26b7ba8d243bfe956216472f572  ./linux/32/bin/test_bitcoin
    279ae1c760b060f42c2c3789c7abb2ce5874d400ae641ff2772faaf828771faf  ./linux/32/bin/test_bitcoin-qt
    f06029f5d95a863ff49ded1b37156d0df758fe968ce1227fb601b2b4f80cef61  ./linux/32/bin/bitcoin-cli
    ee107376b5eeca8ac7e24569ea3d7125ee5915b58c4106a78001ffcab766a0d8  ./linux/32/bin/bitcoin-qt
    59de295f5903ca790c5d33a0fa0a6051811fd46f2b03d18682080959208a02cf  ./linux/src/bitcoin-0.9.99.tar.gz
    d2ab8ae53cac0ede4bdfc83d59572042162c31a493fc7f706161e9a070c0f475  ./linux/64/bin/bitcoind
    6b273dcc060ad92afcd75ef23672890efa6a347e3cb18ab3f119646d08c089e7  ./linux/64/bin/bitcoin-tx
    4f444a47cff4776495f4e9c575d9af951109007b2e1655c6320437ce29294265  ./linux/64/bin/test_bitcoin
    f0a5acfdf90aabd86619c4634dbb9b6992092dc9459cc3266cce05b93c412805  ./linux/64/bin/test_bitcoin-qt
    f7312fae5494c87221c032f3771ac59bdd7058f8653ef6f7272e1aab69f642d8  ./linux/64/bin/bitcoin-cli
    2a63eff15705486222615295a228fd535c5fb7f8d0e8edb94026709ea98ac0dc  ./linux/64/bin/bitcoin-qt
    11a173597d26d9f4b17509a6165599638438d5328f5d9ab80fbfae87c18e2670  ./windows/bitcoin-0.9.99-win64-setup.exe
    9aab88fcc7eb2355678bf5737a43f017ff552f9a8d65ac47de15a16498a974bf  ./windows/32/bin/test_bitcoin.exe
    a45fd1aa8a3061cb303877fef3f0306f2cfe1c96ec7176712865f04d88a3deb5  ./windows/32/bin/bitcoin-tx.exe
    155ca83c12dd629d47f775334e2af477a59946a146ab7bd96caa8b8661d9a51f  ./windows/32/bin/bitcoin-cli.exe
    82c288edf2b0bb3afd9d3ca77ad4ac203b79af7fab2ed243a2101b38ec413121  ./windows/32/bin/test_bitcoin-qt.exe
    3e1f89f224346adf44ddad0ce9f60f65048cec62098f094e0cfc7e7b9db50b88  ./windows/32/bin/bitcoind.exe
    fd600b3e6d0e965a69975ce2a6250b1c0fc4175545d152825253f47c93b687be  ./windows/32/bin/bitcoin-qt.exe
    9242840cccb21d1d629b9b0d60092223a206e7d39f56cb44660800f30702bcc5  ./windows/bitcoin-0.9.99-win32-setup.exe
    59de295f5903ca790c5d33a0fa0a6051811fd46f2b03d18682080959208a02cf  ./windows/src/bitcoin-0.9.99.tar.gz
    7b6f75ddf865fbb3245079bc4b56293a08e8604441fda69f78d82ddc82f321a1  ./windows/64/bin/test_bitcoin.exe
    32ab10f78807edfd2e745e34922412248aa67af505a19fd2d06ea855796cd70f  ./windows/64/bin/bitcoin-tx.exe
    54395ca38e5ba9fea15f03b27a15688430d64cf2e1164119619e674368984516  ./windows/64/bin/bitcoin-cli.exe
    130769de9b3a6cbe45b5bc911e38258208da1422b78c7d6e4ca30ad775fb39cf  ./windows/64/bin/test_bitcoin-qt.exe
    6ed67fb858befcd6a42bbf7119a6da76c4b86433389140b396b7957b45678b64  ./windows/64/bin/bitcoind.exe
    96fd4c5af6ab7fec78247a960b3eea6f6c71229b06b6c707d67c8c79280d648b  ./windows/64/bin/bitcoin-qt.exe
    9a16aea022a188ad9e9a1cf2e9f1e298a448488c6042b52a15022f7fbaec7849  ./macosx/x86_64-apple-darwin11/bin/bitcoind
    8cfd95aea0d72e6eb3321841a92930c7ec9425b23f87312c8fb0390228896485  ./macosx/x86_64-apple-darwin11/bin/bitcoin-tx
    780905c8841d7f0bf96708b8bcfcee28fd4746d14362877ccc9ee76bb91b602c  ./macosx/x86_64-apple-darwin11/bin/test_bitcoin
    db07864d1c98e32e6d4491f9021c5ca6dbdeb664cc4fe8cb90bbda7d71fa98af  ./macosx/x86_64-apple-darwin11/bin/test_bitcoin-qt
    880ff7eb08bd5da4ec1a24bfdfe2a603766b53c2e51070e4fbc81a57b324f28b  ./macosx/x86_64-apple-darwin11/bin/bitcoin-cli
    d83d7372ff9ba3acc281354f60da8e7d06ed9257bcc88f707167ae038ffc65e8  ./macosx/x86_64-apple-darwin11/bin/bitcoin-qt
    5ebcb60e49756dca610dbd20a6cd28add042697de9172bd36ca1c7271c44c94f  ./macosx/Bitcoin-Qt.dmg
    59de295f5903ca790c5d33a0fa0a6051811fd46f2b03d18682080959208a02cf  ./macosx/src/bitcoin-0.9.99.tar.gz
    

52. 
    theuni commented at 2:31 PM on November 19, 2014: member

    Thanks. I'll compare.

53. 
    theuni commented at 7:06 PM on November 19, 2014: member

    All matches except for the source tarballs. Mine match themselves and yours match themselves, but mine don't match yours. Could you upload one of them somewhere so that I can compare?

54. 
    sipa commented at 7:14 PM on November 19, 2014: member

    What do you mean by 'match themselves' ?

55. 
    theuni commented at 7:27 PM on November 19, 2014: member

    @sipa win/linux/osx all output source tarballs, and they all match eachother as they should. mine do too, but they're not the same as @laanwj's

56. theuni force-pushed on Nov 20, 2014
57. depends: Use pic for all linux dependencies

    This avoids textrels, and matches previous gitian behavior.

    4302fa67b1
58. depends: Add a package for qt4.6. Linux uses it by default.

    We're not ready to switch to a static qt5 for Linux yet due to missing plugin
    support. This adds a recipe for building a shared qt4 that we build and link
    against, but don't distribute.
    
    make USE_LINUX_STATIC_QT5=1 can be used to build static qt5 as before.

    5f93ec2028
59. gitian: descriptors overhaul

    Descriptors now make use of the dependencies builder, so results are cached.
    A very new version (>= e9741525c) of Gitian should be used in order to take
    advantage of caching.

    1aead42d41
60. gitian: quick docs update 4bbbdf3244
61. travis: let travis use shared libs for tests c54d647a92
62. gitian: make tarballs deterministic and nuke .la files from build output 246659aff1
63. 
    theuni commented at 4:48 AM on November 20, 2014: member

    I think that should have it worked out.

    From 246659af:

    4b9d79dbb4ad51612676d576cbad96db2611f91af1c5f315829f7407694882e8  ./linux/32/bin/test_bitcoin
    1d5b92f7d9c5499c92542e3dea926a571ec37459cecd69dad250d9c224aed98b  ./linux/32/bin/bitcoin-qt
    4088c5919fd33ab394f65821682704a04afffa37e8eda598f919e33d1b43736e  ./linux/32/bin/bitcoin-tx
    1517ab7e1962407aa719e6ab02f72430135c0ed3dd0af92ec99969da5cf5bd28  ./linux/32/bin/test_bitcoin-qt
    769ccf55d0ae3c84e4aead9dead36a87d57d74ba56967dce3c97fb772e646e92  ./linux/32/bin/bitcoin-cli
    55270757433ede1e49cc95c1ecc97899350b528297cbf43312b734ee46a2e4be  ./linux/32/bin/bitcoind
    00a0d37bf8b27d67df1ebef23135934ca7bbb6d45f24153847f7b460d2827a5c  ./linux/64/bin/test_bitcoin
    6fe8760c80ec8f7828cc0a4b05382f50d7e76c07cddf8f91eff7d8c193ce2edf  ./linux/64/bin/bitcoin-qt
    2464c68474a8678144bb0077f4b7c100cb29014d6cb49bc6a530be97ebf13603  ./linux/64/bin/bitcoin-tx
    b41dea711728f840f57bc27d54c8e68dbc64bd91342c3802d3a041f4d637826a  ./linux/64/bin/test_bitcoin-qt
    ec1d555b838a3e1e3386101a15b0a0fcd428c5f1b6d753b486e9818c32574292  ./linux/64/bin/bitcoin-cli
    4cb43151a15e9e0d84888ad9373f025ccd024fabb6430b230ea068258ddc83ba  ./linux/64/bin/bitcoind
    1f218b21d5cdff8d305248c56f1d0732c3442f579dfafa8e9b0561b4fc942159  ./linux/src/bitcoin-0.9.99.tar.gz
    1f218b21d5cdff8d305248c56f1d0732c3442f579dfafa8e9b0561b4fc942159  ./macosx/src/bitcoin-0.9.99.tar.gz
    ff80ba89bf68c86eac1102c26e4754ea50cca8af38b40515f3b89a28a67d5097  ./macosx/x86_64-apple-darwin11/bin/test_bitcoin
    3d29088b510d03a7e5f13de9fee84d5d1dcefc775438114b9f9a8dc239c57bbc  ./macosx/x86_64-apple-darwin11/bin/bitcoin-qt
    1d7933b2b1cd459216e8559f1f0fcb451162e33904abaf0508f0ee4195ff7d0b  ./macosx/x86_64-apple-darwin11/bin/bitcoin-tx
    d9ce06f00c0b47d4f62a890380a9889fe4765c19a3afcea6ac7b390c7521bbaf  ./macosx/x86_64-apple-darwin11/bin/test_bitcoin-qt
    b313fc3f20519c7de03b337ae37602ec835c1d29daf05f032c594adcf51f69a6  ./macosx/x86_64-apple-darwin11/bin/bitcoin-cli
    e9d6421b0aaf1f6ae1e2612638baa245bfba0e7de71c609afcc9a229ba6b3e36  ./macosx/x86_64-apple-darwin11/bin/bitcoind
    16f2d08444789d08b3096f32df9a5007d490deb0dd8044276d125324633e19e0  ./macosx/Bitcoin-Qt.dmg
    fa70d235b5aed5e79b5ed546dc418c03a7a99fe3ff710f5b5eac209d1e7e0283  ./windows/32/bin/test_bitcoin.exe
    1e8f49ebdc60856ec5a069feed8a4c325aba91d7345488811dbabd88fd3c3042  ./windows/32/bin/bitcoin-qt.exe
    a6fb58f716ec0cd6d8ae61990e03c37cc24af636c86a5f6a2ba4a5542f3ad518  ./windows/32/bin/bitcoin-tx.exe
    994c58a35923e5e6054ed0f1752aed33dd056f087cc6102148813e540f4d94f7  ./windows/32/bin/bitcoin-cli.exe
    11e42d59554c10bfa728b3ce090ad735057827806c155c50bb2b153775292b3f  ./windows/32/bin/test_bitcoin-qt.exe
    24f1ccabf279cf0d6d02d844a551c4f3c519ebe7dc87f3346c205caeffe9054f  ./windows/32/bin/bitcoind.exe
    e77986b91d46c1cc2e3cd7a6a8094c5f0f6806d88e4f93f696c8cba4cfe35d98  ./windows/64/bin/test_bitcoin.exe
    ab880a26513a746fd39a7423498c0dcddf3880f61a0e2fc8eca02995dc9814ff  ./windows/64/bin/bitcoin-qt.exe
    0ae5c8d0c00fdc360599657de89cf116fe86796c0e1eeb43be9170c0cdeae199  ./windows/64/bin/bitcoin-tx.exe
    a7c4a94dd53f5b728be2d1a56eb844e8ce50d25e7a022b190b681b27fb81e9c4  ./windows/64/bin/bitcoin-cli.exe
    67ee3524bea8be1ff5e8a0bb198b9d69915ae38cfd79d7f1583b50cda10fb25b  ./windows/64/bin/test_bitcoin-qt.exe
    b0ffc16a5036f4d87e4ab0a298455122498fad8c89de1dea0009501387102964  ./windows/64/bin/bitcoind.exe
    fe90e7c83393f3387a809832aa6fc81fa8deceda393238cb76843d1dd0469bbb  ./windows/bitcoin-0.9.99-win32-setup.exe
    1f218b21d5cdff8d305248c56f1d0732c3442f579dfafa8e9b0561b4fc942159  ./windows/src/bitcoin-0.9.99.tar.gz
    82695e651b2dbed352ae9b8a3dacd3777b7abbfb35517e79c0468cea505117f9  ./windows/bitcoin-0.9.99-win64-setup.exe
    

64. 
    laanwj commented at 12:24 PM on November 20, 2014: member

    @cfields https://download.visucore.com/bitcoin/bitcoin-0.9.99.tar.gz

65. 
    laanwj commented at 4:07 PM on November 21, 2014: member

    I'm rebuilding the linux gitian with commit 246659a to see if the .tar.gz matches, but I noticed it's also rebuilding Qt. This is taking a long time and that doesn't seem to be necessary? (Edit: oops I was building the wrong commit, master instead of 246659a, although that still doesn't explain why it rebuilt Qt. Anyhow it only happened once)

66. 
    laanwj commented at 4:39 PM on November 21, 2014: member

    .tar.gz (and the rest) matches!

    1f218b21d5cdff8d305248c56f1d0732c3442f579dfafa8e9b0561b4fc942159 src/bitcoin-0.9.99.tar.gz

67. laanwj merged this on Nov 21, 2014
68. laanwj closed this on Nov 21, 2014

    ---

69. laanwj referenced this in commit a574189e2a on Nov 21, 2014
70. 
    theuni commented at 6:37 PM on November 21, 2014: member

    woohoo! @laanwj I'm not sure why qt would've been rebuilt. Best guess is that there was a small change in the last rebase or two, maybe openssl was touched? Otherwise if it rebuild needlessly, that's a bug that I'd want to fix for sure.

71. MarcoFalke locked this on Sep 8, 2021

Contributors

theunilaanwjTheBlueMattMichagogoBitcoinPullTestersipa

Milestone
0.10.0