gitian: build on ubuntu 14.04 #6900

pull laanwj wants to merge 5 commits into bitcoin:master from laanwj:2015_10_gitian_trusty changing 10 files +98 −31
  1. laanwj commented at 8:29 am on October 29, 2015: member

    This changes the VM environment for gitian building to use Ubuntu 14.04 (Trusty).

    • Changes compiler version from GCC 4.6 to GCC 4.8, which makes #6211 (using the relevant subset of c++11) possible, and probably results in better code
    • Removes some 12.04 specific cruft from descriptors
    • The security and symbol checks pass (symbol check reports clock_gettime, but this is the same for old descriptors thus unrelated)
    • Linux and OSX and Windows output is deterministic

    Known problems:

    Windows build is not deterministic

    SOLVED

    Every link, a few bytes in the executable near the end of .rodata are different. These are not a timestamp but apparently random.

    I spent some time tracking this down, eventually used a linker map to find that the bytes come from a ephermal object ertr000001.o. This helped trace this problem to the function pe_create_runtime_relocator_reference in binutils/ld/pe-dll.c which leaks a few bytes of heap to the executable by writing uninitialized data (!).

    This sounds serious, however apparently this was already solved by the Erinn Clark of the Tor project in 2013: https://sourceware.org/bugzilla/show_bug.cgi?id=16192 . Awesome.

    Unfortunately Ubuntu hasn’t included the new binutils in 14.04… @theuni any idea how to handle this? Tor, for a while, manually patched the bytes (well not manually I suppose - in a postprocessing step). Another option would be to use our own binutils, or use an even newer image (but that wouldn’t be long term supported).

  2. laanwj added the label Build system on Oct 29, 2015
  3. wtogami commented at 6:59 pm on October 29, 2015: contributor
    Does this change the minimum compatible system glibc/libstdc++ version of the Linux binary? If the new glibc or libstdc++ added new symbols @theuni may need to add more wrappers to the compat library.
  4. laanwj commented at 8:40 pm on October 29, 2015: member

    Does this change the minimum compatible system glibc/libstdc++ version of the Linux binary?

    It shouldn’t. That’s what the symbols check (contrib/devtools/symbols-check.py) checks for.

  5. theuni commented at 2:52 am on October 30, 2015: member

    Well that was easy :)

    For Windows, patching seems like it’d be the easiest, but I’d worry about breaking checksums. I suppose if Tor did it that way, either it’s not an issue or updating the checksums is doable as well. Will look into it. @sipa mentioned on IRC that depends are currently broken for him on his dev machine, and the fix is the change you made here to qt.mk. Rather than removing the hackish sed, let’s make a patch which properly ifdefs, so that it’ll work on old/new mingw.

    Also, I’d be more comfortable with this if we could wait for Trusty on Travis. I’ve pinged them to try to get an idea of a timeline. Obviously if it’s going to drag on too long, we can just bump the compiler versions we use in our current env in order to simulate a bump to Trusty.

  6. laanwj commented at 9:00 am on October 30, 2015: member

    For Windows, patching seems like it’d be the easiest, but I’d worry about breaking checksums. I suppose if Tor did it that way, either it’s not an issue or updating the checksums is doable as well. Will look into it.

    Tor uses ‘strip’ to recompute the checksum. I’m not sure what they used to find the index of the bytes, as it is not a fixed offset into any section in the final file. Maybe they used the linker map like me. We could ask them.

    Rather than removing the hackish sed, let’s make a patch which properly ifdefs, so that it’ll work on old/new mingw.

    Right - Qt did an attempt at this, but they apparantly got the version threshold wrong. Though it is not impossible that Ubuntu complicated this by backporting things into older releases. From qwindowsdialoghelpers.cpp:

    0/* The correct declaration of the SHGetPathFromIDList symbol is
    1 * being used in mingw-w64 as of r6215, which is a v3 snapshot.  */
    2#if defined(Q_CC_MINGW) && (!defined(__MINGW64_VERSION_MAJOR) || __MINGW64_VERSION_MAJOR < 3)
    3typedef ITEMIDLIST *qt_LpItemIdList;
    4#else
    5typedef PIDLIST_ABSOLUTE qt_LpItemIdList;
    6#endif
    

    Or maybe it is correct! this kind of armoring is missing completely for the use of PIDLIST_ABSOLUTE in qwindowscontext.h, and that is what Travis is failing on.

    Also, I’d be more comfortable with this if we could wait for Trusty on Travis

    I’d like so, too. Depends on the time frame - IMO it would be nice to build 0.12 using the new compiler, as it appears to be pretty much a hassle free upgrade.

  7. laanwj commented at 9:04 am on October 30, 2015: member

    OK, this is Tor’s workaround.

    0  # XXX: What the hell are these three bytes anyways?
    1  sed 's/\x94\x0C\xE7\x6A\xB8\x0C\xE7\x6A\x08...\x04\xBA\xCB\x7E/\x94\x0c\xe7\x6a\xb8\x0c\xe7\x6a\x08\x23\x23\x23\x04\xba\xcb\x7e/M' -i $INSTDIR/App/Firefox/xul.dll
    2  sed 's/\x94\xb1\x24\x65\xb8\xb1\x24\x65\xf8...\x04\xba\xcb\x7e/\x94\xb1\x24\x65\xb8\xb1\x24\x65\xf8\x23\x23\x23\x04\xba\xcb\x7e/M' -i $INSTDIR/App/Firefox/gkmedias.dll
    3  # Stripping again updates the PE header checksums to be correct
    4  i686-w64-mingw32-strip $INSTDIR/App/Firefox/gkmedias.dll
    5  i686-w64-mingw32-strip $INSTDIR/App/Firefox/xul.dll
    

    Using sed for binary patching is not anything I’ve seen before, or would necessarily encourage (fragility as the surrounding data may move). All trace of this is gone in their new builders, so I assume they use a newer version now: https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitian/descriptors/windows/gitian-firefox.yml

    They’re patching binutils: https://gitweb.torproject.org/builders/tor-browser-bundle.git/commit/gitian/descriptors/windows/gitian-firefox.yml?id=fd14de1870b38987c25eba1d3f8ca87b22f3bb8b Then they commented out the hacks: https://gitweb.torproject.org/builders/tor-browser-bundle.git/commit/gitian/descriptors/windows/gitian-firefox.yml?id=95446ee9fef932ba2c443968a90b650a36cd2182

  8. laanwj commented at 12:52 pm on October 30, 2015: member

    Wrote a script that overwrites the bytes given an executable and linker map, then calls strip to recompute the checksum. It was somewhat more involved than I thought as it has to parse the PE sections to find the file offset: https://gist.github.com/laanwj/01dc996c4755d19e134e

    But it works for 32/64 bit, tested using

    0#!/bin/bash
    1$1 mtest.c -o mtest_1 -Wl,-Map=mtest.map
    2$1 mtest.c -o mtest_2 -Wl,-Map=mtest.map                                                                    
    3$1 mtest.c -o mtest_3 -Wl,-Map=mtest.map                                                                    
    4sha256sum mtest_1 mtest_2 mtest_3                                                                                           
    5~/projects/bitcoin/bitcoin/contrib/devtools/bitstomp.py  mtest_1 mtest.map
    6~/projects/bitcoin/bitcoin/contrib/devtools/bitstomp.py  mtest_2 mtest.map                                                    
    7~/projects/bitcoin/bitcoin/contrib/devtools/bitstomp.py  mtest_3 mtest.map                                                    
    8sha256sum mtest_1 mtest_2 mtest_3
    
    0$ ./mtest.sh i686-w64-mingw32-gcc
    1570f42e57f2a5b864f2464eab5f9e49d64545f36992f8429c6a2555ec29272ed  mtest_1
    2ab53006e857d3e8eb14d380a94cdc0a6cc9291a1d4a284961c5fd323bd915dec  mtest_2
    346e6caae3a29378b0bd023124c4000856f6049c4a0602ff599aa2b6035417fa1  mtest_3
    4mtest_1: Patching e8833007 to 00000000 at offset 0000205c (virtual address 000000000040425c)
    5mtest_2: Patching e8735bdb to 00000000 at offset 0000205c (virtual address 000000000040425c)
    6mtest_3: Patching e8638e8d to 00000000 at offset 0000205c (virtual address 000000000040425c)
    7df8a954df2aedb9d004e863449be180aad85490fee5856159102c408e8204ecd  mtest_1
    8df8a954df2aedb9d004e863449be180aad85490fee5856159102c408e8204ecd  mtest_2
    9df8a954df2aedb9d004e863449be180aad85490fee5856159102c408e8204ecd  mtest_3
    
    0$ ./mtest.sh x86_64-w64-mingw32-gcc                                                                                                 
    14647ac5ed9c73aa7cb33324d0222b6c45c3ced70b495e92771c323033f9670bb  mtest_1
    2e43b0fac2bcaa09173c5f5d9b529a9f7ffa2e15e7f67881a8c12226a09058998  mtest_2
    3d8664c15f4a69de0f605a9c8692db954d47480d281205ef946f0879edb705e73  mtest_3
    4mtest_1: Patching e8d60fa24e7f0000 to 0000000000000000 at offset 00002880 (virtual address 0000000000404280)
    5mtest_2: Patching e8d6dbeb997f0000 to 0000000000000000 at offset 00002880 (virtual address 0000000000404280)
    6mtest_3: Patching e836fb54c67f0000 to 0000000000000000 at offset 00002880 (virtual address 0000000000404280)
    751e8cba15304fa7ff52de698b0c9b3ba8a4502df3b099589b56b0e37619e332c  mtest_1
    851e8cba15304fa7ff52de698b0c9b3ba8a4502df3b099589b56b0e37619e332c  mtest_2
    951e8cba15304fa7ff52de698b0c9b3ba8a4502df3b099589b56b0e37619e332c  mtest_3
    
  9. laanwj commented at 7:48 am on October 31, 2015: member
    Now uses another approach (suggested by @theuni) to make the windows build deterministic, until binutils can be updated.
  10. laanwj force-pushed on Nov 1, 2015
  11. laanwj force-pushed on Nov 1, 2015
  12. laanwj force-pushed on Nov 1, 2015
  13. laanwj force-pushed on Nov 1, 2015
  14. laanwj commented at 1:08 pm on November 1, 2015: member

    @theuni I replaced the qt PIDLIST_ABSOLUTE patching with a patch that should work for both old and new mingw.

    Should be ready for merge.

  15. laanwj added this to the milestone 0.12.0 on Nov 2, 2015
  16. laanwj commented at 11:29 am on November 5, 2015: member
    Anyone testing this?
  17. fanquake commented at 11:32 am on November 5, 2015: member

    Yes. Will post back shortly.

    On Thursday, November 5, 2015, Wladimir J. van der Laan < notifications@github.com> wrote:

    Anyone testing this?

    — Reply to this email directly or view it on GitHub #6900 (comment).

  18. jonasschnelli commented at 11:32 am on November 5, 2015: contributor
    I’ll try this within the next couple of hours.
  19. laanwj commented at 2:10 pm on November 10, 2015: member
  20. laanwj commented at 12:28 pm on November 16, 2015: member
    Anyone? I’d like to move forward with this, don’t think there is anything really holding this back anymore.
  21. jonasschnelli commented at 12:49 pm on November 16, 2015: contributor

    I really think this is useful. But i can’t get it running on my Debian 8 jessy machine (see below). I tried to track this down but it seem after a VirtualBox mkfs issue. Couldn’t solve it after investing 1-2h.

     0$bin/make-base-vm --arch amd64 --suite trusty
     1-------------- snip -------------------
     22015-11-16 13:47:09,510 INFO    : Creating file systems
     32015-11-16 13:47:09,666 INFO    : mke2fs 1.42.12 (29-Aug-2014)
     42015-11-16 13:47:09,666 INFO    : The file /dev/mapper/loop0p1 does not exist and no size was specified.
     52015-11-16 13:47:09,667 INFO    : Cleaning up
     62015-11-16 13:47:12,684 ERROR   : Process (['mkfs.ext4', '-F', '/dev/mapper/loop0p1']) returned 1. stdout: , stderr: mke2fs 1.42.12 (29-Aug-2014)
     7The file /dev/mapper/loop0p1 does not exist and no size was specified.
     8
     9Traceback (most recent call last):
    10  File "/usr/local/bin/vmbuilder", line 24, in <module>
    11    cli.main()
    12  File "/usr/local/lib/python2.7/dist-packages/VMBuilder/contrib/cli.py", line 228, in main
    
  22. fanquake commented at 12:53 pm on November 16, 2015: member

    Have you tried the latest Python vmbuilder? That solved that issue for me, check in the gitian guide for the updated link.

    I haven’t had a chance to continue testing after getting that initial issue solved. I can do tomorrow.

    On Monday, November 16, 2015, Jonas Schnelli notifications@github.com wrote:

    I really think this is useful. But i can’t get it running on my Debian 8 jessy machine (see below). I tried to track this down but it seem after a VirtualBox mkfs issue. Couldn’t solve it after investing 1-2h.

    $bin/make-base-vm –arch amd64 –suite trusty ————– snip ——————- 2015-11-16 13:47:09,510 INFO : Creating file systems 2015-11-16 13:47:09,666 INFO : mke2fs 1.42.12 (29-Aug-2014) 2015-11-16 13:47:09,666 INFO : The file /dev/mapper/loop0p1 does not exist and no size was specified. 2015-11-16 13:47:09,667 INFO : Cleaning up 2015-11-16 13:47:12,684 ERROR : Process ([‘mkfs.ext4’, ‘-F’, ‘/dev/mapper/loop0p1’]) returned 1. stdout: , stderr: mke2fs 1.42.12 (29-Aug-2014) The file /dev/mapper/loop0p1 does not exist and no size was specified.

    Traceback (most recent call last): File “/usr/local/bin/vmbuilder”, line 24, in cli.main() File “/usr/local/lib/python2.7/dist-packages/VMBuilder/contrib/cli.py”, line 228, in main

    — Reply to this email directly or view it on GitHub #6900 (comment).

  23. jonasschnelli commented at 1:32 pm on November 16, 2015: contributor

    @fanquake: Thanks! Damit. I forgot that Debian does not have the python vmbuilder tools over apt. Manually updated and it works now…

    Testing gitian building this PR now…

  24. jonasschnelli commented at 2:46 pm on November 16, 2015: contributor

    On my debian 8 machine I got the following errors:

    Linux:

    MinGW:

    0---snip---
    1configure: error: No working boost sleep implementation found.
    

    (check: https://bitcoin.jonasschnelli.ch/pulls/6900/build-win.log)

    OSX:

    works: https://bitcoin.jonasschnelli.ch/pulls/6900/build-osx.log

  25. laanwj force-pushed on Nov 16, 2015
  26. devtools: add libraries for bitcoin-qt to symbol check
    Forgot to add these.
    Also add a short description for each required library.
    9f251b7a9d
  27. depends: qt PIDLIST_ABSOLUTE patch
    Remove sed-based qt PIDLIST_ABSOLUTE workaround, replace by a patch that
    works for both old (such as used by Travis and Ubuntu Precise) and new
    mingw (Ubuntu Trusty).
    0b416c6e9c
  28. gitian: use trusty for building 2e31d74b71
  29. laanwj commented at 3:41 pm on November 16, 2015: member
    Rebased this to master - going to retry my builds
  30. jonasschnelli commented at 3:43 pm on November 16, 2015: contributor
    Re-Kicked the builder… log-links above are invalid now.
  31. laanwj commented at 7:22 am on November 17, 2015: member

    Getting this now on the windows build, since rebase:

    0gcc gen_context.o -o gen_context
    1/usr/bin/x86_64-w64-mingw32-ld: unrecognised emulation mode: elf_x86_64
    2Supported emulations: i386pep i386pe
    3collect2: error: ld returned 1 exit status
    

    This is because of:

    0export COMPILER_PATH=${WRAP_DIR}/${i}
    

    The workaround to set MALLOC_PERTURB_ for linking no longer works. Our collect2 gets invoked for building a native executable too, which obviously fails. @cfields any idea how to fix this? I guess moving the export COMPILER_PATH=xxx to ${tuple}-g++/gcc wrappers may work.

  32. laanwj commented at 6:05 pm on November 18, 2015: member

    Pushed a new commit - all builds including windows should work now.

    (when testing, if you used gitian on 0.12 before, make sure that you remove your caches for 0.12, as they will have been built using previous c++ compiler, causing non-determinism and compile/link errors)

  33. jonasschnelli commented at 9:14 pm on November 18, 2015: contributor
    All three platforms are working here: https://bitcoin.jonasschnelli.ch/pulls/6900/ Now it would be good if someone would post or compare the assert-files to see if it’s really deterministic.
  34. fanquake commented at 5:58 am on November 19, 2015: member
    @jonasschnelli Are you building a release, or this pull request on top of master? My signatures don’t match with yours currently.
  35. laanwj commented at 7:08 am on November 19, 2015: member

    Building commit b082935 with the descriptors of commit b082935 should give:

    082a5e6c87137b7a3f18d4230cc22b1d26b95875c436130de123c272defd5d6c9  bitcoin-0.11.99-linux32.tar.gz
    1f2f81efaeeb16d55dfed2dc6dfa681569cb971b53a8441a3bf083ccb994f82a8  bitcoin-0.11.99-linux64.tar.gz
    267c3bc85ece1ad2905a7f801277fbd76d1e7ac653ad2e021cd7fadf1fa6a6307  bitcoin-0.11.99.tar.gz
    
    067c3bc85ece1ad2905a7f801277fbd76d1e7ac653ad2e021cd7fadf1fa6a6307  bitcoin-0.11.99.tar.gz
    1b2fb5f8b7dcfd9ba1164d13b2aad398fd71b6c7c440b207cf2dfb9302119ff7d  bitcoin-0.11.99-win32-setup-unsigned.exe
    25c74be96221c77a7dfce82c0c04d37672e64de06725a9c3a1e372a1e5062addd  bitcoin-0.11.99-win32.zip
    375f0403001332f2cec8dc07191eb5c67b97d8dc87a4f41624b974c4a35991ade  bitcoin-0.11.99-win64-setup-unsigned.exe
    41592975ac6219793e8c334eb8167c0ccdfa5396e2cf6aa9dd9435ade2cb91b3d  bitcoin-0.11.99-win64.zip
    57fd5e22a794a6fa34defe0cd0e82d8f0ad759fba73e190aa5bd202627fa45bc5  bitcoin-0.11.99-win-unsigned.tar.gz
    
    053a9c868e1e2ba862af05f80471f07a57c90b80ccc88e44ec0161bca95c04d75  bitcoin-0.11.99-osx64.tar.gz
    1b23f22f26c535cc46affd6e6d46f4ef80a33956a93d5e612a0d216537bbfb061  bitcoin-0.11.99-osx-unsigned.dmg
    26cf0b9a04f46bca67defff1f9240c10a1e432b7659f3a5a8e231e0292ec8ea4c  bitcoin-0.11.99-osx-unsigned.tar.gz
    367c3bc85ece1ad2905a7f801277fbd76d1e7ac653ad2e021cd7fadf1fa6a6307  bitcoin-0.11.99.tar.gz
    
  36. jonasschnelli commented at 7:11 am on November 19, 2015: contributor

    @fanquake: right. My build script always pulls a PR on top of master. A match would only be possible if you do the same (check the git HEAD~5 log): https://bitcoin.jonasschnelli.ch/pulls/6900/

    But i’ll try now to build https://github.com/bitcoin/bitcoin/commit/b08293544a207088193de8834bb754f5d212c9bf (to compare with @laanwj ’s hashes)

  37. jonasschnelli commented at 8:13 am on November 19, 2015: contributor

    Linux / OSX: match

    ~~Windows: no match (https://bitcoin.jonasschnelli.ch/pulls/6900/bitcoin-win-0.12-build.assert) ~~ I used

    0b8affff612d645598a4642dcc4eef7d4974c02d73c31a99ba2faa36425142aca  bitcoin-win-0.12-desc.yml
    1git:b08293544a207088193de8834bb754f5d212c9bf bitcoin
    

    Any ideas?

  38. laanwj commented at 8:22 am on November 19, 2015: member
    Which packages differ? Can you upload them? Looking at your assert link, they simply match.
  39. jonasschnelli commented at 8:30 am on November 19, 2015: contributor

    Correction: window build matched. (Compared against the wrong file).

    Nice! Tested ACK.

  40. laanwj force-pushed on Nov 19, 2015
  41. gitian: make windows build deterministic 957c0fd7c0
  42. doc: change suite to trusty in gitian-building.md 2cecb24600
  43. laanwj merged this on Nov 19, 2015
  44. laanwj closed this on Nov 19, 2015

  45. laanwj referenced this in commit c983d6fcb4 on Nov 19, 2015
  46. fanquake commented at 12:05 pm on November 19, 2015: member

    ACK

    On Thursday, November 19, 2015, Wladimir J. van der Laan < notifications@github.com> wrote:

    Merged #6900 #6900.

    — Reply to this email directly or view it on GitHub #6900#event-469081233.

  47. laanwj commented at 12:28 pm on November 19, 2015: member
    Thanks a lot for testing @fanquake @jonasschnelli
  48. theuni commented at 5:38 pm on November 19, 2015: member

    @laanwj Sorry for missing this while traveling.

    For the cache issue, sounds like depends needs to take the output as $(CXX) -v as part of the hash, so that a new compiler invalidates all binaries.

  49. zkbot referenced this in commit 4ee9d712b5 on Oct 17, 2016
  50. fanquake referenced this in commit ac85e81685 on Aug 20, 2019
  51. fanquake referenced this in commit bd3f5a90ec on Sep 23, 2019
  52. MarcoFalke referenced this in commit 4765b91f50 on Oct 14, 2019
  53. sidhujag referenced this in commit a24510d22d on Oct 15, 2019
  54. Fuzzbawls referenced this in commit ce80c763f6 on Jun 23, 2021
  55. MarcoFalke locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-17 12:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me