Make gitian builds consistent across platforms #3546

Michagogo commented at 8:59 PM on January 15, 2014: contributor

Change Linux deps to use a zip archive rather than a gzipped tarball to match win32 Rename Linux descriptor to gitian-linux.yml to match win32

Make gitian builds consistent across platforms

Change Linux deps to use a zip archive rather than a gzipped tarball to
match win32
Rename Linux descriptor to gitian-linux.yml to match win32

f4e72bf8d2

laanwj commented at 9:30 PM on January 15, 2014: member

Agree with the rename, but any specific reason to switch to zip for linux? I personally prefer .tar.gz (it's more linux-ish and it saves a dependency package too).

BitcoinPullTester commented at 9:33 PM on January 15, 2014: none

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/f4e72bf8d20f590738a1307fff0787bb2aa870c8 for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

Michagogo commented at 9:34 PM on January 15, 2014: contributor

@laanwj It's just a matter of consistency... I guess either way is fine, but my decision to use .zip instead of .tar.gz was simply "which one requires fewer changes", since there are several .zips and only one .tar.gz. If you want, I can go and switch them all... (and hopefully not mess up the tar syntax, since I'm not too familiar with that, but I can probably imitate linux deps.)

Michagogo commented at 9:36 PM on January 15, 2014: contributor

Oh, just noticed #3545... Either that PR or this one will probably need to be amended to match...

laanwj commented at 9:42 PM on January 15, 2014: member

I'm fine with switching them to zip, I just think it's a bit pointless.

Anyway, as for the doc/release-process.md that could now be cleaned up to

    ./bin/gbuild ../bitcoin/contrib/gitian-descriptors/deps-linux.yml
    mv build/out/* inputs/
    ./bin/gbuild ../bitcoin/contrib/gitian-descriptors/boost-linux.yml
    mv build/out/* inputs/
    ./bin/gbuild ../bitcoin/contrib/gitian-descriptors/boost-win32.yml
    mv build/out/* inputs/
    ./bin/gbuild ../bitcoin/contrib/gitian-descriptors/deps-win32.yml
    mv build/out/* inputs/
    ./bin/gbuild ../bitcoin/contrib/gitian-descriptors/qt-win32.yml
    mv build/out/* inputs/
    ./bin/gbuild ../bitcoin/contrib/gitian-descriptors/protobuf-win32.yml
    mv build/out/* inputs/

After #3545, no crap is being generated anymore in the output directory (before this, boost put the source files in there as well), so the outputs can simply be copied to the inputs directory without needing to distinguish file types.

wtogami commented at 11:48 PM on January 15, 2014: contributor

@theuni Expressed a desire to change everything from .zip to tarballs. Cory do you still want this?

theuni commented at 12:12 AM on January 16, 2014: member

I did this on another project I worked on (deterministic dmg from gitian) and it turned out to be a bit of a pain. Given that there are drawbacks to both approaches, I'm not really married to either one anymore. So it's probably just easier to stick with zips.

For completeness's sake, here's what it takes to get real-world deterministic .tar.gz's:

export TAR_OPTIONS="-m --mtime="$REFERENCE_DATE\\\ $REFERENCE_TIME""
export GZIP="-9n"
find input-folder | sort | tar --no-recursion -czf output.tar.gz -T -

That tells tar to use a phony timestamp, gzip to use null timestamps, and sorts file entries before tarring them to avoid random input orders from the vfs.

laanwj commented at 12:22 AM on January 16, 2014: member

Ok, let's just stick with zip then. I had no idea .tar.gz was harder to get deterministic.

wtogami commented at 6:33 AM on January 16, 2014: contributor

Close ticket?

laanwj commented at 8:05 AM on January 16, 2014: member

Why close ticket? To me it looks like @Michagogo was doing the right thing all along and this should be merged instead of closed.

laanwj referenced this in commit 417aad4a38 on Jan 16, 2014

laanwj merged this on Jan 16, 2014

laanwj closed this on Jan 16, 2014

Michagogo deleted the branch on Jan 16, 2014

DrahtBot locked this on Sep 8, 2021