Clean up release-process.md and make descriptors more consistent #4271

This addresses some (but not all) of the issues in #4185. It cleans up a few things in release-process.md (such as indents) and makes the descriptors more consistent, in terms of style and also naming (both of the descriptors and the intermediate inputs). This doesn't change any of the actual build scripts, other than to change the file names -- I didn't want to accidentally break anything, though it looks like there are a few things that could be improved and made more consistent. The big change that it would be nice if someone were to make is to switch the OS X intermediates from tarballs to zips for more consistency. I didn't do that, because I don't know the details of how tar and zip/unzip work on the command line, especially when it comes to determinism.

Note that in release-process.md, some parts still need to be updated and OS X added. I only updated the steps that each gitian builder follows.

I'm sure there is a good reason to use .tar.gzs instead of zips for the intermediate dependencies. For example for qt-linux32-4.6.4-gitian I used a .tar.gz because I needed support for symlinks.

@laanwj Well, there may be -- but other things (such as the descriptor names) suggest that the OS X descriptors may have been made independently from the current ones, rather than in coordination, consistent with the current ones. @theuni, could you comment? Was there a specific reason you went with tarballs?

@Michagogo OSX makes extensive use of symlinks, and I was more comfortable doing that with tarballs. If zips get the job done, I don't mind using 'em.

It seems that the OS X intermediates do match... Adding that. Also, I think that before 0.9.2 becomes final, the OS X descriptor should be changed to output a source tarball, which should then be packaged into a versioned zip like Windows and Linux are atm.

@theuni Well, I don't know if they get the job done... I, personally, would prefer a switch to .zips if possible (i.e. doesn't include symlinks?), but it's not that critical.

Bleh. I'll rebase tomorrow or Sunday or something.

On Fri, Jun 6, 2014 at 3:25 AM, BitcoinPullTester notifications@github.com wrote:

Automatic sanity-testing: FAILED MERGE, see http://jenkins.bluematt.me/pull-tester/ff7ed723be85b8492a707092e030c5453a4c6f37 for test log.

This pull does not merge cleanly onto current master

This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

—

Reply to this email directly or view it on GitHub #4271 (comment).

@theuni If we were to standardize on any format for the intermediate dependencies (which I don't see any point of, and wouldn't give any priority), I think it'd be better to standardize on tarballs. Tar wins in capabilities, as it supports symlinks et al by default. Zip is harder to get deterministic if you enable UNIX extensions, no way to force certain permissions/ownership like tar has.

@laanwj Completely agreed (which is why I've used tarballs when given the choice), I was just trying to be agreeable here :)

Tarballs would also be fine, although there's the (very slight) benefit of being able to tell source tarballs apart from intermediate zips.

On Fri, Jun 6, 2014 at 7:39 AM, Cory Fields notifications@github.com wrote:

@laanwj https://github.com/laanwj Completely agreed (which is why I've used tarballs when given the choice), I was just trying to be agreeable here :)

— Reply to this email directly or view it on GitHub #4271 (comment).

@Michagogo Good point there, although something could be said that that distinction should not be made based on file name. The current grab-bag inputs directory is a hack in itself.

It would be better if gitian could source from multiple inputs directories so the intermediate files could live in their own directories. Combined with support for output to a configurable directory, the move/copy step after dependency builds could be skipped completely (also making it impossible to forget it!).

Rebased, I think.

Erm, looks like I missed one conflict when rebasing. Fixed.

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/p4271_17601d359d8dd693cef59cd1d268c1bea0cf4d6d/ for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

Going to close this. @theuni is working on a lot of changes in the gitian dependency handling and pulltester and this would interfere. Any consistency changes should be taken into account then.

@laanwj How would you feel about a version of this that doesn't include the changes to the actual descriptor filenames and output/input names, but rather is limited to the cleanup of r-p.md itself?

No problem with pure documentation changes

@laanwj And what about the change from gitian.sigs/${VERSION}[-win|-osx] to gitian.sigs/${VERSION}-<linux|win|osx>?

Okay, the non-descriptor changes from this PR are now in #4449.