build: simplify tar commands in gitian descriptors #16211

issue fanquake opened this issue on June 14, 2019
  1. fanquake commented at 5:58 AM on June 14, 2019: member

    cfields mentioned that it's likely possible to simplify our use of find, sort, tar and gzip inside our gitian descriptors by taking use of modern command line flags.

    We currently chain multiple commands together like so:

    https://github.com/bitcoin/bitcoin/blob/7524376a81a1955f4da4adb23005f2122b18a270/contrib/gitian-descriptors/gitian-linux.yml#L152

    However it should be possible to drop sort by taking advantage of tar's --sort=name command line flag:

    --sort=ORDER Using --sort=name ensures the member ordering in the created archive is uniform and reproducible.

    We should also be able to drop gzip and instead pass --gzip straight to tar:

    -z, --gzip, --gunzip, --ungzip Filter the archive through gzip(1).

    or, we may need to use -I so we can pass gzip -9n:

    -I, --use-compress-program=COMMAND Filter data through COMMAND. It must accept the -d option, for decompression. The argument can contain command line options.

  2. fanquake added the label Build system on Jun 14, 2019
  3. fanquake added the label good first issue on Jun 14, 2019
  4. MarcoFalke commented at 5:36 PM on June 14, 2019: member

    Is it worth it to make a single line command a bit shorter? Looks like this is working just fine, why change it?

  5. dongcarl commented at 6:39 PM on June 14, 2019: member

    Just for documentation, I've tried those methods and they seem to be less flexible than UNIX compose-ability with pipes. For example, the --sort=name for tar only applies to directory entries, and was a source of non-determinism in my Guix branch. Also, for --use-compress-program=, it's going to make it complicated if we need to pass options to gzip in the future that include IFS and needs escaping or something like that.

    But overall, it's a good thought, and we should look for other places to optimize build time in our gitian descriptors.

  6. hebasto commented at 7:17 AM on June 16, 2019: member

    The Linux piping has clear syntax and robust behavior. It uses the UNIX principle "Do One Thing and Do It Well".

  7. MarcoFalke commented at 1:42 PM on June 16, 2019: member

    Ok, closing for now, unless there is evidence that this brings any advantage.

  8. MarcoFalke closed this on Jun 16, 2019

  9. DrahtBot locked this on Dec 16, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-14 18:14 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me