guix: Add full installation and usage documentation #21711

dongcarl commented at 9:03 PM on April 16, 2021: member

Based on: #21462

Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier.

Current TODOs:

Shell installer option: prompt user to re-login for /etc/profile.d entry to be picked up
Binary tarball option: prompt user to create /etc/profile.d entry and re-login
Fanquake docker option: complete section
Arch Linux AUR option: prompt to start guix-daemon-latest unit after finishing "optional setup" section
Building from source option: Insert dependency tree diagram that I made
Building from source option: redo sectioning, kind of a mess right now
Optional setup: make clear which parts are only needed if building from source
Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key
Overall (after everything): Make the links work.

Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option.

78051301012 commented at 9:27 PM on April 16, 2021: none

.

DrahtBot added the label Scripts and tools on Apr 16, 2021

fanquake deleted a comment on Apr 17, 2021

in contrib/guix/README.md:504 in 57684eda89 outdated

 501 | +
 502 | +- After a `guix install`, it updates
 503 | +  `/var/guix/profiles/per-user/$USER/guix-profile`, and creates a symlink
 504 | +  targeting this directory at `$HOME/.guix-profile`
 505 | +
 506 | +Therefore, in order for these operations to effect your shell/desktop sessions

sipa commented at 4:06 AM on April 17, 2021:

Nit: s/effect/affect/

in contrib/guix/README.md:714 in 57684eda89 outdated

 730 | +key](#authorize-the-signing-keys).
 731 | +
 732 | +Change the default list of substitute servers by starting `guix-daemon` with the
 733 | +`--substitute-urls` option:
 734 | +```
 735 | +guix <cmd> --substitute-urls="https://guix.carldong.io https://ci.guix.gnu.org"

sipa commented at 4:30 AM on April 17, 2021:

I think you mean guix-daemon here.

in contrib/guix/README.md:102 in 57684eda89 outdated

  99 | +
 100 | +Follow the binary installation instructions step-by-step in the Guix Reference
 101 | +Manual
 102 | +[here](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html).
 103 | +Ignore the note about using the shell installer script since that does not
 104 | +apply. This is largely equivalent to maually performing what the shell installer

TheCharlatan commented at 7:41 PM on April 17, 2021:

Nit: s/maually/manually/

in contrib/guix/README.md:577 in 57684eda89 outdated

 574 | +sudo --login guix pull --branch=version-<latest-release-version>
 575 | +# or
 576 | +sudo --login guix pull --commit=<particular-commit>
 577 | +```
 578 | +
 579 | +`guix pull` is quite a long process (espcially if you're using

TheCharlatan commented at 7:47 PM on April 17, 2021:

Nit: s/espcially/especially/

in contrib/guix/README.md:729 in 57684eda89 outdated

 745 | +variable:
 746 | +```
 747 | +export SUBSTITUTE_URLS='https://guix.carldong.io https://ci.guix.gnu.org'
 748 | +```
 749 | +
 750 | +## Option 2: Disabling subsitutes on an ad-hoc basis

TheCharlatan commented at 7:49 PM on April 17, 2021:

Nit: s/subsitutes/substitutes/

fanquake added this to the "Next (Not based on any other PRs)" column in a project

fanquake added this to the milestone 22.0 on Jun 16, 2021

fanquake commented at 5:11 AM on June 17, 2021: member

@dongcarl can you rebase this and address the current batch of comments?

dongcarl force-pushed on Jun 17, 2021

dongcarl commented at 10:07 PM on June 17, 2021: member

Pushed 57684eda89...5ae2760f61

Rebased over master
Addressed #21711#pullrequestreview-638195828, #21711#pullrequestreview-638270471

DrahtBot commented at 2:21 AM on June 18, 2021: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#22418 by fanquake
#21859 by sipa
#21851 by fanquake

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

MarcoFalke commented at 6:22 AM on June 18, 2021: member

Shouldn't this also update release-process.md?

in contrib/guix/README.md:24 in 5ae2760f61 outdated

  21 |    the `HOSTS` environment variable description)
  22 |  
  23 | -## Setup
  24 | +# Installation and Setup
  25 | +
  26 | +At the time of writing, the latest Guix release is v1.2.0, which has a known

achow101 commented at 8:00 PM on June 18, 2021:

In 5ae2760f614c0872a2289574477543070243e236 "guix: Overhaul README"

Latest release is 1.3.0 which fixes the GnuTLS issue, so I think this paragraph is no longer needed.

achow101 commented at 8:07 PM on June 18, 2021: member

Concept ACK

More docs more good

in contrib/guix/README.md:57 in 5ae2760f61 outdated

  54 | +   - Installs any release
  55 | +   - Binary installation only, requires high level of trust
  56 | +4. Using a **distribution-maintained package** [⤓ skip to section]()
  57 | +   - Maintained by distribution's Guix package maintainer
  58 | +   - Normal difficulty (manual setup required)
  59 | +   - Works only on distributions with Guix packaged, at time of writing:

jarolrod commented at 1:00 AM on July 2, 2021:

I don't think we should list the systems that have a guix package available. This will quickly diverge from reality. For example, a quick search shows openSuse should be added to the list: https://software.opensuse.org/package/guix

dongcarl commented at 11:47 PM on July 2, 2021:

I do think that we can give facts "at time of writing," sometimes it's better than nothing. However, in this particular case, perhaps we can simply link to https://repology.org/project/guix/versions ?

in contrib/guix/README.md:100 in 5ae2760f61 outdated

  97 | +
  98 | +## Option 2: Using the official binary tarball
  99 | +
 100 | +Follow the binary installation instructions step-by-step in the Guix Reference
 101 | +Manual
 102 | +[here](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html).

jarolrod commented at 1:09 AM on July 2, 2021:

This link contains the instructions that are found in section Option 1: Using the official shell installer script.

Seems as though this section and the first (Option 1 & 2) can be condensed in some way.

For Option 1, i don't like that we are re-writing the instructions when we could just point to this section of the mentioned link: guix-install-script

dongcarl commented at 11:51 PM on July 2, 2021:

You're right, I'm not too happy with the current situation either... Will fix! Clearer = better

jarolrod commented at 1:10 AM on July 2, 2021: member

concept ACK

Couple of notes. I assume you are already working on an update for 1.3.0 so I will re-review when it's updated

dongcarl commented at 6:52 PM on July 6, 2021: member

The documentation is very close to being done, just some final polishing and making the links work. Please let me know if I'm missing any crucial details!

in contrib/guix/guix-build:21 in d66c110f94 outdated

  17 | @@ -18,7 +18,7 @@ source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
  18 |  # Required non-builtin commands should be invocable
  19 |  ################
  20 |  
  21 | -check_tools cat mkdir make git guix
  22 | +check_tools cat mkdir make getent git guix

jarolrod commented at 11:10 PM on July 6, 2021:

commit 1e3f8d6e23bf1deb27af762e73e9666f8785e62d can really be broken off into its own and focused PR

fanquake commented at 6:58 AM on July 8, 2021:

I think doing this in this PR is fine. However this also needs to be updated to add curl, otherwise on a bare system we'll fail when we try and invoke make -C depends below.

dongcarl commented at 4:49 PM on July 12, 2021:

Fixed in fc4f8449f34e32b0b9ac9d218d6c3264b02467ba

in contrib/guix/INSTALL.md:35 in d66c110f94 outdated

  30 | +   - Works only on distributions with Guix packaged, at time of writing:
  31 | +     - Debian Bullseye or later
  32 | +     - Ubuntu Hirsute or later
  33 | +     - Arch Linux (via AUR)
  34 | +   - Installs a release decided on by package maintainer
  35 | +   - Source or binary installation depending on distribution

jarolrod commented at 11:34 PM on July 6, 2021:

nit 💯

   - Source or binary installation depending on the distribution

in contrib/guix/INSTALL.md:45 in d66c110f94 outdated

  40 | +   - Installs any commit (more granular)
  41 | +   - Source installation, requires lower level of trust
  42 | +
  43 | +## Options 1 and 2: Using the official shell installer script or binary tarball
  44 | +
  45 | +The instructions for both using the offical shell installer script and using

jarolrod commented at 12:33 AM on July 7, 2021:

The installation instructions for both the official shell installer script and the

in contrib/guix/INSTALL.md:46 in d66c110f94 outdated

  41 | +   - Source installation, requires lower level of trust
  42 | +
  43 | +## Options 1 and 2: Using the official shell installer script or binary tarball
  44 | +
  45 | +The instructions for both using the offical shell installer script and using
  46 | +binary tarballs are both found in the GNU Guix Manual's [Binary Installation

jarolrod commented at 12:34 AM on July 7, 2021:

binary tarballs can be found in the GNU Guix Manual's [Binary Installation

in contrib/guix/INSTALL.md:72 in d66c110f94 outdated

  67 | +Note that the `Dockerfile` is largely equivalent to running through the binary
  68 | +tarball installation steps.
  69 | +
  70 | +## Option 4: Using a distribution-maintained package
  71 | +
  72 | +### Debian 11 ("Bullseye")/Ubuntu 21.04 ("Hirsute")

jarolrod commented at 12:35 AM on July 7, 2021:

nit

### Debian 11 (Bullseye)/Ubuntu 21.04 (Hirsute)

in contrib/guix/INSTALL.md:96 in d66c110f94 outdated

  87 | +
  88 | +For up-to-date information on Debian and Ubuntu's release history:
  89 | +- [Debian release history](https://www.debian.org/releases/)
  90 | +- [Ubuntu release history](https://ubuntu.com/about/release-cycle)
  91 | +
  92 | +### Arch Linux

jarolrod commented at 12:39 AM on July 7, 2021:

no opinion, but we don't provide Arch Linux specific instructions anywhere else

dongcarl commented at 4:50 PM on July 12, 2021:

I think we do here: https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md#setup-and-build-example-arch-linux

in contrib/guix/INSTALL.md:75 in d66c110f94 outdated

  70 | +## Option 4: Using a distribution-maintained package
  71 | +
  72 | +### Debian 11 ("Bullseye")/Ubuntu 21.04 ("Hirsute")
  73 | +
  74 | +Guix v1.2.0 is available as a distribution package starting in [Debian
  75 | +11](https://packages.debian.org/bullseye/guix) ("bullseye") and [Ubuntu

jarolrod commented at 12:41 AM on July 7, 2021:

If the nickname is already mentioned in the header along with the os version, we don't need to reiterate it here

11](https://packages.debian.org/bullseye/guix) and [Ubuntu

in contrib/guix/INSTALL.md:76 in d66c110f94 outdated

  71 | +
  72 | +### Debian 11 ("Bullseye")/Ubuntu 21.04 ("Hirsute")
  73 | +
  74 | +Guix v1.2.0 is available as a distribution package starting in [Debian
  75 | +11](https://packages.debian.org/bullseye/guix) ("bullseye") and [Ubuntu
  76 | +21.04](https://packages.ubuntu.com/hirsute/guix) (Hirsute Hippo).

jarolrod commented at 12:42 AM on July 7, 2021:

and here

21.04](https://packages.ubuntu.com/hirsute/guix).

dongcarl force-pushed on Jul 7, 2021

dongcarl marked this as ready for review on Jul 7, 2021

dongcarl commented at 8:37 PM on July 7, 2021: member

This PR is ready for review as of 5016091dd8b827c74fe28212be43d3b22014c5b6

Thanks @jarolrod for the diligent reviews, I will address the nits after others agree that I haven't missed anything crucial!

in doc/release-process.md:64 in 5016091dd8 outdated

  59 | @@ -64,14 +60,14 @@ This will perform a few last-minute consistency checks in the build system files
  60 |  
  61 |  ### First time / New builders
  62 |  
  63 | -If you're using the automated script (found in [contrib/gitian-build.py](/contrib/gitian-build.py)), then at this point you should run it with the "--setup" command. Otherwise ignore this.
  64 | +Install Guix using one of the installation methods detailed in
  65 | +[contrib/guix/README.md](contrib/guix/README.md).

achow101 commented at 10:17 PM on July 7, 2021:

In 5016091dd8b827c74fe28212be43d3b22014c5b6 "guix: Adapt release-process.md to new Guix process"

I think this should refer to contrib/guix/INSTALL.md

dongcarl commented at 4:11 PM on July 8, 2021:

Fixed, thanks!

achow101 commented at 10:22 PM on July 7, 2021: member

ACK 5016091dd8b827c74fe28212be43d3b22014c5b6

fanquake commented at 8:27 AM on July 8, 2021: member

Concept ACK

Looks pretty good. Maybe a little too much detail in some cases, i.e some things feel a bit repetitive, but I think that's ok for now.

In release-process.md, there are still two mentions of gitian in the Update packaging repo section.

In guix-attest, can you address this comment from #22182.

I am running through doing a build using a Ubuntu:Hirsute container, using the following:

docker run -it --rm --privileged ubuntu:hirsute /bin/bash
apt update -y && apt upgrade -y && apt install curl git guix make -y

addgroup guixbuild
useradd -g guixbuild -G guixbuild -d /var/empty -s $(which nologin) -c "Guix build user" --system "guixbuilder"
guix-daemon --no-substitutes --build-users-group=guixbuild &

git clone https://github.com/bitcoin/bitcoin
cd bitcoin
HOSTS="x86_64-linux-gnu" ./contrib/guix/guix-build

I've also updated my Alpine based dockerfile to use Alpine 3.14, and am currently running through a from-scratch build using that. Using a build invocation like: time BASE_CACHE="/guix/base_cache" SOURCES_PATH="/guix/sources" SDK_PATH="/guix/SDKs" ./contrib/guix/guix-build.

fanquake commented at 12:57 PM on July 8, 2021: member

Finished the from-scratch-build in the Alpine container:

real	210m14.468s
user	700m13.502s
sys	51m25.176s
bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
ceaf2111053d48d1024e9c0c7336fc9ee6b081172eca35259b2ee9c3de2b5eec  guix-build-4129134e844f/output/aarch64-linux-gnu/SHA256SUMS.part
d564c8ce3c2132bb1304e75a49b1e9860189b98745ed35e540d7c6ef9f0e5323  guix-build-4129134e844f/output/aarch64-linux-gnu/bitcoin-4129134e844f-aarch64-linux-gnu-debug.tar.gz
8c2cc0b1046d6518b7445a19b775735b688db32b1eefc8bbcd2d3aded3937fd7  guix-build-4129134e844f/output/aarch64-linux-gnu/bitcoin-4129134e844f-aarch64-linux-gnu.tar.gz
3084de9527e95ef9bc074820e190cfacdd1962fa04cef1a79f8ccd42bab0dc6b  guix-build-4129134e844f/output/arm-linux-gnueabihf/SHA256SUMS.part
9b2d147ec30ecd1573d6cd82343cccbd35375eb5ffa269edf34b4404fca8744c  guix-build-4129134e844f/output/arm-linux-gnueabihf/bitcoin-4129134e844f-arm-linux-gnueabihf-debug.tar.gz
bd8da7121fdf38b2da885a53b10103b3a53dfef2d68e38e5ca5dde8db61af641  guix-build-4129134e844f/output/arm-linux-gnueabihf/bitcoin-4129134e844f-arm-linux-gnueabihf.tar.gz
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  guix-build-4129134e844f/output/dist-archive/SKIPATTEST.TAG
5b1ee87a8833130eee8f7c32b0537618b90ac3db815298cc734660bd1821bcd8  guix-build-4129134e844f/output/dist-archive/bitcoin-4129134e844f.tar.gz
78d3c3a94773e3d461ee8797a6d6043c5e48c607ed8cfe5ccaf7aa7f751cd553  guix-build-4129134e844f/output/powerpc64-linux-gnu/SHA256SUMS.part
aad00c9521d461871c33d9f68a41d2a8ba9ec8e962edb93bb846d77c05b3eda1  guix-build-4129134e844f/output/powerpc64-linux-gnu/bitcoin-4129134e844f-powerpc64-linux-gnu-debug.tar.gz
1a326af20ced546fe2d03bd0d5d3b446c81c8c8dfcbde49852d39764748a0524  guix-build-4129134e844f/output/powerpc64-linux-gnu/bitcoin-4129134e844f-powerpc64-linux-gnu.tar.gz
06e409cfefc4add776479b0b70482a0ec684e991685f715a732f85516da40981  guix-build-4129134e844f/output/powerpc64le-linux-gnu/SHA256SUMS.part
237b9ee0ddb60e47b86e505011a9c17b5e898e1c5bfba59abd0e863cfcf03f5b  guix-build-4129134e844f/output/powerpc64le-linux-gnu/bitcoin-4129134e844f-powerpc64le-linux-gnu-debug.tar.gz
98de1dec19a8b6f98f045aa93bc29370b3acd5148a0acd96e9c091e6bdda1e9c  guix-build-4129134e844f/output/powerpc64le-linux-gnu/bitcoin-4129134e844f-powerpc64le-linux-gnu.tar.gz
4855cf747c23dd1d60f6b8af6933ebe464820bbc840e364d4ede5d94677cd3fe  guix-build-4129134e844f/output/riscv64-linux-gnu/SHA256SUMS.part
6e0e781bbd611ad72c085d4e00686b48dff1baf29637c481b11534f9b9bf56f2  guix-build-4129134e844f/output/riscv64-linux-gnu/bitcoin-4129134e844f-riscv64-linux-gnu-debug.tar.gz
96cff3df87b6f62542293a63d169aa29a88dc795d499168273a86c81560ccecb  guix-build-4129134e844f/output/riscv64-linux-gnu/bitcoin-4129134e844f-riscv64-linux-gnu.tar.gz
b40a1645c1601d6afbac60192789ec054ce750dc6eeb5a3f25612f4231c3d7e5  guix-build-4129134e844f/output/x86_64-apple-darwin18/SHA256SUMS.part
4dd21a03916bd42e0369e59aa0f8fb88921ed7b86fc5a1e6702389bda98a8def  guix-build-4129134e844f/output/x86_64-apple-darwin18/bitcoin-4129134e844f-osx-unsigned.dmg
ff3602163bce9934457c031831828dbd016a97d596d5aa76c42b21d9d479c2c2  guix-build-4129134e844f/output/x86_64-apple-darwin18/bitcoin-4129134e844f-osx-unsigned.tar.gz
50b1b0f6f8b8eb16d8c381fb1b66e5fc844d24f2625269f7bb28c80b20eaf125  guix-build-4129134e844f/output/x86_64-apple-darwin18/bitcoin-4129134e844f-osx64.tar.gz
af318a441f563abf1b9e1b3321b13e3a14a857216238133f9f3d923e34fc081e  guix-build-4129134e844f/output/x86_64-linux-gnu/SHA256SUMS.part
3912bb99acf83778aae12d522a0d8c567371754cb31f2f13712284fc5954a443  guix-build-4129134e844f/output/x86_64-linux-gnu/bitcoin-4129134e844f-x86_64-linux-gnu-debug.tar.gz
eb0edfa33b57ecb723b80dfc6a994af7a9cbdde72e7cf05026e98085a7dcb018  guix-build-4129134e844f/output/x86_64-linux-gnu/bitcoin-4129134e844f-x86_64-linux-gnu.tar.gz
c5cf1c6985c1c430df4839db7310ac0a46fd7da79e7de1e5cfc761523900d190  guix-build-4129134e844f/output/x86_64-w64-mingw32/SHA256SUMS.part
59721ac4abcebb51b7c011785e0701e13cb90926437ef60014078889dbec5e22  guix-build-4129134e844f/output/x86_64-w64-mingw32/bitcoin-4129134e844f-win-unsigned.tar.gz
d6f919dd523c407deaf38fc842e7a84240b64bc5c759796965b5b20618e15e2a  guix-build-4129134e844f/output/x86_64-w64-mingw32/bitcoin-4129134e844f-win64-debug.zip
96dc1c8f917b086c2a7eb84a780d4ee718eb1a608c17dfe8c29c07825f057ab6  guix-build-4129134e844f/output/x86_64-w64-mingw32/bitcoin-4129134e844f-win64-setup-unsigned.exe
e0e5925b0a898b9cc963f7a69bf9b4ada766332fffd939db142f357ee294b514  guix-build-4129134e844f/output/x86_64-w64-mingw32/bitcoin-4129134e844f-win64.zip

Then ran a bash-5.1# ./contrib/guix/guix-clean and built again, getting the same results in ~80 minutes iirc.

Unfortunately the --no-substitutes Hirsute build has bombed out, one of the unit tests run at the end of compiling coreutils 8.32 seems to be failing. Something to follow up on later:

================================================
   GNU coreutils 8.32: ./tests/test-suite.log
================================================

# TOTAL: 621
# PASS:  477
# SKIP:  143
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2
...
FAIL: tests/tail-2/inotify-dir-recreate
...
 inotify
-tail: 'dir/file' has become inaccessible: No such file or directory
-tail: directory containing watched file was removed
-tail: inotify cannot be used, reverting to polling
-tail: 'dir/file' has appeared;  following new file
+ fail=1
+ Exit 1

Full log.

guix: Check for a sane services database

On bare systems, it is possible to be lacking a services database. Check
for basic entries before attempting a build.

See the error message in the diff for more context.

263220a85c

guix: Update various check_tools lists fc4f8449f3

tree-wide: Rename gitian-keys to builder-keys 46ce6ce378

dongcarl force-pushed on Jul 8, 2021

dongcarl commented at 4:15 PM on July 8, 2021: member

Pushed 739556bfd4...781812a9cd

Various fixups
Add coreutils/inotify-dir-recreate troubleshooting information

In release-process.md, there are still two mentions of gitian in the Update packaging repo section.

Fixed!

In guix-attest, can you address this comment from #22182.

Fixed!

I am running through doing a build using a Ubuntu:Hirsute container, using the following...

Thanks for testing!

I've also updated my Alpine based dockerfile to use Alpine 3.14, and am currently running through a from-scratch build using that. Using a build invocation like: time BASE_CACHE="/guix/base_cache" SOURCES_PATH="/guix/sources" SDK_PATH="/guix/SDKs" ./contrib/guix/guix-build.

Great to test with a musl-based system too!

Unfortunately the --no-substitutes Hirsute build has bombed out, one of the unit tests run at the end of compiling coreutils 8.32 seems to be failing. Something to follow up on later...

This one I've encountered before when testing Guix's v1.3.0 release. I've added some troubleshooting documentation in the last commit of the latest push: https://github.com/bitcoin/bitcoin/pull/21711/commits/781812a9cd2b85485e300db58a1ca577a0749849

in contrib/guix/INSTALL.md:447 in 781812a9cd outdated

 442 | +
 443 | +However, the Guix init scripts and service configurations for Upstart, systemd,
 444 | +SysV, and OpenRC are installed to launch
 445 | +`${localstatedir}/guix/profiles/per-user/root/current-guix/bin/guix-daemon`,
 446 | +which does not yet exist, and will only exist after `root` performs their first
 447 | +`guix pull`. TODO: Link to `guix pull` as root section

darosior commented at 10:49 PM on July 8, 2021:

nit: missed TODO? :)

dongcarl commented at 3:11 PM on July 12, 2021:

Fixed in 5e23466e04e8c46aba39a1e3bf297d1947455740

in contrib/guix/INSTALL.md:457 in 781812a9cd outdated

 451 | +`/usr/local/bin`).
 452 | +
 453 | +Example for `systemd`, run as `root`:
 454 | +
 455 | +```sh
 456 | +# Create guix-daemon-original.service by modifying guix-daemon.service

darosior commented at 10:55 PM on July 8, 2021:

But guix-daemon.service was never created in this section? Maybe prepand with:

# From the root of the source tree
cp etc/guix-daemon.service /etc/systemd/system/
chmod 644 /etc/systemd/system/guix-daemon.service

dongcarl commented at 3:11 PM on July 12, 2021:

Fixed in 5e23466e04e8c46aba39a1e3bf297d1947455740

in contrib/guix/INSTALL.md:585 in 781812a9cd outdated

 578 | +sudo --login guix pull --branch=version-<latest-release-version>
 579 | +# or
 580 | +sudo --login guix pull --commit=<particular-commit>
 581 | +```
 582 | +
 583 | +`guix pull` is quite a long process (especially if you're using

darosior commented at 9:03 AM on July 9, 2021:

12 hours and counting :p

dongcarl commented at 3:11 PM on July 12, 2021:

:crossed_fingers:

in contrib/guix/INSTALL.md:481 in 4a6d7454e8 outdated

 476 | +section](https://guix.gnu.org/manual/en/html_node/Build-Environment-Setup.html)
 477 | +in the Guix Reference Manual for more details.
 478 | +
 479 | +## Optional setup
 480 | +
 481 | +At this point, you are set up to [use Guix to build bitcoin](./README.md#usage).

MarcoFalke commented at 10:23 AM on July 9, 2021:

4a6d7454e8a2a666216ff0d6f7260922bb72296d:

"build Bitcoin Core"

in contrib/guix/INSTALL.md:167 in 4a6d7454e8 outdated

 162 | +Depending on your distribution, most or all of these dependencies may already be
 163 | +packaged and installable without manually building and installing.
 164 | +
 165 | +For reference, the graphic below outlines Guix v1.3.0's dependency graph:
 166 | +
 167 | +![boostrap map](guix-v1.3.0-bootstrap.png)

MarcoFalke commented at 10:26 AM on July 9, 2021:

Same commit: Is it really needed to add a 0.2 MB binary blob to the git tree? Is it expected that the blob is updated on every new guix version? Background is that .git has to store the file forever, even if it is removed in a later commit.

It could be an external link or at least optimized (still 0.1 MB, though)

![boostrap map](https://user-images.githubusercontent.com/6399679/125064185-a9a59880-e0b0-11eb-82c1-9b8e5dc9950d.png)

MarcoFalke commented at 10:41 AM on July 9, 2021:

(I used https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/optimize-pngs.py for this)

dongcarl commented at 8:12 PM on July 9, 2021:

I think this'll be much less relevant once guix is available as a package on most stable distros.

I don't mind how it's hosted so long as people can access it. Where would be a place that makes sense?

MarcoFalke commented at 4:38 PM on July 12, 2021:

You can use above link ![boostrap map](https://user-images.githubusercontent.com/6399679/125064185-a9a59880-e0b0-11eb-82c1-9b8e5dc9950d.png) if you don't want to upload it anywhere else.

dongcarl commented at 4:42 PM on July 12, 2021:

Done!

MarcoFalke approved

MarcoFalke commented at 10:40 AM on July 9, 2021: member

Approach ACK 9f04eaac6bbf1256a5c0b1c6e818f730d85e239c

darosior commented at 2:16 PM on July 9, 2021: member

A couple of comments on INSTALL.md first, as it looks like managing to guix pull succesfully is going to take me quite some time before i get to the remaining part..

Anyways as someone who didn't follow previous Guix work it's amazing documentation! Hope it'll help to bring more builders :)

dongcarl force-pushed on Jul 9, 2021

in doc/release-process.md:220 in 5e23466e04 outdated

 274 | +       The `*-debug*` files generated by the guix build contain debug symbols
 275 | +       for troubleshooting by developers. It is assumed that anyone that is
 276 | +       interested in debugging can run guix to generate the files for
 277 | +       themselves. To avoid end-user confusion about which file to pick, as well
 278 | +       as save storage space *do not upload these to the bitcoincore.org server,
 279 | +       nor put them in the torrent*.

laanwj commented at 8:36 AM on July 12, 2021:

Agree. I think the main reason to leave -debug out has been because the files are huge, and due to determinism can be generated on demand anyway by anyone with enough dev expertise to need them. But not confusing users with a lot of files makes sense too.

dongcarl commented at 3:13 PM on July 12, 2021:

Yup! To be clear here, it was you who originally wrote this comment 5 years ago :-)

in doc/release-process.md:206 in 5e23466e04 outdated

 246 | -bitcoin-${VERSION}-osx64.tar.gz
 247 | -bitcoin-${VERSION}-osx.dmg
 248 | -bitcoin-${VERSION}.tar.gz
 249 | -bitcoin-${VERSION}-win64-setup.exe
 250 | -bitcoin-${VERSION}-win64.zip
 251 | +$(cat all.SHA256SUMS)

laanwj commented at 8:42 AM on July 12, 2021:

Speaking of filenames, I have one manual rename left in my release script (which prepares the distribution directory for upload to bitcoincore[.]org) at the moment:

mv ${SOURCEDIR}/bitcoin-osx-signed.dmg ${DESTDIR}/bitcoin-${VERSIONTO}-osx.dmg

I guess this is no longer needed with the GUIX build, as the signed output will contain the version number?

dongcarl commented at 3:15 PM on July 12, 2021:

Yes I believe so!

in doc/release-process.md:224 in 5e23466e04 outdated

 286 | -people without access to `bitcoincore.org` to download the binary distribution.
 287 | -Also put it into the `optional_magnetlink:` slot in the YAML file for
 288 | -bitcoincore.org.
 289 | +    2. The combined clear-signed message you just created `SHA256SUMS.asc`
 290 | +
 291 | +- A `.torrent` will appear in the directory after a few minutes. Optionally help

laanwj commented at 8:49 AM on July 12, 2021:

bitcoincore.org does not have functionality to automatically generate torrents. This has been done manually (I use transmission-gtk) for a long time.

laanwj commented at 9:10 AM on July 12, 2021:

FWIW, I use the following trackers:

udp://tracker.openbittorrent.com:80
udp://tracker.opentrackr.org:1337/announce
udp://tracker.coppersurfer.tk:6969/announce

udp://tracker.leechers-paradise.org:6969/announce
udp://explodie.org:6969/announce
udp://tracker.torrent.eu.org:451/announce

udp://tracker.bitcoin.sprovoost.nl:6969

After adding @Sjors's tracker, you need to ping him with the magnet hash so he can whitelist it. (I don't think these details make sense to document in the release process, it seems to be kind of transient information suspect to change, but just FYI)

dongcarl commented at 3:19 PM on July 12, 2021:

Good to have these written down even if it's in a review comment :-) Do you think there's any torrent-related instruction that does belong in release-process.md?

laanwj commented at 1:17 PM on July 13, 2021:

Maybe that we create the torrent from the bitcoin-core-x.x.x directory (so the directory itself is in the torrent, with in it the archives for every platform as well as the SHA256SUMS.asc). When you do this in transmission-gtk it makes a torrent file called bitcoin-core-x.y.z.torrent which I've had to rename manually to bitcoin-x.y.z.torrent before uploading, as that has been the naming scheme e.g. https://bitcoincore.org/bin/bitcoin-core-0.21.1/bitcoin-0.21.1.torrent.

Somewhat amusing: A commonly asked question is why the SHA256SUMS.asc doesn't contain a hash of the torrent, but this is because that would create a chicken-egg problem :smile:

dongcarl commented at 12:40 AM on July 15, 2021:

Added some info in 181ba751f278f5aac94c5ba1b8beeb6b3ec4629e, let me know if I'm missing anything

laanwj commented at 1:12 PM on July 15, 2021:

Looks good to me now!

in contrib/guix/README.md:17 in 5e23466e04 outdated

  15 |  Conservatively, a x86_64 machine with:
  16 |  
  17 |  - 16GB of free disk space on the partition that /gnu/store will reside in
  18 | -- 8GB of free disk space per platform triple you're planning on building (see
  19 | -  the `HOSTS` environment variable description)
  20 | +- 8GB of free disk space **per platform triple** you're planning on building

laanwj commented at 9:34 AM on July 12, 2021:

Thanks for highlighting that this is per platform tuple :smile:

dongcarl force-pushed on Jul 12, 2021

in contrib/guix/INSTALL.md:45 in d766580dfc outdated

  40 | +   - Installs any commit (more granular)
  41 | +   - Source installation, requires lower level of trust
  42 | +
  43 | +## Options 1 and 2: Using the official shell installer script or binary tarball
  44 | +
  45 | +The installation instructions for both the offical shell installer script and

jarolrod commented at 10:09 PM on July 13, 2021:

@( * O * )@

The installation instructions for both the official shell installer script and

dongcarl commented at 12:41 AM on July 15, 2021:

😰 Fixed!

in contrib/guix/INSTALL.md:30 in d766580dfc outdated

  25 | +   - Installs any release
  26 | +   - Binary installation only, requires high level of trust
  27 | +4. Using a **distribution-maintained package** [⤓ skip to section][install-distro-pkg]
  28 | +   - Maintained by distribution's Guix package maintainer
  29 | +   - Normal difficulty (manual setup required)
  30 | +   - Works only on distributions with Guix packaged, at time of writing:

jarolrod commented at 10:15 PM on July 13, 2021:

We link to repology which provides the reader insight into what distributions package guix. I think doing this makes including a list of what distros include Guix at time of writing here. This list could be misleading as in it could make an openSUSE user believe that there is no distro package for them (if they don't look too much into it).

   - Works only on distributions with Guix packaged

dongcarl commented at 12:40 AM on July 15, 2021:

Right, fixed!

in contrib/guix/INSTALL.md:92 in d766580dfc outdated

  90 | +To install:
  91 | +```sh
  92 | +sudo apt install guix
  93 | +```
  94 | +
  95 | +For up-to-date information on Debian and Ubuntu's release history:

jarolrod commented at 10:17 PM on July 13, 2021:

I don't think that this section is necessary here

dongcarl commented at 12:42 AM on July 15, 2021:

Going to leave it there since it doesn't hurt and in case people don't know how Debian/Ubuntu versioning works ("I have bionic, but is that after hirsute or before?")

jarolrod commented at 4:08 AM on July 14, 2021: member

Approach ACK, running through the Build from source instructions now

dongcarl referenced this in commit 181ba751f2 on Jul 15, 2021

dongcarl force-pushed on Jul 15, 2021

laanwj commented at 1:12 PM on July 15, 2021: member

ACK 181ba751f278f5aac94c5ba1b8beeb6b3ec4629e

in contrib/guix/INSTALL.md:24 in bf244f908e outdated

  19 | +   - Installs any release
  20 | +   - Binary installation only, requires high level of trust
  21 | +3. Using fanquake's **Docker image** [↗︎ external instructions][install-fanquake-docker]
  22 | +   - Maintained by fanquake
  23 | +   - Easy (automatically performs *some* setup)
  24 | +   - Works on most Linux distributions

kallewoof commented at 5:12 AM on July 16, 2021:

Commit bf244f908ec92d223fc8272cda66a4dc13545d8c

Works on Mac and (I think) Windows too.

kallewoof commented at 5:21 AM on July 16, 2021: member

ACK 181ba751f278f5aac94c5ba1b8beeb6b3ec4629e

Will attempt to follow instructions on my Mac.

guix: Overhaul README

- Added detailed Guix bootstrap/installation instructions

c2541fd0ca

guix-attest: Produce and sign normalized documents

That way we can easily combine the document and detached signature to
produce cleartext signature files for upload during the release process.

See subsequent commits which modify doc/release-process.md for more
details.

fcab35b229

guix: Adapt release-process.md to new Guix process

Also, clean up release-process.md

318c60700b

guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting 5da2ee49d5

guix/INSTALL: Guix installs init scripts in libdir 5d24cc3d82

doc/release-process: Add torrent creation details

Source: https://github.com/bitcoin/bitcoin/pull/21711#discussion_r668754244

fac4814106

dongcarl force-pushed on Jul 19, 2021

dongcarl commented at 10:25 PM on July 19, 2021: member

Pushed 181ba751f2...fac4814106

Addressed #21711 (review)

fanquake approved

fanquake commented at 3:08 AM on July 20, 2021: member

ACK fac4814106c796b8786dd90053513cc35142dfe5 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release.

fanquake merged this on Jul 20, 2021

fanquake closed this on Jul 20, 2021

fanquake moved this from the "Next (Not based on any other PRs)" to the "Done" column in a project

in contrib/guix/INSTALL.md:283 in fac4814106

 278 | +<!-- v3.0. This can be checked by invoking the following: -->
 279 | +
 280 | +<!-- ``` -->
 281 | +<!-- apt update -->
 282 | +<!-- apt show guile-git -->
 283 | +<!-- ``` -->

MarcoFalke commented at 1:50 PM on July 20, 2021:

What is this?

in contrib/guix/INSTALL.md:127 in fac4814106

 122 | +those looking to minimize trust and maximize customizability (e.g. building a
 123 | +particular commit of Guix). Previous experience with using autotools-style build
 124 | +systems to build packages from source will be helpful. *hic sunt dracones.*
 125 | +
 126 | +I strongly urge you to at least skim through the entire section once before you
 127 | +start issuing commands, as it will save you a lot of unncessary pain and

hebasto commented at 4:03 PM on July 20, 2021:

typo: unncessary ==> unnecessary

in contrib/guix/INSTALL.md:168 in fac4814106

 163 | +Depending on your distribution, most or all of these dependencies may already be
 164 | +packaged and installable without manually building and installing.
 165 | +
 166 | +For reference, the graphic below outlines Guix v1.3.0's dependency graph:
 167 | +
 168 | +![boostrap map](https://user-images.githubusercontent.com/6399679/125064185-a9a59880-e0b0-11eb-82c1-9b8e5dc9950d.png)

hebasto commented at 4:04 PM on July 20, 2021:

typo: boostrap ==> bootstrap

josibake referenced this in commit 50ffba22c6 on Jul 21, 2021

hebasto referenced this in commit 7dc875fcc3 on Jul 22, 2021

sidhujag referenced this in commit 040b06cf6d on Jul 23, 2021

janus referenced this in commit 749626b577 on Nov 5, 2021

gwillen referenced this in commit 7a9b6ec4bf on Jun 1, 2022

DrahtBot locked this on Aug 16, 2022