Guix Release Transition Plan for 22.0 #21145

issue dongcarl openend this issue on February 10, 2021

dongcarl commented at 10:51 pm on February 10, 2021: member
Timeline

The v22.0 Feature Freeze is on June 15th, which is 17 weeks from Feb 11th. With that in mind, what follows is a tentative timeline for us to switch from Gitian release builds to Guix ones for v22.0, subject to change.

Building + User Testing (~9 wks ending April 15th)

Outstanding PRs/Issues
- #20629 | depends: Improve id string robustness
- #21087 | guix: Passthrough BASE_CACHE into container
- #21088 | guix: Jump forwards in time-machine and adapt
- #21089 | guix: Add support for powerpc64{,le}
- #21203 | guix: gnutls-3.6.12 build failures when using --bootstrap --no-substitutes
- #21239 | guix: Add codesigning support for osx+win
- #21271 | guix: Explicitly set umask in build container
- #21272 | guix: Passthrough SDK_PATH into container
- #21273 | guix: Brainstorm w/re guix.sigs repo layout
- #21375 | guix: Misc feedback-based fixes + hier restructuring
- #21304 | guix: Add guix-clean script + establish gc-root for container profiles
- #21462 | guix: Add guix-{attest,verify} scripts
TODOs
- Workflow-related
  
  Setting up a bitcoin-core/guix.sigs GitHub repository
- Documentation-related
  
  guix: Add installation instructions for Guix specific to Bitcoin Core builds The GNU manual is too generic at times and somewhat difficult to navigate
  
  guix: Update all release documentation for new Guix process release-process.md et al.
Non-codesigned dress rehearsal (~2 wks ending April 29nd)

At this point, users should be able to follow release-process.md in order to perform Guix builds and attest to their outputs in guix.sigs.

In order to further improve the usability of this process, I will work with a few brave volunteers to walk through the core parts of release-process.md, namely:
1. Installing Guix on their build machine
2. Setting up a directory structure for Guix builds
3. Performing a non-codesigned Guix build
4. Uploading their SHA256SUMs and signatures to guix.sigs
What to expect should you be brave enough to volunteer:
1. I will open a direct line of communication where you can reach me for help
2. I will document hiccups and open PRs to address common problems
3. Depending on the performance characteristics of your build machine, whether or not you are using substitutes, and whether you request that Guix bootstraps, these builds can take up to a day or two. I know it’s fun to watch build logs scroll by 😄 , but we will most likely start the build on day n and check back on it on day n+1
I would especially like to help volunteers who are not running Debian/Ubuntu, as diverse testing across distros/kernels is always a plus!

Please reach out to me if you are willing to volunteer for this.

Release dress rehearsal + refinement (~2 wks ending May 13th)

After we have performed user testing and ironed out usability kinks, I would like to do a dress rehearsal for the full release process.

Here’s how this differs from just “user testing”:
1. The commit we build will be tagged
2. Codesignatures will be made and uploaded to bitcoin-detached-sigs
3. SHA256SUMs and signatures of codesigned binaries will be uploaded to guix.sigs
This is to make sure that when we do the actual release for v22.0, there are no surprises.

Contingency buffer (~4+ wks ending June 15th)

If we run into any surprises (and I’m sure we will) this contingency buffer will give us some breathing room before the feature freeze.

What I need help with

High-priority
- Review of outstanding PRs
- Volunteers for user testing (Available sometime between March 19th and April 22nd)
- Feedback on this timeline and remaining tasks outlined: am I missing anything? too optimistic?
Normal Priority
- Setting up a bitcoin-core/guix.sigs GitHub repository
- Verifying that Guix debug symbol output works
Nice-to-have’s
- Packaging Guix v1.2.0 for your favourite distribution (please reach out if you do attempt this, I may be able to offer help!)
- #21194 | guix: Make it arch agnostic
dongcarl added this to the "Issues" column in a project
laanwj added this to the milestone 22.0 on Feb 11, 2021
fanquake commented at 12:02 pm on February 17, 2021: member

Looks pretty good. If anyone wants an intro to Guix I have a few resources here as well: https://github.com/fanquake/core-review/tree/master/guix.
hebasto commented at 8:20 am on March 2, 2021: member

@dongcarl

#21272 | guix: Passthrough SDK_PATH into container

Could be checked now :)
laanwj commented at 8:30 am on April 6, 2021: member

Setting up a bitcoin-core/guix.sigs GitHub repository

This was done: https://github.com/bitcoin-core/guix.sigs
MarcoFalke commented at 7:56 am on April 24, 2021: member

The gitian binaries are currently compiled with gcc-8, however the guix ones with gcc-9, which has the memcmp bug (#20005)? Given that gcc-11 has been released (presumably with a fix for that), would it make sense to use gcc-11 over gcc-9 for guix?

For reference, bumping to gcc-9 happened in #21298 IIRC
dongcarl commented at 7:55 pm on April 24, 2021: member
Hmmm, good point! Looking at the GCC repo, it seems this commit is the fix: https://github.com/gcc-mirror/gcc/commit/e4c9aac98611f63847ef6c57916808d9a2d7abcb
0$ git tag --contains e4c9aac98611f63847ef6c57916808d9a2d7abcb 1releases/gcc-10.3.0
So we can also use gcc 10.3 if that’s alright with folks. Guix already has that packaged unlike gcc-11. Going back to gcc-8 should also be possible for Gitian parity.
dongcarl commented at 7:30 pm on April 28, 2021: member

re: #21145 (comment)

I think downgrading to gcc-8 across-the-board might be the best choice for now. Opened: https://github.com/bitcoin/bitcoin/pull/21799
Sjors commented at 9:20 am on May 14, 2021: member

We’ll also need instructions similar to https://github.com/bitcoin-core/docs/blob/master/gitian-building.md

The PR description of #21239#issue-576807425 is a start. It helped me make https://github.com/bitcoin-core/guix.sigs/pull/10. Though I was missing instructions of what needs to be copied over if the build is on a different machine than the signing. The directory guix-build-$VERSION/output seems to be enough.

Alternatively it might make more sense to use NO_SIGN=1 and then copy the right files from the gitian repo and then sign somehow.
dongcarl commented at 8:29 pm on May 14, 2021: member

@Sjors This one desperately requires updating, but most of it should be here: #21711
MarcoFalke commented at 8:48 am on May 25, 2021: member

I’ve left a good first issue in https://github.com/bitcoin-core/guix.sigs/issues/8
MarcoFalke commented at 7:22 am on June 17, 2021: member

Anything left to do here? Can this be closed?
laanwj commented at 12:12 pm on June 17, 2021: member

I think it can be closed now that #22182 is merged. There’s #21711 still open but it isn’t mentioned here and already has the 22.0 milestone.
dongcarl commented at 8:44 pm on June 17, 2021: member

Closing this issue now that all feature-related PRs have been merged. Thanks everyone for reviews, testing, and patience!
dongcarl closed this on Jun 17, 2021
dongcarl moved this from the "Issues" to the "Done" column in a project
DrahtBot locked this on Aug 18, 2022