ci: add Guix builds to CI #36082

Adds Guix builds to CI as a standalone workflow for:

• arm64-apple-darwin
• x86_64-linux-gnu
• x86_64-w64-mingw32

The first commit exports the pinned Guix repository URL and time-machine commit from the common prelude so other scripts can reuse them. This workflow uses the commit in its cache key; automated Guix substitute builders can also use it.

The second commit adds the workflow and Guix installation script, with caches for the Guix bootstrap, time-machine checkouts, depends sources and builds, and macOS SDK.

The final commit adds a codesign environment test to the Windows and macOS jobs. It builds the environment and runs true inside its container, to try and catch manifest errors and broken dependencies without requiring signatures or signing keys.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/36082.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline and AI policy for information on the review process.

If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #36233 (guix: Update time-machine to 60f6956aeffa7f30285745bd0ea615e9acfc74f8 by hebasto)
• #35855 (guix: update time-machine by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

#36006 got auto-closed by a linked PR.

More info on the infra side of things:

I have configured two physically separate bitcoin core substitute servers for better availability (using this module. One server evaluates the guix closure and builds it. Then uses guix copy to copy store items to the second server via ssh.

We could include the default substitute servers (https://bordeaux.guix.gnu.org https://ci.guix.gnu.org), but this makes the CI runs noticeably slower as these (slow) servers are queried even when not used/selected. These servers obviously will never include our custom compilers, which are the slowest thing to build, too.

I tested a run with a single substitute server, shutting the server off part way through, and Guix didn't like that it thought a substitute should be available but then wasn't and so failed. I think our use of --fallback protects us in the case that no substitute servers are available before the run begins; everything is just rebuilt, but have not tested this yet.

<!--85328a0da195eb286784d51f73fa0af9-->

🚧 At least one of the CI tasks failed. _{Task macOS native, fuzz: https://github.com/bitcoin/bitcoin/actions/runs/32941900565/job/98094394757} _{LLM reason (✨ experimental): CI failed because the rpc fuzz target returned exit code 1 due to an “Error processing input” during fuzzing.}

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

@hebasto I think that efbb91102107dd9e758c46b9cd51d7a1de28c38a (the first commit here) and #35929 are duplicating solutions in different ways.

I believe that #35929 is the neater solution, and still covers the multi-toolchain (dual Guix build) scenario via embedding the toolchain info in the build ID as part of:

BASE_CACHE/<host>/<package>/<package>-<version>-<build_id>.tar.gz

Do you agree? If so I think I'll drop efbb91102107dd9e758c46b9cd51d7a1de28c38a from here, and rebase on/review #35929.

@hebasto I think that efbb911 (the first commit here) and #35929 are duplicating solutions in different ways.

I've reviewed and tested efbb91102107dd9e758c46b9cd51d7a1de28c38a. Compared to #35929, it manages to fix the first subissue: "1. All depends are rebuilt when switching to another branch with identical Guix scripts and the depends subdirectory". Unfortunately, it fails to resolve the second one: "2. All non-GUI packages are built twice per host (mostly relevant for non-Linux hosts)".

Another point worth mentioning: for your commit to work as expected, the BASE_CACHE environment variable must be set explicitly.

I believe that #35929 is the neater solution, and still covers the multi-toolchain (dual Guix build) scenario via embedding the toolchain info in the build ID as part of:

BASE_CACHE/<host>/<package>/<package>-<version>-<build_id>.tar.gz

Do you agree? If so I think I'll drop efbb911 from here, and rebase on/review #35929.

That sounds good to me.

@willcl-ark

Please also consider #35929 (comment).

Thanks hebasto, that sounds reasonable to me

Concept ACK

Concept ACK

Added a commit to run true inside the guix-codesign container to try and catch things like #36256 (review)

I don't think there's any more we can do when we don't have signatures.

Marking this as ready for review now as I'd be interested in comments on its current state.

For me the main open question to be answered is:

• Are we happy to rely on the two (physically distinct) Guix substitute servers I'm operating as the only enabled ones in these jobs?

If we add in other upstream servers (the defaults), the job runtime increases a fair amount (about 5 minutes longer to "check substitutes" in all jobs, not to mention that they'll be missing our compiler toolchains as subs so we need to rebuild those anyway which takes ages) but the jobs would become slightly more robust against my two servers being down, or in the case I disappear or whatever.

Another alternative is to have someone else operate another server. This would be my personal preference, if it were possible.