Backport Cirrus runners to 28.x

willcl-ark commented at 1:18 pm on September 16, 2025: member

Backports #32989 to the 28.x branch

DrahtBot added the label Backport on Sep 16, 2025

DrahtBot commented at 1:18 pm on September 16, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33406.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	maflcko

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

fanquake added this to the milestone 28.3 on Sep 16, 2025

willcl-ark force-pushed on Sep 16, 2025

willcl-ark renamed this:
~~WIP: Backport Cirrus runners to 28.x~~
Backport Cirrus runners to 28.x
on Sep 16, 2025

willcl-ark marked this as ready for review on Sep 16, 2025

ci: add configure environment action

Github-Pull: #32989
Rebased-From: b8fcc9fcbcd

d8ad667f94

ci: add caching actions

Github-Pull: #32989
Rebased-From: b232b0fa5e9

Add "Restore" and "Save" caching actions.

These actions reduce boilerplate in the main ci.yml configuration file.

These actions are implemented so that caches will be saved on `push`
only.

When a pull request is opened it will cache hit on the caches from the
lastest push, or in the case of depends will hit on any matching depends
hash, falling back to partial matches.

Depends caches are hashed using
`$(git ls-tree HEAD depends "ci/test/$FILE_ENV" | sha256sum | cut -d' ' -f1)`
and this hash is passed in as an input to the actions. This means we
direct cache hit in cases where depends would not be re-built, otherwise
falling back to a partial match.

Previous releases cache is hashed similarly to depends, but using the
test/get_previous_releases.py file.

The cirruslabs cache action will fallback transparently to GitHub's
cache in the case that the job is not being run on a Cirrus Runner,
making these compatible with running on forks (on free GH hardware).

8d6d70f555

ci: add REPO_USE_CIRRUS_RUNNERS

Github-Pull: #32989
Rebased-From: 33ba073df7a

If set, Cirrus runners will be used on pushes to, and pull requests
against, this repository.

Forks can set this if they have their own cirrus runners.

270191c3c0

ci: add configure-docker action

Github-Pull: #32989
Rebased-From: fdf64e55324

Another action to reduce boilerplate in the main ci.yml file.

This action will set up a docker builder compatible with caching build
layers to a container registry using the `gha` build driver.

It will then configure the docker build cache args.

f8cb31d064

ci: use buildx in ci

Github-Pull: #32989
Rebased-From: 94a09325475

This rebase also includes part of e87429a2d0f23eb59526d335844fa5ff5b50b21f
adding `$DOCKER_BUILD_CACHE_ARG \` to the `docker build` command.

Using buildx is required to properly load the correct driver, for use
with registry caching. Neither build, nor BUILDKIT=1 currently do this
properly.

Use of `docker buildx build` is compatible with podman.

f6ccd895df

ci: have base install run in right dir

Github-Pull: #32989
Rebased-From: 9c2b96e0d03

This sets the build dir at build time so that Apple SDK gets installed
in the correct/expected location for the runtime to find it.

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

30efc95aec

ci: add Cirrus cache host

Github-Pull: #32989
Rebased-From: 020069e6b71

Whilst the action cirruslabs/actions/cache will automatically set this
host, the docker `gha` build cache backend will not be aware of it.

Set the value here, which will later be used in the docker build args to
enable docker build cache on the cirrus cache.

82cfddbc9a

ci: add job to determine runner type

Github-Pull: #32989
Rebased-From: cc1735d7771

To remove multiple occurances of the respository name, against which we
compare `${{ github.repository }}` to check if we should use Cirrus
Runners, introduce a helper job which can check a single environment
variable and output this as an input to subsequent jobs.

Forks can maintain a trivial patch of their repo name against the
`REPO_USE_CIRRUS_RUNNERS` variable in ci.yml if they have Cirrus Runners
of their own, which will then enable cache actions and docker build
cache to use Cirrus Cache.

It's not possible to use `${{ env.USE_CIRRUS_RUNNERS }}` in the
`runs-on:` directive as the context is not supported by GitHub.

If it was, this job would no longer be necessary.

5538ce4f32

ci: port arm job

Github-Pull: #32989
Rebased-From: f253031cb8e

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

bb455c9594

ci: update asan-lsan-ubsan

Github-Pull: #32989
Rebased-From: 884251441bb

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

61bb5180d5

ci: force reinstall of kernel headers in asan

Github-Pull: #32989
Rebased-From: 2c990d84a3d

When using hosted runners in combination with cached docker images,
there is the possibility that the host runner image is updated,
rendering the linux-headers package (stored in the cached docker image)
incompatible.

Fix this by doing a re-install of the headers package in
03_test_script.sh.

If the underlying runner kernel has not changed thie has no effect, but
prevents the job from failing if it has.

8c9048f4fc

ci: port mac-cross-gui-notests

Github-Pull: #32989
Rebased-From: 9c2514de534

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

6237cd537d

ci: port nowallet-libbitcoinkernel

Github-Pull: #32989
Rebased-From: 2a00b12d73b

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

f61cb6be58

ci: port i686-multiprocess-DEBUG

Github-Pull: #32989
Rebased-From: f2068f26c12

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

0fc3fd1eb5

ci: port fuzzer-address-undefined-integer-nodepends

Github-Pull: #32989
Rebased-From: 341196d75c3

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

96cd28f146

ci: port previous-releases-depends-debug

Github-Pull: #32989
Rebased-From: 58e38c3a042

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

fa259b4e72

ci: port 32-bit-centos-dash-gui

Github-Pull: #32989
Rebased-From: 549074bc643

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

c2c69cd6ec

ci: port tidy

Github-Pull: #32989
Rebased-From: bf7d5364527

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

49ff9d7e6e

ci: port tsan-depends-gui

Github-Pull: #32989
Rebased-From: 9bbae61e3b4

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

cf18a500d7

ci: port msan-depends

Github-Pull: #32989
Rebased-From: d290a8e6eab

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

4b3468389b

ci: port lint

Github-Pull: #32989
Rebased-From: bc41848d00f

Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>

99411458b4

ci: port win64-no_gui job 8ab684eeb7

ci: remove .cirrus.yml

Github-Pull: #32989
Rebased-From: 4393ffdd837

Removed as unused.

abaa128095

ci: dynamically match makejobs with cores

Github-Pull: #32989
Rebased-From: 3f339e99e00

Previously jobs were running on a large multi-core server where 10 jobs
as default made sense (or may even have been on the low side).

Using hosted runners with fixed (and lower) numbers of vCPUs we should
adapt compilation to match the number of cpus we have dynamically.

This is cross-platform compatible with macos and linux only.

1bfe9f56c3

doc: Detail configuration of hosted CI runners

Github-Pull: #32989
Rebased-From: f4272844833dd660c2b9db587856baa408889302

a0b6e2ae6b

ci: add ccache hit-rate warning when < 75%

Github-Pull: #32989
Rebased-From: dd1c5903e8d

Print the ccache hit-rate for the job using a GitHub annotation if it
was below 75%.

e4493b15df

ci: fix annoying docker warning

Github-Pull: #32989
Rebased-From: 2aa288efdda

Docker currently warns that we are missing a default value.

Set this to scratch which will error if an appropriate image tag is not
passed in to silence the warning.

189bb39922

ci: remove un-needed lint_run*.sh files

Github-Pull: #32989
Rebased-From: 3c5da69a232

ci/lint_run_all.sh: Only used in .cirrus.yml. Refer to test/lint/README.md on how to run locally.

12eada012b

ci: reduce runner sizes on various jobs

Github Pull: #33319
Rebased-From: 5eeb2facbbbbf68a2c30ef9e6747e39c85d7b116

These jobs can use reduced runner size to avoid wasting CPU, as much of
the long-running part of the job is single-threaded.

Suggested in: https://github.com/bitcoin/bitcoin/pull/32989#discussion_r2321775620

Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>

48761444e2

ci: Checkout latest merged pulls

Github-Pull: #33303
Rebased-From: fa8f081af31

ea4e0aa8c4

willcl-ark force-pushed on Sep 19, 2025

fanquake commented at 2:49 pm on September 24, 2025: member

cc @m3dwards

maflcko commented at 3:59 pm on September 24, 2025: member

lgtm ACK ea4e0aa8c4 🥄

Signature:

0untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
1RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
2trusted comment: lgtm ACK ea4e0aa8c4 🥄
3gzSENNVTLprDXOwKFeI2v4to/hZq+ZGg2mKM2kfXA/MylzKwyCn8yakwkjfYlMdmJAI1JY6vLMBXqmuAqpbcDA==

fanquake merged this on Sep 24, 2025

fanquake closed this on Sep 24, 2025

Backport Cirrus runners to 28.x #33406

Code Coverage & Benchmarks

Reviews