ci: use warp caching on warp runners #35430

pull willcl-ark wants to merge 1 commits into bitcoin:master from willcl-ark:cache-switching changing 6 files +110 −13

willcl-ark commented at 9:50 AM on June 1, 2026: member

The GHA cache is very slow, taking on the order of minutes to save and restore from.

Use WarpBuild's cache instead as this is in the same region and should be much faster.

WarpBuild cache action does not auto-fallback to GHA if not being run on Warp. To allow fork runs to fallback to GHA caching, whilst minimising duplication in the action files, create new "interal" actions which perform the switching logic, and use these in the (renamed) cache|save actions.

Without this we would need the if logic in our prvious actions, 4 times in each of save and restore.
DrahtBot added the label Tests on Jun 1, 2026
DrahtBot commented at 9:50 AM on June 1, 2026: contributor



The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.



Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35430.



Reviews

See the guideline for information on the review process.

Type Reviewers

ACK m3dwards

If your review is incorrectly listed, please copy-paste <code></code> into the comment that the bot should ignore.
fanquake commented at 9:55 AM on June 1, 2026: member

The GHA cache is very slow, taking on the order of minutes to save and restore from.

Yea. I think we've seen some restoring taking up to 16 minutes: https://github.com/bitcoin/bitcoin/actions/runs/26680339841/job/78639514475#step:11:425 ?
willcl-ark force-pushed on Jun 1, 2026
DrahtBot added the label CI failed on Jun 1, 2026
willcl-ark commented at 9:58 AM on June 1, 2026: member

Noting here also that we pay per-minute for runners now, so not wasting (up to) 16 minutes per job per run is important not only for dev UX.
willcl-ark force-pushed on Jun 1, 2026
willcl-ark force-pushed on Jun 1, 2026

Type	Reviewers
ACK	m3dwards

ci: Add dynamic cache switching to warp cache

The GHA cache is very slow, taking on the order of minutes to save and
restore from.

Use WarpBuild's cache instead as this is in the same region and much
faster.

WarpBuild cache action does not auto-fallback to GHA if not being run on
Warp. To allow fork runs to fallback to GHA caching, whilst minimising
duplication in the action files, create new "interal" actions which
perform the switching logic, and use these in the (renamed) cache|save
actions.

Without this we would need the `if` logic in our prvious actions, 4
times in each of save and restore.

Plumb the provider through into the action, as a composite action can't
read `env` (`GITHUB_OUTPUT`) from previous steps.

2ce4ae7d8f

willcl-ark force-pushed on Jun 1, 2026
willcl-ark marked this as ready for review on Jun 1, 2026
DrahtBot removed the label CI failed on Jun 1, 2026
fanquake commented at 1:14 PM on June 1, 2026: member

cc @m3dwards
willcl-ark commented at 1:38 PM on June 1, 2026: member

Run Median Mean Max

This PR 13s 15.1s 36s

master previous 14 days 55s 1m09s 11m10s

edit: I also suspect timings will be much more stable too, but don't have data to back it up (yet).
in .github/actions/cache/restore/internal/action.yml:30 in 2ce4ae7d8f
```
  25 | +  using: 'composite'
  26 | +  steps:
  27 | +    - name: Restore cache with WarpBuild
  28 | +      id: warp
  29 | +      if: ${{ inputs.provider == 'warp' }}
  30 | +      uses: WarpBuilds/cache/restore@v1
```
m3dwards commented at 8:47 PM on June 1, 2026:

Shall we start pinning external actions against commit hashes to prevent supply chain attacks re-using the same tag?

maflcko commented at 9:51 AM on June 2, 2026:

Will this help? IIUC you can tag an action to a hash, but that isn't recursive, so this doesn't help, is verbose and even gives the wrong impression that it helps.

m3dwards commented at 1:54 PM on June 2, 2026:

Even if not fully recursive it still helps doesn't it?

Github docs:

https://docs.github.com/en/actions/reference/security/secure-use#using-third-party-actions

The github and warp cache actions don't call any other actions as far as I can tell (if that's what you mean by recursive in this context)

maflcko commented at 3:36 PM on June 2, 2026:

Even if not fully recursive it still helps doesn't it?

Hmm, I think we should instead not use any GITHUB_TOKEN with write access in the CI.

The github and warp cache actions don't call any other actions as far as I can tell (if that's what you mean by recursive in this context)

Ok, I see. Though, we'd have to review this, and also review every bump? And the benefit seems limited to a belt-and-suspenders check if we (1) leak a GH write token by accident into this action, and (2) warp is compromised by a third-party bad actor. Both seem unlikely by itself and more unlikely in combination, so is this worth the extra review effort?
m3dwards commented at 8:58 PM on June 1, 2026: contributor

https://github.com/bitcoin/bitcoin/compare/28f0ff6cf15c24832750ee709f3fd3e412bb0b38..2ce4ae7d8f006959cb83be93a17b8125e953c30a Nice trick to populate the caches before this was merged!
m3dwards commented at 9:02 PM on June 1, 2026: contributor

This is looking good and is working on my fork and still caching / restoring on GHA. I see that on this main repo that the correct jobs are using Warp cache and the 32 bit arm is correctly still using GHA.
m3dwards commented at 9:15 PM on June 1, 2026: contributor

ACK 2ce4ae7d8f006959cb83be93a17b8125e953c30a

Can add pinning as a follow up
maflcko commented at 9:57 AM on June 2, 2026: member

The GHA cache is very slow, taking on the order of minutes to save and restore from.

Yea. I think we've seen some restoring taking up to 16 minutes: https://github.com/bitcoin/bitcoin/actions/runs/26680339841/job/78639514475#step:11:425 ?

Quite depressing that a paid service (after #35348) can't even offer a download speed of greater than 1MiB/s.

lgtm conceptually, but I haven't reviewed any code at all.
fanquake merged this on Jun 2, 2026
fanquake closed this on Jun 2, 2026
fanquake referenced this in commit c1bf1c0049 on Jun 2, 2026
fanquake commented at 10:14 AM on June 2, 2026: member

Backported to 31.x in #35331.
m3dwards deleted the branch on Jun 2, 2026

Run	Median	Mean	Max
This PR	13s	15.1s	36s
master previous 14 days	55s	1m09s	11m10s

Contributors

Labels

Linked (view graph)

#35331 [31.x] Backports

ci: use warp caching on warp runners #35430

Code Coverage & Benchmarks

Reviews