ci: switch to a sourceware mirror for riscv #35885

pull willcl-ark wants to merge 2 commits into bitcoin:master from willcl-ark:reenable-riscv changing 2 files +14 −0

willcl-ark commented at 7:54 PM on August 4, 2026: member

The https transport is rate-limited to block AI scrapers.

Switch to a live mirror on fish.foo to re-enable the riscv job.
DrahtBot added the label Tests on Aug 4, 2026
DrahtBot commented at 7:54 PM on August 4, 2026: contributor


The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.



Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35885.



Reviews

See the guideline and AI policy for information on the review process.

Type Reviewers

ACK maflcko, sedited

Concept ACK fanquake

If your review is incorrectly listed, please copy-paste <code></code> into the comment that the bot should ignore.



LLM Linter (✨ experimental)

Possible typos and grammar issues:
- # Use a mirror for these submodules as sourceware blocks too many requests combating AI. -> # Use a mirror for these submodules, as sourceware blocks too many requests while combating AI. [The ending phrase is grammatically broken and obscures the intended meaning.]
2026-08-06 11:56:50
willcl-ark commented at 8:02 PM on August 4, 2026: member

Ah, this worked locally, but seems this PR is now bumping into https://github.blog/changelog/2022-03-15-removed-unencrypted-git-protocol-and-certain-ssh-keys/
willcl-ark force-pushed on Aug 4, 2026
DrahtBot added the label CI failed on Aug 4, 2026
willcl-ark renamed this:
~~ci: switch git transport protocol in riscv~~
ci: switch to a sourceware mirror for riscv
on Aug 4, 2026
DrahtBot commented at 8:14 PM on August 4, 2026: contributor


🚧 At least one of the CI tasks failed. Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/30945409441/job/92114034730 LLM reason (✨ experimental): CI failed because IWYU detected required (contentful) header include changes and intentionally returned failure (Failure generated from IWYU).

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:
- Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
- A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
- An intermittent issue.
Leave a comment here, if you need help tracking down a confusing failure.

</details>
willcl-ark force-pushed on Aug 4, 2026
maflcko commented at 8:26 PM on August 4, 2026: member

Fixed the unrelated CI failure in #35886
hebasto referenced this in commit 2f52c2e8c0 on Aug 4, 2026
willcl-ark force-pushed on Aug 4, 2026
DrahtBot removed the label CI failed on Aug 4, 2026

Type	Reviewers
ACK	maflcko, sedited
Concept ACK	fanquake

in ci/test/01_base_install.sh:94 in 95bb01f35f

  87 | @@ -88,8 +88,16 @@ if [[ -n "${USE_INSTRUMENTED_LIBCPP}" ]]; then
  88 |  fi
  89 |  
  90 |  if [[ ${BARE_METAL_RISCV} == "true" ]]; then
  91 | +    # Use a mirror for these submodules as sourceware blocks too many requests combating AI.
  92 |      ${CI_RETRY_EXE} git clone --depth=1 https://github.com/riscv-collab/riscv-gnu-toolchain -b 2026.06.06 /riscv/gcc
  93 |      ( cd /riscv/gcc;
  94 | +      export GIT_CONFIG_COUNT=3;

maflcko commented at 5:48 AM on August 5, 2026:

in d135812519381581e4d06098dbc91984523889c9: All of the redundant trailing ; are a bit confusing and inconsistent maybe remove them wholesale?

willcl-ark commented at 11:56 AM on August 6, 2026:

Done in b283e1751cc

maflcko approved
maflcko commented at 5:48 AM on August 5, 2026: member

lgtm ACK 95bb01f35f741722847d1da5a36d2b8b4e1502f6
fanquake commented at 7:59 AM on August 5, 2026: member

Concept ACK

ci: use mirror for riscv submodules

sourceware is blocking/rate-limiting http(s) git requests to combat AI
scrapers.

Switch to a live mirror hosted at fish.foo

Use exported GIT_CONFIG_* to avoid setting `git config --global` which
could clobber a user running on bare host, but still propagate to child
git processes spawned by `make`.

b283e1751c

Revert "ci: Temporarily remove riscv32 config from GHA matrix"
This reverts commit fa06ea42448d94c6e7ed9fad1adedb439ff6f78d.
81fcecfe45
willcl-ark force-pushed on Aug 6, 2026
willcl-ark marked this as ready for review on Aug 6, 2026
willcl-ark commented at 12:00 PM on August 6, 2026: member

If we are happy to lean on mirrors of the sourceware repos (hosted by myself) then we can re-enable this job using them. It does not seem like sourceware have any plans to ease off on their rate-limiting AFAICT from IRC converstation. The mirror is on a hosted box which I have no plans on deprecating currently. can provide more details if anyone wants them.

We can probably improve the cache layers a bit better here too; ideally building the entire riscv toolchain and caching that early on so it's rarely invalidated by anything else. Although it still doesn't fix the root cause of the repo being rate-limited in any case that the cache misses...
maflcko commented at 12:13 PM on August 6, 2026: member

lgtm ACK 81fcecfe4525f2d57412f07390a139a2acf1c54e
DrahtBot requested review from fanquake on Aug 6, 2026
sedited approved
sedited commented at 12:24 PM on August 6, 2026: contributor

Thank you for the effort towards fixing this and hosting it on a mirror.

ACK 81fcecfe4525f2d57412f07390a139a2acf1c54e

I'm curious about the caching. Why does it seem basically uncached at the moment?
maflcko commented at 12:38 PM on August 6, 2026: member

I'm curious about the caching. Why does it seem basically uncached at the moment?

The CI system basically only has a single layer for the images. The rationale is that usually a base config change (e.g. an edit in ./ci/test/00_setup_env.sh or in ci/test/01_base_install.sh will invalidate all later layers of all images anyway). This can't really be improved upon, unless someone wants to extract all install parts into separate ones.

Or are you asking about genuine cache misses where there shouldn't be ones on warpbuild?
fanquake merged this on Aug 6, 2026
fanquake closed this on Aug 6, 2026

Contributors

Review Requested
fanquake

Labels

Tests

Linked (view graph)

#35886 refactor: Remove unused #include in common/system

ci: switch to a sourceware mirror for riscv #35885

Code Coverage & Benchmarks

Reviews

LLM Linter (✨ experimental)