ci: Disable cache save for pull requests in GitHub Actions #28292

pull hebasto wants to merge 1 commits into bitcoin:master from hebasto:230818-gha-ccache changing 1 files +12 −4
  1. hebasto commented at 6:24 am on August 18, 2023: member

    This PR disable cache save for pull requests in GitHub Actions.

    Otherwise, multiple pull requests fill GitHub Actions cache quota shortly.

    See a discussion here.


    NOTE for the maintainers with “owner” permissions.

    This PR needs the actions/cache/restore@* and actions/cache/save@* acrions to be explicitly allowed in the repository’s Actions permissions.

  2. DrahtBot commented at 6:24 am on August 18, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK MarcoFalke

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

  3. DrahtBot added the label Tests on Aug 18, 2023
  4. in .github/workflows/ci.yml:70 in f1bd112995 outdated
    68+      - name: Save Ccache cache
    69+        uses: actions/cache/save@v3
    70+        if: github.event_name != 'pull_request'
    71+        with:
    72+          path: ${{ env.CCACHE_DIR }}
    73+          key: ${{ github.job }}-ccache-${{ github.run_id }}
    


    MarcoFalke commented at 6:32 am on August 18, 2023:

    Shouldn’t the key be just the task name? There should be no reason to bloat the cache directory with entries that won’t ever be used again.

    See https://github.com/bitcoin/bitcoin/actions/caches


    hebasto commented at 6:44 am on August 18, 2023:
    GitHub Actions caches are immutable. Therefore, this pattern has been used for Ccache caches.

    MarcoFalke commented at 6:45 am on August 18, 2023:
    ugh, fair enough

    MarcoFalke commented at 6:47 am on August 18, 2023:
    0          key: ${{ github.job }}-ccache-${{ github.run_id }}  # https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
    

    hebasto commented at 6:56 am on August 18, 2023:
    The comment has been added.
  5. in .github/workflows/ci.yml:59 in f1bd112995 outdated
    58         with:
    59           path: ${{ env.CCACHE_DIR }}
    60-          key: ${{ github.job }}-ccache-cache-${{ github.run_id }}
    61-          restore-keys: ${{ github.job }}-ccache-cache
    62+          key: ${{ github.job }}-ccache-${{ github.run_id }}
    63+          restore-keys: ${{ github.job }}-ccache-
    


    MarcoFalke commented at 6:32 am on August 18, 2023:
    why does this specify both keys?

    hebasto commented at 6:39 am on August 18, 2023:
    While we are interested in the restore-keys input only, the key input is required, thus, it cannot be avoided.

    MarcoFalke commented at 6:42 am on August 18, 2023:
    ok, maybe use the same value for both, or is there a reason to use different values?

    hebasto commented at 6:51 am on August 18, 2023:

    … is there a reason to use different values?

    Yes, it is. github.run_id is unique for every run. That is why the restore-keys input contains prefixes that are expected to match. The latest matched cache will be restored.

  6. MarcoFalke commented at 6:32 am on August 18, 2023: member
    concept ACK
  7. in .github/workflows/ci.yml:66 in f1bd112995 outdated
    65       - name: CI script
    66         run: ./ci/test_run_all.sh
    67+
    68+      - name: Save Ccache cache
    69+        uses: actions/cache/save@v3
    70+        if: github.event_name != 'pull_request'
    


    MarcoFalke commented at 6:34 am on August 18, 2023:
    Maybe also disable it for branches other than master, to make sure that storage is constant, even if there was an intermittent push to a backport branch?

    MarcoFalke commented at 6:47 am on August 18, 2023:
    ok, nvm. Looks like caches are immutable anyway, so this can’t be achieved, I guess.

    MarcoFalke commented at 6:49 am on August 18, 2023:
    Maybe it is still better to only have caches for a single branch, as they can’t be re-used for other branches anyway?

    hebasto commented at 6:53 am on August 18, 2023:

    In the main repository, caches for backport branches will be evicted after 7 days of non-using them.

    However, the current logic seems useful for fork repos, no?


    MarcoFalke commented at 7:06 am on August 18, 2023:

    I was thinking that (assuming a cache size per push of $n GB, and a total limit of 10GB), 10/n pushes to a backport branch will evict the main cache. For example, for n=5, 2 pushes are enough.

    Though, it is only a cache, so I guess it doesn’t matter.


    MarcoFalke commented at 7:13 am on August 18, 2023:

    However, the current logic seems useful for fork repos, no?

    If you are worried about hard-coding master, I think, if possible, using the GitHub default branch name should achieve the same.


    hebasto commented at 7:16 am on August 18, 2023:

    However, the current logic seems useful for fork repos, no?

    I mean, topic branches in contributors’ fork repos.


    MarcoFalke commented at 9:19 am on August 18, 2023:

    I mean, topic branches in contributors’ fork repos.

    The cache key isn’t included the branch name, so I don’t think this will work.


    fanquake commented at 9:21 am on August 18, 2023:
    Can someone elaborate on why branches in forks of this repository are something we need to consider when deciding on our caching stratergy for this repository? Do the forks share our cache (resources)? Can anyone with a fork clobber our cache?

    hebasto commented at 9:22 am on August 18, 2023:
    On a contributor’s topic branch in their personal repo, the workflow will be triggered by a push event. So the updated push will reuse the cache.

    hebasto commented at 9:24 am on August 18, 2023:

    Can anyone with a fork clobber our cache?

    No.


    hebasto commented at 9:26 am on August 18, 2023:

    Can someone elaborate on why branches in forks of this repository are something we need to consider when deciding on our caching stratergy for this repository?

    I use CI in my personal repo before pushing my branch into the pull request. And I want to use CI with some efficiency.


    MarcoFalke commented at 9:26 am on August 18, 2023:

    On a contributor’s topic branch in their personal repo, the workflow will be triggered by a push event. So the updated push will reuse the cache.

    No? I think it will use the latest cache, which may be from a different branch, because the cache key isn’t included the branch name, so I don’t think this will work.

    Edit: Maybe it does: https://github.com/actions/cache/blob/main/tips-and-workarounds.md#force-deletion-of-caches-overriding-default-cache-eviction-policy


    hebasto commented at 9:30 am on August 18, 2023:

    On a contributor’s topic branch in their personal repo, the workflow will be triggered by a push event. So the updated push will reuse the cache.

    No? I think it will use the latest cache, which may be from a different branch, because the cache key isn’t included the branch name, so I don’t think this will work.

    https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#example-using-multiple-restore-keys: a feature branch is considered before the default branch.

  8. MarcoFalke approved
  9. MarcoFalke commented at 6:48 am on August 18, 2023: member
    lgtm ACK
  10. hebasto force-pushed on Aug 18, 2023
  11. DrahtBot added the label CI failed on Aug 18, 2023
  12. MarcoFalke commented at 7:18 am on August 18, 2023: member
    I guess as an alternative, the ghcr can be used to store a ccache? This would require a bit more code on our side, and maybe a few more permissions to be enabled? Though, if the write-permissions are limited to the master branch, maybe that is fine?
  13. hebasto commented at 7:26 am on August 18, 2023: member

    I guess as an alternative, the ghcr can be used to store a ccache? This would require a bit more code on our side, and maybe a few more permissions to be enabled?

    For now, only two jobs, namely native macOS and Windows, are planned to run on GitHub Actions. It should work fine at this scale, no?

    Though, if the write-permissions are limited to the master branch, maybe that is fine?

    I’m not sure about how to make “the write-permissions are limited to the master branch”. I’ll do my research though.

  14. hebasto commented at 8:57 am on August 18, 2023: member

    the ghcr can be used to store a ccache?

    Anyway, it won’t work with non-Linux runners out-of-the-box.

  15. DrahtBot removed the label CI failed on Aug 18, 2023
  16. DrahtBot commented at 3:03 pm on August 18, 2023: contributor

    Looks like CI is red, but it is not shown here?

    https://github.com/bitcoin/bitcoin/actions/runs/5899744715

  17. hebasto commented at 3:08 pm on August 18, 2023: member

    Looks like CI is red, but it is not shown here?

    bitcoin/bitcoin/actions/runs/5899744715 @achow101 @fanquake

    I’m kindly asking you to update Actions permissions according to this PR description.

  18. fanquake commented at 10:10 am on August 21, 2023: member

    I’m kindly asking you to update Actions permissions according to this PR description.

    Should be available now.

  19. hebasto force-pushed on Aug 21, 2023
  20. ci: Disable cache save for pull requests in GitHub Actions
    Otherwise, multiple pull requests fill GitHub Actions cache quota
    shortly.
    241d6ca34c
  21. hebasto force-pushed on Aug 21, 2023
  22. hebasto requested review from MarcoFalke on Aug 21, 2023
  23. MarcoFalke commented at 2:12 pm on August 21, 2023: member
    lgtm ACK 241d6ca34c60d9003a27bb7960cebfb66366753b
  24. DrahtBot removed review request from MarcoFalke on Aug 21, 2023
  25. fanquake merged this on Aug 21, 2023
  26. fanquake closed this on Aug 21, 2023

  27. hebasto deleted the branch on Aug 21, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me