ci: Use containerfiles? #26918

1. 
   maflcko commented at 10:23 am on January 19, 2023: member

   containerfiles have the advantage that they can be cached locally, so for example an apt operation in an image layer is faster to retrieve from the (local) cache than to run vanilla.

   Converting the ci system to those is non-trivial, because it uses a lot of env vars, different configs, and numerous per-config hacks.

   My understanding of docker is limited, but to pass env vars into a containerfile would require using ARG in the file, as well as code to pass in the ARG at runtime via the command line. For hundreds of args, this should be possible, but verbose, which is why I haven’t looked at it yet.

2. maflcko added the label Feature on Jan 19, 2023
3. maflcko added the label Tests on Jan 19, 2023
4. maflcko added the label Brainstorming on Jan 19, 2023
5. 
   maflcko commented at 10:25 am on January 19, 2023: member

   Also, I wonder, if we use ARG, how this would integrate into Cirrus CI? If it can’t be integrated, containerfiles would only benefit a few devs running locally?
6. 
   maflcko commented at 10:29 am on January 19, 2023: member

   Finally, I wonder, if this needs a complete re-write of the ci system, given that we use source to run everything in one context (https://github.com/bitcoin/bitcoin/blob/master/ci/test_run_all.sh), and splitting this into several layers wouldn’t work, as docker does not retain the env across layers.
7. 
   aureleoules commented at 11:06 am on January 19, 2023: member

   Concept ACK on converting the CI to Dockerfiles. This would allow developers to run the CI easily and locally. #26906 is a good step towards this goal. I tried to run the CI with the Cirrus CLI but it never worked.

   Though, I wonder how ccache and depends_cache can be integrated with Docker and Cirrus. Simple mount volumes maybe? Because I think this is the most important bottleneck, not the apt install overhead.

8. 
   fanquake commented at 11:17 am on January 19, 2023: member

   This would allow developers to run the CI easily and locally.

   Just noting, it’s already possible to run the CI easily and locally, and this already uses Docker. See https://github.com/bitcoin/bitcoin/blob/master/ci/README.md.

   i.e on my Ubuntu machine, I can just run FILE_ENV="./ci/test/00_setup_env_native_tidy.sh" ./ci/test_run_all.sh and this spins up the tidy CI job locally, using Docker.

9. 
   maflcko commented at 4:13 pm on January 19, 2023: member

   Though, I wonder how ccache and depends_cache can be integrated with Docker and Cirrus. Simple mount volumes maybe?

   It may be possible to do that via CCACHE_DIR=/ccache_dir or so? See also https://github.com/bitcoin/bitcoin/blob/b5c88a547996776dbdc2e101bae9b67ac639fd02/.cirrus.yml#L7

   I don’t think that approach will be changing.

10. 
    achow101 commented at 4:59 pm on January 19, 2023: member

    Also, I wonder, if we use ARG, how this would integrate into Cirrus CI?

    ARG values can be provided with a docker_arguments section: https://cirrus-ci.org/guide/docker-builder-vm/#dockerfile-as-a-ci-environment

11. 
    maflcko commented at 5:08 pm on January 19, 2023: member

    Maybe we could build a tool to persist and preserve ci env vars between layers?
12. 
    maflcko commented at 1:56 pm on January 27, 2023: member

    Did something in #26976 for apt
13. 
    maflcko commented at 1:59 pm on January 27, 2023: member

    Is there something other than apt that can be cached in an image layer? If not, closing for now.
14. maflcko closed this on Jan 27, 2023

    ---

15. bitcoin locked this on Jan 27, 2024

Contributors
maflcko aureleoules fanquake achow101

Labels
Feature Brainstorming Tests