lint: modernise lint tooling #34547

pull willcl-ark wants to merge 11 commits into bitcoin:master from willcl-ark:modernise-linter changing 11 files +200 −128
  1. willcl-ark commented at 10:25 am on February 10, 2026: member

    Modernise our lint tooling by:

    - Replacing pyenv + pip with uv for better Python environment and dependency management - Replacing mypy with https://github.com/astral-sh/ty - Replacing the 01_install.sh runtime install script with COPY –from multi-stage Docker image imports for uv, ruff, shellcheck, mlc, and ty - Moving ruff lint rules from hardcoded Rust array (in lint_py.rs) into a top-level ruff.toml, and add ty.toml for the type checker - Extracting all remaining pip dependencies into dedicated ci/lint/requirements.txt - Add a ruff format lint which, if the original file is formatted, will fail if a change un-formats it

    Extra rationale:

    COPY --from pulls pre-built binaries from upstream images instead of compiling/downloading at runtime. Containerfile layer optimisations reduce rebuild frequency further.

    ty is significantly faster/more modern/maintained than mypy, and configured declaratively.

    Adding root-level [ty|ruff].toml config files means contributors can easily run ty check or ruff check locally without running the full linter, along with being accessible to other tooling (similarly for requriements.txt).

    Pinning tool versions in the dockerfile makes it more excplicit and easier to find.

    The tradeoff we make here is that there is no longer a bare install script to install tooling on a local machine. However I think this is OK, as it currently only works for apt-based OSes anyway, and I don’t think running the linter outside of the container is such a valuable use-case as it is with some of the other CI jobs.

    Further work can drop individual rules from ty.toml fixing up the infringing code as necessary.

    I can split this up if wanted, but IMO it makes sense to do it altogether.

  2. DrahtBot added the label Tests on Feb 10, 2026
  3. DrahtBot commented at 10:25 am on February 10, 2026: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #34757 (guix: re-enable riscv exported symbol checking by fanquake)
    • #34550 (guix: update time-machine to f04dc9234c58b064507bb46ab1797f95f09b543e by fanquake)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  4. fanquake commented at 10:39 am on February 10, 2026: member

    Tried running this locally, but it currently fails on arm64, because there is no arm64 variant for becheran/mlc:

    0Error: choosing an image from manifest list docker://becheran/mlc:latest: no image found in image index for architecture "arm64", variant "v8", OS "linux"
    
  5. willcl-ark marked this as a draft on Feb 10, 2026
  6. in ci/lint_imagefile:15 in b63444e4d1 outdated
    11+ENV LC_ALL=C
    12+ENV PATH="/python_env/bin:${PATH}"
    13+ENV VIRTUAL_ENV="/python_env"
    14+
    15+COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
    16+COPY --from=ghcr.io/astral-sh/ruff:0.14.4 /ruff /bin/
    


    maflcko commented at 10:58 am on February 10, 2026:
    Is there any value in having ruff separate? It would appear more consistent to install it next to the other ci/lint/requirements.txt (lief, etc)

    willcl-ark commented at 12:16 pm on February 10, 2026:

    The main advantage is that the COPY --from instructions are standalone layers and are not invalidated when things below change (they are just “replayed” over the top). The more tools we can install from them, the better (usually).

    No strong opinion here, as the end result is the same. But IMO it’s better practice to install from layers like I have it.


    maflcko commented at 12:53 pm on February 10, 2026:

    Sure, makes sense to use COPY –from to keep the layers in the cache.

    I guess the main question is whether we want to force all devs to use the docker image. I think it should be fine for devs to pick pip and then install ci/lint/requirements.txt and get all the stuff they want. But no strong opinion, I rely on the GHA-ci for the lint anyway.

  7. in ci/lint_imagefile:29 in b63444e4d1 outdated
    29+  uv python install && \
    30   echo 'alias lint="./ci/lint/06_script.sh"' >> ~/.bashrc && \
    31-  chmod 755 /entrypoint.sh && \
    32   rm -rf /var/lib/apt/lists/*
    33 
    34+COPY ./ci/lint/container-entrypoint.sh /entrypoint.sh
    


    maflcko commented at 10:59 am on February 10, 2026:
    This seems to conflict with #34427. Maybe that should be merged/closed first?

    willcl-ark commented at 12:59 pm on February 10, 2026:
    Agree. I’ve acked that PR already, and would be happy to rebase this on it (or see it closed) – no strong opinion either way really.
  8. willcl-ark force-pushed on Feb 10, 2026
  9. willcl-ark commented at 12:17 pm on February 10, 2026: member

    Tried running this locally, but it currently fails on arm64, because there is no arm64 variant for becheran/mlc:

    0Error: choosing an image from manifest list docker://becheran/mlc:latest: no image found in image index for architecture "arm64", variant "v8", OS "linux"
    

    This should be fixed in the latest push. We currently (and still here) use an x86_64 MLC binary, but this works in an arm64 container (e.g on Mac) because the Rosetta emulation is passed in/used inside the container.

    I keep this behaviour for now, but ideally MLC woudl start publishing multi-platform docker images.

  10. willcl-ark marked this as ready for review on Feb 10, 2026
  11. DrahtBot added the label Needs rebase on Feb 12, 2026
  12. willcl-ark force-pushed on Feb 12, 2026
  13. DrahtBot removed the label Needs rebase on Feb 12, 2026
  14. DrahtBot added the label Needs rebase on Mar 10, 2026
  15. willcl-ark force-pushed on Mar 12, 2026
  16. DrahtBot removed the label Needs rebase on Mar 12, 2026
  17. lint: install uv from docker and use
    https://docs.astral.sh/uv/
    
    Install python in the linter using uv and a venv.
    This is faster and more simple than building pyenv.
    10332d0129
  18. willcl-ark force-pushed on Mar 12, 2026
  19. lint: install ruff from docker image
    Align with uv tool installation method.
    59bd2f9472
  20. lint: install shellcheck from docker image 87c0592779
  21. lint: add ruff.toml 73eae6f76e
  22. lint: add ty.toml
    Add ty typechecker configuration file.
    
    Includes all needed rules to have `ty` pass on selected files in current
    codebase. This is pretty much all useful rules, but we will remove
    them systematically in follow-ups.
    c0dac30212
  23. lint: use ruff.toml in linter a4200d48bf
  24. lint: install ty typechecker from docker and use
    Remove mypy and switch to `ty` for typechecking.
    
    https://docs.astral.sh/ty/
    https://astral.sh/blog/ty
    
    It's faster and more maintained than mypy. Comes with better editor
    integrations, a language server and is written in rust (tm).
    
    As features are added we may be able to replace vulture with ty too.
    
    Install from docker image as per `uv` and `ruff`.
    49ec173b3d
  25. lint: move remaining install to dockerfile
    Note: we keep MLC install via curl binary download. This is currently
    hardcoded to an x86_64 binary, but this seems to run fine in an arm64
    container (on MacOS), as Rosetta support is passed in (on docker and
    podman) to take care of exactly this.
    
    If upstream MLC start providing a multi-image dockerfile then this
    dependency could be made it's own `FROM --...` layer as most of the
    others are.
    cd94fbb26f
  26. lint: use requirements.txt bea37db4ff
  27. Don't pin Python patch version
    .python-version always matches the minimum supported Python version.
    It's main purpose is to catch accidental use of too modern syntax
    in scripts and functional tests.
    
    We (currently) don't specify a minimum patch version, so it's not
    necessary to do so here. The minor verion is enough.
    
    This also avoids requiring users to keep a potentially unsafe old
    patch version installed.
    5f6938a5ea
  28. lint: add python file format lint check
    Enforce monotonic ruff formatting for Python files: if a file was
    already ruff format-compliant on the base branch, it must stay
    formatted in the PR. Files that aren't yet formatted are not
    enforced, allowing incremental adoption without a big-bang reformat.
    
    This follows the strategy used by nixpkgs to adopt nixfmt without
    merge queues:
    
    1. Add per-file CI check (this commit) — a no-op today since no
       files are ruff format-compliant yet.
    2. Reformat inactive files (files not touched by recent PRs) in a
       batch. The check prevents regressions from that point on.
    3. Reformat remaining files once old PRs have flushed through.
    
    The per-file approach avoids the merge queue requirement: old PRs
    opened before a batch reformat can still merge without breaking
    master, because the check only compares against the base branch
    version of each changed file, not the entire tree.
    c7e4a67086
  29. willcl-ark force-pushed on Mar 12, 2026
  30. DrahtBot added the label CI failed on Mar 12, 2026
  31. willcl-ark commented at 11:59 am on March 12, 2026: member
    bumped to latest ruff (0.15.5) and ty (0.0.21) and added newly-needed rule ignores.
  32. DrahtBot removed the label CI failed on Mar 12, 2026

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-03-12 18:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me