depends: hash local source contents #35765

pull willcl-ark wants to merge 1 commits into bitcoin:master from willcl-ark:guix-depends-hash-check changing 3 files +38 −15
  1. willcl-ark commented at 2:40 PM on July 21, 2026: member

    Fixes #35764.

    Create a normalized temporary archive on each evaluation using the existing CMake dependency, then publish it under a content-addressed filename. This replaces mtime-based invalidation and allows different worktrees to coexist in a shared SOURCES_PATH.

    CMake provides a consistent archive implementation across supported build hosts and allows the archive mtime to be fixed.

    CMake/libarchive records the invoking uid/gid, so cross-user archive reproducibility remains a separate consideration. The reported shared-worktree failure was the same user and host, and I'd expect this to be the most-common case by far.

  2. depends: hash local source contents
    Timestamp comparisons can reuse an archive produced by another worktree, causing Depends to build stale local sources.
    
    Specifically, depends caches packages using `$(package)_local_dir` as a tarball in `SOURCES_PATH`. The tarball is currently recreated only when `find -newer` reports a path newer than the cached archive.
    
    This does not reliably identify directory contents:
    
    - Two worktrees sharing `SOURCES_PATH` can contain different source trees.
    - A tarball created by one worktree may be newer than every path in another.
    - Deleting a file leaves no path whose mtime can be checked.
    - Changed or restored files may have older or equal mtimes.
    - Timestamp granularity and clock differences make ordering unreliable.
    
    `native_libmultiprocess` is currently the only package using `local_dir`.
    
    Local source identity should change when any of the following changes:
    
    - A file is added, removed, or modified.
    - File permissions, including the executable bit, change.
    - A symlink target changes.
    - An empty directory is added or removed.
    
    Filesystem timestamps, ownership paths, and absolute worktree locations should not determine the identity. Identical trees in separate worktrees should produce the same archive checksum.
    
    Different trees using the same shared `SOURCES_PATH` must be able to coexist.
    
    ---
    
    Generate normalized archives with CMake and key them by SHA256 so distinct trees can coexist safely in a shared source cache.
    
    Keep the archive hash as the recipe input so existing package and dependent build ID propagation remains unchanged.
    b1f79c9fa6
  3. DrahtBot added the label Build system on Jul 21, 2026
  4. DrahtBot commented at 2:40 PM on July 21, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35765.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process. A summary of reviews will appear here.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #35618 (depends: Make tarball creation from local directory reproducible by hebasto)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  5. willcl-ark commented at 2:41 PM on July 21, 2026: member

    Not sure if it might be better to set depends mtime to 2000-01-01 12:00:00 UTC as is done by default_build_TOUCH already? Perhaps it could be...

  6. willcl-ark commented at 7:34 AM on July 22, 2026: member

    Build of this PR:

    x86_64
    eb548f6197dee88a0e125c04f7c43b7e3360f963c5b62d6dd7096def4d25ae82  guix-build-b1f79c9fa6dd/output/aarch64-linux-gnu/SHA256SUMS.part
    30109d042a7a31893422649b11b5ac54fc2702d27dabedb9e286dc740e475cc4  guix-build-b1f79c9fa6dd/output/aarch64-linux-gnu/bitcoin-b1f79c9fa6dd-aarch64-linux-gnu-debug.tar.gz
    04c50689d60f5f2d17631a7443cbd4ae5084f31de1934f22d488cdc630721816  guix-build-b1f79c9fa6dd/output/aarch64-linux-gnu/bitcoin-b1f79c9fa6dd-aarch64-linux-gnu.tar.gz
    0a7a24fdf440a63382f1abf83c4a4cd4700bc68e14dbbb2d51ea98c8a0a65ac6  guix-build-b1f79c9fa6dd/output/arm-linux-gnueabihf/SHA256SUMS.part
    9d1383085f267a8bf57842330ef856c4e1f0fd4efa899705a0e6b8e0f82e7144  guix-build-b1f79c9fa6dd/output/arm-linux-gnueabihf/bitcoin-b1f79c9fa6dd-arm-linux-gnueabihf-debug.tar.gz
    e5bc8d5e63a299e64434dce0bcd0d1a953de77d0c12ea87dea1fe4af529f20b9  guix-build-b1f79c9fa6dd/output/arm-linux-gnueabihf/bitcoin-b1f79c9fa6dd-arm-linux-gnueabihf.tar.gz
    da82d78e01c0cd69e6033633dabf3f4ab0fcb5c9d746ea68607f4f4e56bcedb8  guix-build-b1f79c9fa6dd/output/arm64-apple-darwin/SHA256SUMS.part
    13488634f71b849eea3f0ed70fcb96a4c08b7614ed22cd53c17a35030debbd01  guix-build-b1f79c9fa6dd/output/arm64-apple-darwin/bitcoin-b1f79c9fa6dd-arm64-apple-darwin-codesigning.tar.gz
    f7cadfe8ec4e02a569751fefa1a0a257c25cf6edcf9af69dcb73f6c697495e4f  guix-build-b1f79c9fa6dd/output/arm64-apple-darwin/bitcoin-b1f79c9fa6dd-arm64-apple-darwin-unsigned.tar.gz
    2a3f133c5ad11a657286c9f1b377fac100bfb6335f52a7ac0da565ebed7e8ce7  guix-build-b1f79c9fa6dd/output/arm64-apple-darwin/bitcoin-b1f79c9fa6dd-arm64-apple-darwin-unsigned.zip
    0210d6fad3c23629dd44259a318a098ab1d40cd2f614211323da423e00dbd6e2  guix-build-b1f79c9fa6dd/output/dist-archive/bitcoin-b1f79c9fa6dd.tar.gz
    4ff5751126abad7c077158cdc5eae9d8826f61752cb73254ce2d718a5bae841f  guix-build-b1f79c9fa6dd/output/powerpc64-linux-gnu/SHA256SUMS.part
    4b587f04770cb8fb1676de1ac47b2ff7a19dbb28296b6c86e4ead11cca74c103  guix-build-b1f79c9fa6dd/output/powerpc64-linux-gnu/bitcoin-b1f79c9fa6dd-powerpc64-linux-gnu-debug.tar.gz
    627972cee5ff34f41d04fa051a171e9674b28bcc18ec8c2a3224e0aab8265c35  guix-build-b1f79c9fa6dd/output/powerpc64-linux-gnu/bitcoin-b1f79c9fa6dd-powerpc64-linux-gnu.tar.gz
    54cd74d473f49f8bda4b198a8db4bb239b9b2b31dcd604072c9067e15e7676cd  guix-build-b1f79c9fa6dd/output/riscv64-linux-gnu/SHA256SUMS.part
    0309b1be6f6a95e4a577ce4706bc658b150a6114341b260d6239833909f0a2d3  guix-build-b1f79c9fa6dd/output/riscv64-linux-gnu/bitcoin-b1f79c9fa6dd-riscv64-linux-gnu-debug.tar.gz
    d3df748ea3e86ca0a60763955469b92ef496a6f1355c57618aee6ae32f421b4d  guix-build-b1f79c9fa6dd/output/riscv64-linux-gnu/bitcoin-b1f79c9fa6dd-riscv64-linux-gnu.tar.gz
    20b826775a5bc1076e7dbb7b2d3025a98442463642e8e3ca22b736bd048feb14  guix-build-b1f79c9fa6dd/output/x86_64-apple-darwin/SHA256SUMS.part
    8771f090263129505d2bc2aa333ba243439532713b54bfa792b79172513ca2fc  guix-build-b1f79c9fa6dd/output/x86_64-apple-darwin/bitcoin-b1f79c9fa6dd-x86_64-apple-darwin-codesigning.tar.gz
    9fe8f90b64df1a86a0eeda61d3ef25c30c752ff3f3b41f00bf526c181c07339e  guix-build-b1f79c9fa6dd/output/x86_64-apple-darwin/bitcoin-b1f79c9fa6dd-x86_64-apple-darwin-unsigned.tar.gz
    18b6ad45aed36993f4c296e79b04de8591bf325b793b7e797ce1819ae9fa6205  guix-build-b1f79c9fa6dd/output/x86_64-apple-darwin/bitcoin-b1f79c9fa6dd-x86_64-apple-darwin-unsigned.zip
    06bfebb2f8d63caa53587e4612bb75da416740fb52b9cec7aeac75b8378718b1  guix-build-b1f79c9fa6dd/output/x86_64-linux-gnu/SHA256SUMS.part
    bd5a7d537ee343f37eac47aa4b6ce2b762ba11564a8f455993f2ddf673c06e27  guix-build-b1f79c9fa6dd/output/x86_64-linux-gnu/bitcoin-b1f79c9fa6dd-x86_64-linux-gnu-debug.tar.gz
    aa4b4dd950669df51fdb0b0cf0ed555cacc13878416390cc8a6a0258eb1490d3  guix-build-b1f79c9fa6dd/output/x86_64-linux-gnu/bitcoin-b1f79c9fa6dd-x86_64-linux-gnu.tar.gz
    8f5052762e1f72be1c98b334a8cfdcc91a4f96e9fb4468259225347aec6f44f4  guix-build-b1f79c9fa6dd/output/x86_64-w64-mingw32/SHA256SUMS.part
    a534e9c6bf9165c1c093fd9e2a55c2cda1e6a8cfd076beba6ee8717ea61d9168  guix-build-b1f79c9fa6dd/output/x86_64-w64-mingw32/bitcoin-b1f79c9fa6dd-win64-codesigning.tar.gz
    783ab6083576c93e74301574d326a07d36953249e17a90bcebb46d436c99d618  guix-build-b1f79c9fa6dd/output/x86_64-w64-mingw32/bitcoin-b1f79c9fa6dd-win64-debug.zip
    2a5f744b0eeaccf29ad367b5b1cb32c75f5013792a9ca264894569fce6b3c6e5  guix-build-b1f79c9fa6dd/output/x86_64-w64-mingw32/bitcoin-b1f79c9fa6dd-win64-setup-unsigned.exe
    1d2450834b8e53295b9e1317a817e9dddf7a48a5a3352b70fa2e92158a3a8d69  guix-build-b1f79c9fa6dd/output/x86_64-w64-mingw32/bitcoin-b1f79c9fa6dd-win64-unsigned.zip
    

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-07-25 07:50 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me