guix, depends: Omit `GUIX_ENVIRONMENT` from `gen_id` calculations #35929

pull hebasto wants to merge 1 commits into bitcoin:master from hebasto:260807-guix-depends changing 2 files +4 −2
  1. hebasto commented at 1:22 PM on August 7, 2026: member

    This PR fixes a performance regression noticed during the review of #35537.

    Since commit 008a3e29c8844c0be4457279f5c27c1bc57401c7 ("guix: split builds into Linux(gui) and macOS/Windows"), each host is built in two containers: one instantiated from manifest_build.scm and the other from the union of manifest_build.scm and manifest_gui.scm. The resulting Guix profiles necessarily differ, and because $(realpath $(GUIX_ENVIRONMENT)) is hashed into every depends package id, the built-package cache is invalidated whenever the profile changes:

    1. All depends are rebuilt when switching to another branch with identical Guix scripts and the depends subdirectory.

    2. All non-GUI packages are built twice per host, as the GUI container cannot reuse the packages just cached by the base container (this is mostly relevant for non-Linux hosts, as we move towards static builds on Linux).

    Drop GUIX_ENVIRONMENT from the gen_id inputs. The toolchain remains fully accounted for: the verbose compiler output hashed by gen_id embeds the store paths of the toolchain packages themselves, so any change to the toolchain still invalidates the cache. Given the pinned time-machine commit and the manifests, the only remaining user-provided inputs that can alter the build environment are the ADDITIONAL_GUIX_*_FLAGS, which are now conservatively folded into the ids by passing them to gen_id via the {BUILD,HOST}_ID_SALT variables.

    Related: #34228.

  2. hebasto added the label Build system on Aug 7, 2026
  3. DrahtBot commented at 1:22 PM on August 7, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35929.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK willcl-ark

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. hebasto added this to the milestone 32.0 on Aug 7, 2026
  5. willcl-ark commented at 8:26 AM on August 18, 2026: member

    Will this still work if/when we move to static linux builds?

    At that point we might have different compiler/libc targets, so we might still prefer them distictly split as https://github.com/bitcoin/bitcoin/pull/36006/changes/ad0ee317e394f72740307821553cb827c530e5a0

    ad0ee317 does not solve the build twice "issue" which you try to address here though.

  6. fanquake commented at 8:32 AM on August 18, 2026: member

    I think the PR description needs to be updated to clarify that this is only a "regression" for macOS/Windows builds. The point of #35537 was to split the Linux builds, and they wont be re-usable going forward.

  7. hebasto commented at 2:36 PM on August 24, 2026: member

    I think the PR description needs to be updated to clarify that this is only a "regression" for macOS/Windows builds. The point of #35537 was to split the Linux builds, and they wont be re-usable going forward.

    The PR description has been updated.

    Rebuilding packages in depends when switching branches is still a regression, even for Linux hosts. This affects all Guix builders, similar to the issue resolved in #34228.

    Will this still work if/when we move to static linux builds?

    Yes. For the case I mentioned above.

    UPDATE: Please see my revised response here.

  8. guix, depends: Omit `GUIX_ENVIRONMENT` from `gen_id` calculations
    Since commit 008a3e29c8844c0be4457279f5c27c1bc57401c7 ("guix: split
    builds into Linux(gui) and macOS/Windows"), each host is built in two
    containers: one instantiated from `manifest_build.scm` and the other
    from the union of `manifest_build.scm` and `manifest_gui.scm`. The
    resulting Guix profiles necessarily differ, and because
    `$(realpath $(GUIX_ENVIRONMENT))` is hashed into every depends package
    id, the built-package cache is invalidated whenever the profile
    changes:
    1. Depends are rebuilt when switching to another branch with identical
       Guix scripts and the depends subdirectory.
    2. Non-GUI packages are built twice per host, as the GUI container
       cannot reuse the packages just cached by the base container (this is
       mostly relevant for non-Linux hosts, as we move towards static builds
       on Linux).
    
    Drop `GUIX_ENVIRONMENT` from the `gen_id` inputs. The toolchain remains
    fully accounted for: the verbose compiler output hashed by `gen_id`
    embeds the store paths of the toolchain packages themselves, so any
    change to the toolchain still invalidates the cache. Given the pinned
    time-machine commit and the manifests, the only remaining user-provided
    inputs that can alter the build environment are the
    `ADDITIONAL_GUIX_*_FLAGS`, which are now conservatively folded into the
    ids by passing them to `gen_id` via the `{BUILD,HOST}_ID_SALT`
    variables.
    2e605da1a6
  9. hebasto force-pushed on Aug 28, 2026
  10. hebasto commented at 8:49 AM on August 28, 2026: member

    Rebased on top of the recent changes in the depends build subsystem.

  11. hebasto commented at 11:06 AM on August 28, 2026: member

    @willcl-ark

    Will this still work if/when we move to static linux builds?

    At that point we might have different compiler/libc targets, so we might still prefer them distictly split as ad0ee31

    ad0ee31 does not solve the build twice "issue" which you try to address here though.

    For static builds, this approach won't cache non-native non-GUI packages properly. However, that can be mitigated with a single-line change, which follows your idea of separate caches:

    --- a/contrib/guix/libexec/build_linux_gui.sh
    +++ b/contrib/guix/libexec/build_linux_gui.sh
    @@ -15,7 +15,7 @@ gcc_toolchain
     make -C depends --jobs="$JOBS" HOST="$HOST" \
                                        ${V:+V=1} \
                                        ${SOURCES_PATH+SOURCES_PATH="$SOURCES_PATH"} \
    -                                   ${BASE_CACHE+BASE_CACHE="$BASE_CACHE"} \
    +                                   BASE_CACHE="${BASE_CACHE:-$PWD/depends/built}/gui" \
                                        ${build_CC+build_CC="$build_CC"} \
                                        ${build_CXX+build_CXX="$build_CXX"}
     
    
  12. willcl-ark approved
  13. willcl-ark commented at 12:29 PM on August 28, 2026: member

    tACK 2e605da1a653100a0cedc75dde79896918a22761

    Manually tested changing GUIX_ENVIRONMENT here and on master:

    bitcoin/worktrees/pr-35929 on  pr-35929 [$] via △ v4.3.4 via 🐍 v3.14.7 via ❄️  impure (nix-shell-env)
    ❯ GUIX_ENVIRONMENT=/tmp make -C depends --no-print-directory print-build_id
    build_id=fb8808cf406fe741121401c6dca25037e1dfdccf9e6a5094ab3ec3b2e9745f39
    
    bitcoin/worktrees/pr-35929 on  pr-35929 [$] via △ v4.3.4 via 🐍 v3.14.7 via ❄️  impure (nix-shell-env) took 6s
    ❯ GUIX_ENVIRONMENT=/var make -C depends --no-print-directory print-build_id
    build_id=fb8808cf406fe741121401c6dca25037e1dfdccf9e6a5094ab3ec3b2e9745f39
    
    bitcoin/worktrees/pr-35929 on  pr-35929 [$] via △ v4.3.4 via 🐍 v3.14.7 via ❄️  impure (nix-shell-env) took 2s
    ❯ cd bitcoin
    
    core/bitcoin/bitcoin on  master [$?⇕] via △ v4.3.4 via 🐍 v3.14.7 via ❄️  impure (nix-shell-env)
    ❯ GUIX_ENVIRONMENT=/tmp make -C depends --no-print-directory print-build_id
    build_id=055db281a936a1f3617749173a3dfda529852292674e1dd4a0547c09322ad102
    
    core/bitcoin/bitcoin on  master [$?⇕] via △ v4.3.4 via 🐍 v3.14.7 via ❄️  impure (nix-shell-env) took 3s
    ❯ GUIX_ENVIRONMENT=/var make -C depends --no-print-directory print-build_id
    build_id=ea52a88298f7ff28b6d45e44486d1577de149afdabb276263eb08d9d7fec795d
    
  14. willcl-ark commented at 4:12 PM on August 28, 2026: member

    Guix build for good measure:

    x86_64
    52a27582606827acad30bedaeb64d81125e2f069ae97c8fc15b1679bb5463b91  guix-build-2e605da1a653/output/aarch64-linux-gnu/SHA256SUMS.part
    3106d8d7abc7704df689bbd4b553d41e39f647af2c74688e620393832cf6d795  guix-build-2e605da1a653/output/aarch64-linux-gnu/bitcoin-2e605da1a653-aarch64-linux-gnu-debug.tar.gz
    6b4147e8ba221bcc4f4d3ccdba3e597bc6220edc2db98198113ed2f7193e4153  guix-build-2e605da1a653/output/aarch64-linux-gnu/bitcoin-2e605da1a653-aarch64-linux-gnu.tar.gz
    0ff2536a4eccca3a892cdc064f0b82d389add0862ebf5b59d84455bf2c3b10e1  guix-build-2e605da1a653/output/arm-linux-gnueabihf/SHA256SUMS.part
    2c44c5e8e03e25243584683ef84e95f63ea623340ec49ae7284dd052cf0fe9a9  guix-build-2e605da1a653/output/arm-linux-gnueabihf/bitcoin-2e605da1a653-arm-linux-gnueabihf-debug.tar.gz
    d3450a74ff1bd60a6399c1a4a64fbbe77c2cb7513c1665b6df0152c8957d2e5e  guix-build-2e605da1a653/output/arm-linux-gnueabihf/bitcoin-2e605da1a653-arm-linux-gnueabihf.tar.gz
    5155416ed81d9b75d8691ccdf5a7e817832b4eb9d2403e819830dd27f195dd1c  guix-build-2e605da1a653/output/arm64-apple-darwin/SHA256SUMS.part
    9622f3054edb5ef4b055259400ee612aa34d4d0250b19b0e0b00285f6d2967e9  guix-build-2e605da1a653/output/arm64-apple-darwin/bitcoin-2e605da1a653-arm64-apple-darwin-codesigning.tar.gz
    52114da05c6355f033b1000c6a34bb8091dcdc97444104f97c101fd304f469d4  guix-build-2e605da1a653/output/arm64-apple-darwin/bitcoin-2e605da1a653-arm64-apple-darwin-unsigned.tar.gz
    27e2d133aaf7c62f557f94416557f084f37882a24d9c92ed9527125b6fd44b99  guix-build-2e605da1a653/output/arm64-apple-darwin/bitcoin-2e605da1a653-arm64-apple-darwin-unsigned.zip
    3041d76bba1f83000e72f1b5bc35f773bb5ce47b858a71ed6028676f92ad9a24  guix-build-2e605da1a653/output/dist-archive/bitcoin-2e605da1a653.tar.gz
    24735eaa4cf8222da930eba23af74b4cac8607cccce4fdf424221468ec4a0b31  guix-build-2e605da1a653/output/powerpc64-linux-gnu/SHA256SUMS.part
    2059aeebda7c74eaa85e890a97123b7d7cbf064f9fd923af570051cca686ee0b  guix-build-2e605da1a653/output/powerpc64-linux-gnu/bitcoin-2e605da1a653-powerpc64-linux-gnu-debug.tar.gz
    04745d8bde8772d274df6cb50dc14471647d296ef5d9873cdb4cc40dc03095c9  guix-build-2e605da1a653/output/powerpc64-linux-gnu/bitcoin-2e605da1a653-powerpc64-linux-gnu.tar.gz
    c0500744d4f4619cfdf84af1b45c10e7dbd3d4a2b4d93d614f379241f1f31403  guix-build-2e605da1a653/output/riscv64-linux-gnu/SHA256SUMS.part
    7e21b0b07032a2f6e9f7f2e684f8264521bec879f653d54cb3bdb11ac7a95357  guix-build-2e605da1a653/output/riscv64-linux-gnu/bitcoin-2e605da1a653-riscv64-linux-gnu-debug.tar.gz
    e14e20b43a11fb04028d252a7f4d09add5bec5a9cd4d4e0caaae4429924dc0b2  guix-build-2e605da1a653/output/riscv64-linux-gnu/bitcoin-2e605da1a653-riscv64-linux-gnu.tar.gz
    17d0070b7497a52d0e8148933f3cc69940aaf8c601e07289493241d7ccf224f3  guix-build-2e605da1a653/output/x86_64-apple-darwin/SHA256SUMS.part
    dc92119f5929e81cdb14d3fc012baa64b6a6bde154a2ea82755ac83dbd2b9c53  guix-build-2e605da1a653/output/x86_64-apple-darwin/bitcoin-2e605da1a653-x86_64-apple-darwin-codesigning.tar.gz
    3a6859dac6131988f9163165b0c6a5b82b09f20532199498924ed7482982def3  guix-build-2e605da1a653/output/x86_64-apple-darwin/bitcoin-2e605da1a653-x86_64-apple-darwin-unsigned.tar.gz
    a9019c9d837d6e27593230602e216101db83e1b210edf9ea46063c85cc7c07cd  guix-build-2e605da1a653/output/x86_64-apple-darwin/bitcoin-2e605da1a653-x86_64-apple-darwin-unsigned.zip
    ba136010df1fc24d5c40ce77b7c2f6b1c6a522f056c0df0a183877e427261be6  guix-build-2e605da1a653/output/x86_64-linux-gnu/SHA256SUMS.part
    e2e31b7c3087c009626c4dd90426f290e3add2767348c87e012db224c6a8ac97  guix-build-2e605da1a653/output/x86_64-linux-gnu/bitcoin-2e605da1a653-x86_64-linux-gnu-debug.tar.gz
    133cf528ba9923facad4d375b7079e09cd69fd63d7730568a481ce3319b6d818  guix-build-2e605da1a653/output/x86_64-linux-gnu/bitcoin-2e605da1a653-x86_64-linux-gnu.tar.gz
    922b967eb4df77bbdd0bb8bf3a14a189686209885ec8d764715925c80af2c935  guix-build-2e605da1a653/output/x86_64-w64-mingw32/SHA256SUMS.part
    2aee52621f2f5422b684d355039b5223cb7d92993e4ef0638b61b7e52b593643  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-codesigning.tar.gz
    92a488ad32aaaf8b81d4d59fb47ce3428303cf14029dc9ec0ab4e88a397f98ab  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-debug.zip
    aa04ba875eaef202a5dc4b1f50b996a770c8b9e670b3db4ba6214be38afc76a1  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-setup-unsigned.exe
    519357dd2b0103d495da0d7bd7817e6bc286f8de6a9dc78fd372e282a7000a63  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-unsigned.zip
    
  15. hebasto commented at 10:03 AM on August 29, 2026: member

    My Guix build:

    $ uname -m && find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
    aarch64
    52a27582606827acad30bedaeb64d81125e2f069ae97c8fc15b1679bb5463b91  guix-build-2e605da1a653/output/aarch64-linux-gnu/SHA256SUMS.part
    3106d8d7abc7704df689bbd4b553d41e39f647af2c74688e620393832cf6d795  guix-build-2e605da1a653/output/aarch64-linux-gnu/bitcoin-2e605da1a653-aarch64-linux-gnu-debug.tar.gz
    6b4147e8ba221bcc4f4d3ccdba3e597bc6220edc2db98198113ed2f7193e4153  guix-build-2e605da1a653/output/aarch64-linux-gnu/bitcoin-2e605da1a653-aarch64-linux-gnu.tar.gz
    0ff2536a4eccca3a892cdc064f0b82d389add0862ebf5b59d84455bf2c3b10e1  guix-build-2e605da1a653/output/arm-linux-gnueabihf/SHA256SUMS.part
    2c44c5e8e03e25243584683ef84e95f63ea623340ec49ae7284dd052cf0fe9a9  guix-build-2e605da1a653/output/arm-linux-gnueabihf/bitcoin-2e605da1a653-arm-linux-gnueabihf-debug.tar.gz
    d3450a74ff1bd60a6399c1a4a64fbbe77c2cb7513c1665b6df0152c8957d2e5e  guix-build-2e605da1a653/output/arm-linux-gnueabihf/bitcoin-2e605da1a653-arm-linux-gnueabihf.tar.gz
    5155416ed81d9b75d8691ccdf5a7e817832b4eb9d2403e819830dd27f195dd1c  guix-build-2e605da1a653/output/arm64-apple-darwin/SHA256SUMS.part
    9622f3054edb5ef4b055259400ee612aa34d4d0250b19b0e0b00285f6d2967e9  guix-build-2e605da1a653/output/arm64-apple-darwin/bitcoin-2e605da1a653-arm64-apple-darwin-codesigning.tar.gz
    52114da05c6355f033b1000c6a34bb8091dcdc97444104f97c101fd304f469d4  guix-build-2e605da1a653/output/arm64-apple-darwin/bitcoin-2e605da1a653-arm64-apple-darwin-unsigned.tar.gz
    27e2d133aaf7c62f557f94416557f084f37882a24d9c92ed9527125b6fd44b99  guix-build-2e605da1a653/output/arm64-apple-darwin/bitcoin-2e605da1a653-arm64-apple-darwin-unsigned.zip
    3041d76bba1f83000e72f1b5bc35f773bb5ce47b858a71ed6028676f92ad9a24  guix-build-2e605da1a653/output/dist-archive/bitcoin-2e605da1a653.tar.gz
    24735eaa4cf8222da930eba23af74b4cac8607cccce4fdf424221468ec4a0b31  guix-build-2e605da1a653/output/powerpc64-linux-gnu/SHA256SUMS.part
    2059aeebda7c74eaa85e890a97123b7d7cbf064f9fd923af570051cca686ee0b  guix-build-2e605da1a653/output/powerpc64-linux-gnu/bitcoin-2e605da1a653-powerpc64-linux-gnu-debug.tar.gz
    04745d8bde8772d274df6cb50dc14471647d296ef5d9873cdb4cc40dc03095c9  guix-build-2e605da1a653/output/powerpc64-linux-gnu/bitcoin-2e605da1a653-powerpc64-linux-gnu.tar.gz
    c0500744d4f4619cfdf84af1b45c10e7dbd3d4a2b4d93d614f379241f1f31403  guix-build-2e605da1a653/output/riscv64-linux-gnu/SHA256SUMS.part
    7e21b0b07032a2f6e9f7f2e684f8264521bec879f653d54cb3bdb11ac7a95357  guix-build-2e605da1a653/output/riscv64-linux-gnu/bitcoin-2e605da1a653-riscv64-linux-gnu-debug.tar.gz
    e14e20b43a11fb04028d252a7f4d09add5bec5a9cd4d4e0caaae4429924dc0b2  guix-build-2e605da1a653/output/riscv64-linux-gnu/bitcoin-2e605da1a653-riscv64-linux-gnu.tar.gz
    17d0070b7497a52d0e8148933f3cc69940aaf8c601e07289493241d7ccf224f3  guix-build-2e605da1a653/output/x86_64-apple-darwin/SHA256SUMS.part
    dc92119f5929e81cdb14d3fc012baa64b6a6bde154a2ea82755ac83dbd2b9c53  guix-build-2e605da1a653/output/x86_64-apple-darwin/bitcoin-2e605da1a653-x86_64-apple-darwin-codesigning.tar.gz
    3a6859dac6131988f9163165b0c6a5b82b09f20532199498924ed7482982def3  guix-build-2e605da1a653/output/x86_64-apple-darwin/bitcoin-2e605da1a653-x86_64-apple-darwin-unsigned.tar.gz
    a9019c9d837d6e27593230602e216101db83e1b210edf9ea46063c85cc7c07cd  guix-build-2e605da1a653/output/x86_64-apple-darwin/bitcoin-2e605da1a653-x86_64-apple-darwin-unsigned.zip
    ba136010df1fc24d5c40ce77b7c2f6b1c6a522f056c0df0a183877e427261be6  guix-build-2e605da1a653/output/x86_64-linux-gnu/SHA256SUMS.part
    e2e31b7c3087c009626c4dd90426f290e3add2767348c87e012db224c6a8ac97  guix-build-2e605da1a653/output/x86_64-linux-gnu/bitcoin-2e605da1a653-x86_64-linux-gnu-debug.tar.gz
    133cf528ba9923facad4d375b7079e09cd69fd63d7730568a481ce3319b6d818  guix-build-2e605da1a653/output/x86_64-linux-gnu/bitcoin-2e605da1a653-x86_64-linux-gnu.tar.gz
    922b967eb4df77bbdd0bb8bf3a14a189686209885ec8d764715925c80af2c935  guix-build-2e605da1a653/output/x86_64-w64-mingw32/SHA256SUMS.part
    2aee52621f2f5422b684d355039b5223cb7d92993e4ef0638b61b7e52b593643  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-codesigning.tar.gz
    92a488ad32aaaf8b81d4d59fb47ce3428303cf14029dc9ec0ab4e88a397f98ab  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-debug.zip
    aa04ba875eaef202a5dc4b1f50b996a770c8b9e670b3db4ba6214be38afc76a1  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-setup-unsigned.exe
    519357dd2b0103d495da0d7bd7817e6bc286f8de6a9dc78fd372e282a7000a63  guix-build-2e605da1a653/output/x86_64-w64-mingw32/bitcoin-2e605da1a653-win64-unsigned.zip
    

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-08-31 18:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me