RFC: Hazards of the modern unprivileged Guix daemon configuration #36239

issue hebasto opened this issue on September 13, 2026
  1. hebasto commented at 4:31 PM on September 13, 2026: member

    According to the Guix documentation, there are two ways to configure the Guix daemon:

    1. running guix-daemon as "root", letting it run build processes as unprivileged users taken from a pool of build users...
    2. running guix-daemon as a separate unprivileged user, relying on Linux's unprivileged user namespace functionality to set up isolated environments...

    The latter is the option chosen when installing Guix on a systemd-based distribution with the installation script.

    Unfortunately, the two configurations produce observably different build environments. Consequently, some packages' test suites fail on the unprivileged daemon. Some of those packages are dependencies of our Guix environments. For example, here is the test failure for python-distlib:

    =================================== FAILURES ===================================
    _______________________ FileOpsTestCase.test_is_writable _______________________
    
    self = <test_util.FileOpsTestCase testMethod=test_is_writable>
    
    def test_is_writable(self):
      sd = 'subdir'
      ssd = 'subsubdir'
      path = os.path.join(self.workdir, sd, ssd)
      os.makedirs(path)
      path = os.path.join(path, 'test')
      self.assertTrue(self.fileop.is_writable(path))
      if os.name == 'posix':
    >           self.assertFalse(self.fileop.is_writable('/etc'))
    E           AssertionError: True is not false
    tests/test_util.py:721: AssertionError
    

    Users with an unprivileged daemon encounter such errors whenever a dependency is built locally rather than substituted, either because they pass --no-substitutes or because no substitutes exist for their platform (for example, on RISC-V systems). On x86_64 and aarch64 the problem is masked by substitutes, which are built on Guix's build farm with the privileged daemon.

    Although failures of this class are known upstream, we should address the issue at the Bitcoin Core project level as well.

    Here are some possible approaches:

    1. Provide a shell script that detects the Guix daemon configuration and warns about the unprivileged one.
    2. Document a migration path from the unprivileged daemon to the privileged one (note that Guix documents the migration in the opposite direction only).
  2. hebasto added the label Build system on Sep 13, 2026
  3. hebasto commented at 8:58 PM on September 13, 2026: member

    For example, here is the test failure for python-distlib:...

    The test failure for this package has been fixed in 482b131.

  4. willcl-ark commented at 9:43 AM on September 14, 2026: member

    Would the ideal solution here not be that we (and Guix) nudge users towards the more secure unprivileged solution, and that this worked to build correctly, i.e. the opposite of what you are recommending? Working under the assumption that these failures will be ironed out as the (relatively new) rootless-daemon approach gets more usage/testing.

    If we must choose between one or the other today though, and the rootless approach is broken, then I'd agree that perhaps we can detect/warn towards requiring the root approach.

    FWIW the nixpkgs Guix service uses the privileged approach, with no built-in option to change.

  5. fanquake commented at 9:56 AM on September 14, 2026: member

    Document a migration path from the unprivileged daemon to the privileged one

    I don't think telling users to use Guix in a less secure way, is a great solution, and that goes in the opposite direction of what Guix itself is doing.

    Here are some possible approaches:

    I think one of the options should be patching the (I assume few?) packages in our dependency graph to work around this for now (and submitting relevant changes upstream). That seems better than introducing a warning that the build might fail (option 1).

  6. hebasto commented at 11:27 AM on September 14, 2026: member

    Would the ideal solution here not be that we (and Guix) nudge users towards the more secure unprivileged solution, and that this worked to build correctly, i.e. the opposite of what you are recommending? Working under the assumption that these failures will be ironed out as the (relatively new) rootless-daemon approach gets more usage/testing.

    Document a migration path from the unprivileged daemon to the privileged one

    I don't think telling users to use Guix in a less secure way, is a great solution, and that goes in the opposite direction of what Guix itself is doing.

    I do not mean to nudge or force users toward permanently using a less secure Guix build daemon configuration. My goal is simply to ensure that people facing the same issue are aware of possible workarounds.

    Here are some possible approaches:

    I think one of the options should be patching the (I assume few?) packages in our dependency graph to work around this for now (and submitting relevant changes upstream). That seems better than introducing a warning that the build might fail (option 1).

    Patching packages upstream, as is done in 482b131, doesn't change the fact that different Guix build daemon configurations provide different package build environments, which in turn can lead to non-reproducible builds.

  7. fanquake commented at 11:37 AM on September 14, 2026: member

    non-reproducible builds.

    Isn't this a case of the build either succeeding (reproducible), or it not completing at all?

    Patching packages upstream, as is done in 482b131, doesn't change the fact

    In any case, why wouldn't the issue here be mitigated by:

    patching the (I assume few?) packages in our dependency graph

    If our manifest is fixed, why would the different build daemons produce different results?

  8. Ronin95 commented at 5:29 PM on September 14, 2026: none

    Thanks for the discussion, everyone. I understand there are differing opinions here on whether to nudge users toward the privileged daemon or not. Since my goal is just to ensure users don't waste hours on broken builds, I am happy to take this PR in whatever direction you all agree on.

    Option A: Leave the PR as is (Warning + Documentation) as @hebasto mentioned, even if upstream packages are patched, different daemon configurations fundamentally provide different build environments, which arguably goes against the spirit of reproducible builds. This PR ensures everyone is compiling in the exact same privileged environment.

    Option B: Bump the Guix Commit natively - if the consensus is that we should absolutely not warn against the unprivileged daemon, the best solution is to just inherit the upstream Guix fixes. For example, Guix explicitly fixed the python-distlib unprivileged daemon in test failure 2 months ago (in Guix commit 482b131). We could simply bump the GUIX_COMMIT in contrib/guix/libexec/prelude.bash to the latest Guix master hash (e.g., 7e74121a40a8308166e328a23647cf6f3768e6c8) to pull in these fixes naturally without touching our manifests (contrib/guix/manifest_build.scm).

    However, I understand that bumping the Guix commit affects the entire toolchain and is typically reserved for its own dedicated PR.

    Let me know which of these two paths you prefer, and I'll update or close the PR accordingly!

  9. maflcko added the label Brainstorming on Sep 17, 2026
  10. maflcko commented at 3:11 PM on September 17, 2026: member

    For example, here is the test failure for python-distlib:...

    The test failure for this package has been fixed in 482b131.

    I think this is also the correct fix to the general problem here: If there is a bug in guix upstream (or even in an upstream package), the bug should be fixed there. In the meantime, any of the already documented workarounds can be used during installation (e.g. https://github.com/bitcoin/bitcoin/blob/master/contrib/guix/INSTALL.md#workaround-3-disable-the-tests-in-the-guix-source-code-for-this-single-derivation)

    Same goes for non-reproducibility, but I guess I am just repeating what was already said above by the others?

    Maybe this issue can be closed?

  11. hebasto commented at 2:36 PM on September 30, 2026: member

    non-reproducible builds.

    Isn't this a case of the build either succeeding (reproducible), or it not completing at all?

    Patching packages upstream, as is done in 482b131, doesn't change the fact

    In any case, why wouldn't the issue here be mitigated by:

    patching the (I assume few?) packages in our dependency graph

    If our manifest is fixed, why would the different build daemons produce different results?

    Theoretically, the daemon mode is an unpinned input to the build environment. For example, a toolchain component can observe the difference in the build environment at configure time and choose a different code path, which in turn changes its behaviour and therefore the Bitcoin Core binaries it produces.

    I don't expect this to be the case for our current dependency graph, but it can only be verified by building with --no-substitutes under the two daemon configurations and comparing the results.

  12. Sjors commented at 3:24 PM on September 30, 2026: member

    FWIF I always use --no-substitutes

  13. hebasto commented at 3:31 PM on September 30, 2026: member

    FWIF I always use --no-substitutes

    I'm wondering what your current Guix daemon configuration is?

  14. Sjors commented at 4:02 PM on September 30, 2026: member

    @hebasto guix-daemon.service:

    [Service]
    ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon \
        --build-users-group=guixbuild --discover=no --cores=3 --no-substitutes
    Environment='GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale' LC_ALL=en_US.utf8
    

    So it's privileged IIUC. I'll look into changing that...

  15. willcl-ark commented at 4:31 PM on September 30, 2026: member

    Theoretically, the daemon mode is an unpinned input to the build environment. For example, a toolchain component can observe the difference in the build environment at configure time and choose a different code path, which in turn changes its behaviour and therefore the Bitcoin Core binaries it produces.

    our build environment is containerised though, right. so this shouldn’t be an issue for us?

    or do you mean guix builds a different derivation depending on daemon or not (this would seem ver concerning!)

  16. hebasto commented at 6:19 PM on September 30, 2026: member

    Consider the following MRE.

    ;; daemon-probe.scm
    (use-modules (guix gexp))
    
    (computed-file "daemon-probe"
      #~(begin
          (mkdir #$output)
          (call-with-output-file (string-append #$output "/probe")
            (lambda (port)
              (format port "uid=~a~%etc-owner=~a~%etc-writable=~a~%"
                      (getuid)
                      (stat:uid (stat "/etc"))
                      (access? "/etc" W_OK))))))
    

    With the privileged Guix daemon on Fedora 44:

    $ guix time-machine --commit=60f6956aeffa7f30285745bd0ea615e9acfc74f8 -- build -f daemon-probe.scm
    /gnu/store/1in4vadwx49v5akvyvz0falxgaf09sq9-daemon-probe
    $ cat /gnu/store/1in4vadwx49v5akvyvz0falxgaf09sq9-daemon-probe/probe
    uid=973
    etc-owner=0
    etc-writable=#f
    

    With the unprivileged Guix daemon on Ubuntu 26.04:

    $ guix time-machine --commit=60f6956aeffa7f30285745bd0ea615e9acfc74f8 -- build -f daemon-probe.scm
    /gnu/store/1in4vadwx49v5akvyvz0falxgaf09sq9-daemon-probe
    $ cat /gnu/store/1in4vadwx49v5akvyvz0falxgaf09sq9-daemon-probe/probe
    uid=30001
    etc-owner=30001
    etc-writable=#t
    

    Note that the derivation and the output path are identical, yet the contents of probe differ. Both builds succeed, so this is a different-output case, not a build-failure case.

  17. willcl-ark commented at 10:47 AM on October 6, 2026: member

    I think I'd expect outputs to have different permissions as a difference between the two privilege models?

    Surely the entire idea is that outputs are not root-owned, unless I'm missing something here...

  18. maflcko commented at 11:21 AM on October 6, 2026: member

    Right. It is one thing if a package embeds the uid at compile time into itself. It seems certainly a bug if a package then embeds that stuff in some way or another into Bitcoin Core. Such a bug should be fixed and reported upstream, just like any other upstream bug?

    Not sure what needs to be done here. I think a few builders can follow https://guix.gnu.org/manual/devel/en/html_node/Build-Environment-Setup.html#Migrating-to-the-Unprivileged-Daemon if they want (and then share that they did so in their guix builds).

  19. willcl-ark commented at 2:16 PM on October 6, 2026: member

    I have patched my NixOS system to use an local nixpkgs with an unprivileged Guix package.

    Will kick a Guix build off later this evening and report back...

  20. willcl-ark commented at 11:32 AM on October 7, 2026: member

    Will kick a Guix build off later this evening and report back...

    This seems to currently work correctly per results in: #36029 (comment)

  21. maflcko removed the label Brainstorming on Oct 7, 2026
  22. hebasto commented at 2:23 PM on October 13, 2026: member

    Will kick a Guix build off later this evening and report back...

    This seems to currently work correctly per results in: #36029 (comment)

    You also have to disable substitutes.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-10-23 04:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me