unable to make a Guix build #150

issue ryanofsky openend this issue on February 4, 2025
  1. ryanofsky commented at 1:06 pm on February 4, 2025: collaborator

    Originally posted by @Sjors in https://github.com/bitcoin/bitcoin/issues/31741#issuecomment-2633426678

    I’m unable to make a Guix build. I’m using a branch sjors/2025/02/ipc-guix which enables multiprocess in the simplest possible way, without modifying the depends system, by setting MULTIPROCESS=1 in the pre-download and depends make step.

    It downloads and builds capnp as expected, but in the build says:

     0make: Entering directory '/bitcoin/depends'
 1Checksum missing or mismatched for native_libmultiprocess source. Forcing re-download.
 2...
 3Extracting native_capnp..
 4...
 5Caching native_capnp...
 6Fetching from
 7/gnu/store/7k364lx15sq567ivd79pp9dmnallcxz1-bash-minimal-5.1.16/bin/sh: line 1: curl: command not found
 8Fetching from https://bitcoincore.org/depends-sources
 9/gnu/store/7k364lx15sq567ivd79pp9dmnallcxz1-bash-minimal-5.1.16/bin/sh: line 1: curl: command not found
10make: *** [funcs.mk:324: /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash] Error 127

    I can add more debugging if you can’t reproduce.

  2. ryanofsky commented at 1:10 pm on February 4, 2025: collaborator

    I can add more debugging if you can’t reproduce.

    Thanks, I will try to reproduce this, but if you can add debugging that would be helpful. Output from https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2625576039 would provide more information:

     0--- a/depends/Makefile
 1+++ b/depends/Makefile
 2@@ -251,8 +251,9 @@ endef
 3 
 4 define check_or_remove_sources
 5   mkdir -p $($(package)_source_dir); cd $($(package)_source_dir); \
 6-  test -f $($(package)_fetched) && ( $(build_SHA256SUM) -c $($(package)_fetched) >/dev/null 2>/dev/null || \
 7+  test -f $($(package)_fetched) && ( $(build_SHA256SUM) -c $($(package)_fetched) || \
 8     ( echo "Checksum missing or mismatched for $(package) source. Forcing re-download."; \
 9+      set -x; cat $($(package)_fetched); : $($(package)_sha256_hash); ls -ld --full-time $(CURDIR)/$($(package)_local_dir) $($(package)_fetched) $($(package)_source); \
10       rm -f $($(package)_all_sources) $($(1)_fetched))) || true
11 endef
12 
13--- a/depends/funcs.mk
14+++ b/depends/funcs.mk
15@@ -40,7 +40,7 @@ define fetch_file
16 endef
17 
18 define fetch_local_dir_sha256
19-    if ! [ -f $($(1)_source) ] || [ $($(1)_source) -ot $($(1)_local_dir) ]; then \
20+    set -x; if ! [ -f $($(1)_source) ] || [ $($(1)_source) -ot $($(1)_local_dir) ]; then \
21         mkdir -p $(dir $($(1)_source)) && \
22         $(build_TAR) -c -f $($(1)_source) -C $($(1)_local_dir) . && \
23         rm -f $($(1)_fetched); \
  3. Sjors commented at 1:22 pm on February 4, 2025: member
    Mmm, this patch doesn’t apply on 8d63c140ebc9c2a09aa874fdd048c7d45bb5521f.
  4. ryanofsky commented at 1:32 pm on February 4, 2025: collaborator
    Yes funcs.mk changed a bit. The only difference in funcs.mk is adding “set -x; " before “if”
  5. Sjors commented at 2:54 pm on February 4, 2025: member 
     0diff --git a/depends/funcs.mk b/depends/funcs.mk
 1index 1907bab03a..373f296c9b 100644
 2--- a/depends/funcs.mk
 3+++ b/depends/funcs.mk
 4@@ -40,7 +40,7 @@ define fetch_file
 5 endef
 6 
 7 define fetch_local_dir_sha256
 8-    if ! [ -f $($(1)_source) ] || [ -n "$$(find $($(1)_local_dir) -newer $($(1)_source) | head -n1)" ]; then \
 9+    set -x; if ! [ -f $($(1)_source) ] || [ -n "$$(find $($(1)_local_dir) -newer $($(1)_source) | head -n1)" ]; then \
10         mkdir -p $(dir $($(1)_source)) && \
11         $(build_TAR) -c -f $($(1)_source) -C $($(1)_local_dir) . && \
12         rm -f $($(1)_fetched); \

    Which gives me the following logs:

     0make: Entering directory '/home/sjors/bitcoin/depends'
 1+ [ -f /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar ]
 2+ find ../src/ipc/libmultiprocess -newer /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar
 3+ head -n1
 4+ [ -n  ]
 5+ [ -f /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash ]
 6+ head -n1
 7+ find /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar -newer /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
 8+ [ -n  ]
 9+ cut -d  -f1 /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
10make[1]: Entering directory '/home/sjors/bitcoin/depends'
11+ [ -f /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar ]
12+ find ../src/ipc/libmultiprocess -newer /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar
13+ head -n1
14+ [ -n  ]
15+ [ -f /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash ]
16+ head -n1
17+ find /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar -newer /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
18+ [ -n  ]
19+ cut -d  -f1 /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
20boost_1_81_0.tar.gz: OK
21libevent-2.1.12-stable.tar.gz: OK
22qtbase-everywhere-opensource-src-5.15.16.tar.xz: OK
23qttranslations-everywhere-opensource-src-5.15.16.tar.xz: OK
24qttools-everywhere-opensource-src-5.15.16.tar.xz: OK
25qrencode-4.1.1.tar.gz: OK
26db-4.8.30.NC.tar.gz: OK
27sqlite-autoconf-3460100.tar.gz: OK
28zeromq-4.3.5.tar.gz: OK
29capnproto-cxx-1.1.0.tar.gz: OK
30sha256sum: /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash: no properly formatted checksum lines found
31Checksum missing or mismatched for native_libmultiprocess source. Forcing re-download.
32+ cat /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
33+ :
34+ ls -ld --full-time /home/sjors/bitcoin/depends/../src/ipc/libmultiprocess /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar
35drwxrwxr-x 9 sjors sjors   4096 2025-02-04 09:44:29.396999115 +0000 /home/sjors/bitcoin/depends/../src/ipc/libmultiprocess
36-rw-r--r-- 1 sjors sjors      0 2025-02-04 10:00:26.154937995 +0000 /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
37-rw-rw-r-- 1 sjors sjors 296960 2025-02-04 10:00:03.689024413 +0000 /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar
38+ rm -f /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
39capnproto-cxx-1.1.0.tar.gz: OK
40Fetching from
41curl: (3) URL rejected: No host part in the URL
42Fetching from https://bitcoincore.org/depends-sources
43  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
44                                 Dload  Upload   Total   Spent    Left  Speed
45  0  308k    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
46curl: (22) The requested URL returned error: 403
47make[1]: *** [funcs.mk:324: /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash] Error 22
48make[1]: Leaving directory '/home/sjors/bitcoin/depends'
49make: *** [Makefile:281: download-osx] Error 2
50make: Leaving directory '/home/sjors/bitcoin/depends'
  6. Sjors commented at 2:58 pm on February 4, 2025: member

    After a git clean -dfx:

     0make: Entering directory '/home/sjors/bitcoin/depends'
 1+ [ -f /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar ]
 2+ mkdir -p /home/sjors/bitcoin/depends/sources/
 3+ tar -c -f /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar -C ../src/ipc/libmultiprocess .
 4+ rm -f /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
 5+ [ -f /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash ]
 6+ mkdir -p /home/sjors/bitcoin/depends/sources/download-stamps/
 7+ sha256sum /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar
 8+ cut -d  -f1 /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
 9make[1]: Entering directory '/home/sjors/bitcoin/depends'
10+ [ -f /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar ]
11+ find ../src/ipc/libmultiprocess -newer /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar
12+ head -n1
13+ [ -n  ]
14+ [ -f /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash ]
15+ head -n1
16+ find /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar -newer /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
17+ [ -n  ]
18+ cut -d  -f1 /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
19/home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar: OK
20Fetching boost_1_81_0.tar.gz from https://archives.boost.io/release/1.81.0/source/
21...
22/home/sjors/bitcoin/depends/work/download/capnp-1.1.0/capnproto-cxx-1.1.0.tar.gz.temp: OK
23make[1]: Leaving directory '/home/sjors/bitcoin/depends'
24make: Leaving directory '/home/sjors/bitcoin/depends'
25INFO: Building 8d63c140ebc9 for platform triple arm64-apple-darwin:
26...
27make: Entering directory '/bitcoin/depends'
28+ '[' -f /bitcoin/depends/sources/src-ipc-libmultiprocess.tar ']'
29++ find ../src/ipc/libmultiprocess -newer /bitcoin/depends/sources/src-ipc-libmultiprocess.tar
30++ head -n1
31+ '[' -n '' ']'
32+ '[' -f /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash ']'
33++ find /bitcoin/depends/sources/src-ipc-libmultiprocess.tar -newer /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
34++ head -n1
35+ '[' -n '' ']'
36+ cut '-d ' -f1 /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
37boost_1_81_0.tar.gz: OK
38libevent-2.1.12-stable.tar.gz: OK
39...
40Caching native_capnp...
41Fetching from
42/gnu/store/7k364lx15sq567ivd79pp9dmnallcxz1-bash-minimal-5.1.16/bin/sh: line 1: curl: command not found
43Fetching from https://bitcoincore.org/depends-sources
44/gnu/store/7k364lx15sq567ivd79pp9dmnallcxz1-bash-minimal-5.1.16/bin/sh: line 1: curl: command not found
45make: *** [funcs.mk:324: /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash] Error 127
46make: Leaving directory '/bitcoin/depends'
  7. ryanofsky commented at 3:14 pm on February 4, 2025: collaborator

    Thanks so in first log which shows a non-clean build https://github.com/chaincodelabs/libmultiprocess/issues/150#issuecomment-2634221309 it looks like creation of tarball and checksum files are skipped because they are up to date. But the at some point after that there are errors:

    sha256sum: /home/sjors/bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash: no properly formatted checksum lines found Checksum missing or mismatched for native_libmultiprocess source. Forcing re-download.

    Happening because .stamp_fetched-native_libmultiprocess-.hash seems to have size 0.

    Second log https://github.com/chaincodelabs/libmultiprocess/issues/150#issuecomment-2634233791 on clean build seems to show tarball and checksum being created in way you would expect and no “Checksum missing” error (or maybe that part of the output is just omitted) and then is an error attempting to build .stamp_fetched-native_libmultiprocess-.hash which should have been generated earlier and not supposed to be getting built.

    If you see “Checksum missing or mismatched for native_libmultiprocess source. Forcing re-download.” error in the clean build it would be helpful to see output around it. Otherwise I probably have to debug. Am in the middle of trying to debug https://github.com/bitcoin/bitcoin/issues/31772 so haven’t tried this yet.

  8. Sjors commented at 3:41 pm on February 4, 2025: member

    I forgot to copy that bit:

     0capnproto-cxx-1.1.0.tar.gz: OK
 1sha256sum: /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar: No such file or directory
 2/home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar: FAILED open or read
 3sha256sum: WARNING: 1 listed file could not be read
 4Checksum missing or mismatched for native_libmultiprocess source. Forcing re-download.
 5+ cat /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
 6625aad21a6dac1005501646b8077776c782c7df35f4f00b64e5d67aea36d2326  /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar
 7+ : 625aad21a6dac1005501646b8077776c782c7df35f4f00b64e5d67aea36d2326
 8+ ls -ld --full-time /bitcoin/depends/../src/ipc/libmultiprocess /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash /bitcoin/depends/sources/src-ipc-libmultiprocess.tar
 9drwxrwxr-x 9 sjors users   4096 2025-02-04 09:44:29.396999115 +0000 /bitcoin/depends/../src/ipc/libmultiprocess
10-rw-rw-r-- 1 sjors users    130 2025-02-04 14:55:19.385601822 +0000 /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
11-rw-rw-r-- 1 sjors users 296960 2025-02-04 14:55:19.383601814 +0000 /bitcoin/depends/sources/src-ipc-libmultiprocess.tar
12+ rm -f /bitcoin/depends/sources/download-stamps/.stamp_fetched-native_libmultiprocess-.hash
13capnproto-cxx-1.1.0.tar.gz: OK
  9. ryanofsky commented at 4:06 pm on February 4, 2025: collaborator

    Thanks that is interesting. It seems to be looking in the wrong location for the tarball. There is a “no such file or directory” error about:

    /home/sjors/bitcoin/depends/sources/src-ipc-libmultiprocess.tar

    And directory listing shows file is instead located in:

    /bitcoin/depends/sources/src-ipc-libmultiprocess.tar

    So I think path in checksum file needs to be corrected. It looks like other hash files use relative paths and this one is created with an absolute path.

    (Also I started trying to repoduce this myself, but the build is taking a lot of time to download things so I haven’t seen an error yet.)

  10. ryanofsky commented at 4:56 pm on February 4, 2025: collaborator

    I think following should fix the problem. It switched the checksum paths to be relative instead of absolute. (I haven’t been able to fully test this yet because the guix run is still building/downloading stuff on my machine.). Will squash into #31741 if this works

     0--- a/depends/funcs.mk
 1+++ b/depends/funcs.mk
 2@@ -47,7 +47,8 @@ define fetch_local_dir_sha256
 3     fi && \
 4     if ! [ -f $($(1)_fetched) ] || [ -n "$$(find $($(1)_source) -newer $($(1)_fetched) | head -n1)" ]; then \
 5         mkdir -p $(dir $($(1)_fetched)) && \
 6-        $(build_SHA256SUM) $($(1)_source) > $($(1)_fetched); \
 7+        cd $($(1)_source_dir) && \
 8+        $(build_SHA256SUM) $($(1)_all_sources) > $($(1)_fetched); \
 9     fi && \
10     cut -d" " -f1 $($(1)_fetched)
11 endef
12@@ -82,7 +83,8 @@ define int_get_build_properties
13 $(1)_build_subdir?=.
14 $(1)_download_file?=$($(1)_file_name)
15 $(1)_source_dir:=$(SOURCES_PATH)
16-$(1)_source:=$$($(1)_source_dir)/$(if $($(1)_local_dir),$(subst $(null) $(null),-,$(strip $(subst ., ,$(subst /, ,$($(1)_local_dir))))).tar,$($(1)_file_name))
17+$(if $($(1)_file_name),,$(eval $(1)_file_name:=$(if $($(1)_local_dir),$(subst $(null) $(null),-,$(strip $(subst ., ,$(subst /, ,$($(1)_local_dir))))).tar,)))
18+$(1)_source:=$$($(1)_source_dir)/$($(1)_file_name)
19 $(1)_download_dir:=$(base_download_dir)/$(1)-$($(1)_version)
20 $(1)_prefixbin:=$($($(1)_type)_prefix)/bin/
21 $(1)_all_sources=$($(1)_file_name) $($(1)_extra_sources)
  11. Sjors commented at 5:10 pm on February 4, 2025: member

    That did the trick for completing the depends build.

    Though the bitcoin core build itself still fails:

    0/gnu/store/7k364lx15sq567ivd79pp9dmnallcxz1-bash-minimal-5.1.16/bin/sh: line 1: /bitcoin/depends/arm64-apple-darwin/bin/capnp: cannot execute binary file: Exec format error
1make[2]: *** [src/ipc/libmultiprocess/CMakeFiles/multiprocess.dir/build.make:74: src/ipc/libmultiprocess/include/mp/proxy.capnp.c++] Error 126
2make[1]: *** [CMakeFiles/Makefile2:1477: src/ipc/libmultiprocess/CMakeFiles/multiprocess.dir/all] Error 2
3make[1]: *** Waiting for unfinished jobs....

    For that I needed https://github.com/bitcoin/bitcoin/pull/30975/commits/a69e49310ffd0ebcdc2e0ca5f63c44b391bc3084, like in https://github.com/bitcoin/bitcoin/pull/30975#issuecomment-2634149344.

  12. Sjors commented at 5:13 pm on February 4, 2025: member

    I’m now running a guix build for all 5 platforms that I care about for the purpose of initial Stratum v2 binaries: aarch64-apple-darwin x86_64-apple-darwin x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu

    Will let you know if any fail.

  13. ryanofsky commented at 5:22 am on February 5, 2025: collaborator
    The guix fix above is now included in the https://github.com/bitcoin/bitcoin/pull/31741#issuecomment-2635727935 update. So hopefully this issue is fixed, but can leave it open until that is confirmed
  14. Sjors commented at 2:10 pm on February 5, 2025: member
    I was able to run the guix build after rebasing on your latest changes.
  15. ryanofsky commented at 2:40 pm on February 7, 2025: collaborator
    Thanks for testing, will close this issue
  16. ryanofsky closed this on Feb 7, 2025


ryanofsky Sjors

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin-core/libmultiprocess. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-12-04 19:30 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me