deps: Bump lief to 0.16.6

davidgumberg commented at 0:59 am on May 7, 2025: contributor

Partially resolves #30520, updating lief to 0.16.6.

DrahtBot commented at 0:59 am on May 7, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32431.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	laanwj, hebasto

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#32644 (doc: miscellaneous changes by fanquake)
#25573 ([POC] guix: produce a fully -static-pie bitcoind by fanquake)
#24123 (guix: Pointer Authentication and Branch Target Identification for aarch64 Linux by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

davidgumberg force-pushed on May 7, 2025

DrahtBot added the label CI failed on May 7, 2025

DrahtBot commented at 1:29 am on May 7, 2025: contributor

🚧 At least one of the CI tasks failed. Task lint: https://github.com/bitcoin/bitcoin/runs/41764401882 LLM reason (✨ experimental): (empty)

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

davidgumberg force-pushed on May 7, 2025

laanwj commented at 5:51 am on May 7, 2025: member

Concept ACK.

Fixes an existing bug, where bcrypt.dll was not one of the expected symbols in contrib/devtools/symbol-check.py, even though this symbol gets included by way of secp256k1:

From what i understand, that’s only used for the example, not for the library itself. The bcrypt-using code (fill_random in examples/examples_util.h) is never included in the library.

It’s unlikely that the LIEF version changes anything there.

laanwj added the label Build system on May 7, 2025

hebasto commented at 6:13 am on May 7, 2025: member

~~Also the LIEF version must be bumped here~~

in contrib/guix/manifest.scm:163 in 3d1465f074 outdated

157@@ -158,7 +158,7 @@ chain for " target " development."))
158 (define-public python-lief
159   (package
160     (name "python-lief")
161-    (version "0.13.2")
162+    (version "0.16.5")

hebasto commented at 6:16 am on May 7, 2025:

Does Guix build succeed?

davidgumberg commented at 6:20 am on May 7, 2025:

In the process of testing this, I’ll mark as draft in the meanwhile.

hebasto commented at 6:17 am on May 7, 2025: member

Concept ACK.

davidgumberg force-pushed on May 7, 2025

davidgumberg commented at 6:23 am on May 7, 2025: contributor

From what i understand, that’s only used for the example, not for the library itself. The bcrypt-using code (fill_random in examples/examples_util.h) is never included in the library.

It’s unlikely that the LIEF version changes anything there.

Oops, my bad, I must have ran symbol-check.py against a stale build of the bcrypt branch, rebased to drop adding bcrypt to symbol-check.py, marking draft until I test this as working in Guix.

davidgumberg marked this as a draft on May 7, 2025

DrahtBot removed the label CI failed on May 7, 2025

fanquake commented at 8:05 am on May 7, 2025: member

Add usage notes for contrib/devtools/security-check.py

I think you can actually remove these docs. The symbol / security scripts are not for general developer usage, and there’s no expectation that the scripts should pass, outside of a Guix build. We could move them out of contrib to make this more clear.

Fixes an existing bug, where bcrypt.dll was not one of the expected symbols in contrib/devtools/symbol-check.py, even though this symbol gets included by way of secp256k1:

This seems to be a symptom of the above. Dependencies in libsecp256k1 example code, are not relevant for our release builds. Looking at a Guix build of master, I cannot see any dependency on bcrypt.dll in any binary.

in contrib/devtools/security-check.py:277 in ae6c79c319 outdated

283 
284 if __name__ == '__main__':
285     retval: int = 0
286     for filename in sys.argv[1:]:
287         binary = lief.parse(filename)
288+        if binary is None:

fanquake commented at 8:12 am on May 7, 2025:

Was this added to appease a linter? There should be no need for code like this, as every file passed to these scripts in Guix, is a binary. Leaving the script to fail if .parse fails is fine.

davidgumberg commented at 0:56 am on May 9, 2025:

Yes, but it was to appease the CI linter, I understand now that this script is not meant for general use, so I changed this to just cast to lief.Binary without checking

davidgumberg force-pushed on May 9, 2025

davidgumberg commented at 0:51 am on May 9, 2025: contributor

I think you can actually remove these docs.

Thanks, fixed. I did not understand that these were not meant for use outside of Guix builds,

I definitely opened this PR prematurely, thinking I could get away with just bumping the version and fixing the python scripts 😂.

Working on getting guix builds working right, I referenced prior art by @willcl-ark here (https://github.com/willcl-ark/bitcoin/commit/ede9aa93fe6339fe6285c954dc8fbd9ae8623916).

The python build is convinced that a build option is set in config-settings, even when I override configure-flags, which is where config-settings is supposed to get it’s value from (https://github.com/fanquake/guix/blob/ac2d792aae241f5233ee3fdfa29cd3dbaeb9338c/guix/build/pyproject-build-system.scm#L107-L111) :

 0starting phase `set-pythonpath'
 1phase `set-pythonpath' succeeded after 0.0 seconds
 2starting phase `change-directory'
 3phase `change-directory' succeeded after 0.0 seconds
 4starting phase `build'
 5Using 'setup' to build wheels, auto-detected 'setup', override '#f'.
 6ERROR: Unrecognized options in config-settings:
 7  build -> Did you mean: build-dir?
 8error: in phase 'build': uncaught exception:
 9%exception #<&invoke-error program: "python" arguments: ("-c" "import sys, importlib, json\nconfig_settings = json.loads (sys.argv[3])\nbuilder = importlib.import_module(sys.argv[1])\nbuilder.build_wheel(sys.argv[2], config_settings=config_settings)" "setup" "dist" "[]") exit-status: 7 term-signal: #f stop-signal: #f>
10phase `build' failed after 0.1 seconds
11command "python" "-c" "import sys, importlib, json\nconfig_settings = json.loads (sys.argv[3])\nbuilder = importlib.import_module(sys.argv[1])\nbuilder.build_wheel(sys.argv[2], config_settings

davidgumberg force-pushed on May 9, 2025

laanwj commented at 11:17 am on May 9, 2025: member

The symbol / security scripts are not for general developer usage, and there’s no expectation that the scripts should pass, outside of a Guix build. We could move them out of contrib to make this more clear.

i think this makes sense, especially for the symbol check. It doesn’t have any use outside the guix build. The security check may have some value as a more general “binary security check”. But moving both to a guix-specific path would be fine with me.

fanquake commented at 12:54 pm on May 9, 2025: member

But moving both to a guix-specific path would be fine with me.

See #32458.

DrahtBot added the label Needs rebase on May 12, 2025

willcl-ark commented at 12:43 pm on May 13, 2025: member

Hey @davidgumberg just getting around to the ping here (sorry).

I have a local manifest which builds 0.16.3 lief: https://gist.github.com/willcl-ark/3ac07722025b696910adba256d813cb6

As with the changes to Lief themselves I began, they def need a second look over, but perhaps it will be of some help.

davidgumberg force-pushed on May 20, 2025

davidgumberg renamed this:
~~deps: Bump lief to 0.16.5~~
deps: Bump lief to 0.16.4
on May 20, 2025

davidgumberg commented at 7:48 am on May 20, 2025: contributor

I have a local manifest which builds 0.16.3 lief: https://gist.github.com/willcl-ark/3ac07722025b696910adba256d813cb6

I was using basically this manifest taken from your branch (https://github.com/willcl-ark/bitcoin/tree/bump-lief), but it seems some build changes were made in lief 0.16.5 that prevent this from working, I’ve rebased, bumping to 0.16.4

davidgumberg marked this as ready for review on May 20, 2025

DrahtBot removed the label Needs rebase on May 20, 2025

fanquake commented at 4:14 pm on May 20, 2025: member

I think it’d be worth investigating any issues with 0.16.5, rather than deferring. Seeing some build output with this branch, which is unexpected:

0[100%] Built target bitcoin-qt
1Checking binary security...
2have_gnu_relro:#True and have_bindnow: True
3have_gnu_relro:#True and have_bindnow: True
4have_gnu_relro:#True and have_bindnow: True
5have_gnu_relro:#True and have_bindnow: True
6have_gnu_relro:#True and have_bindnow: True
7have_gnu_relro:#True and have_bindnow: True
8have_gnu_relro:#True and have_bindnow: True
9[100%] Built target check-security

davidgumberg force-pushed on May 20, 2025

davidgumberg commented at 6:39 pm on May 20, 2025: contributor

I think it’d be worth investigating any issues with 0.16.5, rather than deferring.

Sounds good, marking the PR as draft again while I investigate.

I’ve bisected to this commit in lief: https://github.com/lief-project/LIEF/commit/f23ced2f4ffc170d0a6f40ff4a1bee575e3447cf

Seems related to scikit-build 0.10 changing cmake.targets to build.targets, and defining build.targets explains why build is defined in config_settings.

Seeing some build output with this branch, which is unexpected:

[…]

Left some logging in, fixed.

davidgumberg marked this as a draft on May 20, 2025

davidgumberg renamed this:
~~deps: Bump lief to 0.16.4~~
deps: Bump lief to 0.16.5
on May 20, 2025

DrahtBot added the label CI failed on May 20, 2025

DrahtBot removed the label CI failed on May 23, 2025

in ci/lint/04_install.sh:41 in 2d26b4545e outdated

37@@ -38,7 +38,7 @@ python3 --version
38 
39 ${CI_RETRY_EXE} pip3 install \
40   codespell==2.2.6 \
41-  lief==0.13.2 \
42+  lief==0.16.4 \

hebasto commented at 4:44 pm on May 27, 2025:

0  lief==0.16.5 \

davidgumberg commented at 2:10 am on May 28, 2025:

Thanks, fixed.

in contrib/guix/manifest.scm:184 in 2d26b4545e outdated

182-    (build-system python-build-system)
183-    (native-inputs (list cmake-minimal python-tomli))
184+                "1wq57xgnvpqpjnld03a095wb0fjl719sqrg51n80dv0lx0868n2z"))))
185+    (build-system pyproject-build-system)
186+    (native-inputs (list cmake-minimal
187+                         python-tomli

hebasto commented at 4:44 pm on May 27, 2025:

Is python-tomli really necessary?

davidgumberg commented at 2:10 am on May 28, 2025:

Good catch, I’ve removed this.

hebasto commented at 4:46 pm on May 27, 2025: member

I think it’d be worth investigating any issues with 0.16.5, rather than deferring.

Sounds good, marking the PR as draft again while I investigate.

I’ve bisected to this commit in lief: lief-project/LIEF@f23ced2

Seems related to scikit-build 0.10 changing cmake.targets to build.targets, and defining build.targets explains why build is defined in config_settings.

Seeing some build output with this branch, which is unexpected: […]

Left some logging in, fixed.

This requires bumping the Guix time machine to commit a786cd333c738dacdf156752b428f572b31382b0 or later to ensure the appropriate version of the python-scikit-build-core package.

davidgumberg referenced this in commit e96d6a5314 on May 28, 2025

davidgumberg force-pushed on May 28, 2025

DrahtBot added the label Needs rebase on May 28, 2025

davidgumberg force-pushed on May 28, 2025

davidgumberg referenced this in commit 9171600924 on May 28, 2025

davidgumberg commented at 2:16 am on May 28, 2025: contributor

Thanks for the suggested fix @hebasto! Bumping the guix time machine to a786cd333c738dacdf156752b428f572b31382b0 seems to fix building lief 0.16.5, but it introduces other unrelated issues with the guix build, so I’ve rebased on the time machine change, but leaving this as draft for now while investigating.

 0 - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/setjmp/rtld-__longjmp.os
 1a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/signal/rtld-sigaction.os
 2a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strchr.os
 3a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strcmp.os
 4a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strcspn.os
 5a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strdup.os
 6a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strlen.os
 7a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strnlen.os
 8a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strncmp.os
 9a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-memchr.os
10a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-memcmp.os
11a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-memmove.os
12a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-memset.os
13a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-stpcpy.os
14a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-memcpy.os
15a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-wordcopy.os
16a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-rawmemchr.os
17a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/string/rtld-strchrnul.os
18a - /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/time/rtld-setitimer.os
19mv -f /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/rtld-libc.aT /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/rtld-libc.a
20make[3]: Leaving directory '/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/source/elf'
21riscv64-linux-gnu-gcc   -nostdlib -nostartfiles -r -o /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/librtld.os '-Wl,-(' /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/dl-allobjs.os /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/rtld-libc.a -lgcc '-Wl,-)' \
22          -Wl,-Map,/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/librtld.os.map
23riscv64-linux-gnu-gcc   -nostdlib -nostartfiles -shared -o /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld.so.new         \
24          -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,now    \
25          /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/librtld.os -Wl,--version-script=/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/ld.map              \
26          -Wl,-soname=ld-linux-riscv64-lp64d.so.1                       \
27          -Wl,-defsym=_begin=0
28riscv64-linux-gnu-readelf -s /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld.so.new \
29  | gawk '($7 ~ /^UND(|EF)$/ && $1 != "0:" && $4 != "REGISTER") { print; p=1 } END { exit p != 0 }'
30mv -f /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld.so.new /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld.so
31rm -f /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld-linux-riscv64-lp64d.so.1.new
32riscv64-linux-gnu-gcc   -shared -static-libgcc -Wl,-O1  -Wl,-z,defs -Wl,-dynamic-linker=/gnu/store/q9fb29wi0nvi1y91fiypc2lcf54nryz5-glibc-cross-riscv64-linux-gnu-2.31/lib/ld-linux-riscv64-lp64d.so.1  -B/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/csu/  -Wl,--version-script=/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/libc.map -Wl,-soname=libc.so.6 -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -Wl,-z,now -nostdlib -nostartfiles -e __libc_main -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/math -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/dlfcn -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/nss -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/nis -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/rt -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/resolv -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/mathvec -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/support -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/crypt -L/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/nptl -Wl,-rpath-link=/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/math:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/dlfcn:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/nss:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/nis:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/rt:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/resolv:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/mathvec:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/support:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/crypt:/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/nptl -o /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/libc.so -T /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/shlib.lds /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/csu/abi-note.o /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/soinit.os /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/libc_pic.os /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/interp.os /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld.so -lgcc /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/sofini.os
33/gnu/store/3jhfhxdf6v5ms10x5zmnl166dh3yhbr1-bash-minimal-5.1.16/bin/sh ../scripts/rellns-sh /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld.so /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld-linux-riscv64-lp64d.so.1.new
34mv -f /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld-linux-riscv64-lp64d.so.1.new /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/elf/ld-linux-riscv64-lp64d.so.1
35riscv64-linux-gnu-ld: relocation R_RISCV_RVC_JUMP against `__sigsetjmp' which may bind externally can not be used when making a shared object; recompile with -fPIC
36riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
37riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
38riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
39riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
40riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
41riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
42riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
43riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
44riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
45riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
46riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
47riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
48riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
49riscv64-linux-gnu-ld: BFD (GNU Binutils) 2.41 assertion fail elfnn-riscv.c:3288
50collect2: error: ld returned 1 exit status
51make[2]: *** [../Makerules:699: /tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/build/libc.so] Error 1
52make[2]: Leaving directory '/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/source/elf'
53make[1]: *** [Makefile:470: elf/subdir_lib] Error 2
54make[1]: Leaving directory '/tmp/guix-build-glibc-cross-riscv64-linux-gnu-2.31.drv-0/source'
55make: *** [Makefile:9: all] Error 2
56error: in phase 'build': uncaught exception:
57%exception #<&invoke-error program: "make" arguments: ("-j" "32") exit-status: 2 term-signal: #f stop-signal: #f>
58phase `build' failed after 27.5 seconds
59command "make" "-j" "32" failed with status 2
60build process 18 exited with status 256

DrahtBot removed the label Needs rebase on May 28, 2025

fanquake commented at 1:01 pm on May 28, 2025: member

Thanks for the suggested fix @hebasto! Bumping the guix time machine to

I don’t think we need to bump anything here (putting aside the fact that the suggested change breaks the riscv & windows builds, and will make many other changes to the release env). We can just apply a one line patch to LIEF. i.e:

 0Partially revert f23ced2f4ffc170d0a6f40ff4a1bee575e3447cf
 1
 2Restore compat with python-scikit-build-core 0.10.x
 3Can be dropped when using python-scikit-build-core 0.11.x
 4
 5--- a/api/python/backend/setup.py
 6+++ b/api/python/backend/setup.py
 7@@ -101,12 +101,12 @@ def _get_hooked_config(is_editable: bool) -> Optional[dict[str, Union[str, List[
 8     config_settings = {
 9         "logging.level": "DEBUG",
10         "build-dir": config.build_dir,
11-        "build.targets": config.build.targets,
12         "install.strip": config.strip,
13         "backport.find-python": "0",
14         "wheel.py-api":  config.build.py_api,
15         "cmake.source-dir": SRC_DIR.as_posix(),
16         "cmake.build-type": config.build.build_type,
17+        "cmake.targets": config.build.targets,
18         "cmake.args": [
19             *config.cmake_generator,
20             *config.get_cmake_args(is_editable),

(see https://github.com/fanquake/bitcoin/tree/scikit_compat).

hebasto commented at 1:47 pm on May 28, 2025: member

0Restore compat with python-scikit-build-core 0.10.x
1Can be dropped when using python-scikit-build-core 0.11.x

It seems it should be:

0Restore compat with python-scikit-build-core 0.9.x
1Can be dropped when using python-scikit-build-core 0.10.x

davidgumberg force-pushed on May 28, 2025

davidgumberg commented at 3:01 am on May 29, 2025: contributor

Thanks for the patch @fanquake, I’ve rebased to include https://github.com/fanquake/bitcoin/tree/scikit_compat, but squashed to avoid a broken commit.

It seems it should be:

0Restore compat with python-scikit-build-core 0.9.x
1Can be dropped when using python-scikit-build-core 0.10.x

I see, in theory, it should be sufficient to fast-forward to 35c5f07e967155d2276c7ec58e5108e4da02c974, but some other build issue with python-pydantic prevents building lief at that commit hash.

I believe there is a regression in parsing big endian PPC64 headers in LIEF: https://github.com/lief-project/LIEF/issues/1217, but I’ve added a work-around to contrib/guix/security-check.py for this.

Leaving this as draft for now until #32634 is merged.

davidgumberg force-pushed on May 29, 2025

hebasto commented at 8:21 am on May 29, 2025: member

Tested 38d7664dc1a00e76b75186f4146affa4dc538bae:

0Checking binary security...
1Traceback (most recent call last):
2  File "/distsrc-base/distsrc-38d7664dc1a0-powerpc64-linux-gnu/contrib/guix/security-check.py", line 285, in <module>
3    for (name, func) in CHECKS[etype][arch]:
4KeyError: ARCHITECTURES.UNKNOWN

UPD. Oh, I missed https://github.com/lief-project/LIEF/issues/1217.

davidgumberg force-pushed on May 29, 2025

hebasto commented at 11:09 am on May 29, 2025: member

~~I believe the manifest could be further simplified:~~

 0--- a/contrib/guix/manifest.scm
 1+++ b/contrib/guix/manifest.scm
 2@@ -18,7 +18,7 @@
 3              ((gnu packages python-build) #:select (python-poetry-core))
 4              ((gnu packages python-crypto) #:select (python-asn1crypto))
 5              ((gnu packages python-science) #:select (python-scikit-build-core))
 6-             ((gnu packages python-xyz) #:select (python-pydantic-2 python-pydantic-core))
 7+             ((gnu packages python-xyz) #:select (python-pydantic-2))
 8              ((gnu packages tls) #:select (openssl))
 9              ((gnu packages version-control) #:select (git-minimal))
10              (guix build-system cmake)
11@@ -183,18 +183,12 @@ chain for " target " development."))
12     (build-system pyproject-build-system)
13     (native-inputs (list cmake-minimal
14                          python-scikit-build-core
15-                         python-pydantic-core
16                          python-pydantic-2))
17     (arguments
18      (list
19       #:tests? #f                  ;needs network
20       #:phases #~(modify-phases %standard-phases
21-                   (add-before 'build 'set-pythonpath
22-                     (lambda _
23-                       (setenv "PYTHONPATH"
24-                         (string-append (string-append (getcwd) "/api/python/backend")
25-                                      ":" (or (getenv "PYTHONPATH") "")))))
26-                  (add-after 'set-pythonpath 'change-directory
27+                   (add-before 'build 'change-directory
28                      (lambda _
29                        (chdir "api/python"))))))
30     (home-page "https://github.com/lief-project/LIEF")

UPD. For some reason, it works in a separate derivation, but not in our manifest. Therefore, ignore this diff.

fanquake commented at 12:04 pm on May 29, 2025: member

I believe there is a regression in parsing big endian PPC64 headers in LIEF: https://github.com/lief-project/LIEF/issues/1217, but I’ve added a work-around to contrib/guix/security-check.py for this.

You should be able to pull in https://github.com/lief-project/LIEF/commit/a1ec0f815fb7ddefc8b53af9f7bb990884655964.

fanquake commented at 3:21 pm on May 29, 2025: member

Actually, we can now use 0.16.6: https://github.com/lief-project/LIEF/releases/tag/0.16.6, which includes that change.

davidgumberg force-pushed on May 29, 2025

davidgumberg renamed this:
~~deps: Bump lief to 0.16.5~~
deps: Bump lief to 0.16.6
on May 29, 2025

davidgumberg commented at 2:11 am on May 30, 2025: contributor

Rebased against master to include https://github.com/bitcoin/bitcoin/pull/32634/commits/dbb2d4c3d54759f8c346882b6f98d26d5bb8e816
Bumped LIEF to 0.16.6 which resolves https://github.com/lief-project/LIEF/issues/1217
Added a workaround for https://github.com/lief-project/LIEF/pull/1218, this is acceptable and won’t need to be resolved once the upstream patch is merged, since this change tightens our MachO has_nx checks to both the stack and heap.
Added a commit to enable a previously disabled check which an upstream fix has been merged for a long time.

Marking this as ready for review.

Guix hashes

I rebased and these hashes are now invalid, redoing the build.

0$ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum

 0329bfb5ac252bd35d6a1fac4b7beb96eed963b9c745508f9e2ec27f84eb4d613  guix-build-b72926849b73/output/aarch64-linux-gnu/SHA256SUMS.part
 13e03e68e9ba0ccc90da474fcb7608b1f946d3de9edcea2e9166cc6e6ff2f5ffc  guix-build-b72926849b73/output/aarch64-linux-gnu/bitcoin-b72926849b73-aarch64-linux-gnu-debug.tar.gz
 2c1b7af10db7333a6b01c8b1f1600f4631172f9b52547d62330e75cf76401387f  guix-build-b72926849b73/output/aarch64-linux-gnu/bitcoin-b72926849b73-aarch64-linux-gnu.tar.gz
 31b5fa8c89de424267a7b382b2283d9ec4cd2c3c73bb258049ebe273089e78f02  guix-build-b72926849b73/output/arm-linux-gnueabihf/SHA256SUMS.part
 4bb9781f7cd00f454a77a3f29d28b892494ca32381b5e22f60139015771a5f5f2  guix-build-b72926849b73/output/arm-linux-gnueabihf/bitcoin-b72926849b73-arm-linux-gnueabihf-debug.tar.gz
 58ac4ddcd0dcd46667331a5b924182c49330bb9215f20ebc07ec1c99a8bb3d977  guix-build-b72926849b73/output/arm-linux-gnueabihf/bitcoin-b72926849b73-arm-linux-gnueabihf.tar.gz
 67fe75d1d0f60c53a1f07eee253f4cc61de0b515e119ac12e77637b90356c3522  guix-build-b72926849b73/output/arm64-apple-darwin/SHA256SUMS.part
 775c6694568d93c62a92e8a7e11efb4cc50ad608ab1ff09d297d7eccee8a49c9a  guix-build-b72926849b73/output/arm64-apple-darwin/bitcoin-b72926849b73-arm64-apple-darwin-codesigning.tar.gz
 84aba767d08bce80056fd3f5645a4b120f6208903684ad352c70635f1b0e7c353  guix-build-b72926849b73/output/arm64-apple-darwin/bitcoin-b72926849b73-arm64-apple-darwin-unsigned.tar.gz
 9539a69f884f288a174a09b167a7b16675aba40b2e42fe14ba3930305b7fd7118  guix-build-b72926849b73/output/arm64-apple-darwin/bitcoin-b72926849b73-arm64-apple-darwin-unsigned.zip
107807302d3f1174a46864a564bc230d0bfe7654b4721a29a3359bcc82d8a476e8  guix-build-b72926849b73/output/dist-archive/bitcoin-b72926849b73.tar.gz
11ae312d931c3ff090bb6cdcfaefcc55ae706d205a46bf8c570b76e89d0b0a3f73  guix-build-b72926849b73/output/powerpc64-linux-gnu/SHA256SUMS.part
124f5f81b5cfa848c57bb42da462f409dfb3558a229a45d732ce1ff7c2cc98f236  guix-build-b72926849b73/output/powerpc64-linux-gnu/bitcoin-b72926849b73-powerpc64-linux-gnu-debug.tar.gz
136d16058a39e5f5bd5734bcf947cba090232dc1d75f021c31a216b71245ff5d24  guix-build-b72926849b73/output/powerpc64-linux-gnu/bitcoin-b72926849b73-powerpc64-linux-gnu.tar.gz
14971e27d2a637e9f51eb1b664c044dace2bdf2e75ff2609409043ac6fdd96c42a  guix-build-b72926849b73/output/riscv64-linux-gnu/SHA256SUMS.part
15b711bf4724d4e4cec5b4c89c4d757bd22882cea9fd8602a86fee5f6685f6e8f0  guix-build-b72926849b73/output/riscv64-linux-gnu/bitcoin-b72926849b73-riscv64-linux-gnu-debug.tar.gz
16c3b2926b39d530e8f18d0be1ab036f37ed7f0afda5570994ba2b0f063674f6c3  guix-build-b72926849b73/output/riscv64-linux-gnu/bitcoin-b72926849b73-riscv64-linux-gnu.tar.gz
17b41d93d5c1f2d27a295bdffa99bac5e1edc46feb2f7da997c8cf41784f929cb5  guix-build-b72926849b73/output/x86_64-apple-darwin/SHA256SUMS.part
18154ddf968565e6400a72e94ea673dac8556a64a0e2ade9dc1fd00a3d037b0bf5  guix-build-b72926849b73/output/x86_64-apple-darwin/bitcoin-b72926849b73-x86_64-apple-darwin-codesigning.tar.gz
1977bff1e4ade35f561fd1a42d0be05ceec3e7b6b6d85296f030da4729add4ea58  guix-build-b72926849b73/output/x86_64-apple-darwin/bitcoin-b72926849b73-x86_64-apple-darwin-unsigned.tar.gz
208353c0e4eb112ba3f1dcac34f074421789a1ec546f05b1b46a36a1d7d540b349  guix-build-b72926849b73/output/x86_64-apple-darwin/bitcoin-b72926849b73-x86_64-apple-darwin-unsigned.zip
2101d6f21c57f7de5ce1723cd6d3cf51040d85b5462a16537b476c6cd270ffeff7  guix-build-b72926849b73/output/x86_64-linux-gnu/SHA256SUMS.part
222279870ee6a8f72a3db8a1bef777b52c785da3ed0da24a56dc8b3f00fac150fe  guix-build-b72926849b73/output/x86_64-linux-gnu/bitcoin-b72926849b73-x86_64-linux-gnu-debug.tar.gz
23099790f9f05959e0cdd375db46bc42451b350a2e11088a6f7a5e83e4b7d143b1  guix-build-b72926849b73/output/x86_64-linux-gnu/bitcoin-b72926849b73-x86_64-linux-gnu.tar.gz
24dfce99265caf6229b115728ca009a42e91636abe7f2719aea077e9aca81685ce  guix-build-b72926849b73/output/x86_64-w64-mingw32/SHA256SUMS.part
2585bd305df55b8bbd0518a1c9c1f0ffe373f0ce00fab08f0ebf63f17ebcc0cbcd  guix-build-b72926849b73/output/x86_64-w64-mingw32/bitcoin-b72926849b73-win64-codesigning.tar.gz
26e390a2876cca056286b14c7942fe154e77f790da52570b32096a2ae130a186e2  guix-build-b72926849b73/output/x86_64-w64-mingw32/bitcoin-b72926849b73-win64-debug.zip
274ab469d1b5a82940085cb02362f8036e99a38fb700f01bce36ba824d0a1b65a9  guix-build-b72926849b73/output/x86_64-w64-mingw32/bitcoin-b72926849b73-win64-setup-unsigned.exe
28becbb9f15416397bdc42ceba464b7880ad88579f0af1aa33483f357e1c3bcb12  guix-build-b72926849b73/output/x86_64-w64-mingw32/bitcoin-b72926849b73-win64-unsigned.zip

davidgumberg marked this as ready for review on May 30, 2025

deps: bump lief to 0.16.6

Some of the primary changes are:
- lief.EXE_FORMATS became lief.Binary.FORMATS IN 0.14.0
        - https://github.com/lief-project/LIEF/blob/494f116c6b8c9163fc4b5f6497233f18424981ee/doc/sphinx/changelog.rst?plain=1#L702
- lief.ARCHITECTURES became lief.Header.ARCHITECTURES in 0.16.0
        - https://github.com/lief-project/LIEF/blob/494f116c6b8c9163fc4b5f6497233f18424981ee/doc/sphinx/changelog.rst?plain=1#L226C18-L227C18
- lief.ELF.ARCH.x86_64 became lief.ELF.ARCH.X86_64

This commit includes a workaround for the bug fixed in
https://github.com/lief-project/LIEF/pull/1218, but the workaround can
be kept, since it makes `has_nx` checks stricter by enforcing both heap
and stack are non-executable.

This change also requires a patch to partially revert a commit to LIEF
(https://github.com/lief-project/LIEF/commit/f23ced2f4ffc170d0a6f40ff4a1bee575e3447cf)
which broke compatibility with versions of scikit-build-core <= 0.10.x.

This patch can be dropped once the guix time machine advances to or
beyond https://codeberg.org/guix/guix/commit/35c5f07e967155d2276c7ec58e5108e4da02c974,
which bumps the scikit-build-core version in guix from 0.9.3 to 0.10.7.

Co-authored-by: willcl-ark <will8clark@gmail.com>
Co-authored-by: fanquake <fanquake@gmail.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>

d64c7fd8fb

contrib: Re-enable FORTIFY check for RISCV

Skipping this check is no longer necessary since https://github.com/lief-project/LIEF/commit/ab85865f279cf02648018417ec8afa12bd0bef24
resolved the issue.

b355454739

refactor: contrib: Move FORTIFY check to BASE_ELF 240fe8754d

davidgumberg force-pushed on May 30, 2025

in contrib/guix/security-check.py:210 in 240fe8754d

206@@ -206,7 +207,10 @@ def check_NX(binary) -> bool:
207     '''
208     Check for no stack execution
209     '''
210-    return binary.has_nx
211+    if binary.format == lief.Binary.FORMATS.MACHO:

fanquake commented at 12:40 pm on May 30, 2025:

In d64c7fd8fb82e6f2d0bbf3e88b079446119e9ea5:

but the workaround can be kept, since it makes has_nx checks stricter by enforcing both heap and stack are non-executable.

If we do this, can you add a note, for why MACHO has a different check (given has_nx exists for it). It could also be worth seeing if upstream would just combine the has_nx_heap check into has_nx for MACHO?

hebasto commented at 3:24 pm on May 30, 2025: member

My Guix build:

 0aarch64
 174539281616c591d2b7157152ad5959cb7a68b9505ea3fcb74004fb19df34ca4  guix-build-240fe8754d21/output/aarch64-linux-gnu/SHA256SUMS.part
 208a9f2153670c0a07819b73694857f448baffdea276f94e2be6e24732bfd80ef  guix-build-240fe8754d21/output/aarch64-linux-gnu/bitcoin-240fe8754d21-aarch64-linux-gnu-debug.tar.gz
 370026671b2e698c28d11426d35da71422b4163b52524a3c4442f7dd2606f1c84  guix-build-240fe8754d21/output/aarch64-linux-gnu/bitcoin-240fe8754d21-aarch64-linux-gnu.tar.gz
 4376d53e484e2c5a353fb9022256e2ea64b7bff91c2d4c37ad7c0af93ff874923  guix-build-240fe8754d21/output/arm-linux-gnueabihf/SHA256SUMS.part
 5f14547ebf708ef2bb1eb95189b66311d49accacbcd4812daba51ff29f97a2758  guix-build-240fe8754d21/output/arm-linux-gnueabihf/bitcoin-240fe8754d21-arm-linux-gnueabihf-debug.tar.gz
 6c18762bf9b59aaa109b0305d32f79a7d80e7907e2e35abdb2d9d6d9d8d05965b  guix-build-240fe8754d21/output/arm-linux-gnueabihf/bitcoin-240fe8754d21-arm-linux-gnueabihf.tar.gz
 767cb395211298f632b5c0e1751a0c78e625ab938eb6070f8cc330c2e237222fe  guix-build-240fe8754d21/output/arm64-apple-darwin/SHA256SUMS.part
 884f7dc819af3beb04142047789553f873eac29dde82538e58f32e2b1bca066e0  guix-build-240fe8754d21/output/arm64-apple-darwin/bitcoin-240fe8754d21-arm64-apple-darwin-codesigning.tar.gz
 965024f81a5925a4f8110f6b242895295fbb7ff086dca1f99c1a16a8007c91bad  guix-build-240fe8754d21/output/arm64-apple-darwin/bitcoin-240fe8754d21-arm64-apple-darwin-unsigned.tar.gz
10e7b20667cc7a1472cc4da020cbf66685e7ec3c0b1f1d63d7d8737f04e0bbee55  guix-build-240fe8754d21/output/arm64-apple-darwin/bitcoin-240fe8754d21-arm64-apple-darwin-unsigned.zip
1177de042a9cbb81ccf76d547559f3872d0de45e5288092f7b9e6a76850826dbfd  guix-build-240fe8754d21/output/dist-archive/bitcoin-240fe8754d21.tar.gz
12297a295a0c01e8d577a62fd972ec0d968891755d17c5a45d50de614564aeb281  guix-build-240fe8754d21/output/powerpc64-linux-gnu/SHA256SUMS.part
1334109f1e6f20cca085bcb8a11d4ca7c34c089f6afbbfbcce3cc644f2701f20e2  guix-build-240fe8754d21/output/powerpc64-linux-gnu/bitcoin-240fe8754d21-powerpc64-linux-gnu-debug.tar.gz
14e9975759dc6ed1ea2634ad4076f90290370d5aecd987117718482122e215f638  guix-build-240fe8754d21/output/powerpc64-linux-gnu/bitcoin-240fe8754d21-powerpc64-linux-gnu.tar.gz
156b973e0ca6b29574f0f80c4266505de02d6ed858f5ab9098db184fc769783a64  guix-build-240fe8754d21/output/riscv64-linux-gnu/SHA256SUMS.part
16dd045635ae141c14c370669fc17b2b5c15d7a7c22354a8b343054a05ef4c2eda  guix-build-240fe8754d21/output/riscv64-linux-gnu/bitcoin-240fe8754d21-riscv64-linux-gnu-debug.tar.gz
176dd007bea1ef72ee94f3028b1abea0692e86056a4b0544134de647c803bb05a8  guix-build-240fe8754d21/output/riscv64-linux-gnu/bitcoin-240fe8754d21-riscv64-linux-gnu.tar.gz
186c3e51a29b64c763f7a529088b8c41e6e6f463eb4a12ffcf09882fce5160cb09  guix-build-240fe8754d21/output/x86_64-apple-darwin/SHA256SUMS.part
194c57a09b1e9cfff512a527c9b40b4dc076383022d4ed46e85f21bfbf963dffa1  guix-build-240fe8754d21/output/x86_64-apple-darwin/bitcoin-240fe8754d21-x86_64-apple-darwin-codesigning.tar.gz
20c4a99aa5a5051ae673baa6ba9512cbd3df6795c4f40ca2187925b9ecfeec0b89  guix-build-240fe8754d21/output/x86_64-apple-darwin/bitcoin-240fe8754d21-x86_64-apple-darwin-unsigned.tar.gz
21aa620c8eea8c0876e6b404226e8e71b6fbced3f8300433c03f28b4230f5f8241  guix-build-240fe8754d21/output/x86_64-apple-darwin/bitcoin-240fe8754d21-x86_64-apple-darwin-unsigned.zip
225fb402e9988e784b4587fc67a5c3d07729bf33a3b298b87b6c7f77f957105935  guix-build-240fe8754d21/output/x86_64-linux-gnu/SHA256SUMS.part
2347dbc6c6f0db5f65a9981055a6d08b12bab425d8563074405a4b5302d3d0223d  guix-build-240fe8754d21/output/x86_64-linux-gnu/bitcoin-240fe8754d21-x86_64-linux-gnu-debug.tar.gz
24c3805bcb2165a36729518dab307ff0be303967c7eba7a932002e96ae5e34bbc7  guix-build-240fe8754d21/output/x86_64-linux-gnu/bitcoin-240fe8754d21-x86_64-linux-gnu.tar.gz
2544302cce79525b1ed622eed8b616f9aca9009f0769ebcf3b41267cb8e02d0b42  guix-build-240fe8754d21/output/x86_64-w64-mingw32/SHA256SUMS.part
267bb4f33f8cbfd3b8e6923f6acd78fdfe5b1e01c4ae2cf23b2a8cb648cdd8789a  guix-build-240fe8754d21/output/x86_64-w64-mingw32/bitcoin-240fe8754d21-win64-codesigning.tar.gz
27194e63fbb72e884b0039082300f7bbe87848472faebc6561f54e43fa05cfb46e  guix-build-240fe8754d21/output/x86_64-w64-mingw32/bitcoin-240fe8754d21-win64-debug.zip
28456a05dba1c09f3c451ed820b34da1195c290369e6e634e0810bb87208951dc6  guix-build-240fe8754d21/output/x86_64-w64-mingw32/bitcoin-240fe8754d21-win64-setup-unsigned.exe
29be932477dc006a3accaa71d9c7fc02fb5a72d809892c0cb5cffb1ca1792c08f3  guix-build-240fe8754d21/output/x86_64-w64-mingw32/bitcoin-240fe8754d21-win64-unsigned.zip

hebasto commented at 8:52 pm on May 30, 2025: member

ninja is already listed as a dependency in manifest.scm, so patching it out doesn’t seem necessary:

 0--- a/contrib/guix/manifest.scm
 1+++ b/contrib/guix/manifest.scm
 2@@ -167,21 +167,13 @@ chain for " target " development."))
 3                     (url "https://github.com/lief-project/LIEF")
 4                     (commit version)))
 5               (file-name (git-file-name name version))
 6-              (modules '((guix build utils)))
 7-              (snippet
 8-               '(begin
 9-                  ;; Configure build for Python bindings.
10-                  (substitute* "api/python/config-default.toml"
11-                    (("(ninja         = )true" all m)
12-                     (string-append m "false"))
13-                    (("(parallel-jobs = )0" all m)
14-                     (string-append m (number->string (parallel-job-count)))))))
15               (sha256
16                (base32
17                 "1pq9nagrnkl1x943bqnpiyxmkd9vk99znfxiwqp6vf012b50bz2a"))
18               (patches (search-our-patches "lief-scikit-0-9.patch"))))
19     (build-system pyproject-build-system)
20     (native-inputs (list cmake-minimal
21+                         ninja
22                          python-scikit-build-core
23                          python-pydantic-core
24                          python-pydantic-2))

deps: Bump lief to 0.16.6 #32431

Code Coverage & Benchmarks

Reviews

Conflicts

Guix hashes