guix: Always canonicalize HOST using `./depends/config.sub`

dongcarl commented at 8:26 pm on April 13, 2021: member

Closes #21347

Note: this is probably not absolutely necessary for Guix release, but it’s nice to have!

dongcarl added the label Build system on Apr 13, 2021

fanquake added this to the "Next (Not based on any other PRs)" column in a project

fanquake commented at 2:13 am on April 14, 2021: member

Concept ACK. We should also update the HOSTS listing in the Guix README.

dongcarl force-pushed on Apr 14, 2021

dongcarl commented at 2:13 pm on April 14, 2021: member

Pushed c9ea4c79c3dfb6418bc3a5b6533d81ac94793e65 -> 9e4e638750fb17abb173aae2f6a33d838befd7c3

Addressed: #21671 (comment)

laanwj commented at 3:05 pm on April 14, 2021: member

Concept ACK. Using canonicalized architecture tuples makes sense, I guess, it’s good to rule out ambiguity when the architecture (at least the part that we care about) is the same but have slightly different tuples. I think the whole thing (architecture tuples in general, I mean) is a mess but that’s how it is.

DrahtBot commented at 7:25 am on April 29, 2021: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#22392 by fanquake
#22365 by dongcarl
#22237 by dgoncharov
#21711 by dongcarl

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

dongcarl force-pushed on May 3, 2021

dongcarl force-pushed on May 4, 2021

dongcarl commented at 3:50 pm on May 4, 2021: member

Updated to be based on: #21799

guix: Build fixes for HOST canonicalization

1. Bump time-machine to a commit that includes:

   commit 58452d08ff3e0044e9a32e6d5b3663cf185d8b33
   Author: Carl Dong <contact@carldong.me>
   Date:   Sun Mar 28 10:51:16 2021 -0700

       gnu: cross-base: Relax check for powerpc64le.

       * gnu/packages/cross-base.scm (cross-gcc-arguments): When conditionally adding
       "--with-long-double-128", check for "powerpc64le-" prefix instead of matching
       full target.

       Signed-off-by: Chris Marusich <cmmarusich@gmail.com>

2. Explicitly supply CC_BUILD to freetype's configure invocation,
   otherwise it erroneously use CC for the target build if it thinks
   that host=build. However, when Guix-building, even if host=build, we
   need to use a different CC for the target, as the Guix's default
   GCC's specfile (which inserts a lot of rpaths) is not suitable for
   our purposes.

dfaa9b33c4

guix: Always canonicalize HOST

We need to specify the qt_config_opts_aarch64_linux because the default
Qt platform profile for aarch64 assumes that gcc is named
aarch64-linux-gnu-gcc, but it's actually named
aarch64-unknown-linx-gnu-gcc if platform triples are correctly
canonicalized.

aa0f832835

dongcarl force-pushed on May 5, 2021

dongcarl marked this as ready for review on May 5, 2021

dongcarl commented at 7:42 pm on May 7, 2021: member

Pushed 69d4d9ca9b5d51ef4db04d486333aefab2fdf2e1 -> aa0f83283504c69b57ea20f4cb127c4ec4d993f3

Rebased after merge of #21799

laanwj commented at 12:48 pm on June 3, 2021: member

Code review ACK aa0f83283504c69b57ea20f4cb127c4ec4d993f3

fanquake commented at 6:00 am on June 10, 2021: member

Tested that hosts are properly passed through, i.e x86_64-linux-gnu becomes x86_64-pc-linux-gnu. However this doesn’t build the x86_64-pc-linux-gnu host for me (aa0f83283504c69b57ea20f4cb127c4ec4d993f3):

 0time HOSTS="x86_64-pc-linux-gnu" BASE_CACHE="/guix/base_cache" SOURCES_PATH="/guix/sources" SDK_PATH="/guix/SDKs" ./contrib/guix/guix-build
 1...
 2INFO: Building aa0f83283504 for platform triple x86_64-pc-linux-gnu:
 3      ...using reference timestamp: 1618345007
 4      ...running at most 8 jobs
 5      ...from worktree directory: '/bitcoin'
 6          ...bind-mounted in container to: '/bitcoin'
 7      ...in build directory: '/bitcoin/guix-build-aa0f83283504/distsrc-aa0f83283504-x86_64-pc-linux-gnu'
 8          ...bind-mounted in container to: '/distsrc-base/distsrc-aa0f83283504-x86_64-pc-linux-gnu'
 9      ...outputting in: '/bitcoin/guix-build-aa0f83283504/output/x86_64-pc-linux-gnu'
10          ...bind-mounted in container to: '/outdir-base/x86_64-pc-linux-gnu'
11Required environment variables as seen inside the container:
12    DIST_ARCHIVE_BASE: /outdir-base/dist-archive
13    DISTNAME: bitcoin-aa0f83283504
14    HOST: x86_64-pc-linux-gnu
15    SOURCE_DATE_EPOCH: 1618345007
16    JOBS: 8
17    DISTSRC: /distsrc-base/distsrc-aa0f83283504-x86_64-pc-linux-gnu
18    OUTDIR: /outdir-base/x86_64-pc-linux-gnu
19make: Entering directory '/bitcoin/depends'
20...
21checking whether we are cross compiling... configure: error: in `/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119':
22configure: error: cannot run C compiled programs.
23If you meant to cross compile, use `--host'.
24See `config.log' for more details
25make: *** [funcs.mk:282: /bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119/./.stamp_configured] Error 1
26make: Leaving directory '/bitcoin/depends'

config.log

 0configure:2676: checking for a BSD-compatible install
 1configure:2744: result: /root/.guix-profile/bin/install -c
 2configure:2755: checking whether build environment is sane
 3configure:2810: result: yes
 4configure:2959: checking for a thread-safe mkdir -p
 5configure:2998: result: /root/.guix-profile/bin/mkdir -p
 6configure:3005: checking for gawk
 7configure:3021: found /root/.guix-profile/bin/gawk
 8configure:3032: result: gawk
 9configure:3043: checking whether make sets $(MAKE)
10configure:3065: result: yes
11configure:3094: checking whether make supports nested variables
12configure:3111: result: yes
13configure:3248: checking whether make supports nested variables
14configure:3265: result: yes
15configure:3291: checking whether make supports the include directive
16configure:3306: make -f confmf.GNU && cat confinc.out
17make[1]: Entering directory '/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119'
18make[1]: Leaving directory '/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119'
19this is the am__doit target
20configure:3309: $? = 0
21configure:3328: result: yes (GNU style)
22configure:3358: checking for x86_64-pc-linux-gnu-gcc
23configure:3385: result: x86_64-pc-linux-gnu-gcc
24configure:3654: checking for C compiler version
25configure:3663: x86_64-pc-linux-gnu-gcc --version >&5
26x86_64-pc-linux-gnu-gcc (GCC) 8.4.0
27Copyright (C) 2018 Free Software Foundation, Inc.
28This is free software; see the source for copying conditions.  There is NO
29warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
30
31configure:3674: $? = 0
32configure:3663: x86_64-pc-linux-gnu-gcc -v >&5
33Using built-in specs.
34COLLECT_GCC=x86_64-pc-linux-gnu-gcc
35COLLECT_LTO_WRAPPER=/gnu/store/jyg0s2g2gil3w8d7ym2i6lhz45cj5ncx-gcc-cross-x86_64-pc-linux-gnu-8.4.0/libexec/gcc/x86_64-pc-linux-gnu/8.4.0/lto-wrapper
36Target: x86_64-pc-linux-gnu
37Configured with: 
38Thread model: posix
39gcc version 8.4.0 (GCC) 
40configure:3674: $? = 0
41configure:3663: x86_64-pc-linux-gnu-gcc -V >&5
42x86_64-pc-linux-gnu-gcc: error: unrecognized command line option '-V'
43x86_64-pc-linux-gnu-gcc: fatal error: no input files
44compilation terminated.
45configure:3674: $? = 1
46configure:3663: x86_64-pc-linux-gnu-gcc -qversion >&5
47x86_64-pc-linux-gnu-gcc: error: unrecognized command line option '-qversion'; did you mean '--version'?
48x86_64-pc-linux-gnu-gcc: fatal error: no input files
49compilation terminated.
50configure:3674: $? = 1
51configure:3694: checking whether the C compiler works
52configure:3716: x86_64-pc-linux-gnu-gcc -pipe -O2   -I/bitcoin/depends/x86_64-pc-linux-gnu/include   -L/bitcoin/depends/x86_64-pc-linux-gnu/lib conftest.c  >&5
53configure:3720: $? = 0
54configure:3768: result: yes
55configure:3771: checking for C compiler default output file name
56configure:3773: result: a.out
57configure:3779: checking for suffix of executables
58configure:3786: x86_64-pc-linux-gnu-gcc -o conftest -pipe -O2   -I/bitcoin/depends/x86_64-pc-linux-gnu/include   -L/bitcoin/depends/x86_64-pc-linux-gnu/lib conftest.c  >&5
59configure:3790: $? = 0
60configure:3812: result: 
61configure:3834: checking whether we are cross compiling
62configure:3842: x86_64-pc-linux-gnu-gcc -o conftest -pipe -O2   -I/bitcoin/depends/x86_64-pc-linux-gnu/include   -L/bitcoin/depends/x86_64-pc-linux-gnu/lib conftest.c  >&5
63configure:3846: $? = 0
64configure:3853: ./conftest
65./conftest: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
66configure:3857: $? = 127
67configure:3864: error: in `/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119':
68configure:3866: error: cannot run C compiled programs.
69If you meant to cross compile, use `--host'.
70See `config.log' for more details

All other hosts seem to build fine.

dongcarl commented at 2:59 pm on June 15, 2021: member

Very curious as I did not encounter this before (or perhaps I did and did not notice?)

I will investigate further but the following patch fixes the problem (not in any way that we’d want in our repo), which is interesting.

 0diff --git a/depends/funcs.mk b/depends/funcs.mk
 1index 34a030fab7..e5fcb352a8 100644
 2--- a/depends/funcs.mk
 3+++ b/depends/funcs.mk
 4@@ -146,7 +146,7 @@ $(1)_stage_env+=PATH=$(build_prefix)/bin:$(PATH)
 5 # config.guess, which is what we set it too here. This also quells autoconf
 6 # warnings, "If you wanted to set the --build type, don't use --host.",
 7 # when using versions older than 2.70.
 8-$(1)_autoconf=./configure --build=$(BUILD) --host=$($($(1)_type)_host) --prefix=$($($(1)_type)_prefix) $$($(1)_config_opts) CC="$$($(1)_cc)" CXX="$$($(1)_cxx)"
 9+$(1)_autoconf=./configure --build=$(BUILD) --host=$($($(1)_type)_host) --prefix=$($($(1)_type)_prefix) $$($(1)_config_opts) CC="$$($(1)_cc)" CXX="$$($(1)_cxx)" cross_compiling=yes
10 ifneq ($($(1)_nm),)
11 $(1)_autoconf += NM="$$($(1)_nm)"
12 endif

depends: Recognize and pass AUTOCONFFLAGS 754fd2bbbe

guix: Supply cross_compiling=yes to Autoconf

Abstractly, our Guix builds do not treat native-builds and cross-builds
differently. We always build 2 toolchains:

1. A native toolchain (in depends terms: build) (namely, the
   `gcc-toolchain` package, with the command line name being simply
   `gcc`), and
2. A cross toolchain (in depends terms: host) (namely, packages returned
   by the `make-cross-toolchain` procedure in `manifest.scm`, with the
   command line name being something like `x86_64-pc-linux-gnu-gcc`)

This means that even if we are technically not cross
compiling (`build==host`), we do not coalesce or reuse the toolchains in
any way. This is necessary because our release binaries (produced by the
cross toolchain) are built to:

1. Have a dependency on a minimal number of shared libraries
2. Expect the dynamic linker/loader `ld-linux.so(8)` to be somewhere
   like `/lib{,64}/ld*`

This is not the case for binaries produced by Guix's native
`gcc-toolchain` package. They are built to work best in Guix containers,
where the binaries reach for libraries and `ld-linux.so(8)` in
`/gnu/store` and have dependencies recorded as RUNPATHs in their dynamic
section (since paths are deterministic).

In other words, Guix's native `gcc-toolchain` package can be thought of
as a different platform triple: `x86_64-guix-linux-gnu` (PLEASE don't
actually use this triple this is not standard just a way to think about
it).

We can therefore think of performing a Guix build on `x86_64` for
`x86_64` as a cross-compilation with build=`x86_64-guix-linux-gnu` and
host=`x86_64-pc-linux-gnu` since even when the architecture and OS
matches, binaries built with the native toolchain will not run on normal
distros, and binaries built with the cross toolchain will not run in
Guix environments.

---

With this context in mind, we can now discuss our particular issue.

Prior to canonicalizing `$HOST` (this PR), autoconf's barely-working
cross compilation detection (which is marked `# FIXME: To remove some
day.`) set `cross_compiling=yes` if `$host` is not the exact same string
as `$build`:
http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=blob;f=lib/autoconf/general.m4;h=053130b3d0ba7af8b9177da6ff850d8c20ac6cbe;hb=HEAD#l972.
Which actually worked in our favor since we were not canonicalizing
`host` and specifying `build=x86_64-pc-linux-gnu` and
`host=x86_64-linux-gnu`, and the missing `-pc-` made autoconf assume
that it was a cross-compilation.

After canonicalizing `$HOST` autoconf instead sees that `$build==$host`
and sets `cross_compiling=no`, which leads to later configure tests to
compile `conftest` binaries using `CC` (namely, our cross-toolchain
`x86_64-pc-linux-gnu-gcc`) and attempt to run them inside the Guix
container. This leads to the problems fanquake identified here:
https://github.com/bitcoin/bitcoin/pull/21671#issuecomment-858331108

The solution is to be explicit and supply `cross_compiling=yes` to
`./configure`, which is us saying: "yes i'm on the same architecture,
OS, and userspace as the target, but there are important differences in
what we want in our dynamic sections which lead to binaries not being
able to find the right libraries to load if they were copied to the
other environment so just assume that I'm cross compiling and don't try
to be smart"

d0e2378c1a

dongcarl commented at 6:11 pm on June 18, 2021: member

Pushed aa0f832835..d0e2378c1a

Wrote up a description of my investigation

Abstractly, our Guix builds do not treat native-builds and cross-builds differently. We always build 2 toolchains:

A native toolchain (in depends terms: build) (namely, the gcc-toolchain package, with the command line name being simply gcc), and
A cross toolchain (in depends terms: host) (namely, packages returned by the make-cross-toolchain procedure in manifest.scm, with the command line name being something like x86_64-pc-linux-gnu-gcc)

This means that even if we are technically not cross compiling (build==host), we do not coalesce or reuse the toolchains in any way. This is necessary because our release binaries (produced by the cross toolchain) are built to:

Have a dependency on a minimal number of shared libraries
Expect the dynamic linker/loader ld-linux.so(8) to be somewhere like /lib{,64}/ld*

This is not the case for binaries produced by Guix’s native gcc-toolchain package. They are built to work best in Guix containers, where the binaries reach for libraries and ld-linux.so(8) in /gnu/store and have dependencies recorded as RUNPATHs in their dynamic section (since paths are deterministic).

In other words, Guix’s native gcc-toolchain package can be thought of as a different platform triple: x86_64-guix-linux-gnu (PLEASE don’t actually use this triple this is not standard just a way to think about it).

We can therefore think of performing a Guix build on x86_64 for x86_64 as a cross-compilation with build=x86_64-guix-linux-gnu and host=x86_64-pc-linux-gnu since even when the architecture and OS matches, binaries built with the native toolchain will not run on normal distros, and binaries built with the cross toolchain will not run in Guix environments.

With this context in mind, we can now discuss our particular issue.

Prior to canonicalizing $HOST (this PR), autoconf’s barely-working cross compilation detection (which is marked # FIXME: To remove some day.) set cross_compiling=yes if $host is not the exact same string as $build: http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=blob;f=lib/autoconf/general.m4;h=053130b3d0ba7af8b9177da6ff850d8c20ac6cbe;hb=HEAD#l972. Which actually worked in our favor since we were not canonicalizing host and specifying build=x86_64-pc-linux-gnu and host=x86_64-linux-gnu, and the missing -pc- made autoconf assume that it was a cross-compilation.

After canonicalizing $HOST autoconf instead sees that $build==$host and sets cross_compiling=no, which leads to later configure tests to compile conftest binaries using CC (namely, our cross-toolchain x86_64-pc-linux-gnu-gcc) and attempt to run them inside the Guix container. This leads to the problems fanquake identified here: #21671 (comment)

The solution is to be explicit and supply cross_compiling=yes to ./configure, which is us saying: “yes i’m on the same architecture, OS, and userspace as the target, but there are important differences in what we want in our dynamic sections which lead to binaries not being able to find the right libraries to load if they were copied to the other environment so just assume that I’m cross compiling and don’t try to be smart”

DrahtBot commented at 4:40 pm on July 5, 2021: member

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a “draft”.

DrahtBot added the label Needs rebase on Jul 5, 2021

DrahtBot commented at 1:06 pm on March 21, 2022: member

Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.

dongcarl closed this on Mar 21, 2022

dongcarl added the label Up for grabs on Mar 21, 2022

fanquake referenced this in commit c68af6d498 on Aug 4, 2022

fanquake referenced this in commit 10717c68a9 on Aug 11, 2022

DrahtBot locked this on Mar 21, 2023

guix: Always canonicalize HOST using ./depends/config.sub #21671

Conflicts

guix: Always canonicalize HOST using `./depends/config.sub` #21671