depends: Switch from multilib to platform-specific toolchains

hebasto commented at 1:34 pm on March 29, 2025: member

Using the multilib GCC toolchain, as currently documented in depends/README.md, has several issues, such as:

The g++-multilib package conflicts with platform-specific cross-compiler packages. This means it is not possible to cross compile for i686 and other platforms using the same set of installed packages.
The g++-multilib package is not available for arm64:

0$ sudo apt install g++-multilib
1Reading package lists... Done
2Building dependency tree... Done
3Reading state information... Done
4E: Unable to locate package g++-multilib

Managing the multilib GCC toolchain requires additional code in both depends and Guix scripts.

This PR addresses all the issues mentioned above by switching from multilib to platform-specific toolchains.

Also see #22456.

Here are examples of building for different scenarions:

Linux, x86_64 or arm64, building with depends natively:

0$ gmake -C depends -j $(nproc)
1$ cmake -B build --toolchain depends/$(./depends/config.sub $(./depends/config.guess))/toolchain.cmake
2$ cmake --build build -j $(nproc)

Linux, x86_64 or arm64, cross compiling for i686-pc-linux-gnu:

0$ sudo apt install g++-i686-linux-gnu binutils-i686-linux-gnu
1$ export HOST=i686-linux-gnu
2$ gmake -C depends -j $(nproc)
3$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
4$ cmake --build build-${HOST} -j $(nproc)

Linux, x86_64, cross compiling for arm64:

0$ sudo apt install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu
1$ export HOST=aarch64-linux-gnu
2$ gmake -C depends -j $(nproc)
3$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
4$ cmake --build build-${HOST} -j $(nproc)

Linux, arm64, cross compiling for x86_64:

0$ sudo apt install g++-x86-64-linux-gnu binutils-x86-64-linux-gnu
1$ export HOST=x86_64-linux-gnu
2$ gmake -C depends -j $(nproc)
3$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
4$ cmake --build build-${HOST} -j $(nproc)

hebasto added the label Build system on Mar 29, 2025

DrahtBot commented at 1:35 pm on March 29, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32162.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	laanwj

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#32086 (Shuffle depends instructions and recommend modern make for macOS by Sjors)
#31802 (Add bitcoin-{node,gui} to release binaries for IPC by Sjors)
#25573 ([POC] guix: produce a fully -static-pie bitcoind by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

hebasto force-pushed on Mar 29, 2025

DrahtBot added the label CI failed on Mar 29, 2025

hebasto marked this as a draft on Mar 29, 2025

hebasto force-pushed on Mar 29, 2025

hebasto marked this as ready for review on Mar 29, 2025

DrahtBot removed the label CI failed on Mar 29, 2025

in depends/README.md:30 in 1925dc1a3c outdated

34-- `aarch64-linux-gnu` for Linux ARM 64 bit
35-- `powerpc64-linux-gnu` for Linux POWER 64 bit (big endian)
36-- `powerpc64le-linux-gnu` for Linux POWER 64 bit (little endian)
37-- `riscv32-linux-gnu` for Linux RISC-V 32 bit
38-- `riscv64-linux-gnu` for Linux RISC-V 64 bit
39+- `arm-linux-gnueabihf` for Linux ARM 32-bit

laanwj commented at 7:11 am on March 31, 2025:

No strong opinion on XX-bit versus XX bit, but mind that i made the opposite change (for consistency) recently in a0c9595810c7d8bb17d8b5bea8d916db194b5239

hebasto commented at 7:26 am on March 31, 2025:

I referred to the following sources:

Happy to revert if requested.

laanwj commented at 7:57 am on March 31, 2025:

No it’s fine with me let’s just not get into an edit war 😄

in ci/test/00_setup_env_i686_multiprocess.sh:9 in 1925dc1a3c outdated

 5@@ -6,11 +6,12 @@
 6 
 7 export LC_ALL=C.UTF-8
 8 
 9-export HOST=i686-pc-linux-gnu
10+export HOST=i686-linux-gnu

laanwj commented at 7:13 am on March 31, 2025:

i’ve always been confused about the -pc- in the architecture tuple-is it directly related to use (or non-use) of multilib?

hebasto commented at 7:37 am on March 31, 2025:

It’s hard to say.

Referring to x86_64-pc-linux-gnu is still necessary when building depends natively on x86_64, because:

0$ uname -m
1x86_64
2$ ./depends/config.sub $(./depends/config.guess)
3x86_64-pc-linux-gnu

However, dropping the -pc- infix helped to with Clang’s paths in the OSS-Fuzz environment.

laanwj commented at 8:09 am on March 31, 2025:

Hrm, yea, config.sub adds it for x86 archs (but unknown for others) so it’s likely canonical somehow.

0$ ./config.sub i386-linux-gnu
1i386-pc-linux-gnu
2$ ./config.sub x86_64-linux-gnu
3x86_64-pc-linux-gnu
4$ ./config.sub arm-linux-gnueabihf
5arm-unknown-linux-gnueabihf
6$ ./config.sub riscv64-linux-gnu
7riscv64-unknown-linux-gnu

But seems fine to leave it out in the README.md because it doesn’t include the -unknown- infixes either.

hebasto commented at 8:20 am on March 31, 2025:

Also, please note that depends/README.md lists triplets after specifically mentioning “for cross compilation”. In such cases: https://github.com/bitcoin/bitcoin/blob/4c1906a500cacab385b09e780b54271b0addaf4b/depends/hosts/default.mk#L1-L3 and toolchain prefixes usually do not include the -pc- infix.

laanwj commented at 7:16 am on March 31, 2025: member

Concept ACK, multilib is more or less specific to x86, it’s better if multi-platform is handled in a more architecture-agnostic and consistent way.

DrahtBot added the label Needs rebase on Apr 2, 2025

depends: Switch from multilib to platform-specific toolchains b3ea4f94a3

hebasto force-pushed on Apr 2, 2025

hebasto commented at 2:43 pm on April 2, 2025: member

Rebased due to a conflict with the merged bitcoin/bitcoin#30997.

DrahtBot removed the label Needs rebase on Apr 2, 2025

maflcko added the label DrahtBot Guix build requested on Apr 16, 2025

DrahtBot commented at 5:54 am on April 19, 2025: contributor

Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]

File	commit 247e9de62228bd1cb0e2fb5e3bd9a906b3056167(master)	commit e1e71deac46db729a483b23055d31a60c0003b06(pull/32162/merge)
*-aarch64-linux-gnu-debug.tar.gz	`22a79eff8e36e1bf...`	`15c15579e5a5982e...`
*-aarch64-linux-gnu.tar.gz	`cbb887b1068cbcb2...`	`984fa45e0d72837f...`
*-arm-linux-gnueabihf-debug.tar.gz	`d5b1583e9e776bdb...`	`fe94c6c34bfeacd4...`
*-arm-linux-gnueabihf.tar.gz	`067ec12478462110...`	`fe1f3ec4a8c78b41...`
*-arm64-apple-darwin-codesigning.tar.gz	`dd98a2df7a886aa2...`	`dce09a1f2e9e297a...`
*-arm64-apple-darwin-unsigned.tar.gz	`578d53467f1964e4...`	`e1d8f1e3a3f04f6e...`
*-arm64-apple-darwin-unsigned.zip	`f7324f4fbf04094a...`	`4ccfe1b8ee80835f...`
*-powerpc64-linux-gnu-debug.tar.gz	`4d1e9e946f74f4de...`	`8748afd25ef3b857...`
*-powerpc64-linux-gnu.tar.gz	`95073776ba9e496f...`	`7b7ade31f33089c7...`
*-riscv64-linux-gnu-debug.tar.gz	`b9aa3f6b161a9a22...`	`77731a8aaf459437...`
*-riscv64-linux-gnu.tar.gz	`4359303e00e93dd6...`	`71e3fe87a310e406...`
*-x86_64-apple-darwin-codesigning.tar.gz	`796e72140cca465e...`	`e293e011394b3620...`
*-x86_64-apple-darwin-unsigned.tar.gz	`cd0a846310165114...`	`4e57320428a685bb...`
*-x86_64-apple-darwin-unsigned.zip	`3f81d9630b346346...`	`4ba511e9d082d14c...`
*-x86_64-linux-gnu-debug.tar.gz	`b9600b433f60e512...`	`729ce3b41986971a...`
*-x86_64-linux-gnu.tar.gz	`051c8ae1c97f699f...`	`68598760c55ab395...`
*.tar.gz	`a4f9396cd4278eca...`	`11cff29d34c63af4...`
SHA256SUMS.part	`e34ac75fc6f95298...`	`7c1b2a222dbfe423...`
guix_build.log	`295cf795a514f994...`	`d288c9663291bf72...`
guix_build.log.diff		`181053a7600bf88e...`

DrahtBot removed the label DrahtBot Guix build requested on Apr 19, 2025

depends: Switch from multilib to platform-specific toolchains #32162

Code Coverage & Benchmarks

Reviews

Conflicts

Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]