depends: Switch from multilib to platform-specific toolchains #32162

pull hebasto wants to merge 1 commits into bitcoin:master from hebasto:250329-cross changing 4 files +29 −49
  1. hebasto commented at 1:34 pm on March 29, 2025: member

    Using the multilib GCC toolchain, as currently documented in depends/README.md, has several issues, such as:

    1. The g++-multilib package conflicts with platform-specific cross-compiler packages. This means it is not possible to cross compile for i686 and other platforms using the same set of installed packages.

    2. The g++-multilib package is not available for arm64:

    0$ sudo apt install g++-multilib
    1Reading package lists... Done
    2Building dependency tree... Done
    3Reading state information... Done
    4E: Unable to locate package g++-multilib
    
    1. Managing the multilib GCC toolchain requires additional code in both depends and Guix scripts.

    This PR addresses all the issues mentioned above by switching from multilib to platform-specific toolchains.

    Also see #22456.


    Here are examples of building for different scenarions:

    • Linux, x86_64 or arm64, building with depends natively:
    0$ gmake -C depends -j $(nproc)
    1$ cmake -B build --toolchain depends/$(./depends/config.sub $(./depends/config.guess))/toolchain.cmake
    2$ cmake --build build -j $(nproc)
    
    • Linux, x86_64 or arm64, cross compiling for i686-pc-linux-gnu:
    0$ sudo apt install g++-i686-linux-gnu binutils-i686-linux-gnu
    1$ export HOST=i686-linux-gnu
    2$ gmake -C depends -j $(nproc)
    3$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
    4$ cmake --build build-${HOST} -j $(nproc)
    
    • Linux, x86_64, cross compiling for arm64:
    0$ sudo apt install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu
    1$ export HOST=aarch64-linux-gnu
    2$ gmake -C depends -j $(nproc)
    3$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
    4$ cmake --build build-${HOST} -j $(nproc)
    
    • Linux, arm64, cross compiling for x86_64:
    0$ sudo apt install g++-x86-64-linux-gnu binutils-x86-64-linux-gnu
    1$ export HOST=x86_64-linux-gnu
    2$ gmake -C depends -j $(nproc)
    3$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
    4$ cmake --build build-${HOST} -j $(nproc)
    
  2. hebasto added the label Build system on Mar 29, 2025
  3. DrahtBot commented at 1:35 pm on March 29, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32162.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK laanwj

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #32262 (build: Restore cross-compilation for Android by hebasto)
    • #31802 (Add bitcoin-{node,gui} to release binaries for IPC by Sjors)
    • #25573 ([POC] guix: produce a fully -static-pie bitcoind by fanquake)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  4. hebasto force-pushed on Mar 29, 2025
  5. DrahtBot added the label CI failed on Mar 29, 2025
  6. hebasto marked this as a draft on Mar 29, 2025
  7. hebasto force-pushed on Mar 29, 2025
  8. hebasto force-pushed on Mar 29, 2025
  9. hebasto marked this as ready for review on Mar 29, 2025
  10. DrahtBot removed the label CI failed on Mar 29, 2025
  11. in depends/README.md:113 in 1925dc1a3c outdated
    34-- `aarch64-linux-gnu` for Linux ARM 64 bit
    35-- `powerpc64-linux-gnu` for Linux POWER 64 bit (big endian)
    36-- `powerpc64le-linux-gnu` for Linux POWER 64 bit (little endian)
    37-- `riscv32-linux-gnu` for Linux RISC-V 32 bit
    38-- `riscv64-linux-gnu` for Linux RISC-V 64 bit
    39+- `arm-linux-gnueabihf` for Linux ARM 32-bit
    


    laanwj commented at 7:11 am on March 31, 2025:
    No strong opinion on XX-bit versus XX bit, but mind that i made the opposite change (for consistency) recently in a0c9595810c7d8bb17d8b5bea8d916db194b5239

    hebasto commented at 7:26 am on March 31, 2025:

    I referred to the following sources:

    Happy to revert if requested.


    laanwj commented at 7:57 am on March 31, 2025:
    No it’s fine with me let’s just not get into an edit war 😄
  12. in ci/test/00_setup_env_i686_multiprocess.sh:9 in 1925dc1a3c outdated
     5@@ -6,11 +6,12 @@
     6 
     7 export LC_ALL=C.UTF-8
     8 
     9-export HOST=i686-pc-linux-gnu
    10+export HOST=i686-linux-gnu
    


    laanwj commented at 7:13 am on March 31, 2025:
    i’ve always been confused about the -pc- in the architecture tuple-is it directly related to use (or non-use) of multilib?

    hebasto commented at 7:37 am on March 31, 2025:

    It’s hard to say.

    Referring to x86_64-pc-linux-gnu is still necessary when building depends natively on x86_64, because:

    0$ uname -m
    1x86_64
    2$ ./depends/config.sub $(./depends/config.guess)
    3x86_64-pc-linux-gnu
    

    However, dropping the -pc- infix helped to with Clang’s paths in the OSS-Fuzz environment.


    laanwj commented at 8:09 am on March 31, 2025:

    Hrm, yea, config.sub adds it for x86 archs (but unknown for others) so it’s likely canonical somehow.

    0$ ./config.sub i386-linux-gnu
    1i386-pc-linux-gnu
    2$ ./config.sub x86_64-linux-gnu
    3x86_64-pc-linux-gnu
    4$ ./config.sub arm-linux-gnueabihf
    5arm-unknown-linux-gnueabihf
    6$ ./config.sub riscv64-linux-gnu
    7riscv64-unknown-linux-gnu
    

    But seems fine to leave it out in the README.md because it doesn’t include the -unknown- infixes either.


    hebasto commented at 8:20 am on March 31, 2025:
    Also, please note that depends/README.md lists triplets after specifically mentioning “for cross compilation”. In such cases: https://github.com/bitcoin/bitcoin/blob/4c1906a500cacab385b09e780b54271b0addaf4b/depends/hosts/default.mk#L1-L3 and toolchain prefixes usually do not include the -pc- infix.
  13. laanwj commented at 7:16 am on March 31, 2025: member
    Concept ACK, multilib is more or less specific to x86, it’s better if multi-platform is handled in a more architecture-agnostic and consistent way.
  14. DrahtBot added the label Needs rebase on Apr 2, 2025
  15. hebasto force-pushed on Apr 2, 2025
  16. hebasto commented at 2:43 pm on April 2, 2025: member
    Rebased due to a conflict with the merged bitcoin/bitcoin#30997.
  17. DrahtBot removed the label Needs rebase on Apr 2, 2025
  18. maflcko added the label DrahtBot Guix build requested on Apr 16, 2025
  19. DrahtBot commented at 5:54 am on April 19, 2025: contributor

    Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]

    File commit 247e9de62228bd1cb0e2fb5e3bd9a906b3056167(master) commit e1e71deac46db729a483b23055d31a60c0003b06(pull/32162/merge)
    *-aarch64-linux-gnu-debug.tar.gz 22a79eff8e36e1bf... 15c15579e5a5982e...
    *-aarch64-linux-gnu.tar.gz cbb887b1068cbcb2... 984fa45e0d72837f...
    *-arm-linux-gnueabihf-debug.tar.gz d5b1583e9e776bdb... fe94c6c34bfeacd4...
    *-arm-linux-gnueabihf.tar.gz 067ec12478462110... fe1f3ec4a8c78b41...
    *-arm64-apple-darwin-codesigning.tar.gz dd98a2df7a886aa2... dce09a1f2e9e297a...
    *-arm64-apple-darwin-unsigned.tar.gz 578d53467f1964e4... e1d8f1e3a3f04f6e...
    *-arm64-apple-darwin-unsigned.zip f7324f4fbf04094a... 4ccfe1b8ee80835f...
    *-powerpc64-linux-gnu-debug.tar.gz 4d1e9e946f74f4de... 8748afd25ef3b857...
    *-powerpc64-linux-gnu.tar.gz 95073776ba9e496f... 7b7ade31f33089c7...
    *-riscv64-linux-gnu-debug.tar.gz b9aa3f6b161a9a22... 77731a8aaf459437...
    *-riscv64-linux-gnu.tar.gz 4359303e00e93dd6... 71e3fe87a310e406...
    *-x86_64-apple-darwin-codesigning.tar.gz 796e72140cca465e... e293e011394b3620...
    *-x86_64-apple-darwin-unsigned.tar.gz cd0a846310165114... 4e57320428a685bb...
    *-x86_64-apple-darwin-unsigned.zip 3f81d9630b346346... 4ba511e9d082d14c...
    *-x86_64-linux-gnu-debug.tar.gz b9600b433f60e512... 729ce3b41986971a...
    *-x86_64-linux-gnu.tar.gz 051c8ae1c97f699f... 68598760c55ab395...
    *.tar.gz a4f9396cd4278eca... 11cff29d34c63af4...
    SHA256SUMS.part e34ac75fc6f95298... 7c1b2a222dbfe423...
    guix_build.log 295cf795a514f994... d288c9663291bf72...
    guix_build.log.diff 181053a7600bf88e...
  20. DrahtBot removed the label DrahtBot Guix build requested on Apr 19, 2025
  21. DrahtBot added the label CI failed on Apr 29, 2025
  22. DrahtBot removed the label CI failed on May 2, 2025
  23. DrahtBot added the label Needs rebase on May 6, 2025
  24. hebasto force-pushed on May 6, 2025
  25. hebasto commented at 9:38 pm on May 6, 2025: member
    Rebased due to a conflict with the merged bitcoin/bitcoin#32086.
  26. in depends/README.md:138 in 3819d12b5c outdated
    132@@ -133,29 +133,35 @@ For more information, see [SDK Extraction](../contrib/macdeploy/README.md#sdk-ex
    133 
    134     apt install g++-mingw-w64-x86-64-posix
    135 
    136-#### For linux (including i386, ARM) cross compilation
    137+#### For Linux cross compilation
    138 
    139-Common linux dependencies:
    140+Please note that package availability might depend on your arch+OS your are building on.
    


    maflcko commented at 9:52 pm on May 6, 2025:
    0Please note that package availability might depend on your arch+OS you are building on.
    

    hebasto commented at 10:05 pm on May 6, 2025:
    Thanks! Fixed.
  27. maflcko added the label DrahtBot Guix build requested on May 6, 2025
  28. depends: Switch from multilib to platform-specific toolchains 5f061f5abd
  29. hebasto force-pushed on May 6, 2025
  30. DrahtBot removed the label Needs rebase on May 6, 2025
  31. DrahtBot commented at 12:43 pm on May 7, 2025: contributor

    Guix builds (on x86_64) [untrusted test-only build, possibly unsafe, not for production use]

    File commit 59d3e4ed34eb55cb40928d524cb0bd5e183ed85a(master) commit 76a1cebc2e6b3bbdd13b771ed1f801ca04ea48b0(pull/32162/merge)
    *-aarch64-linux-gnu-debug.tar.gz e0193ecd1871d425... 7a82e5a9fe68f727...
    *-aarch64-linux-gnu.tar.gz 6e3bb180153dc291... 90e05fbc4cb05a53...
    *-arm-linux-gnueabihf-debug.tar.gz 3291410869705bfe... 0f35cc75eb515270...
    *-arm-linux-gnueabihf.tar.gz cf1082edfd687943... 1b0242183b303c38...
    *-arm64-apple-darwin-codesigning.tar.gz 9df0349e64311daf... 39b930c018e5f352...
    *-arm64-apple-darwin-unsigned.tar.gz 8bbc34ab226fd025... 950b27fc25cea644...
    *-arm64-apple-darwin-unsigned.zip c3807d59f991285a... 696d7aa5459a6cb1...
    *-powerpc64-linux-gnu-debug.tar.gz 2e1f34ac367034c5... 04875bec78b1a74f...
    *-powerpc64-linux-gnu.tar.gz e6c66e531c536299... a56f0391c58193d8...
    *-riscv64-linux-gnu-debug.tar.gz 86c5575bd0883e42... 36fb9ae99e4f6148...
    *-riscv64-linux-gnu.tar.gz e7566fe0b4ece149... 3e0bcb95d23aace7...
    *-x86_64-apple-darwin-codesigning.tar.gz 5c36d57f3c69ca68... 21a45c63587412db...
    *-x86_64-apple-darwin-unsigned.tar.gz afdf8c906ae02d95... a274ba4b177f267c...
    *-x86_64-apple-darwin-unsigned.zip d475a6afead07841... 91c95e729938e18f...
    *-x86_64-linux-gnu-debug.tar.gz 8a8a4b4d829a1271... 05c753dad68740e0...
    *-x86_64-linux-gnu.tar.gz f38000b830dfc210... 2b80c1a0a35e177b...
    *.tar.gz 7f8bff0f749c6e47... c5cf6251153ddd87...
    SHA256SUMS.part e173338664f1a383... 60dffb2b65e4d6b0...
    guix_build.log 44abee820cfd68a6... b9ede0afdbf80413...
    guix_build.log.diff 9f1908318d7a3496...
  32. DrahtBot removed the label DrahtBot Guix build requested on May 7, 2025
  33. hebasto commented at 3:37 pm on May 7, 2025: member

    My Guix build:

     0aarch64
     144c90cd188bf6f78d06831c61d74b141cdefb12d2a786b4d65235b310862ccff  guix-build-5f061f5abdee/output/aarch64-linux-gnu/SHA256SUMS.part
     2347230a10b4f1a169442f41ff6ebdd09565d9b6239e8894e653fd08f16d6f9ca  guix-build-5f061f5abdee/output/aarch64-linux-gnu/bitcoin-5f061f5abdee-aarch64-linux-gnu-debug.tar.gz
     3cee271b441733749e6070ece4b14dd2090d3a273ebc86d5331baf8c991bdd8fe  guix-build-5f061f5abdee/output/aarch64-linux-gnu/bitcoin-5f061f5abdee-aarch64-linux-gnu.tar.gz
     49f7c97483735be92ea37bf91e0f33f4296430dedee90ae75a9a87132e5850d7d  guix-build-5f061f5abdee/output/arm-linux-gnueabihf/SHA256SUMS.part
     54fe25a0ec3605ae368b9d25c98e252befd815dc49dececd7af1c653738f7305a  guix-build-5f061f5abdee/output/arm-linux-gnueabihf/bitcoin-5f061f5abdee-arm-linux-gnueabihf-debug.tar.gz
     691bb88b50bcb23ca00af5721f9442a93a00aa7018e86e86ae1b2bfcc13379153  guix-build-5f061f5abdee/output/arm-linux-gnueabihf/bitcoin-5f061f5abdee-arm-linux-gnueabihf.tar.gz
     72f96e499b4c750da2c632929e26f93c7739847c48d8c9c07c060a0d1ba4ac08f  guix-build-5f061f5abdee/output/arm64-apple-darwin/SHA256SUMS.part
     809d94ae1ce764d556b6c6d20e726ac65d15c0725caae97566c4dca6057d17ae3  guix-build-5f061f5abdee/output/arm64-apple-darwin/bitcoin-5f061f5abdee-arm64-apple-darwin-codesigning.tar.gz
     96181faca58ca5089cc5d34e59630d75b8f5e46ac9bae584a5bd6175b3ce2ca07  guix-build-5f061f5abdee/output/arm64-apple-darwin/bitcoin-5f061f5abdee-arm64-apple-darwin-unsigned.tar.gz
    103980b61031975e029877d63cece663dccfad5c9733418c7158ad4251146bd48f  guix-build-5f061f5abdee/output/arm64-apple-darwin/bitcoin-5f061f5abdee-arm64-apple-darwin-unsigned.zip
    11d3712f5f362328eeb9850e30356d0e06af488e8ebf257b9a8d57d7d96a38ed60  guix-build-5f061f5abdee/output/dist-archive/bitcoin-5f061f5abdee.tar.gz
    12a120d84e90db95c8e64417f45a97e45c0c0b2d4f7dcb2de8f1ee8fdd8d8d5428  guix-build-5f061f5abdee/output/powerpc64-linux-gnu/SHA256SUMS.part
    13cda0f3b011d221785b16a744e253a94d715f7b5ec3eec6ac087ec62dc37405cc  guix-build-5f061f5abdee/output/powerpc64-linux-gnu/bitcoin-5f061f5abdee-powerpc64-linux-gnu-debug.tar.gz
    14829efde0993e381d901c199bdcf26551a11fc78b41467b6266cecba4250c78bc  guix-build-5f061f5abdee/output/powerpc64-linux-gnu/bitcoin-5f061f5abdee-powerpc64-linux-gnu.tar.gz
    151d7cd1af1791ba2726666ae3c3e7fc8d7ca80882077522135c8b1853685c84ab  guix-build-5f061f5abdee/output/riscv64-linux-gnu/SHA256SUMS.part
    16ccf76fd7c0e01d53a5fe5f345ca7ede783447bf99ad8f70c50ea5b2fe1f747a1  guix-build-5f061f5abdee/output/riscv64-linux-gnu/bitcoin-5f061f5abdee-riscv64-linux-gnu-debug.tar.gz
    17b5ae53a00fac193eb820c8921d53db51c2e364b3b6a3625e8643821dd8dc8fb8  guix-build-5f061f5abdee/output/riscv64-linux-gnu/bitcoin-5f061f5abdee-riscv64-linux-gnu.tar.gz
    18f315dbdf97467ba580d9b97007805318d08ea103f89c4582eaa2c15185b00650  guix-build-5f061f5abdee/output/x86_64-apple-darwin/SHA256SUMS.part
    19670fce3cc1aa4a5d4da0281bf448e2680a012c9c5ff9f951bd396ff778d0c504  guix-build-5f061f5abdee/output/x86_64-apple-darwin/bitcoin-5f061f5abdee-x86_64-apple-darwin-codesigning.tar.gz
    20507f169cd5950aa539e77b8f23ef967d0e738554da299871d31566deb35db2e6  guix-build-5f061f5abdee/output/x86_64-apple-darwin/bitcoin-5f061f5abdee-x86_64-apple-darwin-unsigned.tar.gz
    2192defdd3d1420ab78e909b8d98dec89eedbbb9aa7f11b5a1d0e378a2dad3bd54  guix-build-5f061f5abdee/output/x86_64-apple-darwin/bitcoin-5f061f5abdee-x86_64-apple-darwin-unsigned.zip
    225c59779906b3021f7bd96583268236ca3b61a18a4405b6489ce08ee109b05cac  guix-build-5f061f5abdee/output/x86_64-linux-gnu/SHA256SUMS.part
    23ad9dcfc2c842280bc1544b8051174d80ec1efdd12c9222ea2af6893258b0c640  guix-build-5f061f5abdee/output/x86_64-linux-gnu/bitcoin-5f061f5abdee-x86_64-linux-gnu-debug.tar.gz
    24ea921eea083aec87d6971f90423213a825dc3728b57bc289f9b0e8b7e9641f6e  guix-build-5f061f5abdee/output/x86_64-linux-gnu/bitcoin-5f061f5abdee-x86_64-linux-gnu.tar.gz
    256234d51100d4f9ee5654e672fed4c044b1a808f6ec3de2cd46d5d2eab217b68a  guix-build-5f061f5abdee/output/x86_64-w64-mingw32/SHA256SUMS.part
    268cc1033a153ccbe04a2ff6edae922b618d4ff4ca44eaaecffde26d678fcbcddd  guix-build-5f061f5abdee/output/x86_64-w64-mingw32/bitcoin-5f061f5abdee-win64-codesigning.tar.gz
    2728dfc83b73cf4fab5fe8992a125f1bab0439ba7151acc9c8830b880ec8307324  guix-build-5f061f5abdee/output/x86_64-w64-mingw32/bitcoin-5f061f5abdee-win64-debug.zip
    28f9fb4b3c2114333e494a322251497fd586796617070da7c74e4f3addae87cb12  guix-build-5f061f5abdee/output/x86_64-w64-mingw32/bitcoin-5f061f5abdee-win64-setup-unsigned.exe
    29d48ec312c8cfbf2700951e3be1c0f0d4327087712df958c0a5f7f93fca99d6c5  guix-build-5f061f5abdee/output/x86_64-w64-mingw32/bitcoin-5f061f5abdee-win64-unsigned.zip
    

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-05-11 15:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me