build: Fix x86_64 <-> arm64 cross-compiling for macOS #23603

1. 
   hebasto commented at 12:22 pm on November 26, 2021: member

   Currently, on master (111c3e06b35b7867f2e0f740e988f648ac6c325d), dependencies are built for the build system architecture, not the provided host.

   On Intel-based macOS Big Sur 11.6.1 (20G224):

   0% make -C depends HOST=arm64-apple-darwin20
   1% lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a 
   2Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64
   

   On M1-based macOS Monterey 12.0.1 (21A559) the boost package building fails with multiple errors like that:

   0% make -C depends boost HOST=x86_64-apple-darwin19
   1...
   2error: option 'cf-protection=return' cannot be specified on this target
   3error: option 'cf-protection=branch' cannot be specified on this target
   42 errors generated.
   

   This PR allows to cross-compile as follows:

   • on Intel-based macOS Big Sur 11.6.1 (20G224):

   0% make -C depends HOST=arm64-apple-darwin20
   1% lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a 
   2Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64
   3% CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure
   4% make
   5% lipo -info src/qt/bitcoin-qt 
   6Non-fat file: src/qt/bitcoin-qt is architecture: arm64
   

   • on M1-based macOS Monterey 12.0.1 (21A559):

   0% make -C depends HOST=x86_64-apple-darwin19
   1% CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure
   2% make
   3% lipo -info src/qt/bitcoin-qt
   4Non-fat file: src/qt/bitcoin-qt is architecture: x86_64
   

   No behavior change for other builder-host pairs.

   This is an alternative to bitcoin/bitcoin#22506.

2. build: Fix x86_64 <-> arm64 cross-compiling for macOS 19f3896c9a
3. hebasto added the label macOS on Nov 26, 2021
4. hebasto added the label Build system on Nov 26, 2021
5. 
   hebasto commented at 5:47 pm on November 26, 2021: member

   Guix builds:

    0$ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
    1cd821fb2efbb8ead184ef32559fd6cf8a0bb87fa8bb0bf8d7952de440c32e44e  guix-build-19f3896c9a6a/output/aarch64-linux-gnu/SHA256SUMS.part
    213d1ea066972d4c39090183e8a5053a7c32f1faf5fcca00549aefc3ca43d25c4  guix-build-19f3896c9a6a/output/aarch64-linux-gnu/bitcoin-19f3896c9a6a-aarch64-linux-gnu-debug.tar.gz
    3bf3bc56e29a9bbde1c9dab5fc3573da79ba25c1256720fdaff66bca649e275b1  guix-build-19f3896c9a6a/output/aarch64-linux-gnu/bitcoin-19f3896c9a6a-aarch64-linux-gnu.tar.gz
    449b03284cd373b6e18b6fcc79a419ff8b769f5c63f1c1bbf2328e2ee6aa3fe9e  guix-build-19f3896c9a6a/output/arm-linux-gnueabihf/SHA256SUMS.part
    5970215788561cbbf50b7e285548db76bbcc00660615305d5a0f4abcc643799b6  guix-build-19f3896c9a6a/output/arm-linux-gnueabihf/bitcoin-19f3896c9a6a-arm-linux-gnueabihf-debug.tar.gz
    69bf1cff65d07e41230fc676d5eb3a22a68dec56294cfb1c2dd613e60ef8c2626  guix-build-19f3896c9a6a/output/arm-linux-gnueabihf/bitcoin-19f3896c9a6a-arm-linux-gnueabihf.tar.gz
    70c98403967aa48610fbe953df1aa56d30d11da0c9013640d5e9df867348793b4  guix-build-19f3896c9a6a/output/dist-archive/bitcoin-19f3896c9a6a.tar.gz
    8d598262841072b5663e3a67fe55365e506dce1ee6b6467af1e23cb3aa7ed7c80  guix-build-19f3896c9a6a/output/powerpc64-linux-gnu/SHA256SUMS.part
    9df9f13656780a2e2f4ba127280cb3f417c4e43c55efb9413e67fe27738b52dc1  guix-build-19f3896c9a6a/output/powerpc64-linux-gnu/bitcoin-19f3896c9a6a-powerpc64-linux-gnu-debug.tar.gz
   10862b7b05bd622b2b6f4e9425afd08fa2219bb08187dcf454c7e52dd37d142efd  guix-build-19f3896c9a6a/output/powerpc64-linux-gnu/bitcoin-19f3896c9a6a-powerpc64-linux-gnu.tar.gz
   1145184c8e2ec75712473a6e8d7de1676fcfd051867130230f49f16bdfa74bea98  guix-build-19f3896c9a6a/output/powerpc64le-linux-gnu/SHA256SUMS.part
   122b842d75c5965f571cb9bf61b748605485fa3c93fb5a0ddd5c00b6c35c65307b  guix-build-19f3896c9a6a/output/powerpc64le-linux-gnu/bitcoin-19f3896c9a6a-powerpc64le-linux-gnu-debug.tar.gz
   1352b8aa6ed35338d6a5e38db4d6476b56aaf3298b690d0aed5413a944ca483a5d  guix-build-19f3896c9a6a/output/powerpc64le-linux-gnu/bitcoin-19f3896c9a6a-powerpc64le-linux-gnu.tar.gz
   14cb591a7ddc87eb9068b9a52b87f6300b6a8ad27bb08cf80ffa2c4f47b3c5dbb7  guix-build-19f3896c9a6a/output/riscv64-linux-gnu/SHA256SUMS.part
   15c4e4083462ec34d9770e0a8ce823eb5046d5879fc82d87a0d696f62bd6bc578c  guix-build-19f3896c9a6a/output/riscv64-linux-gnu/bitcoin-19f3896c9a6a-riscv64-linux-gnu-debug.tar.gz
   16297b5c58b56d8e3397bed42dd09d9a386a85568f084df69fbd869d5b86981472  guix-build-19f3896c9a6a/output/riscv64-linux-gnu/bitcoin-19f3896c9a6a-riscv64-linux-gnu.tar.gz
   1761e4f3e657ac1b42431167c02e816bc874eb27597b15b4f16ee4ce43076e1795  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/SHA256SUMS.part
   183d1117e5e195b3ad48a70af2a9b362b100774562bd43f290d2591fcc7c8d2cff  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/bitcoin-19f3896c9a6a-osx-unsigned.dmg
   198e96a48f5e4d72ebe8823fd7e02dbb70c0de91ca391050152da5724c090d46cd  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/bitcoin-19f3896c9a6a-osx-unsigned.tar.gz
   2027cdaf70d1897520aca1f079abc39012ee71c4048886236e50b5906373ddb29e  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/bitcoin-19f3896c9a6a-osx64.tar.gz
   21ae36a6ff945092150878bbd509cd4e4f3ff1596d21855aced1be1e4846a8c103  guix-build-19f3896c9a6a/output/x86_64-linux-gnu/SHA256SUMS.part
   22c0a415f2ac58e1af4d195ba27652014de147a6039ca26dcdca3027ee315c1347  guix-build-19f3896c9a6a/output/x86_64-linux-gnu/bitcoin-19f3896c9a6a-x86_64-linux-gnu-debug.tar.gz
   23f96a59ebab15b3936c966553365caf4b198626d8b8a7fbf4dd24e88b0db9dc66  guix-build-19f3896c9a6a/output/x86_64-linux-gnu/bitcoin-19f3896c9a6a-x86_64-linux-gnu.tar.gz
   241e23c0cf4cefc80ab1706eccf68aeba3575e510bc69565bd4bc2c29a1a6c5ad6  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/SHA256SUMS.part
   2530372abc5e1d810a5b52cac195fa13723f4fb0a0bb327ac18a123ca5c21fbd44  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win-unsigned.tar.gz
   26ba7b57dbfe7b61d349d1eeb46a923fc53ac5897b7cc2f1a770765ffa186e0572  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win64-debug.zip
   274be16b33b432d34e40a018d196e86210e393df476691fc23c9f1ec69b7dc7b0b  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win64-setup-unsigned.exe
   2845628090057e319751ccc940b880c93c6bc8510bf93f863b0f6435106e589ec7  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win64.zip
   

6. 
   jarolrod commented at 9:58 pm on November 26, 2021: member

   Interesting, concept ACK

   GUIX hashes, mine match @hebasto:

    0cd821fb2efbb8ead184ef32559fd6cf8a0bb87fa8bb0bf8d7952de440c32e44e  guix-build-19f3896c9a6a/output/aarch64-linux-gnu/SHA256SUMS.part
    113d1ea066972d4c39090183e8a5053a7c32f1faf5fcca00549aefc3ca43d25c4  guix-build-19f3896c9a6a/output/aarch64-linux-gnu/bitcoin-19f3896c9a6a-aarch64-linux-gnu-debug.tar.gz
    2bf3bc56e29a9bbde1c9dab5fc3573da79ba25c1256720fdaff66bca649e275b1  guix-build-19f3896c9a6a/output/aarch64-linux-gnu/bitcoin-19f3896c9a6a-aarch64-linux-gnu.tar.gz
    349b03284cd373b6e18b6fcc79a419ff8b769f5c63f1c1bbf2328e2ee6aa3fe9e  guix-build-19f3896c9a6a/output/arm-linux-gnueabihf/SHA256SUMS.part
    4970215788561cbbf50b7e285548db76bbcc00660615305d5a0f4abcc643799b6  guix-build-19f3896c9a6a/output/arm-linux-gnueabihf/bitcoin-19f3896c9a6a-arm-linux-gnueabihf-debug.tar.gz
    59bf1cff65d07e41230fc676d5eb3a22a68dec56294cfb1c2dd613e60ef8c2626  guix-build-19f3896c9a6a/output/arm-linux-gnueabihf/bitcoin-19f3896c9a6a-arm-linux-gnueabihf.tar.gz
    60c98403967aa48610fbe953df1aa56d30d11da0c9013640d5e9df867348793b4  guix-build-19f3896c9a6a/output/dist-archive/bitcoin-19f3896c9a6a.tar.gz
    7d598262841072b5663e3a67fe55365e506dce1ee6b6467af1e23cb3aa7ed7c80  guix-build-19f3896c9a6a/output/powerpc64-linux-gnu/SHA256SUMS.part
    8df9f13656780a2e2f4ba127280cb3f417c4e43c55efb9413e67fe27738b52dc1  guix-build-19f3896c9a6a/output/powerpc64-linux-gnu/bitcoin-19f3896c9a6a-powerpc64-linux-gnu-debug.tar.gz
    9862b7b05bd622b2b6f4e9425afd08fa2219bb08187dcf454c7e52dd37d142efd  guix-build-19f3896c9a6a/output/powerpc64-linux-gnu/bitcoin-19f3896c9a6a-powerpc64-linux-gnu.tar.gz
   1045184c8e2ec75712473a6e8d7de1676fcfd051867130230f49f16bdfa74bea98  guix-build-19f3896c9a6a/output/powerpc64le-linux-gnu/SHA256SUMS.part
   112b842d75c5965f571cb9bf61b748605485fa3c93fb5a0ddd5c00b6c35c65307b  guix-build-19f3896c9a6a/output/powerpc64le-linux-gnu/bitcoin-19f3896c9a6a-powerpc64le-linux-gnu-debug.tar.gz
   1252b8aa6ed35338d6a5e38db4d6476b56aaf3298b690d0aed5413a944ca483a5d  guix-build-19f3896c9a6a/output/powerpc64le-linux-gnu/bitcoin-19f3896c9a6a-powerpc64le-linux-gnu.tar.gz
   13cb591a7ddc87eb9068b9a52b87f6300b6a8ad27bb08cf80ffa2c4f47b3c5dbb7  guix-build-19f3896c9a6a/output/riscv64-linux-gnu/SHA256SUMS.part
   14c4e4083462ec34d9770e0a8ce823eb5046d5879fc82d87a0d696f62bd6bc578c  guix-build-19f3896c9a6a/output/riscv64-linux-gnu/bitcoin-19f3896c9a6a-riscv64-linux-gnu-debug.tar.gz
   15297b5c58b56d8e3397bed42dd09d9a386a85568f084df69fbd869d5b86981472  guix-build-19f3896c9a6a/output/riscv64-linux-gnu/bitcoin-19f3896c9a6a-riscv64-linux-gnu.tar.gz
   1661e4f3e657ac1b42431167c02e816bc874eb27597b15b4f16ee4ce43076e1795  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/SHA256SUMS.part
   173d1117e5e195b3ad48a70af2a9b362b100774562bd43f290d2591fcc7c8d2cff  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/bitcoin-19f3896c9a6a-osx-unsigned.dmg
   188e96a48f5e4d72ebe8823fd7e02dbb70c0de91ca391050152da5724c090d46cd  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/bitcoin-19f3896c9a6a-osx-unsigned.tar.gz
   1927cdaf70d1897520aca1f079abc39012ee71c4048886236e50b5906373ddb29e  guix-build-19f3896c9a6a/output/x86_64-apple-darwin19/bitcoin-19f3896c9a6a-osx64.tar.gz
   20ae36a6ff945092150878bbd509cd4e4f3ff1596d21855aced1be1e4846a8c103  guix-build-19f3896c9a6a/output/x86_64-linux-gnu/SHA256SUMS.part
   21c0a415f2ac58e1af4d195ba27652014de147a6039ca26dcdca3027ee315c1347  guix-build-19f3896c9a6a/output/x86_64-linux-gnu/bitcoin-19f3896c9a6a-x86_64-linux-gnu-debug.tar.gz
   22f96a59ebab15b3936c966553365caf4b198626d8b8a7fbf4dd24e88b0db9dc66  guix-build-19f3896c9a6a/output/x86_64-linux-gnu/bitcoin-19f3896c9a6a-x86_64-linux-gnu.tar.gz
   231e23c0cf4cefc80ab1706eccf68aeba3575e510bc69565bd4bc2c29a1a6c5ad6  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/SHA256SUMS.part
   2430372abc5e1d810a5b52cac195fa13723f4fb0a0bb327ac18a123ca5c21fbd44  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win-unsigned.tar.gz
   25ba7b57dbfe7b61d349d1eeb46a923fc53ac5897b7cc2f1a770765ffa186e0572  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win64-debug.zip
   264be16b33b432d34e40a018d196e86210e393df476691fc23c9f1ec69b7dc7b0b  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win64-setup-unsigned.exe
   2745628090057e319751ccc940b880c93c6bc8510bf93f863b0f6435106e589ec7  guix-build-19f3896c9a6a/output/x86_64-w64-mingw32/bitcoin-19f3896c9a6a-win64.zip
   

7. 
   DrahtBot commented at 5:23 am on November 27, 2021: member

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #22486 (build: Avoid rebuilding makefiles by dgoncharov)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

8. fanquake approved
9. 
   fanquake commented at 1:22 pm on December 9, 2021: member

   ACK 19f3896c9a6a710bdd49b65a4a4947d90eb8a692 - this is definitely cleaner than the approach in #22506. I’ve tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it’s scoped to building for darwin targets, on darwin. Not sure if this is something we’ll actually see used very often.
10. fanquake merged this on Dec 9, 2021
11. fanquake closed this on Dec 9, 2021

    ---

12. hebasto deleted the branch on Dec 9, 2021
13. sidhujag referenced this in commit b1a5c6ce07 on Dec 10, 2021
14. 
    hebasto commented at 10:44 am on December 19, 2021: member

    Unfortunately, this PR introduced a regression which is fixed in #23817.
15. RandyMcMillan referenced this in commit eff47bf252 on Dec 23, 2021
16. fanquake referenced this in commit a213bd63ca on Dec 27, 2021
17. sidhujag referenced this in commit 7bd7471c3d on Dec 28, 2021
18. PastaPastaPasta referenced this in commit aa2c3bb13e on Apr 7, 2022
19. PastaPastaPasta referenced this in commit 3a8271c1c2 on Apr 7, 2022
20. PastaPastaPasta referenced this in commit 36e01e3ef3 on Apr 7, 2022
21. PastaPastaPasta referenced this in commit 125d914a63 on Apr 7, 2022
22. PastaPastaPasta referenced this in commit 378bc5877e on Apr 7, 2022
23. PastaPastaPasta referenced this in commit d424cc4f8d on Apr 7, 2022
24. DrahtBot locked this on Dec 19, 2022

Contributors
hebasto jarolrod DrahtBot fanquake

Labels
macOS Build system