With this change, I’m now seeing cross-architecture reproducibility from our Guix build (release) process. Given the same code, doing a Guix build on x86_64
and aarch64
(arm64 Apple M1) hardware now gives the exact same binaries.
The code change to our build script is a bit yuck (hence draft), and generating the correct store paths is a multi-phase process, as modifying glibc to insert the debug-prefix option for the native compiler, changes the store path hashes of the then-rebuilt GCC 10 toolchain, however once we have them, they should be stable for a given commit in our Guix time-machine.
Given that glibc startup code ends up in our binaries, we also need to patch out any /gnu/store/*
paths that may end up in the debug info for it’s objects/libs i.e:
0 The Directory Table (offset 0xff5619):
1 1 /gnu/store/2d2r6jcsdjf6i3nzbrmf13psxmpwz55w-gcc-cross-sans-libc-x86_64-linux-gnu-7.5.0-lib/lib/gcc/x86_64-linux-gnu/7.5.0/include
2
3 The File Name Table (offset 0xff569c):
4 Entry Dir Time Size Name
5 1 0 0 0 elf-init.c
6 2 1 0 0 stddef.h
Two points on that:
- We currently use a GCC 7 native compiler, which is why
-fdebug-prefix-map
is used instead of-ffile-prefix-map=
. The later is only supported by GCC 8 and older. - Newer glibcs have actually introduced an option (
--with-nonshared-cflags=
) to make it easier to:
Use additional compiler flags cflags to build the parts of the library which are always statically linked into applications and libraries even with shared linking.
which means in future we could potentially use this, as an additional glibc configure option, rather than patching a glibc Makefile.
Related to #21194.
Guix build on x86_64:
0uname -a
1Linux 1ab0f609970a 5.13.0-1017-aws [#19](/bitcoin-bitcoin/19/)~20.04.1-Ubuntu SMP Mon Mar 7 12:53:12 UTC 2022 x86_64 Linux
0e3bb0544efc5e3d377cee5714e0e8bbe048ebedcb8981ae738055fd61e2422e1 guix-build-9cf6d7d89390/output/dist-archive/bitcoin-9cf6d7d89390.tar.gz
1aa994b2edfbca77269e71e5a8d3464d3429ceb252cb3939bc301e1194636e566 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/SHA256SUMS.part
23f53813e74c11c528a677f454873e17a873fc2c2dedcc11c3ee3e7db64355da5 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390-x86_64-linux-gnu-debug.tar.gz
39ada6333aa20f0a7124ca08471a02759951d017e6af12a5554693e8efb4f15bd guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390-x86_64-linux-gnu.tar.gz
43990edfa2fa4d29acc332f61ad95996c7760ec77563130fe3daafd222eb393d3 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/README.md
51b797c31235c1034d2df2729e0c68d7de600131727e3c9a42521d48a1a24f3c7 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-cli
611d01327ef0c87296c01f46f2ef3843b4895091341f41c5e6635fba7b105941d guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-cli.dbg
773569b03799c50ffc92c80489b935d9643165786e0e7724844ee8638d49e7f95 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-qt
841ec7c787a7ddac8214970b7b32e87b00b069b8a274871b724f6d3ed0a4e7a90 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-qt.dbg
9f905cdb727cce29166502ea70d1917a284143d2bb1297895f074aec0fcf13e5f guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-tx
1086ebbc36d64bf8d94a1761e5fac3f6edd63e59f0022c2cbd9ed8995b0b14f294 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-tx.dbg
11eb49cd79f143bf574eabfba8f7112323eb85a0f2fddea1d28357f02af7739ed0 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-util
1240d4a822e6a7dfd75a2a86c0a4685fec7cf56480f962917bc7f7ecd193cc5af4 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-util.dbg
13a87b7b4220379e6ef6c9b8744c0de1d5cadaf48f41dce6a3cd4057b12f6127f7 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-wallet
1424d79d1c0920335f944bcb180d126150151d80d6375c8a5a1020bf5d238ac291 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-wallet.dbg
15175a1ec3e721e457bcec95791beb8cfc5573621a5ddd78e9c9f65d43c8332899 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoind
1635d6a79bfcc6f7591eed887bd4e6a3f89d6f874704d27ef63083dc70ae123eaa guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoind.dbg
173d97a53717c9ca9460a6f1ea6802f64a3bf9bcdfc3977b6beca5d462df9dc89f guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/test_bitcoin
1886073797f1c5a9df4e72a20f008c6deae159fb5c70c5c76e825e1d38bd06a411 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/test_bitcoin.dbg
1936127300d74413c6e0e16bc3a1331598128ff93b103cd033ad4ade3cdeeadbec guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/include/bitcoinconsensus.h
20cb0ade6c79774358efb502fadaef7d02894e43d96110cd914a61a659f81c7f95 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/lib/libbitcoinconsensus.so.0.0.0
211101b54af2d1ed9110227d782614cf687329404ff04e95d6b5556df8d328010a guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/lib/libbitcoinconsensus.so.0.0.0.dbg
226722c631147686338875b2384ef9cdaf976b5df9f0d79eb03a8d2ec8218005f6 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-cli.1
234c8e6a876222d5451574abefaeb716fd1735e2001cb0dbd40c98a296eec763b1 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-qt.1
24f59c03e2dc314adbded43bfb94fb5d3f94b64064ec409683ef1a4bae3ebd9be1 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-tx.1
255356bea504330cd7c07e9598d45e689d8531d72eb82460536604b878a5d44954 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-util.1
26c6a810006423b2923b9d13d5c6e5ae24f5c93ce2527041d2dda88fd779fadb9e guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-wallet.1
2700537e3fb6e839c0f29a7d3b989d8908d6a390cf9a87831f815401346e4f7878 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoind.1
Guix build on arm64 (Debian in Docker on Apple M1)
0uname -a
1Linux 3b26b9608b88 5.10.104-linuxkit [#1](/bitcoin-bitcoin/1/) SMP PREEMPT Wed Mar 9 19:01:25 UTC 2022 aarch64 GNU/Linux
0e3bb0544efc5e3d377cee5714e0e8bbe048ebedcb8981ae738055fd61e2422e1 guix-build-9cf6d7d89390/output/dist-archive/bitcoin-9cf6d7d89390.tar.gz
1aa994b2edfbca77269e71e5a8d3464d3429ceb252cb3939bc301e1194636e566 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/SHA256SUMS.part
23f53813e74c11c528a677f454873e17a873fc2c2dedcc11c3ee3e7db64355da5 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390-x86_64-linux-gnu-debug.tar.gz
39ada6333aa20f0a7124ca08471a02759951d017e6af12a5554693e8efb4f15bd guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390-x86_64-linux-gnu.tar.gz
43990edfa2fa4d29acc332f61ad95996c7760ec77563130fe3daafd222eb393d3 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/README.md
51b797c31235c1034d2df2729e0c68d7de600131727e3c9a42521d48a1a24f3c7 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-cli
611d01327ef0c87296c01f46f2ef3843b4895091341f41c5e6635fba7b105941d guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-cli.dbg
773569b03799c50ffc92c80489b935d9643165786e0e7724844ee8638d49e7f95 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-qt
841ec7c787a7ddac8214970b7b32e87b00b069b8a274871b724f6d3ed0a4e7a90 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-qt.dbg
9f905cdb727cce29166502ea70d1917a284143d2bb1297895f074aec0fcf13e5f guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-tx
1086ebbc36d64bf8d94a1761e5fac3f6edd63e59f0022c2cbd9ed8995b0b14f294 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-tx.dbg
11eb49cd79f143bf574eabfba8f7112323eb85a0f2fddea1d28357f02af7739ed0 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-util
1240d4a822e6a7dfd75a2a86c0a4685fec7cf56480f962917bc7f7ecd193cc5af4 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-util.dbg
13a87b7b4220379e6ef6c9b8744c0de1d5cadaf48f41dce6a3cd4057b12f6127f7 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-wallet
1424d79d1c0920335f944bcb180d126150151d80d6375c8a5a1020bf5d238ac291 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoin-wallet.dbg
15175a1ec3e721e457bcec95791beb8cfc5573621a5ddd78e9c9f65d43c8332899 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoind
1635d6a79bfcc6f7591eed887bd4e6a3f89d6f874704d27ef63083dc70ae123eaa guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/bitcoind.dbg
173d97a53717c9ca9460a6f1ea6802f64a3bf9bcdfc3977b6beca5d462df9dc89f guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/test_bitcoin
1886073797f1c5a9df4e72a20f008c6deae159fb5c70c5c76e825e1d38bd06a411 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/bin/test_bitcoin.dbg
1936127300d74413c6e0e16bc3a1331598128ff93b103cd033ad4ade3cdeeadbec guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/include/bitcoinconsensus.h
20cb0ade6c79774358efb502fadaef7d02894e43d96110cd914a61a659f81c7f95 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/lib/libbitcoinconsensus.so.0.0.0
211101b54af2d1ed9110227d782614cf687329404ff04e95d6b5556df8d328010a guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/lib/libbitcoinconsensus.so.0.0.0.dbg
226722c631147686338875b2384ef9cdaf976b5df9f0d79eb03a8d2ec8218005f6 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-cli.1
234c8e6a876222d5451574abefaeb716fd1735e2001cb0dbd40c98a296eec763b1 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-qt.1
24f59c03e2dc314adbded43bfb94fb5d3f94b64064ec409683ef1a4bae3ebd9be1 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-tx.1
255356bea504330cd7c07e9598d45e689d8531d72eb82460536604b878a5d44954 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-util.1
26c6a810006423b2923b9d13d5c6e5ae24f5c93ce2527041d2dda88fd779fadb9e guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoin-wallet.1
2700537e3fb6e839c0f29a7d3b989d8908d6a390cf9a87831f815401346e4f7878 guix-build-9cf6d7d89390/output/x86_64-linux-gnu/bitcoin-9cf6d7d89390/share/man/man1/bitcoind.1