guix: Build out of source tree

hebasto commented at 7:50 pm on August 4, 2021: member

This PR allows use an annotated git tag, e.g., v22.0rc2, as a package version, instead of a commit hash for Guix, and, actually, it restores the behavior as it was for gitian builds.

Fixes #22623.

Note for reviewers. Before testing you should add an annotated tag to the top commit locally. But Guix builds should be performed with no tags to be deterministic.

Guix builds

 0$ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
 1dd827d09a6c72d74018868a35e3da4112c31b4613d9a1795d9cf39c8be50f24d  guix-build-81f81b27d88d/output/aarch64-linux-gnu/SHA256SUMS.part
 2f72a1f25cce34669555d3945bc7572604bd436782f62b4110b2feac75db5e88e  guix-build-81f81b27d88d/output/aarch64-linux-gnu/bitcoin-81f81b27d88d-aarch64-linux-gnu-debug.tar.gz
 3397cc49877548bf2d9775d2dc75a2a749d2d8efa333671e517923dde8696e372  guix-build-81f81b27d88d/output/aarch64-linux-gnu/bitcoin-81f81b27d88d-aarch64-linux-gnu.tar.gz
 4984908990c7026f0b5628c0cdc1845fcc40593eaedcca9ac78c4583e2076249f  guix-build-81f81b27d88d/output/arm-linux-gnueabihf/SHA256SUMS.part
 53468bf605a4b1179eb51ce10972c73d2ad23108a7d48cd4ea26e188af65429eb  guix-build-81f81b27d88d/output/arm-linux-gnueabihf/bitcoin-81f81b27d88d-arm-linux-gnueabihf-debug.tar.gz
 612c12359b3afd843c9b0ea6094d34a297aef4cea938a085c07575ebc8ef99bbe  guix-build-81f81b27d88d/output/arm-linux-gnueabihf/bitcoin-81f81b27d88d-arm-linux-gnueabihf.tar.gz
 7d5d52d531b798b32f8c80ad01423d46764deaff73c1aa683001c600d59a36911  guix-build-81f81b27d88d/output/dist-archive/bitcoin-81f81b27d88d.tar.gz
 8d731ea0561f0e7f3235acdc2ea52d47706eaefcb5668f26f0efe59f030135a98  guix-build-81f81b27d88d/output/powerpc64-linux-gnu/SHA256SUMS.part
 9bc4a98db971262901e61349beaf60334653962dc26e43107e3dde7d0c04042a6  guix-build-81f81b27d88d/output/powerpc64-linux-gnu/bitcoin-81f81b27d88d-powerpc64-linux-gnu-debug.tar.gz
10114ba0344977451c14bb9cb69dbd590716da341314fc78c7b362300e46c851ea  guix-build-81f81b27d88d/output/powerpc64-linux-gnu/bitcoin-81f81b27d88d-powerpc64-linux-gnu.tar.gz
1194a5a4f16e4b11a8cbcc25b4016c120f8a0dbfd99a63291dfade97e4b3d11adf  guix-build-81f81b27d88d/output/powerpc64le-linux-gnu/SHA256SUMS.part
1221f648c9e49ab1c74b3f46742983900341fdeec7481cbc3f4dcfb7593f479717  guix-build-81f81b27d88d/output/powerpc64le-linux-gnu/bitcoin-81f81b27d88d-powerpc64le-linux-gnu-debug.tar.gz
13103efeb96afa3dee3ef0ea85487d2f77ea322a9f7120b2882eb58924a8271ac6  guix-build-81f81b27d88d/output/powerpc64le-linux-gnu/bitcoin-81f81b27d88d-powerpc64le-linux-gnu.tar.gz
14867feb9574b8e51b022cc87438b50f32fc1010487ec678cc10e2b814fbb1f85f  guix-build-81f81b27d88d/output/riscv64-linux-gnu/SHA256SUMS.part
15d45797e99f68a5b4c991b10c7bdcf4750216825d6c5f9001a11e81876000af71  guix-build-81f81b27d88d/output/riscv64-linux-gnu/bitcoin-81f81b27d88d-riscv64-linux-gnu-debug.tar.gz
16a0d666f3ce83c1cec1fd1a8b782471f59811a14a8d83d86b53063951ae1546f9  guix-build-81f81b27d88d/output/riscv64-linux-gnu/bitcoin-81f81b27d88d-riscv64-linux-gnu.tar.gz
1727a1c714614f815b3c4914a2ede955cb6704509053b0f57b1fabfab6c4b5aa99  guix-build-81f81b27d88d/output/x86_64-apple-darwin18/SHA256SUMS.part
180adf5f317deeeead6188145e3be8a34bc168c8d712da6ea300872366253c0521  guix-build-81f81b27d88d/output/x86_64-apple-darwin18/bitcoin-81f81b27d88d-osx-unsigned.dmg
196cc8f90fc30e096fcc1411152e27538bb0d7b8f365013c8481772e32d6d8746a  guix-build-81f81b27d88d/output/x86_64-apple-darwin18/bitcoin-81f81b27d88d-osx-unsigned.tar.gz
20df9fd9e3f53343d3ec99059634e7fea9a3b9197379f1786f6812618232af23fe  guix-build-81f81b27d88d/output/x86_64-apple-darwin18/bitcoin-81f81b27d88d-osx64.tar.gz
211fbdc6761c10ed16159ba2e826a9ca7fef8abb7e246cb32fdc4c8d57ccf74e7d  guix-build-81f81b27d88d/output/x86_64-linux-gnu/SHA256SUMS.part
222b367b265fe580a0e215523fcfc7df2c3c544d50eb8a9956e9d3fe1456d64218  guix-build-81f81b27d88d/output/x86_64-linux-gnu/bitcoin-81f81b27d88d-x86_64-linux-gnu-debug.tar.gz
2357816755675c573f1de4060499c5d12d2e133991e387309be1a62b144a5a5553  guix-build-81f81b27d88d/output/x86_64-linux-gnu/bitcoin-81f81b27d88d-x86_64-linux-gnu.tar.gz
24379c2d9e7586b859f555b70c1b351e1005603771b4c556ce01192e5b14dcfa36  guix-build-81f81b27d88d/output/x86_64-w64-mingw32/SHA256SUMS.part
25b3ae00f55da42f025a174441d60a5f601199f6ce6cd9f058b5c89c845650e889  guix-build-81f81b27d88d/output/x86_64-w64-mingw32/bitcoin-81f81b27d88d-win-unsigned.tar.gz
264ae0bd74c148e9709abfe57c8336382cf6eacac4261b127bf8ceddb52774a822  guix-build-81f81b27d88d/output/x86_64-w64-mingw32/bitcoin-81f81b27d88d-win64-debug.zip
272309c163dfe83f662cd4673876b7b21a066dcab1a7dcbcb98216c537da787219  guix-build-81f81b27d88d/output/x86_64-w64-mingw32/bitcoin-81f81b27d88d-win64-setup-unsigned.exe
28ead1878e88bc6be593c7d3405bd07f7d8ccb6988849eaab056ba2acb8907ccae  guix-build-81f81b27d88d/output/x86_64-w64-mingw32/bitcoin-81f81b27d88d-win64.zip

DrahtBot added the label Scripts and tools on Aug 4, 2021

hebasto marked this as a draft on Aug 4, 2021

MarcoFalke added the label DrahtBot Guix build requested on Aug 4, 2021

hebasto force-pushed on Aug 4, 2021

hebasto marked this as ready for review on Aug 4, 2021

hebasto force-pushed on Aug 4, 2021

hebasto marked this as a draft on Aug 4, 2021

build: Fix `make deploy` for Windows when building out of source tree 2646c062ba

guix: Build out of source tree

This change allows use an annotated git tag, e.g., v22.0rc2, as a
package version, instead of a commit hash.

81f81b27d8

hebasto force-pushed on Aug 4, 2021

hebasto marked this as ready for review on Aug 4, 2021

hebasto commented at 3:37 pm on August 5, 2021: member

Is this PR a potential candidate for backporting to 22.x ?

DrahtBot commented at 6:37 pm on August 8, 2021: member

Guix builds

File	commit 03826aecc56c5c5c76570805897c2ddf92e11ab6(master)	commit c6de839a6111425fe8cc6e2d58b35fb654d1c83a(master and this pull)
SHA256SUMS.part	`0dc999bdf2259e95...`	`60389bfc4b08e888...`
*-aarch64-linux-gnu-debug.tar.gz	`c214b75f2cbdf64a...`	`61789faf421b2818...`
*-aarch64-linux-gnu.tar.gz	`3eb68efb739f6123...`	`e9f2b0d217d2e759...`
*-arm-linux-gnueabihf-debug.tar.gz	`515677e37be1a840...`	`cc5059e0c73cc732...`
*-arm-linux-gnueabihf.tar.gz	`66411c0f2272cc76...`	`c0ba4e1256452c40...`
*-osx-unsigned.dmg	`93bf4940c82a67ac...`	`c64b04c77ef38bb0...`
*-osx-unsigned.tar.gz	`b83bf8f597499eb9...`	`54b384eab7099ddc...`
*-osx64.tar.gz	`e5f0e3dbfe2fbba3...`	`daecce0ea82b31b5...`
*-powerpc64-linux-gnu-debug.tar.gz	`c0be02de5de1116f...`	`9e756f5c47376186...`
*-powerpc64-linux-gnu.tar.gz	`1943a0458a051225...`	`3468976716670e17...`
*-powerpc64le-linux-gnu-debug.tar.gz	`fea39d1ba38735e8...`	`7f70172d505a7ee6...`
*-powerpc64le-linux-gnu.tar.gz	`4cfcd5867fb30b8e...`	`51a284fc3a76ff92...`
*-riscv64-linux-gnu-debug.tar.gz	`b9582bf270f15472...`	`59f342cfa153e93d...`
*-riscv64-linux-gnu.tar.gz	`47ec6694c797180f...`	`cdb686391aefb436...`
*-win-unsigned.tar.gz	`c19895cd7f71186a...`	`701d0b5300dab116...`
*-win64-debug.zip	`a1571d27ee9dcb3c...`	`c3fbef3ef65b739a...`
*-win64-setup-unsigned.exe	`6c15b3adf685321b...`	`7986c27ae8154e4f...`
*-win64.zip	`a0eb0bc26011feed...`	`472508785ec249a5...`
*-x86_64-linux-gnu-debug.tar.gz	`07a8993cfdbedc47...`	`b05e204da2a4783f...`
*-x86_64-linux-gnu.tar.gz	`33dd67fd9d933fe3...`	`40d0543e84732ee1...`
*.tar.gz	`ff9aed316ea9a7e8...`	`0ae5b28f006ee2b0...`
guix_build.log	`2511e0c773bb8260...`	`fa582f47b8912e75...`
guix_build.log.diff		`3bc1faafb9a83764...`

DrahtBot removed the label DrahtBot Guix build requested on Aug 8, 2021

achow101 commented at 7:11 pm on August 9, 2021: member

ACK 81f81b27d88dc438fb08bda68844b752ff793991

Tested that the resulting guix build does use the tag name as the version rather than the commit hash. Agree that this should be backported.

fanquake requested review from dongcarl on Aug 10, 2021

dongcarl commented at 8:28 pm on August 10, 2021: member

Sorry, I’m NACK on this based on my understanding.

It would seem that ./autogen.sh and ./configure both modify their physical parent directory, even when executed from another directory. This seems like a bad idea because we’ll have all HOSTs modifying the same parent directory when configuring/autoreconf-ing, which seems fragile and is a recipe for broken builds in the future or non-reproducibility.

It seems that the underlying problem is that our source tarball produced by git-archive (say for v22.0), when extracted and used to build, does not result in a binary with a version string that’s just a tag name with no commit hash suffix. I think the fix likely lies in changing the genbuild.sh/clientversion.cpp logic.

fanquake commented at 2:00 am on August 11, 2021: member

Agree with Carl, that this doesn’t seem like the right solution. We should look at #22680 instead.

fanquake closed this on Aug 11, 2021

hebasto commented at 9:03 am on August 11, 2021: member

I agree that #22680 looks preferable.

It would seem that ./autogen.sh and ./configure both modify their physical parent directory, even when executed from another directory. This seems like a bad idea because we’ll have all HOSTs modifying the same parent directory when configuring/autoreconf-ing, which seems fragile and is a recipe for broken builds in the future or non-reproducibility.

Yes, ./autogen.sh does modify the source directory, when executed from another directory. But such modifications are the same for each HOST. We even could run ./autogen.sh once only for all HOSTs.

No, ./configure does not modify the source directory, when executed from another directory. That is, actually, the goal of the “building out of source tree” – not touching the source tree, right?

DrahtBot locked this on Aug 18, 2022

guix: Build out of source tree #22625

Guix builds

Guix builds