build: Update Guix build free space requirements #23861

hebasto commented at 5:48 PM on December 24, 2021: member

After bumping Qt from 5.12 to 5.15 the actual disk space that are consumed by Guix builds (measured with a tweaked guix-build script):

x86_64-linux-gnu -- 6080844 KiB
arm-linux-gnueabihf -- 4775864 KiB
aarch64-linux-gnu -- 7266380 KiB
riscv64-linux-gnu -- 6583008 KiB
powerpc64-linux-gnu -- 7330884 KiB
powerpc64le-linux-gnu -- 7293340 KiB
x86_64-w64-mingw32 -- 8667608 KiB
x86_64-apple-darwin -- 430280 KiB

This PR adjusts the required_KiB variable values accordingly.

build: Update Guix build free space requirements

It is required after bumping Qt from 5.12 to 5.15.

aa9a48bb8a

DrahtBot added the label Build system on Dec 24, 2021

DrahtBot added the label Scripts and tools on Dec 24, 2021

MarcoFalke added the label DrahtBot Guix build requested on Dec 27, 2021

in contrib/guix/guix-build:154 in aa9a48bb8a

 152 | +        *darwin*)       required_KiB=440000 ;;
 153 | +        *mingw*)        required_KiB=8700000 ;;
 154 | +        *x86_64-linux*) required_KiB=6100000 ;;
 155 | +        *arm-linux*)    required_KiB=4800000 ;;
 156 | +        *riscv64*)      required_KiB=6600000 ;;
 157 | +        *)              required_KiB=7400000 ;;

josibake commented at 1:48 PM on December 29, 2021:

why is this 7400000? this line is for matching any host, correct? shouldn't it be the max required, then (8700000, in this case)

hebasto commented at 3:23 PM on December 29, 2021:

this line is for matching any host, correct?

For matching any unspecified above host.

josibake commented at 3:40 PM on December 29, 2021:

gotcha. wouldn't it still be better to make this the same as the max value of the specified hosts? the system doing the building can already handle that value because we are specifying it for a known host, so why not play it safe for an unspecified host?

i guess i don't understand how this value was arrived at or why we would make it less than the max

hebasto commented at 3:51 PM on December 29, 2021:

The default value required_KiB=7400000 is specified for the sake of the code brevity. Actually it covers the following hosts with similar free disk space requirements:

aarch64-linux-gnu -- 7266380 KiB
powerpc64-linux-gnu -- 7330884 KiB
powerpc64le-linux-gnu -- 7293340 KiB

Overestimating of the required free disk space seems suboptimal in cases when Guix builds are being done in virtual machines with limited resources.

Alternatively, each host can be specified explicitly.

josibake commented at 4:02 PM on December 29, 2021:

Thank you for the explanation. I'm not sure I totally agree and would prefer that if the HOST is an unspecified HOST, we set total_required_KiB to avail_KiB (line 145).

I think it would be far worse to fail during a long-running process due to underestimating the space required vs finishing the build but with more space than was needed.

Said differently, if we haven't specified anything for a host, assume the worst and use all the space available.

hebasto commented at 4:06 PM on December 29, 2021:

Said differently, if we haven't specified anything for a host, assume the worst and use all the space available.

The HOSTS variable is always defined here: https://github.com/bitcoin/bitcoin/blob/aa9a48bb8af7a7f1bef98051a1590de3713c7789/contrib/guix/guix-build#L76-L79

josibake commented at 4:09 PM on December 29, 2021:

right, but I can also run HOSTS=foo-bar-baz ./contrib/guix-build. In this case, I think it would be better for the default to be required_KiB = avail_KiB

josibake commented at 4:41 PM on December 29, 2021:

AH, my apologies. I misunderstood your explanation. The last case is to cover:

aarch64-linux-gnu -- 7266380 KiB
powerpc64-linux-gnu -- 7330884 KiB
powerpc64le-linux-gnu -- 7293340 KiB

in that case, i would recommend *-linux-gnu instead of *. this makes it more readable/explicit as to what the code is doing and which subgroup from HOSTS is being matched.

josibake commented at 1:49 PM on December 29, 2021: member

Concept ACK

running a build now, just had one question in-line (more for my own understanding).

DrahtBot commented at 4:32 PM on December 30, 2021: member

Guix builds

File	commit 587cbca826d7331633a30b5a6cce170e3dbe2612<br>(master)	commit cc5b96fba4fad9036db1806877572dbb8e0c8e51<br>(master and this pull)
SHA256SUMS.part	`54bf0fba61b0789f...`	`5dd69ef5b56ae899...`
*-aarch64-linux-gnu-debug.tar.gz	`99943d6bf15b523b...`	`b97cca8be0ac5904...`
*-aarch64-linux-gnu.tar.gz	`5af252a5fa34b686...`	`a451f6b4cf1e45db...`
*-arm-linux-gnueabihf-debug.tar.gz	`9ed7eea7b54765e4...`	`4f5b2045f1c4129f...`
*-arm-linux-gnueabihf.tar.gz	`42855682e42af5f8...`	`75606a35812d7897...`
*-osx-unsigned.dmg	`a62bbe4777e83f15...`	`da4c9de2468462e4...`
*-osx-unsigned.tar.gz	`65fa556f437dcbde...`	`ca152b8cc4e565de...`
*-osx64.tar.gz	`73a0ad1b30fdce75...`	`8e4da13c5dcefc79...`
*-powerpc64-linux-gnu-debug.tar.gz	`d7c5c16039d0ffc3...`	`6d2a4f173bf55dfe...`
*-powerpc64-linux-gnu.tar.gz	`ffc000354f60afd5...`	`bea36845e8d4ee74...`
*-powerpc64le-linux-gnu-debug.tar.gz	`c273e299164508b2...`	`9386a84000a9dee1...`
*-powerpc64le-linux-gnu.tar.gz	`c3a2cda7e9b4ef7c...`	`e8708c59feb7342b...`
*-riscv64-linux-gnu-debug.tar.gz	`7299d460303154ab...`	`483d7820fa3fa1e8...`
*-riscv64-linux-gnu.tar.gz	`610783f6c2937a43...`	`8af5aef87455ac9f...`
*-win-unsigned.tar.gz	`f485dff0967f6346...`	`849a72f55ba066a4...`
*-win64-debug.zip	`473f1631e5448937...`	`796d178db33a43c5...`
*-win64-setup-unsigned.exe	`20537ab5eecc1ce3...`	`430f27693043b4fe...`
*-win64.zip	`b9b5f15df3c6abd7...`	`bdb3d7a540e2c8eb...`
*-x86_64-linux-gnu-debug.tar.gz	`b51a03099a61d75c...`	`5c82f5a2aaef1f05...`
*-x86_64-linux-gnu.tar.gz	`42c2f600eff3d781...`	`d50129dd0582d2de...`
*.tar.gz	`3dc92e1b19e7cbd9...`	`180da84535e13adb...`
guix_build.log	`d1b1af42e2c0bc4b...`	`82bbaacca331853d...`
guix_build.log.diff		`bfca71c4e0d37db3...`

DrahtBot removed the label DrahtBot Guix build requested on Dec 30, 2021

luke-jr commented at 2:16 AM on January 12, 2022: member

Did dropping rust/rsvg/imagemagick reduce these?

hebasto closed this on Mar 10, 2022

DrahtBot locked this on Mar 10, 2023