depends: Improve id string robustness

dongcarl commented at 8:29 pm on December 11, 2020: member

0Environment variables and search paths can drastically effect the
1operation of build tools.
2
3Include these in our id string to mitigate against false cache hits.

Note to builders: This will invalidate all depends output caches in BASE_CACHE

dongcarl added the label Build system on Dec 11, 2020

DrahtBot commented at 11:12 pm on December 11, 2020: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

laanwj commented at 11:58 am on December 14, 2020: member

Concept ACK. If I understand it correctly, this includes only specific targeted environment variables and not the entire env?

luke-jr commented at 6:04 pm on December 14, 2020: member

gcc -v -E - seems to lose COLLECT_LTO_WRAPPER…

dongcarl commented at 8:14 pm on December 14, 2020: member

Concept ACK. If I understand it correctly, this includes only specific targeted environment variables and not the entire env?

Yes! It automatically includes any env var that starts with the name of the tool (e.g. STRIP_* for strip), and if there are particular ones we know about (e.g. ZERO_AR_DATE) we can just append it like so: 6ee5ee4 (#20629)

dongcarl commented at 8:32 pm on December 14, 2020: member

gcc -v -E - seems to lose COLLECT_LTO_WRAPPER…

Right, perhaps someone knows a better invocation here? I’m guessing COLLECT_LTO_WRAPPER is omitted because the linker is not going to run?

dongcarl force-pushed on Dec 16, 2020

dongcarl commented at 9:08 pm on December 16, 2020: member

Pushed 4b9aaca598aea561d44215eecd3c4a01d08ddd5a -> f792069bc117f11dff0db11f9386ec216581d2a2

Added an invocation of CC/CXX with just “-v” to pick up COLLECT_LTO_WRAPPER
Added comment about COLLECT_LTO_WRAPPER and the weird fd indirection trick

MarcoFalke added the label Needs gitian build on Dec 17, 2020

MarcoFalke added the label Needs Guix build on Dec 17, 2020

DrahtBot commented at 5:22 am on December 18, 2020: member

Guix builds

File	commit 8452f922d211155b012b04256e1304c4afb6b913(master)	commit bcf4efe812e41b04c682dfc96e45a1e8d7e7dc8a(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz	`47fe896930c0d771...`	`4f389e051495a470...`
*-aarch64-linux-gnu.tar.gz	`e09c8c469d3844de...`	`d1644b8f32f58e0f...`
*-arm-linux-gnueabihf-debug.tar.gz	`e167617ff0806e89...`	`70159bbc83ab5357...`
*-arm-linux-gnueabihf.tar.gz	`6328582ec8f6a043...`	`96cea72f7b75e077...`
*-riscv64-linux-gnu-debug.tar.gz	`cdc1795bca5d99ad...`	`f930151ff6a3b114...`
*-riscv64-linux-gnu.tar.gz	`6af595f2cfc8f2d1...`	`c376f2f162571024...`
*-win-unsigned.tar.gz	`40811556f4d465fb...`	`d44c161c5528ab26...`
*-win64-debug.zip	`e7f88111b53f15c3...`	`a79a8366828ec6a7...`
*-win64-setup-unsigned.exe	`2afa8ac9334944ba...`	`47ce1c26ea8e2a24...`
*-win64.zip	`5a69a9f481267790...`	`6e3be0d25c9292a5...`
*-x86_64-linux-gnu-debug.tar.gz	`279d5ed7554ac6d2...`	`0ae3bdefa4302546...`
*-x86_64-linux-gnu.tar.gz	`b3281357f0ba632a...`	`9e09f09629cf3413...`
*.tar.gz	`6e9d2dc19aa6fb92...`	`0198417c9dd7136d...`
guix_build.log	`0193e388f34b83ef...`	`4403833a99495890...`
guix_build.log.diff		`c952affa422fa987...`

DrahtBot removed the label Needs Guix build on Dec 18, 2020

laanwj commented at 11:52 am on December 18, 2020: member

Code review ACK f792069bc117f11dff0db11f9386ec216581d2a2

DrahtBot commented at 0:22 am on December 20, 2020: member

Gitian builds

File	commit 816314ef0f7bdf50a6596ef893ac1a1d2d8723bf(master)	commit a130addf675b96eeed202c57aef9bdb43818704f(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz	`92c094a57f28b2fe...`	`1548d056613d4282...`
*-aarch64-linux-gnu.tar.gz	`6406c70036869155...`	`35279dc260fdb7d0...`
*-arm-linux-gnueabihf-debug.tar.gz	`fa1dcb7738417481...`	`ca60e75f26aec626...`
*-arm-linux-gnueabihf.tar.gz	`ba0552af3463d066...`	`75f4dbc9379560a9...`
*-osx-unsigned.dmg	`b38904a51c9c12d5...`	`daff62c8b99e1c3f...`
*-osx64.tar.gz	`81c94a20c6fe25d9...`	`91278b789e63ee3f...`
*-riscv64-linux-gnu-debug.tar.gz	`5199aceb15600fb9...`	`96a09768b7d1d42f...`
*-riscv64-linux-gnu.tar.gz	`79881c3a59eecd00...`	`e3d5f74e06f90228...`
*-win64-debug.zip	`9f6dc265cf93e377...`	`1edd6655f5b1714b...`
*-win64-setup-unsigned.exe	`4f64508a4629cde1...`	`37a5366b9495a77b...`
*-win64.zip	`85b0b7372d64928b...`	`42f64b3e2ceb78a2...`
*-x86_64-linux-gnu-debug.tar.gz	`db8b3d3f3967dbb4...`	`de7b27b3711a7318...`
*-x86_64-linux-gnu.tar.gz	`10c06c92f10e4d8d...`	`9b4bdc05eda43091...`
*.tar.gz	`b47efbff1aff7eeb...`	`0a9228076efb94af...`
bitcoin-core-linux-22-res.yml	`6b70a61e243f8979...`	`51e0aace21f10ff7...`
bitcoin-core-osx-22-res.yml	`c492c16c3445ba68...`	`7d32f7a68583a9ce...`
bitcoin-core-win-22-res.yml	`a4afeb86eac5227b...`	`e52f2cf7e4e4b43b...`
linux-build.log	`6e3648941ec583a8...`	`92e7f663c31a113b...`
osx-build.log	`951cd893b8181a69...`	`5d243fadaa77c6d9...`
win-build.log	`09e44a321a721c70...`	`3bcbf90cdf01d0df...`
bitcoin-core-linux-22-res.yml.diff		`1bae578e73833145...`
bitcoin-core-osx-22-res.yml.diff		`7ae483317f86820a...`
bitcoin-core-win-22-res.yml.diff		`4991a0017bd60e5a...`
linux-build.log.diff		`419e5a5bf80f4867...`
osx-build.log.diff		`f19ccbc6242d8138...`
win-build.log.diff		`311f56737cd53c51...`

DrahtBot removed the label Needs gitian build on Dec 20, 2020

DrahtBot added the label Needs rebase on Jan 22, 2021

build: Proper quoting for var printing targets

Previously, if the value contained syntax that was meaningful to make,
the printing would fail. Quoting properly avoids this.

b3bdff42b5

depends: Improve id string robustness

Environment variables and search paths can drastically effect the
operation of build tools.

Include these in our id string to mitigate against false cache hits.

4c7d418588

depends: Include GUIX_ENVIRONMENT in id string 5200929bfe

dongcarl force-pushed on Feb 5, 2021

DrahtBot removed the label Needs rebase on Feb 5, 2021

laanwj renamed this:
~~depends: Improve id string robustness~~
depends: Improve id string robustness
on Feb 9, 2021

laanwj commented at 10:42 am on February 15, 2021: member

re-ACK 5200929bfe26c549d7da92c0adf8adf61e143416

laanwj merged this on Feb 15, 2021

laanwj closed this on Feb 15, 2021

sidhujag referenced this in commit 58144a8550 on Feb 15, 2021

DrahtBot locked this on Aug 16, 2022

depends: Improve id string robustness #20629

Conflicts

Guix builds

Gitian builds