Homebrew attempts to check for outdated dependents or those with broken linkage. Such behavior might lead to failures when Homebrew updates them on old macOS images. For example, https://github.com/bitcoin/bitcoin/actions/runs/7199058794/job/19609891263 using the macOS image version 20231025.2.
This PR prevents such behavior.
<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--006a51241073e994b41acfe9ec718e94-->
For detailed information about the code coverage, see the test coverage report.
<!--021abf342d371248e50ceaed478a90ca-->
See the guideline for information on the review process.
| Type | Reviewers |
|---|---|
| ACK | maflcko, ismaelsadeeq |
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.
utACK f3800ba21914dc91174c096291690847c1b47a92
Homebrew attempts to check for outdated dependents or those with broken
linkage. Such behavior might lead to failures when Homebrew updates them
on old macOS images.
This change prevents such behavior.
Sorry, HOMEBREW_NO_AUTO_UPDATE has been replaced with HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK, which does fix the issue. See https://github.com/bitcoin/bitcoin/actions/runs/7208749840/job/19638344437.
lgtm ACK 43c3246af774bda284111056268a814477f9b256
when Homebrew updates them on old macOS images.
Do we use old images?
What other side-effects could this cause due to not updating dependencies, and broken linage/other issues?
when Homebrew updates them on old macOS images.
Do we use old images?
The images are from GHA/Microsoft.
What other side-effects could this cause due to not updating dependencies, and broken linage/other issues?
This also speeds up the build by 5 Minutes.
Do we use old images?
The GHA chooses an image from a set of available images when setting up a new job.
It is not rare case when more than one images are available simultaneously. One of them is older than others.
It is not rare case when more than one images are available simultaneously. One of them is older than others.
What does "older" mean here? Aren't we pinned to a specific version to avoid exactly this issue?
It is not rare case when more than one images are available simultaneously. One of them is older than others.
What does "older" mean here? Aren't we pinned to a specific version to avoid exactly this issue?
For example,
##[group]Operating System
macOS
13.6
22G120
##[endgroup]
##[group]Runner Image
Image: macos-13
Version: 20231025.2
Included Software: https://github.com/actions/runner-images/blob/macos-13/20231025.2/images/macos/macos-13-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-13%2F20231025.2
##[endgroup]
vs
##[group]Operating System
macOS
13.6.1
22G313
##[endgroup]
##[group]Runner Image
Image: macos-13
Version: 20231204.4
Included Software: https://github.com/actions/runner-images/blob/macos-13/20231204.4/images/macos/macos-13-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-13%2F20231204.4
##[endgroup]
We pin only macos-13.
For example,. Image Release: https://github.com/actions/runner-images/releases/tag/macos-13%2F20231025.2 vs Image Release: https://github.com/actions/runner-images/releases/tag/macos-13%2F20231204.4
I still don't really understand this fix, and assume it could lead to obscure breakage in the future, i.e a missing dep, or a dep trying to use an outdated sub dep. (it also just makes our CI further differ from nomal macOS / dev setups).
I don't really understand how a new image being released, causes failures with the previous image, especially when, the image you're linking to above (20231204), was never even released? See: https://github.com/actions/runner-images/pull/8960
The release was cancelled, the badge was updated incorrectly - we are working to update the status and eliminate the causes of this "event" itself.
It also doesn't appear in https://github.com/actions/runner-images/tree/main/images/macos, so I don't understand how anything could be using this image, or how it's causing failures.
... so I don't understand how anything could be using this image, or how it's causing failures.
This PR does nothing about new 20231204 release, because the failure was caused by the older 20231025 one.
I think that the GHA image release process issues are irrelevant to this PR change.
I still don't really understand this fix, and assume it could lead to obscure breakage in the future, i.e a missing dep, or a dep trying to use an outdated sub dep. (it also just makes our CI further differ from nomal macOS / dev setups).
Our dependencies are installed explicitly. ~We do not use any of other tools preinstalled via Homebrew. Why should we care about their dependencies?~
I still don't really understand this fix
This fix makes Homebrew skip upgrading the aws-sam-cli package that we do not use.
We do not use any of other tools preinstalled via Homebrew.
Don't we use at least git, curl and Python? If any those are probably less-prone to breakage.
Don't we use at least git, curl and Python?
Right. Here is the brew list -1 command output for the image version 20231025.2:
ant
aom
apr
apr-util
argon2
aria2
aspell
autoconf
aws-sam-cli
azure-cli
bazelisk
berkeley-db
bicep
brotli
c-ares
ca-certificates
cairo
carthage
cffi
cmake
composer
curl
fontconfig
freetds
freetype
gcc
gcc@11
gcc@12
gd
gdbm
geckodriver
gettext
gh
giflib
git
git-lfs
glib
gmp
gnu-tar
gnupg
gnutls
gradle
graphite2
harfbuzz
highway
httpd
icu4c
imath
isl
jpeg-turbo
jpeg-xl
jq
kotlin
krb5
libassuan
libavif
libevent
libgcrypt
libgpg-error
libidn2
libksba
libmpc
libnghttp2
libpng
libpq
libsodium
libssh2
libtasn1
libtiff
libtool
libunistring
libusb
libuv
libvmaf
libx11
libxau
libxcb
libxdmcp
libxext
libxrender
libyaml
libzip
little-cms2
llvm@15
lz4
lzo
m4
maven
mpdecimal
mpfr
nettle
node@18
npth
oniguruma
openblas
openexr
openjdk
openldap
openssl@1.1
openssl@3
p11-kit
p7zip
packer
pcre2
perl
php
pinentry
pipx
pixman
pkg-config
pycparser
pygments
python-argcomplete
python-certifi
python-cryptography
python-packaging
python-pytz
python-setuptools
python-typing-extensions
python@3.10
python@3.11
python@3.12
pyyaml
r
readline
rtmpdump
ruby@3.0
rustup-init
selenium-server
six
sqlite
swiftformat
swiftlint
tcl-tk
tidy-html5
unbound
unixodbc
unxip
webp
wget
xorgproto
xz
yq
zstd
firefox
google-chrome
julia
microsoft-edge
session-manager-plugin
As CI does not suffer from unrelated failures now, there is time to investigate underlying issue more thoroughly and suggest a more robust alternative solution.
This, or some other solution will also need to be backported to 26.x. I think a simpler solution would have just been to uninstall aws-sam-cli as the first step. Which should solve the problem, have no other side-effects, and retain the speedup, given it's related to upgrading it's deps.
Won't the other dependencies also be upgraded and thus slow down the CI?
I don't know. However If the issue was sub dependencies (Fetching dependencies for aws-sam-cli: python-setuptools, python-attrs, python-dateutil, python-networkx, python-pbr, python-mpmath, python-sympy, python-urllib3, ca-certificates, cfn-lint, pygments, python-click, python-markupsafe, python-jinja, python-certifi, python-charset-normalizer, python-idna, python-requests, cookiecutter and python-cryptography) of something being upgraded (aws-sam-cli), also being upgraded, if (aws-sam-cli) is no-longer installed / being upgraded, I wouldn't think it's dependencies should be either?
This fix makes Homebrew skip upgrading the aws-sam-cli package that we do not use.
Ok the issue here also isn't specifically related to aws-sam-cli, because upgrading any Python will fail, due to symlinks not being able to be written.
Looks like ultimately, we can't avoid the failure by removing packages we don't use.
The root cause, is that qt@5 has a dependency on python@3.11, and after we install qt@5, brew will try and upgrade python@3.11, which will then fails:
==> Upgrading python@3.11
3.11.6 -> 3.11.6_1
==> Pouring python@3.11--3.11.6_1.ventura.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/2to3
Target /usr/local/bin/2to3
already exists. You may want to remove it:
rm '/usr/local/bin/2to3'
To force the link and overwrite all conflicting files:
brew link --overwrite python@3.11
To list all files that would be deleted:
brew link --overwrite --dry-run python@3.11
Possible conflicting files are:
/usr/local/bin/2to3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/2to3
/usr/local/bin/2to3-3.11 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/2to3-3.11
/usr/local/bin/idle3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/idle3
/usr/local/bin/idle3.11 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/idle3.11
/usr/local/bin/pydoc3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/pydoc3
/usr/local/bin/pydoc3.11 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/pydoc3.11
/usr/local/bin/python3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/python3
/usr/local/bin/python3-config -> /Library/Frameworks/Python.framework/Versions/3.12/bin/python3-config
/usr/local/bin/python3.11 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11
/usr/local/bin/python3.11-config -> /Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11-config