ci: Reword stale readme #36574

faa4478 ci: Add a note on pinning dependencies:

I don't agree with this, but I won't block the change. These are all valid statements, but it's not complete.

pins can delay normal and security updates until the next bump

They can also do the opposite silently update to a buggy version - we've seen this before, it's not theoretical.

It does not constrain recursive downloads at runtime

Sure, they don't fix every single thing that could go wrong, but when things do, it helps in narrowing down what caused the inconsistency.

it could create a false sense of reproducibility and security

I don't see how that's different from what we're doing now, we're already pinning some versions and commits and hashes, where is the threshold for "sense of security"? In the same way as we're pinning C++ includes lately to reduce the number of moving parts, I would push towards pinning as much as we can in the externally controllable dependencies so that more changes are under our control - even if it won't ever be 100%. I don't like that each CI run is magically slightly different without notice...

They can also do the opposite silently update to a buggy version - we've seen this before, it's not theoretical.

I don't think this was ever seen in this CI. Do you mind providing a link where a buggy version lead to a CI bug that would have been prevented by a pin?

Sure, they don't fix every single thing that could go wrong, but when things do, it helps in narrowing down what caused the inconsistency.

We don't even consistently track stuff that goes wrong in internal test code. E.g. #35919#event-33782256873 silently re-runs an intermittent failure without reporting it.

I don't mind pinning, but if someone wants to do it, they'll have to explain what the expectations are around the review effort required for the maintenance churn. Ideally with a real-world observable benefit (e.g. an instance in the past where it would have helped).

I don't see how that's different from what we're doing now, we're already pinning some versions and commits and hashes, where is the threshold for "sense of security"?

Those pins are not for security. They are to catch transport and storage errors.

I don't like that each CI run is magically slightly different without notice...

The CI should be robust to pass on diverse underlying hardware and software, and it has extended logging, so that if something goes wrong, one can trace the logs to find the exact difference that caused a different outcome.

I don't feel too strongly here, so I am happy to drop this commit, but I think it can't hurt to explain the status-quo and it should be trivial to adjust or remove the doc in the future, if there is a rationale.

Do you mind providing a link where a buggy version lead to a CI bug that would have been prevented by a pin?

When the socket availability in the 32-bit Docker containers was suddenly changed in #35202

E.g. #35919#event-33782256873 silently re-runs an intermittent failure without reporting it.

It was a Windows timeout, see https://github.com/bitcoin/bitcoin/actions/runs/39458100061/job/118756715796

The CI should be robust to pass on diverse underlying hardware and software

Yes, but we should know what that software is, it shouldn't just change without notice.

When the socket availability in the 32-bit Docker containers was suddenly changed in #35202

docker isn't pinned and I don't think it can be pinned, as this is something the sysadmin provides and not the ci system. So I don't think this is relevant to pinning. In fact, it is a rationale against pinning, because that very docker update had a "CVE fix", so if docker were somehow pinned, users would continue running with the "vulnerable" docker, unless a bump was submitted and users backported that bump to all commit they are running the ci on.

Again, I don't mind pinning, but my preference would be:

• to have a rationale for it, ideally with an applicable real-world example; and
• to have a policy explained to minimize the review effort and maintenance churn.

so if docker were somehow pinned, users would continue running with the "vulnerable" docker

No, we would provide a PR to fix it, just like we did when it broke when third parties decided to push the changes to our CI - except that we wouldn't be spectators.