Historical scripted-diff failures (trivial) #29692

I ran through all 313 historical scripted-diff: commits as of 71b63195b30b2fa0dff20ebb262ce7566dd5d673.

I found several that don't pass, none are problematic:

• fa21fc60c292ab947b2200e54201440f16230566 (2022): fails because of missing file. Works if you run it manually and drop ren nSinceLastTry. It was indirectly noticed during review: #24697 (review)
• 3aa258109e3f3e0b1bfc4f811cbadfd6d516208c (2022): commit message misses a trailing -, but script does work
• f471a3be00c2b6433b8c258b716982c0539da13f (2020): scripted diff instead of scripted-diff marker. I can't get the script to run, but the commit just changes a few RPC strings.
• fb65dde147f63422c4148b089c2f5be0bf5ba80f (2021): throws "sed: no input files" a few times, but does produce the right result
• fa9d5902f7d72e8cce105dd1b1f5a1062e304b10 (2020): uses clang-format-diff which presumably has changed over time
• 728667b771e80388812d521eca03682c9366ce4b (2018): malformed commit message, but works

I don't think any action is required, other than perhaps double checking that merge scripts always run this. So this can be closed soon(tm).

I also found two instances that pass on Ubuntu, but not on macOS (despite having GNU sed and grep, see #29689):

• f0a2fb3c5dbf3c4bec7faf934baff3e723734b3f (2022): and several others in this PR
• 3704433c4f5ecf9f196860b2ccecae0d2c8b5f6e (2020): and several others in this PR

On macOS 14.4 with git 2.44.0 (via homebrew) and GNU grep 3.11 the line find_regex="\bpindexBestHeader\b" git grep -l -E "$find_regex" -- src returns nothing. Tried with bash and zsh. Doesn't really matter anymore.

Check for yourself (mostly copied from verify-commits.py):

(but be careful because you're about to run 300+ bash scripts)

#!/bin/sh
set -e
for commit in $(git rev-list HEAD); do
    # Skip known failures
    if test "$commit" = "fa21fc60c292ab947b2200e54201440f16230566"; then continue; fi
    if test "$commit" = "3aa258109e3f3e0b1bfc4f811cbadfd6d516208c"; then continue; fi
    if test "$commit" = "f471a3be00c2b6433b8c258b716982c0539da13f"; then continue; fi
    if test "$commit" = "fb65dde147f63422c4148b089c2f5be0bf5ba80f"; then continue; fi
    if test "$commit" = "fa9d5902f7d72e8cce105dd1b1f5a1062e304b10"; then continue; fi
    if test "$commit" = "728667b771e80388812d521eca03682c9366ce4b"; then continue; fi

    if git rev-list -n 1 --pretty="%s" "$commit" | grep -q "^scripted-diff:"; then
        git checkout --quiet "$commit"^ || exit
        SCRIPT="$(git rev-list --format=%b -n1 "$commit" | sed '/^-BEGIN VERIFY SCRIPT-$/,/^-END VERIFY SCRIPT-$/{//!b};d')"
        if test -z "$SCRIPT"; then
            echo "Error: missing script for: $commit"
            echo "Failed"
            exit 1
        else
            echo "Running script for: $commit"
            # If PyEnv is installed, ignore missing Python versions
            # for (a few) scripts that use Python
            if test -f ".python-version"; then
                pyenv local 3.12.1
            fi
            echo "$SCRIPT"
            (eval "$SCRIPT")
            if test -f ".python-version"; then
                git checkout HEAD .python-version
            fi
            git --no-pager diff --exit-code "$commit" && echo "OK" || (echo "Failed"; exit 1)
        fi
        git reset --quiet --hard HEAD
    else
        if git rev-list "--format=%b" -n1 "$commit" | grep -q '^-\(BEGIN\|END\)[ a-zA-Z]*-$'; then
            echo "Error: script block marker but no scripted-diff in title of commit $commit"
            echo "Failed"
            exit 1
        fi
    fi
done

How did they pass CI?

@maflcko good question, I initially thought it was because of this: https://github.com/bitcoin/bitcoin/pull/29660/files#r1533676542

Perhaps an earlier version of the CI scripts did get the commit range wrong? Or maybe there were many spurious CI failures that causes a failure to be overlooked? Though that seems unlikely for the linter job.

Closed at request of author.

IIRC the exit code is ignored either way, because ./test/lint/commit-script-check.sh doesn't set -e, but you do.

That may also be the reason they were not found by CI.