ci: Enable pipefail in 03_test

maflcko commented at 12:45 pm on March 11, 2026: member

The CI script is problematic, because it is written in Bash, without pipefail enabled. Thus, some failures are silently ignored.

Enabling pipefail is a bit tedious, because:

The IWYU task has no (--verbose) ccache output, so the pipe fails after grep [1]. Also, right now on master, the if silently skips: ci/test/03_test_script.sh: line 122: [: : integer expression expected.
The Alpine task has Hits: twice in the output, so the pipe fails after head -1 [2]

Not sure what the easiest way to fix this would be. Some options are:

Just use tail -1 and 0 as fallback: hit_rate=$(ccache --show-stats | grep "Hits:" | tail -1 | sed 's/.*($.*$%).*/\1/' || echo "0")
Properly parse, using Python and --print-stats (this pull)

[1]

 0+ ccache --version
 1+ head -n 1
 2ccache version 4.11.2
 3+ ccache --show-stats --verbose
 4Cache directory:    /home/admin/actions-runner/_work/_temp/ccache_dir
 5Config file:        /home/admin/actions-runner/_work/_temp/ccache_dir/ccache.conf
 6System config file: /etc/ccache.conf
 7Stats updated:      Tue Feb 17 08:40:20 2026
 8Local storage:
 9  Cache size (GB):  0.0 / 2.0 ( 0.00%)
10  Files:              0
11  Hits:               0
12  Misses:             0
13  Reads:              0
14  Writes:             0
15++ ccache --show-stats
16++ grep Hits:
17++ head -1
18++ sed 's/.*(\(.*\)%).*/\1/'
19+ hit_rate=
20Command '['docker', 'exec', '--env', 'DANGER_RUN_CI_ON_HOST=1', 'f5e8f319c22101ada5be9d4c5fd7d883ce37b830e86ec64627cb7d2b96749053', '/home/admin/actions-runner/_work/_temp/ci/test/03_test_script.sh']' returned non-zero exit status 1.
21Error: Process completed with exit code 1.

[2]

 0+ ccache --version
 1+ head -n 1
 2ccache version 4.12.1
 3+ ccache --show-stats --verbose
 4Cache directory:    /home/admin/actions-runner/_work/_temp/ccache_dir
 5Config file:        /home/admin/actions-runner/_work/_temp/ccache_dir/ccache.conf
 6System config file: /etc/ccache.conf
 7Stats updated:      Tue Feb 17 08:40:35 2026
 8Cacheable calls:     873 / 873 (100.0%)
 9  Hits:              846 / 873 (96.91%)
10    Direct:          822 / 846 (97.16%)
11    Preprocessed:     24 / 846 ( 2.84%)
12  Misses:             27 / 873 ( 3.09%)
13Successful lookups:
14  Direct:            822 / 873 (94.16%)
15  Preprocessed:       24 /  51 (47.06%)
16Local storage:
17  Cache size (GB):   2.0 / 2.0 (99.95%)
18  Files:            2580
19  Cleanups:           13
20  Hits:              846 / 873 (96.91%)
21  Misses:             27 / 873 ( 3.09%)
22  Reads:            1772
23  Writes:             52
24++ ccache --show-stats
25++ grep Hits:
26++ head -1
27++ sed 's/.*(\(.*\)%).*/\1/'
28+ hit_rate=96.91
29Command '['docker', 'exec', '--env', 'DANGER_RUN_CI_ON_HOST=1', '272a66a48206f1f6096612e196127ce46ea4dbff5dc14be3a4a20c4ee523956f', '/home/admin/actions-runner/_work/_temp/ci/test/03_test_script.sh']' returned non-zero exit status 141.
30Error: Process completed with exit code 1.

ci: Enable pipefail in 03_test_script.sh

This requires re-writing the --show-stats parsing to use --print-stats
parsing, to avoid pipefail failures.

0509ac7743

DrahtBot added the label Tests on Mar 11, 2026

DrahtBot commented at 12:45 pm on March 11, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Approach ACK	BrandonOdiwuor

If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

maflcko commented at 12:46 pm on March 11, 2026: member

Draft for now, because CI fails here IIRC.

DrahtBot added the label CI failed on Mar 11, 2026

willcl-ark commented at 8:21 pm on March 11, 2026: member

How about:

 0ccache --version
 1ccache --show-stats --verbose
 2ccache --print-stats | python3 -c '
 3import os
 4import sys
 5
 6hits = 0
 7miss = 0
 8
 9for line in sys.stdin:
10    key, value = line.split("\t", 1)
11    if key in ("local_storage_hit", "primary_storage_hit"):
12        hits = int(value)
13    elif key in ("local_storage_miss", "primary_storage_miss"):
14        miss = int(value)
15
16calls = hits + miss
17rate = (hits / calls * 100) if calls else 0
18print(f"{rate:.2f}")
19if rate < 75:
20    container = os.environ["CONTAINER_NAME"]
21    print(
22        "::notice title=low ccache hitrate::"
23        f"Ccache hit-rate in {container} was {rate:.2f}%"
24    )
25'

No possible pipe failures in this non-critical section, I think :)

(slightly more verbose logs, but I think that’s fine?)

BrandonOdiwuor commented at 5:55 am on March 27, 2026: contributor

Approach ACK 0509ac7743b8234964db52ae31464edf2f614ae1

Enabling pipefail in ci/test/03_test_script.sh and replacing the fragile ccache hit-rate shell parsing with a clean Python script (using --print-stats) is a nice cleanup.

ci: Enable pipefail in 03_test_script.sh #34801

Reviews