ci: run native fuzz with MSAN job

fanquake commented at 4:57 pm on October 14, 2025: member

I think this job should exist in this repo (not just qa-assets), if the alternative is double-handling changes to the interpreter. #32998 made changes which were then re-changed in #33600, to work around a false positive.

The unchached runtime of this job with -lg is ~32m, with -md it’s ~43m.

Timeout is set to 150m, as the slow GHA runners were close to hitting a 120m limit.

DrahtBot added the label Tests on Oct 14, 2025

DrahtBot commented at 4:58 pm on October 14, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33626.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Stale ACK	dergoegge

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

dergoegge commented at 5:06 pm on October 14, 2025: member

Concept ACK

fanquake commented at 5:22 pm on October 14, 2025: member

This has actually failed in a different way:

 0+ LD_LIBRARY_PATH=/home/admin/actions-runner/_work/_temp/depends/x86_64-pc-linux-gnu/lib
 1+ /home/admin/actions-runner/_work/_temp/build/test/fuzz/test_runner.py -j16 -l DEBUG /home/admin/actions-runner/_work/_temp/ci/scratch/qa-assets/fuzz_corpora/ --empty_min_time=60
 2==11758==WARNING: MemorySanitizer: use-of-uninitialized-value
 3    [#0](/bitcoin-bitcoin/0/) 0x56180d820d70 in SetArgs(int, char**) /home/admin/actions-runner/_work/_temp/build/src/test/fuzz/util/./test/fuzz/fuzz.cpp:52:5
 4    [#1](/bitcoin-bitcoin/1/) 0x56180d820d70 in LLVMFuzzerInitialize /home/admin/actions-runner/_work/_temp/build/src/test/fuzz/util/./test/fuzz/fuzz.cpp:223:5
 5    [#2](/bitcoin-bitcoin/2/) 0x56180ca378cb in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/admin/actions-runner/_work/_temp/build/bin/fuzz+0xe368cb) (BuildId: 1247816ab83eefd9a1d26eb30ceb8bc67293109d)
 6    [#3](/bitcoin-bitcoin/3/) 0x56180ca66056 in main (/home/admin/actions-runner/_work/_temp/build/bin/fuzz+0xe65056) (BuildId: 1247816ab83eefd9a1d26eb30ceb8bc67293109d)
 7    [#4](/bitcoin-bitcoin/4/) 0x7f307471b1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 274eec488d230825a136fa9c4d85370fed7a0a5e)
 8    [#5](/bitcoin-bitcoin/5/) 0x7f307471b28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 274eec488d230825a136fa9c4d85370fed7a0a5e)
 9    [#6](/bitcoin-bitcoin/6/) 0x56180ca2e264 in _start (/home/admin/actions-runner/_work/_temp/build/bin/fuzz+0xe2d264) (BuildId: 1247816ab83eefd9a1d26eb30ceb8bc67293109d)
10
11  Member fields were destroyed
12    [#0](/bitcoin-bitcoin/0/) 0x56180caa53a1 in __sanitizer_dtor_callback_fields (/home/admin/actions-runner/_work/_temp/build/bin/fuzz+0xea43a1) (BuildId: 1247816ab83eefd9a1d26eb30ceb8bc67293109d)
13    [#1](/bitcoin-bitcoin/1/) 0x56180ca28e3e in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /cxx_build/include/c++/v1/string:903:3
14    [#2](/bitcoin-bitcoin/2/) 0x56180ca28e3e in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /cxx_build/include/c++/v1/string:1208:3
15    [#3](/bitcoin-bitcoin/3/) 0x56180ca28e3e in std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, script_verify_flag_name>::~pair() /cxx_build/include/c++/v1/__utility/pair.h:90:8
16    [#4](/bitcoin-bitcoin/4/) 0x56180ca28e3e in __cxx_global_var_init.18 /home/admin/actions-runner/_work/_temp/build/src/./script/interpreter.cpp:2167:54
17    [#5](/bitcoin-bitcoin/5/) 0x56180ca28e3e in _GLOBAL__sub_I_interpreter.cpp /home/admin/actions-runner/_work/_temp/build/src/./script/interpreter.cpp
18    [#6](/bitcoin-bitcoin/6/) 0x7f307471b303 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a303) (BuildId: 274eec488d230825a136fa9c4d85370fed7a0a5e)
19    [#7](/bitcoin-bitcoin/7/) 0x56180ca2e264 in _start (/home/admin/actions-runner/_work/_temp/build/bin/fuzz+0xe2d264) (BuildId: 1247816ab83eefd9a1d26eb30ceb8bc67293109d)
20
21SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/admin/actions-runner/_work/_temp/build/src/test/fuzz/util/./test/fuzz/fuzz.cpp:52:5 in SetArgs(int, char**)
22Exiting

Looks like #30760?

maflcko commented at 6:32 am on October 15, 2025: member

No objection, but my worry is that the error is not exactly trivial to understand and actionable:

In https://github.com/bitcoin/bitcoin/commit/9cb9651d92ddb5d92724f6a52440601c7a0bbcf8 it was fixed by changing the linker. While the patch looks harmless and is easy to test to fix the issue, I don’t think it is straightforward to see why a runtime sanitizer issue is addressed by link-time changes.
In #29676 (comment) it was fixed by accident as a side product of a compiler version bump.

So making this task a blocker for all pull requests could mean a pull request is blocked for unrelated and unclear reasons. Maybe that is fine, though, I just wanted to mention it.

fanquake commented at 9:32 am on October 15, 2025: member

my worry is that the error is not exactly trivial to understand and actionable:

I agree that it’s odd, and I’m wondering why it’s not happening (seemingly at all?) in the qa-assets repo?

fanquake force-pushed on Oct 15, 2025

maflcko commented at 9:42 am on October 15, 2025: member

I agree that it’s odd, and I’m wondering why it’s not happening (seemingly at all?) in the qa-assets repo?

I can see it happening here: https://github.com/bitcoin-core/qa-assets/actions/runs/18426954137/job/52509387674#step:7:5257:

 0==11740==WARNING: MemorySanitizer: use-of-uninitialized-value
 1    [#0](/bitcoin-bitcoin/0/) 0x55e39ba5db80 in SetArgs(int, char**) /home/runner/work/_temp/build/src/test/fuzz/util/./test/fuzz/fuzz.cpp:52:5
 2    [#1](/bitcoin-bitcoin/1/) 0x55e39ba5db80 in LLVMFuzzerInitialize /home/runner/work/_temp/build/src/test/fuzz/util/./test/fuzz/fuzz.cpp:223:5
 3    [#2](/bitcoin-bitcoin/2/) 0x55e39ac758bb in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/runner/work/_temp/build/bin/fuzz+0xe328bb) (BuildId: 922987ad2b7e961f9d98e26bd1c329dc3b74d487)
 4    [#3](/bitcoin-bitcoin/3/) 0x55e39aca4046 in main (/home/runner/work/_temp/build/bin/fuzz+0xe61046) (BuildId: 922987ad2b7e961f9d98e26bd1c329dc3b74d487)
 5    [#4](/bitcoin-bitcoin/4/) 0x7f617ba6c1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 274eec488d230825a136fa9c4d85370fed7a0a5e)
 6    [#5](/bitcoin-bitcoin/5/) 0x7f617ba6c28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 274eec488d230825a136fa9c4d85370fed7a0a5e)
 7    [#6](/bitcoin-bitcoin/6/) 0x55e39ac6c254 in _start (/home/runner/work/_temp/build/bin/fuzz+0xe29254) (BuildId: 922987ad2b7e961f9d98e26bd1c329dc3b74d487)
 8
 9  Member fields were destroyed
10    [#0](/bitcoin-bitcoin/0/) 0x55e39ace3391 in __sanitizer_dtor_callback_fields (/home/runner/work/_temp/build/bin/fuzz+0xea0391) (BuildId: 922987ad2b7e961f9d98e26bd1c329dc3b74d487)
11    [#1](/bitcoin-bitcoin/1/) 0x55e39ac66e2e in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /cxx_build/include/c++/v1/string:903:3
12    [#2](/bitcoin-bitcoin/2/) 0x55e39ac66e2e in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /cxx_build/include/c++/v1/string:1208:3
13    [#3](/bitcoin-bitcoin/3/) 0x55e39ac66e2e in std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, script_verify_flag_name>::~pair() /cxx_build/include/c++/v1/__utility/pair.h:90:8
14    [#4](/bitcoin-bitcoin/4/) 0x55e39ac66e2e in __cxx_global_var_init.18 /home/runner/work/_temp/build/src/./script/interpreter.cpp:2167:54
15    [#5](/bitcoin-bitcoin/5/) 0x55e39ac66e2e in _GLOBAL__sub_I_interpreter.cpp /home/runner/work/_temp/build/src/./script/interpreter.cpp
16    [#6](/bitcoin-bitcoin/6/) 0x7f617ba6c303 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a303) (BuildId: 274eec488d230825a136fa9c4d85370fed7a0a5e)
17    [#7](/bitcoin-bitcoin/7/) 0x55e39ac6c254 in _start (/home/runner/work/_temp/build/bin/fuzz+0xe29254) (BuildId: 922987ad2b7e961f9d98e26bd1c329dc3b74d487)
18
19SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/runner/work/_temp/build/src/test/fuzz/util/./test/fuzz/fuzz.cpp:52:5 in SetArgs(int, char**)
20Exiting
21Traceback (most recent call last):
22  File "/home/runner/work/_temp/build/test/fuzz/test_runner.py", line 404, in <module>
23    main()
24  File "/home/runner/work/_temp/build/test/fuzz/test_runner.py", line 111, in main
25    test_list_all = parse_test_list(
26                    ^^^^^^^^^^^^^^^^
27  File "/home/runner/work/_temp/build/test/fuzz/test_runner.py", line 390, in parse_test_list
28    test_list_all = subprocess.run(
29                    ^^^^^^^^^^^^^^^
30  File "/usr/lib/python3.12/subprocess.py", line 571, in run
31    raise CalledProcessError(retcode, process.args,
32subprocess.CalledProcessError: Command '/home/runner/work/_temp/build/bin/fuzz' returned non-zero exit status 1.
33Command '['./ci/test/02_run_container.sh']' returned non-zero exit status 1.
34Error: Process completed with exit code 1.

fanquake force-pushed on Oct 15, 2025

in .github/workflows/ci.yml:534 in 5c7bd7fadd outdated

518@@ -519,6 +519,12 @@ jobs:
519             timeout-minutes: 120
520             file-env: './ci/test/00_setup_env_native_tsan.sh'
521 
522+          - name: 'MSan, fuzz'
523+            cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-lg'

maflcko commented at 11:24 am on October 15, 2025:

0            cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'

I wonder if md gives a speedup, similar to the valgrind task? #33461 (review)

Also on GHA runners, the timeout is short a few minutes of being hit?

fanquake commented at 1:10 pm on October 29, 2025:

Pushed up -md for a look.

fanquake force-pushed on Oct 22, 2025

fanquake force-pushed on Oct 23, 2025

fanquake marked this as ready for review on Oct 23, 2025

fanquake force-pushed on Oct 24, 2025

dergoegge approved

dergoegge commented at 10:19 am on October 29, 2025: member

utACK 362587409767eb349cd4f679db71a1e5bf407bb8

ci: use LLVM libcxx 21.1.4 677f0dc462

fanquake force-pushed on Oct 29, 2025

ci: run native fuzz with MSAN job

Set the timout to 150, to give some leeway to the slow GHA runners, that
were close to timing out with a 120m limit.

f043bedee1

in .github/workflows/ci.yml:531 in 400db134f2

524@@ -525,6 +525,12 @@ jobs:
525             timeout-minutes: 120
526             file-env: './ci/test/00_setup_env_native_tsan.sh'
527 
528+          - name: 'MSan, fuzz'
529+            cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
530+            fallback-runner: 'ubuntu-24.04'
531+            timeout-minutes: 120

maflcko commented at 1:23 pm on October 29, 2025:

on the slow GHA runners, the timeout is short a few minutes of being hit?

fanquake commented at 1:27 pm on October 29, 2025:

Ah, will bump this to ~150.

fanquake force-pushed on Oct 29, 2025

fanquake commented at 2:47 pm on October 29, 2025: member

-md runtime was 43 minutes with no caches (libccxx/depends/cacche).

fanquake requested review from dergoegge on Oct 30, 2025

ci: run native fuzz with MSAN job #33626

Code Coverage & Benchmarks

Reviews

Conflicts