ci: switch MSAN to use prebuilt Clang binaries #31850

1. 
   fanquake commented at 5:45 pm on February 12, 2025: member

   Use the packages from https://apt.llvm.org/.
2. 
   DrahtBot commented at 5:45 pm on February 12, 2025: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Code Coverage & Benchmarks

   For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31850.

   Reviews

   See the guideline for information on the review process. A summary of reviews will appear here.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #31906 (ci: Switch to gcr.io mirror to avoid rate limits by maflcko)
   • #31802 (Add bitcoin-{node,gui} to release binaries for IPC by Sjors)
   • #30975 (ci: build multiprocess on most jobs by Sjors)
   • #28710 (Remove the legacy wallet and BDB dependency by achow101)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

3. DrahtBot added the label Tests on Feb 12, 2025
4. fanquake force-pushed on Feb 12, 2025
5. DrahtBot added the label CI failed on Feb 12, 2025
6. 
   DrahtBot commented at 5:51 pm on February 12, 2025: contributor

   🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/37113437498

   Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

   • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

   • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

   • An intermittent issue.

   Leave a comment here, if you need help tracking down a confusing failure.

7. DrahtBot removed the label CI failed on Feb 12, 2025
8. in ci/test/01_base_install.sh:52 in 4fdb96f10b outdated

   62-  ninja -C /msan/clang_build/ install-runtimes
   63-
   64-  update-alternatives --install /usr/bin/clang++ clang++ /msan/clang_build/bin/clang++ 100
   65-  update-alternatives --install /usr/bin/clang clang /msan/clang_build/bin/clang 100
   66-  update-alternatives --install /usr/bin/llvm-symbolizer llvm-symbolizer /msan/clang_build/bin/llvm-symbolizer 100
   67+  ${CI_RETRY_EXE} git clone --depth=1 https://github.com/llvm/llvm-project -b "llvmorg-${APT_LLVM_V}.1.0-rc1" /msan/llvm-project
   

   ---

   ---

   

   fanquake commented at 10:11 pm on February 12, 2025:

   https://github.com/llvm/llvm-project/releases/tag/llvmorg-20.1.0-rc2
9. maflcko approved
10. 
    maflcko commented at 8:37 am on February 13, 2025: member

    lgtm. The clang compilation was done back when APT_LLVM_V wasn’t available. Seems fine to use now and should be trivial to revert if there is an issue.
11. 
    willcl-ark commented at 10:12 am on February 13, 2025: member

    Also looks good to me!

    A docker build (with --no-cache set, so only using a cached Ubuntu base image) on my 11700k went from:

    0[+] Building 1521.2s (9/9) FINISHED                                          
    

    to

    0[+] Building 196.8s (9/9) FINISHED
    

    Should be a nice win on CI runs (and for users) where a rebuild would have previously been triggered (which can be caused by almost anything).

12. fanquake force-pushed on Feb 13, 2025
13. 
    fanquake commented at 11:17 am on February 13, 2025: member

    Swapped to rc2. Needs a fixup for MSAN fuzz.
14. 
    willcl-ark commented at 11:21 am on February 13, 2025: member

    For fuzzing end-users, you might consider touching up the wording in:

    https://github.com/bitcoin/bitcoin/blob/55cf39e4c54da6639a8f1f7c813c2909454cada1/doc/fuzzing.md?plain=1#L104-L112

    …to directly mention using pre-built llvm bins (although it does already say to use the CI job as a reference which is and will remain accurate).

15. ci: switch MSAN to use prebuilt Clang binaries

    Use the packages from https://apt.llvm.org/.

    3d5da9f28c
16. fanquake force-pushed on Feb 18, 2025
17. 
    maflcko commented at 3:16 pm on February 18, 2025: member

    Needs a fixup for MSAN fuzz.

    What is the error?

18. 
    maflcko commented at 9:48 am on February 20, 2025: member

    Needs a fixup for MSAN fuzz.

    What is the error?

    Yeah, I don’t think this works. This is probably https://github.com/bitcoin/bitcoin/issues/24289

19. 
    fanquake commented at 3:14 pm on February 20, 2025: member

    Yea, I don’t think so. We can circle back around to this soon.
20. fanquake closed this on Feb 20, 2025

    ---

Contributors
fanquake DrahtBot maflcko willcl-ark

Labels
Tests