test: LLVM/Clang 16 for MSAN jobs #27436

fanquake commented at 3:27 PM on April 7, 2023: member

Similar to other CI infra changes we've made recently. Move to LLVM/Clang 16 for the MSAN jobs (which is currently using LLVM 12).

-fsanitize-memory-param-retval is turned on by default. With -fsanitize=memory, passing uninitialized variables to functions and returning uninitialized variables from functions is more aggressively reported. -fno-sanitize-memory-param-retval restores the previous behavior.

DrahtBot commented at 3:27 PM on April 7, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	dergoegge

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#27340 (ci: Use Cirrus CI dockerfile env by MarcoFalke)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot added the label Tests on Apr 7, 2023

in ci/test/01_base_install.sh:53 in dd542d6459 outdated

  54 | -  git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-12.0.0 "${BASE_SCRATCH_DIR}"/msan/llvm-project
  55 | -  cd "${BASE_SCRATCH_DIR}"/msan/build/ && cmake -DLLVM_ENABLE_PROJECTS='libcxx;libcxxabi' -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=MemoryWithOrigins -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_TARGETS_TO_BUILD=X86 ../llvm-project/llvm/
  56 | -  cd "${BASE_SCRATCH_DIR}"/msan/build/ && make "$MAKEJOBS" cxx
  57 | +  update-alternatives --install /usr/bin/clang++ clang++ "$(which clang++-16)" 100
  58 | +  update-alternatives --install /usr/bin/clang clang "$(which clang-16)" 100
  59 | +  git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-16.0.0 "${BASE_SCRATCH_DIR}"/msan/llvm-project

fanquake commented at 3:34 PM on April 7, 2023:

Could also bump this to 16.0.1 now https://github.com/llvm/llvm-project/compare/llvmorg-16.0.0...llvmorg-16.0.1.

fanquake commented at 3:47 PM on April 7, 2023:

Moved to 16.0.1.

in ci/test/00_setup_env_native_msan.sh:20 in dd542d6459 outdated

  19 |  # BDB generates false-positives and will be removed in future
  20 | -export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
  21 | +export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang-16' CXX='clang++-16' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
  22 |  export GOAL="install"
  23 | -export BITCOIN_CONFIG="--with-sanitizers=memory --disable-hardening --with-asm=no CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
  24 | +export BITCOIN_CONFIG="--with-sanitizers=memory --disable-hardening --with-asm=no CC=clang-16 CXX=clang++-16 CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"

maflcko commented at 3:42 PM on April 7, 2023:

any reason to change this here and elsewhere? Either you use update-alternatives, or hardcode the version, but doing both seems not needed?

fanquake commented at 3:47 PM on April 7, 2023:

Right, no need to change these with update-alternatives. Dropped.

maflcko approved

maflcko commented at 3:42 PM on April 7, 2023: member

lgtm

fanquake force-pushed on Apr 7, 2023

test: LLVM/Clang 16 for MSAN jobs

Sync up with other CI infra.

676671527f

in ci/test/01_base_install.sh:54 in c0aa5f7105 outdated

  55 | -  cd "${BASE_SCRATCH_DIR}"/msan/build/ && cmake -DLLVM_ENABLE_PROJECTS='libcxx;libcxxabi' -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=MemoryWithOrigins -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_TARGETS_TO_BUILD=X86 ../llvm-project/llvm/
  56 | -  cd "${BASE_SCRATCH_DIR}"/msan/build/ && make "$MAKEJOBS" cxx
  57 | +  update-alternatives --install /usr/bin/clang++ clang++ "$(which clang++-16)" 100
  58 | +  update-alternatives --install /usr/bin/clang clang "$(which clang-16)" 100
  59 | +  git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-16.0.1 "${BASE_SCRATCH_DIR}"/msan/llvm-project
  60 | +  cmake -B "${BASE_SCRATCH_DIR}"/msan/build/ -DLLVM_ENABLE_RUNTIMES='libcxx;libcxxabi' -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=MemoryWithOrigins -DCMAKE_C_COMPILER=clang-16 -DCMAKE_CXX_COMPILER=clang++-16 -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF -S "${BASE_SCRATCH_DIR}"/msan/llvm-project/runtimes

maflcko commented at 9:44 AM on April 10, 2023:

  cmake -B "${BASE_SCRATCH_DIR}"/msan/build/ -DLLVM_ENABLE_RUNTIMES='libcxx;libcxxabi' -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=MemoryWithOrigins -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF -S "${BASE_SCRATCH_DIR}"/msan/llvm-project/runtimes

nit?

fanquake commented at 9:57 AM on April 10, 2023:

Yea. Fixed

fanquake force-pushed on Apr 10, 2023

dergoegge commented at 9:42 AM on April 11, 2023: member

utACK 676671527f08ef8113af3661de73db583f6ea9e2

fanquake merged this on Apr 11, 2023

fanquake closed this on Apr 11, 2023

fanquake deleted the branch on Apr 11, 2023

sidhujag referenced this in commit eaa6da3dc7 on Apr 11, 2023

maflcko commented at 6:52 AM on April 13, 2023: member

ci_native_fuzz_msan fails for me locally

bitcoin locked this on Apr 12, 2024