GCC + ASan debug builds: runtime SEGV in `sha256_sse4::Transform` on CPUs without SHA-NI #34881

Building Bitcoin Core with GCC + ASan in debug mode (CMAKE_BUILD_TYPE=Debug) causes an immediate SEGV on startup on CPUs without SHA-NI. The crash happens inside sha256_sse4::Transform during SHA256AutoDetect()'s self-test, before the node does anything else.

This only affects CPUs that use the SSE4 SHA256 code path (those without SHA-NI). The existing no_sanitize("address") workaround in sha256_sse4.cpp only covers Clang (#30097, #32437) - GCC is not handled.

I hit this while running the peer-observer/infra-library NixOS VM test suite (context), which builds Bitcoin Core with GCC and ASan enabled.

Crash trace

Excerpt from my original reproduction (NixOS, GCC 13.3.0):

AddressSanitizer:DEADLYSIGNAL
=================================================================
==680269==ERROR: AddressSanitizer: SEGV on unknown address 0x00006a09e607 (pc 0x55c20e591c00 bp 0x7fff925f4380 sp 0x7fff925f4250 T0)
==680269==The signal is caused by a WRITE memory access.
    [#0](/bitcoin-bitcoin/0/) 0x55c20e591c00 in sha256_sse4::Transform(unsigned int*, unsigned char const*, unsigned long) /home/deadmanoz/bitcoin/src/crypto/sha256_sse4.cpp:54
    [#1](/bitcoin-bitcoin/1/) 0x55c20e58dfb2 in SelfTest /home/deadmanoz/bitcoin/src/crypto/sha256.cpp:538
    [#2](/bitcoin-bitcoin/2/) 0x55c20e58ea47 in SHA256AutoDetect[abi:cxx11](sha256_implementation::UseImplementation) /home/deadmanoz/bitcoin/src/crypto/sha256.cpp:688
    [#3](/bitcoin-bitcoin/3/) 0x55c20b3fb220 in operator() /home/deadmanoz/bitcoin/src/kernel/context.cpp:19
    ...
    [#14](/bitcoin-bitcoin/14/) 0x55c20b235126 in AppInit /home/deadmanoz/bitcoin/src/bitcoind.cpp:199
    [#15](/bitcoin-bitcoin/15/) 0x55c20b2366bb in main /home/deadmanoz/bitcoin/src/bitcoind.cpp:283

SUMMARY: AddressSanitizer: SEGV /home/deadmanoz/bitcoin/src/crypto/sha256_sse4.cpp:54 in sha256_sse4::Transform(unsigned int*, unsigned char const*, unsigned long)
==680269==ABORTING

How to reproduce

On a CPU without SHA-NI:

cmake -B build -G Ninja \
  -DCMAKE_C_COMPILER=gcc \
  -DCMAKE_CXX_COMPILER=g++ \
  -DCMAKE_BUILD_TYPE=Debug \
  -DSANITIZERS=address
cmake --build build --target test_bitcoin -j$(nproc)
./build/bin/test_bitcoin --run_test=crypto_tests  # SEGV

On CPUs with SHA-NI, first force the SSE4 path. This patches SHA256AutoDetect() to skip SHA-NI detection, so the function falls through to sha256_sse4::Transform where the bug lives:

sed -i 's/have_x86_shani = (ebx >> 29) & 1;/have_x86_shani = false;/' src/crypto/sha256.cpp

Release builds have not been observed to trigger the crash.

Tested configurations

Reproduced across four configurations on three machines (GCC 13, 14, and 15) (Haswell CPUs without SHA-NI, NixOS and Ubuntu 24.04):

A CI matrix across GCC 13/14 confirms the pattern:

What's happening

In upstream master, the no_sanitize("address") attribute in sha256_sse4.cpp (lines 16-26) is wrapped in #if defined(__clang__). GCC skips that block entirely, so ASan instruments the function. ASan's instrumentation is incompatible with this inline assembly in the failing configuration.

The same no_sanitize("address") attribute works for both compilers. The only difference is how each compiler detects that ASan is active: Clang uses __has_feature(address_sanitizer), GCC uses __SANITIZE_ADDRESS__.

Suggested fix

Branch: fix/gcc-asan-sha256-sse4-only (PR to follow)

Add a GCC branch to the existing guard:

#if defined(__clang__)
  #if __has_feature(address_sanitizer)
    __attribute__((no_sanitize("address")))
  #endif
#elif defined(__GNUC__) && defined(__SANITIZE_ADDRESS__)
  __attribute__((no_sanitize("address")))
#endif
void Transform(uint32_t* s, const unsigned char* chunk, size_t blocks)
{

Related fix: src/util/check.h

Branch: fix/gcc-asan-check-h (separate PR, pending feedback on whether this is worth including?)

In upstream master, src/util/check.h has a similar gap for GCC < 14. The ASAN_POISON_MEMORY_REGION / ASAN_UNPOISON_MEMORY_REGION macros are gated by __has_feature(address_sanitizer), which GCC < 14 doesn't support. On those versions, the macros silently become no-ops, and the pool allocator's manual ASan poisoning (#32581) is inactive.

GCC 14+ added __has_feature, so this only affects GCC 13 and older. The fix adds __SANITIZE_ADDRESS__ as a fallback:

#if defined(__has_feature)
#    if __has_feature(address_sanitizer)
#       include <sanitizer/asan_interface.h>
#    endif
#elif defined(__SANITIZE_ADDRESS__)
#    include <sanitizer/asan_interface.h>
#endif

CI coverage comment

Bitcoin Core's CI only tests ASan with Clang. If GCC + ASan testing is not a goal, these fixes are still worth considering for consistency - downstream projects building with GCC + ASan (such as NixOS-based test infrastructure) will otherwise hit the crash or silently lose pool allocator coverage.

Related

• #29801 - Compilation failure with -O0 + -fsanitize=address due to inline asm
• #31913 - build: x86 afl++ ASan build broken "error: inline assembly requires more registers than available"
• #30097 - crypto: disable asan for sha256_sse4 with clang and -O0
• #32437 - crypto: disable ASan for sha256_sse4 with Clang
• llvm/llvm-project#92182 - clang: "expected relocatable expression" when compiling large inline asm with -O0 and address sanitizer

Is there a way to test it on hardware with SHA-NI? Tricking cmake into thinking shani isn't supported wasn't enough