crypto: disable ASan instrumentation of SSE4 SHA256 for GCC (matching Clang) #34953

1. 
   deadmanoz commented at 3:15 am on March 30, 2026: contributor

   Fix the runtime crash described in #34881.

   Upstream already disables ASan instrumentation for sha256_sse4::Transform() under Clang. This extends the same workaround to GCC by adding an #elif branch for __GNUC__ / __SANITIZE_ADDRESS__ that applies the same no_sanitize("address") attribute.

   Testing:

   • reproduced the crash before the fix with GCC 13, 14, and 15 on Haswell-class machines / guests without SHA-NI (including by forcing the SSE4 implementation on GitHub CI)
   • SEGV in debug builds regardless of optimization level (tested -O0, -O1, -O2, -O3)
   • verified that the GCC + ASan debug configurations that previously crashed pass with this change

   Issue #34881 has more details about the issue.

   Note: the original Clang code placed the __attribute__ between the function declarator and the opening brace. GCC’s Attribute Syntax documentation notes that this position in a function definition “may, in future, be permitted,” so it is not currently supported. Placing the attribute at the start of the function definition is valid form for both GCC and Clang.

2. crypto: disable ASan instrumentation of SSE4 SHA256 for GCC

   The existing Clang-only no_sanitize("address") guard is extended to
   also cover GCC.  When GCC compiles this file with -fsanitize=address
   in debug builds, the instrumented inline assembly causes a SEGV during
   SHA256AutoDetect()'s self-test on CPUs that use the SSE4 code path
   (i.e. those without SHA-NI support), regardless of optimization level.
   
   The original Clang code placed the attribute between the function
   declarator and the opening brace.  GCC's Attribute Syntax
   documentation notes that this position in a function definition
   "may, in future, be permitted," so it is not currently supported.
   The attribute is moved to the start of the function definition,
   which is valid form for both GCC and Clang.
   
   The preprocessor guards are restructured so each compiler branch is
   explicit: __clang__ with __has_feature, and __GNUC__ with
   __SANITIZE_ADDRESS__.

   80160abe68
3. DrahtBot added the label Utils/log/libs on Mar 30, 2026
4. 
   DrahtBot commented at 3:15 am on March 30, 2026: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Reviews

   See the guideline for information on the review process. A summary of reviews will appear here.

Contributors
deadmanoz DrahtBot maflcko

Labels
Utils/log/libs