doc: document workaround and fallback for macOS fuzzing #32084

pull l0rinc wants to merge 1 commits into bitcoin:master from l0rinc:l0rinc/libfuzzer-nosan-mac changing 1 files +29 −10
  1. l0rinc commented at 12:18 pm on March 17, 2025: contributor

    After the recent adjustment to fuzzing docs in #31954, most of the tests started working on macOS again.

    A few of them are still failing for weird reasons, let’s document how to work around those:

    Since libfuzzer-nosan builds to a different folder, I’ve added the full build steps after configuration. I’ve also deleted the brew install llvm duplication, fixed a typo (non-systems clang), and adjusted the fuzzer link for mac in Quickstart guide

  2. DrahtBot commented at 12:18 pm on March 17, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32084.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

  3. DrahtBot added the label Docs on Mar 17, 2025
  4. brunoerg commented at 12:41 pm on March 17, 2025: contributor

    “Changing the preset to avoid the other sanitizers seems to solve the issue.”

    It would be good to understand why it doesn’t work anymore first.

  5. l0rinc commented at 1:19 pm on March 17, 2025: contributor

    Thanks for the hint @brunoerg, I did a bisect to see if it’s the commits or the update, which gave me contradictory results. Clearing all build caches however fixed it for master as well:

    0ccache -C && git clean -fxd && git reset --hard

    Guess we can’t rely on cache invalidation here.

  6. l0rinc closed this on Mar 17, 2025
  7. hebasto commented at 1:33 pm on March 17, 2025: member

    Thanks for the hint @brunoerg, I did a bisect to see if it’s the commits or the update, which gave me contradictory results. Clearing all build caches however fixed it for master as well:

    0ccache -C && git clean -fxd && git reset --hard

    Guess we can’t rely on cache invalidation here.

    I’d appreciate it if you could provide steps to reproduce the scenario where Ccache cache causes a build issue.

  8. l0rinc commented at 1:39 pm on March 17, 2025: contributor
    I can’t, I deleted everything as the above steps show. But it was likely the local build folder and not ccache (based on how git bisect behaved, i.e. after the first build passed, everything did).
  9. l0rinc deleted the branch on Mar 17, 2025
  10. brunoerg commented at 1:20 pm on March 18, 2025: contributor

    Thanks for the hint @brunoerg, I did a bisect to see if it’s the commits or the update, which gave me contradictory results. Clearing all build caches however fixed it for master as well:

    0ccache -C && git clean -fxd && git reset --hard

    Guess we can’t rely on cache invalidation here.

    I’d appreciate it if you could provide steps to reproduce the scenario where Ccache cache causes a build issue.

    I’m getting the same issue and it works when I explicity set -DAPPEND_LDFLAGS="-fsanitize=..."

    e.g.:

    0cmake --preset=libfuzzer-nosan \
1   -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
2   -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
3   -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" \
4   -DAPPEND_LDFLAGS="-fsanitize=fuzzer"
  11. l0rinc commented at 8:15 pm on March 25, 2025: contributor

    I’m getting the same issue and it works when

    It’s still not working for me in a few cases, no matter how many caches I delete or how much I go back in time. And the failures don’t seem to make any sense to me:

    0rm -rfd build_fuzz \
1  && cmake --preset=libfuzzer \
2  -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
3  -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
4  -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" \
5  && cmake --build build_fuzz \
6  && FUZZ=psbt_base64_decode build_fuzz/bin/fuzz

    which fails immediately with

     0fuzz(70461,0x1ec764840) malloc: nano zone abandoned due to inability to reserve vm space.
 1INFO: Running with entropic power schedule (0xFF, 100).
 2INFO: Seed: 1288419648
 3INFO: Loaded 1 modules   (1253405 inline 8-bit counters): 1253405 [0x107024000, 0x10715601d), 
 4INFO: Loaded 1 PC tables (1253405 PCs): 1253405 [0x107156020,0x1084761f0), 
 5INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
 6libc++abi: terminating due to uncaught exception of type std::__1::ios_base::failure: DataStream::read(): end of data: unspecified iostream_category error
 7==70461== ERROR: libFuzzer: deadly signal
 8    [#0](/bitcoin-bitcoin/0/) 0x00010a3c9248 in __sanitizer_print_stack_trace+0x28 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x5d248)
 9    [#1](/bitcoin-bitcoin/1/) 0x00010661c544 in fuzzer::PrintStackTrace()+0x2c (fuzz:arm64+0x1037a4544)
10    [#2](/bitcoin-bitcoin/2/) 0x00010660fef8 in fuzzer::Fuzzer::CrashCallback()+0x54 (fuzz:arm64+0x103797ef8)
11    [#3](/bitcoin-bitcoin/3/) 0x000182e2ade0 in _sigtramp+0x34 (libsystem_platform.dylib:arm64+0x3de0)
12    [#4](/bitcoin-bitcoin/4/) 0x9d18800182df3f6c  (<unknown module>)
13    [#5](/bitcoin-bitcoin/5/) 0x7c15800182d00904  (<unknown module>)
14    [#6](/bitcoin-bitcoin/6/) 0xd303800182daa448  (<unknown module>)
15    [#7](/bitcoin-bitcoin/7/) 0x1925000182d98a20  (<unknown module>)
16    [#8](/bitcoin-bitcoin/8/) 0x1e31800182a413f0  (<unknown module>)
17    [#9](/bitcoin-bitcoin/9/) 0x242b000182da970c  (<unknown module>)
18    [#10](/bitcoin-bitcoin/10/) 0x3e28000182daccd8  (<unknown module>)
19    [#11](/bitcoin-bitcoin/11/) 0xc62c800182dacc80  (<unknown module>)
20    [#12](/bitcoin-bitcoin/12/) 0x7e1e80010309ac20  (<unknown module>)
21    [#13](/bitcoin-bitcoin/13/) 0x000103248c84 in void PartiallySignedTransaction::Unserialize<DataStream>(DataStream&)+0x224 (fuzz:arm64+0x1003d0c84)
22    [#14](/bitcoin-bitcoin/14/) 0x000103e66ad0 in DecodeRawPSBT(PartiallySignedTransaction&, std::__1::span<std::byte const, 18446744073709551615ul>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&)+0x200 (fuzz:arm64+0x100feead0)
23    [#15](/bitcoin-bitcoin/15/) 0x000103e6645c in DecodeBase64PSBT(PartiallySignedTransaction&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&)+0x1dc (fuzz:arm64+0x100fee45c)
24    [#16](/bitcoin-bitcoin/16/) 0x000102ec7a68 in psbt_base64_decode_fuzz_target(std::__1::span<unsigned char const, 18446744073709551615ul>)+0x270 (fuzz:arm64+0x10004fa68)
25    [#17](/bitcoin-bitcoin/17/) 0x000103b324d8 in LLVMFuzzerTestOneInput+0x198 (fuzz:arm64+0x100cba4d8)
26    [#18](/bitcoin-bitcoin/18/) 0x0001066114a4 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long)+0x12c (fuzz:arm64+0x1037994a4)
27    [#19](/bitcoin-bitcoin/19/) 0x000106612614 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>&)+0x220 (fuzz:arm64+0x10379a614)
28    [#20](/bitcoin-bitcoin/20/) 0x000106612c9c in fuzzer::Fuzzer::Loop(std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>&)+0x98 (fuzz:arm64+0x10379ac9c)
29    [#21](/bitcoin-bitcoin/21/) 0x000106609654 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1dd0 (fuzz:arm64+0x103791654)
30    [#22](/bitcoin-bitcoin/22/) 0x00010661cf48 in main+0x24 (fuzz:arm64+0x1037a4f48)
31    [#23](/bitcoin-bitcoin/23/) 0x000182a74270  (<unknown module>)
32    [#24](/bitcoin-bitcoin/24/) 0x9d3afffffffffffc  (<unknown module>)
33
34NOTE: libFuzzer has rudimentary signal handlers.
35      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
36SUMMARY: libFuzzer: deadly signal
37MS: 0 ; base unit: 0000000000000000000000000000000000000000
38
39
40artifact_prefix='./'; Test unit written to ./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709
41Base64:

    Which doesn’t make sense since PartiallySignedTransaction::Unserialize in DecodeRawPSBT is called inside a try/catch (const std::exception& e).

    Similarly to #32089, --preset=libfuzzer-nosan works with some warnings:

    0[100%] Built target fuzz
1WARNING: Failed to find function "__sanitizer_acquire_crash_state". Reason dlsym(RTLD_DEFAULT, __sanitizer_acquire_crash_state): symbol not found.
2WARNING: Failed to find function "__sanitizer_print_stack_trace". Reason dlsym(RTLD_DEFAULT, __sanitizer_print_stack_trace): symbol not found.
3WARNING: Failed to find function "__sanitizer_set_death_callback". Reason dlsym(RTLD_DEFAULT, __sanitizer_set_death_callback): symbol not found.
4INFO: Running with entropic power schedule (0xFF, 100).
  12. l0rinc restored the branch on Mar 25, 2025
  13. l0rinc commented at 8:18 pm on March 25, 2025: contributor
    I’ll reopen since others seem to have a similar problem as well - once we figure out what’s causing it exactly, we can go back to --preset=libfuzzer on mac as well.
  14. l0rinc reopened this on Mar 25, 2025
  15. l0rinc force-pushed on Mar 25, 2025
  16. fjahr commented at 2:28 pm on March 26, 2025: contributor

    I am currently having similar problems (also on macos). I am not sure if I want to change the docs though, obviously it would be great if we could figure out the actual issue instead.

    Maybe a bit more context that I can add, I am seeing the same error as above when running without providing a corpus but with a corpus I see this:

     0$ FUZZ=base32_encode_decode build_fuzz/bin/fuzz ../qa-assets/fuzz_corpora/base32_encode_decode/
 1fuzz(54938,0x1fedb8840) malloc: nano zone abandoned due to inability to reserve vm space.
 2INFO: Running with entropic power schedule (0xFF, 100).
 3INFO: Seed: 2131471744
 4INFO: Loaded 1 modules   (1253368 inline 8-bit counters): 1253368 [0x10867c000, 0x1087adff8),
 5INFO: Loaded 1 PC tables (1253368 PCs): 1253368 [0x1087adff8,0x109acdf78),
 6=================================================================
 7==54938==ERROR: AddressSanitizer: container-overflow on address 0x608000000ae8 at pc 0x000104518ef4 bp 0x00016b92e6f0 sp 0x00016b92e6e8
 8WRITE of size 8 at 0x608000000ae8 thread T0
 9    [#0](/bitcoin-bitcoin/0/) 0x000104518ef0 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::__init_copy_ctor_external(char const*, unsigned long)+0x1c4 (fuzz:arm64+0x100048ef0)
10    [#1](/bitcoin-bitcoin/1/) 0x000107c66314 in fuzzer::ListFilesInDirRecursive(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, long*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>*, bool)+0x26c (fuzz:arm64+0x103796314)
11    [#2](/bitcoin-bitcoin/2/) 0x000107c655dc in fuzzer::GetSizedFilesFromDir(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>*)+0x2c (fuzz:arm64+0x1037955dc)
12    [#3](/bitcoin-bitcoin/3/) 0x000107c61154 in fuzzer::ReadCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)+0x4c (fuzz:arm64+0x103791154)
13    [#4](/bitcoin-bitcoin/4/) 0x000107c60fbc in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1dbc (fuzz:arm64+0x103790fbc)
14    [#5](/bitcoin-bitcoin/5/) 0x000107c748c4 in main+0x24 (fuzz:arm64+0x1037a48c4)
15    [#6](/bitcoin-bitcoin/6/) 0x0001950c8270  (<unknown module>)
16    [#7](/bitcoin-bitcoin/7/) 0xf297ffffffffffc  (<unknown module>)
17
180x608000000ae8 is located 72 bytes inside of 96-byte region [0x608000000aa0,0x608000000b00)
19allocated by thread T0 here:
20    [#0](/bitcoin-bitcoin/0/) 0x00010b9952c4 in _Znwm+0x6c (libclang_rt.asan_osx_dynamic.dylib:arm64+0x612c4)
21    [#1](/bitcoin-bitcoin/1/) 0x000104a95160 in std::__1::__split_buffer<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>&)+0xf0 (fuzz:arm64+0x1005c5160)
22    [#2](/bitcoin-bitcoin/2/) 0x000104decb48 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>* std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::__push_back_slow_path<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)+0x244 (fuzz:arm64+0x10091cb48)
23    [#3](/bitcoin-bitcoin/3/) 0x000107c66284 in fuzzer::ListFilesInDirRecursive(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, long*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>*, bool)+0x1dc (fuzz:arm64+0x103796284)
24    [#4](/bitcoin-bitcoin/4/) 0x000107c655dc in fuzzer::GetSizedFilesFromDir(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>*)+0x2c (fuzz:arm64+0x1037955dc)
25    [#5](/bitcoin-bitcoin/5/) 0x000107c61154 in fuzzer::ReadCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)+0x4c (fuzz:arm64+0x103791154)
26    [#6](/bitcoin-bitcoin/6/) 0x000107c60fbc in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1dbc (fuzz:arm64+0x103790fbc)
27    [#7](/bitcoin-bitcoin/7/) 0x000107c748c4 in main+0x24 (fuzz:arm64+0x1037a48c4)
28    [#8](/bitcoin-bitcoin/8/) 0x0001950c8270  (<unknown module>)
29    [#9](/bitcoin-bitcoin/9/) 0xf297ffffffffffc  (<unknown module>)
30
31HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
32If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
33SUMMARY: AddressSanitizer: container-overflow (fuzz:arm64+0x100048ef0) in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::__init_copy_ctor_external(char const*, unsigned long)+0x1c4
34Shadow bytes around the buggy address:
35  0x608000000800: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
36  0x608000000880: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
37  0x608000000900: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
38  0x608000000980: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
39  0x608000000a00: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
40=>0x608000000a80: fa fa fa fa 00 00 00 00 00 00 00 00 00[fc]fc fc
41  0x608000000b00: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
42  0x608000000b80: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
43  0x608000000c00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
44  0x608000000c80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
45  0x608000000d00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
46Shadow byte legend (one shadow byte represents 8 application bytes):
47  Addressable:           00
48  Partially addressable: 01 02 03 04 05 06 07
49  Heap left redzone:       fa
50  Freed heap region:       fd
51  Stack left redzone:      f1
52  Stack mid redzone:       f2
53  Stack right redzone:     f3
54  Stack after return:      f5
55  Stack use after scope:   f8
56  Global redzone:          f9
57  Global init order:       f6
58  Poisoned by user:        f7
59  Container overflow:      fc
60  Array cookie:            ac
61  Intra object redzone:    bb
62  ASan internal:           fe
63  Left alloca redzone:     ca
64  Right alloca redzone:    cb
65==54938==ABORTING
66Abort trap: 6

    So there seems to be some problem with reading the corpus when trying to provide a corpus.

  17. brunoerg commented at 4:49 pm on March 26, 2025: contributor
    @fjahr Did you try with ASAN_OPTIONS=detect_container_overflow=0? If so, did you get any issue?
  18. fjahr commented at 8:59 pm on March 27, 2025: contributor

    @fjahr Did you try with ASAN_OPTIONS=detect_container_overflow=0? If so, did you get any issue?

    I actually hadn’t because for what I was working on I didn’t need the sanitiziers so I went with nosan right away but I just tried it now and it does indeed fix it for me with this with or without corpus, e.g. this works:

    0$ cmake --preset=libfuzzer -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld"
1$ ASAN_OPTIONS=detect_container_overflow=0 FUZZ=mini_miner_selection build_fuzz/bin/fuzz ../qa-assets/fuzz_corpora/mini_miner_selection/

    If that works for you as well @l0rinc maybe that’s a better docs change to suggest trying that.

  19. l0rinc commented at 9:37 pm on March 27, 2025: contributor

    ASAN_OPTIONS=detect_container_overflow=0 that’s a weird syntax - unfortunately that still fails for me.

    0ASAN_OPTIONS=detect_container_overflow=0 FUZZ=psbt_base64_decode build_fuzz/bin/fuzz

     0fuzz(61762,0x1fa5e4840) malloc: nano zone abandoned due to inability to reserve vm space.
 1INFO: Running with entropic power schedule (0xFF, 100).
 2INFO: Seed: 3061578036
 3INFO: Loaded 1 modules   (1253375 inline 8-bit counters): 1253375 [0x106344000, 0x106475fff), 
 4INFO: Loaded 1 PC tables (1253375 PCs): 1253375 [0x106476000,0x107795ff0), 
 5INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
 6libc++abi: terminating due to uncaught exception of type std::__1::ios_base::failure: DataStream::read(): end of data: unspecified iostream_category error
 7==61762== ERROR: libFuzzer: deadly signal
 8    [#0](/bitcoin-bitcoin/0/) 0x0001097fd248 in __sanitizer_print_stack_trace+0x28 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x5d248)
 9    [#1](/bitcoin-bitcoin/1/) 0x00010593be34 in fuzzer::PrintStackTrace()+0x2c (fuzz:arm64+0x1037a3e34)
10    [#2](/bitcoin-bitcoin/2/) 0x00010592f7e8 in fuzzer::Fuzzer::CrashCallback()+0x54 (fuzz:arm64+0x1037977e8)
11    [#3](/bitcoin-bitcoin/3/) 0x000190caade0 in _sigtramp+0x34 (libsystem_platform.dylib:arm64+0x3de0)
12    [#4](/bitcoin-bitcoin/4/) 0xd05c000190c73f6c  (<unknown module>)
13    [#5](/bitcoin-bitcoin/5/) 0x1644800190b80904  (<unknown module>)
14    [#6](/bitcoin-bitcoin/6/) 0x9817000190c2a448  (<unknown module>)
15    [#7](/bitcoin-bitcoin/7/) 0xd109000190c18a20  (<unknown module>)
16    [#8](/bitcoin-bitcoin/8/) 0xe8378001908c13f0  (<unknown module>)
17    [#9](/bitcoin-bitcoin/9/) 0x2d14800190c2970c  (<unknown module>)
18    [#10](/bitcoin-bitcoin/10/) 0x2d7a800190c2ccd8  (<unknown module>)
19    [#11](/bitcoin-bitcoin/11/) 0xe77800190c2cc80  (<unknown module>)
20    [#12](/bitcoin-bitcoin/12/) 0xed628001023bac20  (<unknown module>)
21    [#13](/bitcoin-bitcoin/13/) 0x000102568c84 in void PartiallySignedTransaction::Unserialize<DataStream>(DataStream&)+0x224 (fuzz:arm64+0x1003d0c84)
22    [#14](/bitcoin-bitcoin/14/) 0x000103186ad0 in DecodeRawPSBT(PartiallySignedTransaction&, std::__1::span<std::byte const, 18446744073709551615ul>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&)+0x200 (fuzz:arm64+0x100feead0)
23    [#15](/bitcoin-bitcoin/15/) 0x00010318645c in DecodeBase64PSBT(PartiallySignedTransaction&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&)+0x1dc (fuzz:arm64+0x100fee45c)
24    [#16](/bitcoin-bitcoin/16/) 0x0001021e7a68 in psbt_base64_decode_fuzz_target(std::__1::span<unsigned char const, 18446744073709551615ul>)+0x270 (fuzz:arm64+0x10004fa68)
25    [#17](/bitcoin-bitcoin/17/) 0x000102e524d8 in LLVMFuzzerTestOneInput+0x198 (fuzz:arm64+0x100cba4d8)
26    [#18](/bitcoin-bitcoin/18/) 0x000105930d94 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long)+0x12c (fuzz:arm64+0x103798d94)
27    [#19](/bitcoin-bitcoin/19/) 0x000105931f04 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>&)+0x220 (fuzz:arm64+0x103799f04)
28    [#20](/bitcoin-bitcoin/20/) 0x00010593258c in fuzzer::Fuzzer::Loop(std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>&)+0x98 (fuzz:arm64+0x10379a58c)
29    [#21](/bitcoin-bitcoin/21/) 0x000105928f44 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1dd0 (fuzz:arm64+0x103790f44)
30    [#22](/bitcoin-bitcoin/22/) 0x00010593c838 in main+0x24 (fuzz:arm64+0x1037a4838)
31    [#23](/bitcoin-bitcoin/23/) 0x0001908f4270  (<unknown module>)
32    [#24](/bitcoin-bitcoin/24/) 0xaa107ffffffffffc  (<unknown module>)
33
34NOTE: libFuzzer has rudimentary signal handlers.
35      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
36SUMMARY: libFuzzer: deadly signal
37MS: 0 ; base unit: 0000000000000000000000000000000000000000
38
39
40artifact_prefix='./'; Test unit written to ./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709
41Base64:

    But the libfuzzer-nosan fallback does work, I’ve documented both, thanks for the hints - coauthored @brunoerg and @fjahr.

  20. l0rinc force-pushed on Mar 27, 2025
  21. l0rinc renamed this:
    doc: use `libfuzzer-nosan` for macOS
    doc: document workaround and fallback for macOS fuzzing
    on Mar 27, 2025
  22. brunoerg commented at 9:57 pm on March 27, 2025: contributor
    I think we have two different issues here: @l0rinc is getting “libc++abi: terminating due to uncaught exception of type std::__1::ios_base::failure: DataStream::read(): end of data: unspecified iostream_category error”. Me and @fjahr were getting the AddressSanitizer: container-overflow one. For both nosan should work, but I think that the container-overflow is a false positive so might be good to document the ASAN_OPTIONS=detect_container_overflow=0.
  23. l0rinc commented at 10:01 pm on March 27, 2025: contributor

    might be good to document the ASAN_OPTIONS=detect_container_overflow=0

    Already pushed, please review the changes.

  24. fjahr commented at 10:01 pm on March 27, 2025: contributor
    Yeah, I do still get the error that @l0rinc sees when I test psbt_base64_decode. When I test mini_miner_selection which I am working on then it is fixed with the ASAN option with and without corpus.
  25. in doc/fuzzing.md:150 in 8b7c8a293c outdated 
    144@@ -146,23 +145,43 @@ Every single pull request submitted against the Bitcoin Core repo is automatical
145 ## macOS hints for libFuzzer
146 
147 The default Clang/LLVM version supplied by Apple on macOS does not include
148-fuzzing libraries, so macOS users will need to install a full version, for
149-example using `brew install llvm`.
    fjahr commented at 10:03 pm on March 27, 2025:
    Why remove the brew hint here? In other places the examples still include brew references.
    l0rinc commented at 10:16 pm on March 27, 2025:
    we’re repeating the same a few lines below
  26. in doc/fuzzing.md:152 in 8b7c8a293c outdated 
    144@@ -146,23 +145,43 @@ Every single pull request submitted against the Bitcoin Core repo is automatical
145 ## macOS hints for libFuzzer
146 
147 The default Clang/LLVM version supplied by Apple on macOS does not include
148-fuzzing libraries, so macOS users will need to install a full version, for
149-example using `brew install llvm`.
150+fuzzing libraries, so macOS users will need to install a full version.
151 
152-You may also need to take care of giving the correct path for `clang` and
    fjahr commented at 10:04 pm on March 27, 2025:
    Do I see correctly that the content hasn’t chnaged here? It’s annoying to review this reformatting when it doesn’t have any effect.
    l0rinc commented at 10:17 pm on March 27, 2025:
    I’ve fixed a typo (non-systems clang) and rearranged the lines based on sentence structure (which caused the typo to be invisible until now I think)
  27. in doc/fuzzing.md:11 in 8b7c8a293c outdated 
     6@@ -7,9 +7,8 @@ To quickly get started fuzzing Bitcoin Core using [libFuzzer](https://llvm.org/d
 7 ```sh
 8 $ git clone https://github.com/bitcoin/bitcoin
 9 $ cd bitcoin/
10+# macOS users: make sure to read ["macOS hints for libFuzzer"](#macos-hints-for-libfuzzer)
11 $ cmake --preset=libfuzzer
12-# macOS users: If you have problem with this step then make sure to read "macOS hints for
    fjahr commented at 10:11 pm on March 27, 2025:
    I liked the old version better where it’s only there in case you run into problems. There is no need to send people there preemptively. For example maybe the issues are fixed with the next llvm version or so. Then we would save users time with the old comment.
    l0rinc commented at 10:19 pm on March 27, 2025:

    I don’t think we can run these instructions on a mac currently - can we? I always had to jump to the mac section.

    maybe the issues are fixed with the next llvm version or so

    You mean we will be able to remove the special mac section in that case? We can remove the comment as well in that case, but I think we always have to jump over if on a mac - please correct me if I’m wrong.

  28. fjahr commented at 10:13 pm on March 27, 2025: contributor
    Left some comments but I am still ~0 on this change and would much rather prefer that someone tries to spend more time on investigating the issue. I tried for a bit but wasn’t successful so far.
  29. brunoerg commented at 10:15 pm on March 27, 2025: contributor
    If we’re going to put ASAN_OPTIONS=detect_container_overflow=0 into the documentation, I think it would be good to mention https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow#false-positives
  30. doc: document workaround and fallback for macOS fuzzing 
    On macOS, running fuzz targets with the default `libfuzzer` preset can fail due to linker errors or AddressSanitizer issues such as:
* Linker did not accept requested flags, you are missing required libraries
* ==54938==ERROR: AddressSanitizer: container-overflow on address 0x608000000ae8 at pc 0x000104518ef4 bp 0x00016b92e6f0 sp 0x00016b92e6e8

Documented a workaround using:
* ASAN_OPTIONS=detect_container_overflow=0;
* Reverting to the `libfuzzer-nosan` preset.

Co-authored-by: brunoerg <brunoely.gc@gmail.com>
Co-authored-by: Fabian Jahr <fjahr@protonmail.com>
    89477d0abf
  31. l0rinc force-pushed on Mar 27, 2025
  32. l0rinc commented at 10:21 pm on March 27, 2025: contributor

    it would be good to mention Wiki: AddressSanitizerContainerOverflow (false positives) (google/sanitizers)

    I’ve put this in the description already, but added it to the doc as well now

  33. maflcko commented at 7:18 am on March 28, 2025: member

    Left some comments but I am still ~0 on this change and would much rather prefer that someone tries to spend more time on investigating the issue. I tried for a bit but wasn’t successful so far.

    I tend to agree. It would be good to check:

    • Does it happen with a self-compiled clang. If yes, then it is likely an upstream brew issues. If no, then it is likely an upstream clang/llvm issue.

    Unrelated to that, it would be good to minimize the DataStream crash, so that there is a single minimal main.cpp file (or two files) that reproduce it. This makes it easier to spot if the error is in this codebase or somewhere else.


l0rinc DrahtBot brunoerg hebasto fjahr maflcko

Labels
Docs

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-03-28 15:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me