tests: Add fuzzing harness testing the locale independence of the strencodings.h functions #18126

1. 
   practicalswift commented at 2:50 pm on February 12, 2020: contributor

   Context: C and C++ locale assumptions in bitcoind and bitcoin-qt

   Add fuzzing harness for locale independence testing of functions in strencodings.h and tinyformat.h.

   Test this PR using:

   0$ make distclean
   1$ ./autogen.sh
   2$ CC=clang CXX=clang++ ./configure --enable-fuzz \
   3      --with-sanitizers=address,fuzzer,undefined
   4$ make
   5$ src/test/fuzz/locale
   6…
   

   The tested functions (ParseInt32(…), ParseInt64(…), atoi(const std::string&), atoi64(const std::string& str), i64tostr(const char*), itostr(…), strprintf(…)) all call locale dependent functions (such as strtol(…), strtoll(…), atoi(const char*), etc.) but are assumed to do so in a way that the tested functions return same results regardless of the chosen C locale (setlocale).

   This fuzzer aims to test that those assumptions hold up also in practice now and over time.

2. practicalswift force-pushed on Feb 12, 2020
3. practicalswift force-pushed on Feb 12, 2020
4. fanquake added the label Tests on Feb 12, 2020
5. 
   DrahtBot commented at 6:09 pm on February 12, 2020: member

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #10443 (Add fee_est tool for debugging fee estimation code by ryanofsky)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

6. practicalswift force-pushed on Feb 15, 2020
7. practicalswift force-pushed on Feb 15, 2020
8. practicalswift force-pushed on Feb 18, 2020
9. DrahtBot added the label Needs rebase on Mar 5, 2020
10. tests: Add fuzzing harness for locale independence testing 259e290db8
11. practicalswift force-pushed on Mar 6, 2020
12. 
    practicalswift commented at 1:45 pm on March 6, 2020: contributor

    Rebased! :)
13. 
    MarcoFalke commented at 2:07 pm on March 6, 2020: member

    The OP says “all call locale dependent functions”, the title says “locale independence”. Which is correct?

    What is the point of a test that tests the functions are not locale independent?

14. DrahtBot removed the label Needs rebase on Mar 6, 2020
15. 
    practicalswift commented at 3:37 pm on March 6, 2020: contributor

    @MarcoFalke They all call functions that are known to be locale dependent (see list in lint-locale-dependence.sh) but they are supposed to do so in a way that guarantees that their own output is locale independent. Does that answer your question? :)
16. MarcoFalke merged this on Mar 6, 2020
17. MarcoFalke closed this on Mar 6, 2020

    ---

18. 
    fanquake commented at 11:01 pm on March 6, 2020: member

    This has broken Travis:

    0Run locale with args ['valgrind', '--quiet', '--error-exitcode=1', '/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/locale', '-runs=1', '/home/travis/build/bitcoin/bitcoin/ci/scratch//qa-assets/fuzz_seed_corpus/locale']
    1Output: INFO: Seed: 818858521
    2INFO: Loaded 1 modules   (1402 inline 8-bit counters): 1402 [0x3a7c40, 0x3a81ba), 
    3INFO: Loaded 1 PC tables (1402 PCs): 1402 [0x3a81c0,0x3ad960), 
    4No such file or directory: /home/travis/build/bitcoin/bitcoin/ci/scratch//qa-assets/fuzz_seed_corpus/locale; exiting
    5INFO: Seed: 818858521
    6INFO: Loaded 1 modules   (1402 inline 8-bit counters): 1402 [0x3a7c40, 0x3a81ba), 
    7INFO: Loaded 1 PC tables (1402 PCs): 1402 [0x3a81c0,0x3ad960), 
    8No such file or directory: /home/travis/build/bitcoin/bitcoin/ci/scratch//qa-assets/fuzz_seed_corpus/locale; exiting
    9Target "locale" failed with exit code 1: valgrind --quiet --error-exitcode=1 /home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/locale -runs=1 /home/travis/build/bitcoin/bitcoin/ci/scratch//qa-assets/fuzz_seed_corpus/locale
    

    I assume seeds just need to be added to https://github.com/bitcoin-core/qa-assets?

19. fanquake referenced this in commit 3d28c886f0 on Mar 7, 2020
20. sidhujag referenced this in commit e2de813c58 on Mar 7, 2020
21. jasonbcox referenced this in commit 34e2cee828 on Nov 4, 2020
22. sidhujag referenced this in commit cea25aeec6 on Nov 10, 2020
23. practicalswift deleted the branch on Apr 10, 2021
24. PastaPastaPasta referenced this in commit ed97363e78 on Sep 17, 2021
25. PastaPastaPasta referenced this in commit a71feacf02 on Sep 18, 2021
26. thelazier referenced this in commit 086d92b635 on Sep 25, 2021
27. kittywhiskers referenced this in commit 2c989171a1 on Feb 27, 2022
28. kittywhiskers referenced this in commit 4a22e9d10e on Feb 27, 2022
29. kittywhiskers referenced this in commit dbe3b8655c on Feb 28, 2022
30. kittywhiskers referenced this in commit a36e0560b4 on Feb 28, 2022
31. kittywhiskers referenced this in commit 7fab86b613 on Feb 28, 2022
32. kittywhiskers referenced this in commit b3dba520bc on Feb 28, 2022
33. kittywhiskers referenced this in commit cd080fa998 on Mar 13, 2022
34. kittywhiskers referenced this in commit f284388f67 on Mar 24, 2022
35. kittywhiskers referenced this in commit b28395f23b on Mar 24, 2022
36. DrahtBot locked this on Aug 18, 2022

Contributors
practicalswift DrahtBot MarcoFalke fanquake

Labels
Tests