test: fix creation of "std::string"s with \0s #20000

pull vasild wants to merge 1 commits into bitcoin:master from vasild:fix_base32_tests changing 6 files +59 −44
  1. vasild commented at 6:44 AM on September 23, 2020: member

    A string literal "abc" contains a terminating \0, so that is 4 bytes. There is no need to write "abc\0" unless two terminating \0s are necessary.

    std::string objects do not internally contain a terminating \0, so std::string("abc") creates a string with size 3 and is the same as std::string("abc", 3).

    In "\01" the 01 part is interpreted as one number (1) and that is the same as "\1" which is a string like {1, 0} whereas "\0z" is a string like {0, 'z', 0}. To create a string like {0, '1', 0} one must use "\0" "1".

    Adjust the tests accordingly.

  2. MarcoFalke requested review from practicalswift on Sep 23, 2020
  3. fanquake added the label Tests on Sep 23, 2020
  4. eriknylund commented at 7:14 AM on September 23, 2020: contributor

    Concept ACK. I browsed over these decode tests when extending the Base32 encoding tests with support for no padding for #19845 and I found them hard to comprehend. I think this is an improvement.

  5. vasild force-pushed on Sep 23, 2020
  6. vasild commented at 7:26 AM on September 23, 2020: member

    This is actually mode widespread. I checked all tests and fixed some others too. @eriknylund, this is how stumbled on that too! :)

  7. practicalswift approved
  8. practicalswift commented at 9:13 AM on September 23, 2020: contributor

    ACK b7f1cc6c911b8c18d1181e6ede8e1b7ed3c6ea14 - patch looks correct

    Thanks for cleaning up these tests! :)

  9. in src/test/base32_tests.cpp:26 in b7f1cc6c91 outdated
      22 | @@ -23,14 +23,14 @@ BOOST_AUTO_TEST_CASE(base32_testvectors)
      23 |  
      24 |      // Decoding strings with embedded NUL characters should fail
      25 |      bool failure;
      26 | -    (void)DecodeBase32(std::string("invalid", 7), &failure);
      27 | -    BOOST_CHECK_EQUAL(failure, true);
      28 | -    (void)DecodeBase32(std::string("AWSX3VPP", 8), &failure);
      29 | -    BOOST_CHECK_EQUAL(failure, false);
      30 | +    (void)DecodeBase32(std::string("invalid", 8), &failure);
    


    MarcoFalke commented at 10:21 AM on September 23, 2020:

    This fails before and after, indicating that the failure has nothing to do with NUL, no?


    eriknylund commented at 12:24 PM on September 23, 2020:

    Good point. I also don't fully understand what any of these tests are trying to achieve? Wouldn't something like

    diff --git a/src/test/base32_tests.cpp b/src/test/base32_tests.cpp
    index 26ed78e2c2..4a5af0fd4d 100644
    --- a/src/test/base32_tests.cpp
    +++ b/src/test/base32_tests.cpp
    @@ -23,13 +23,13 @@ BOOST_AUTO_TEST_CASE(base32_testvectors)
     
         // Decoding strings with embedded NUL characters should fail
         bool failure;
    -    (void)DecodeBase32(std::string("invalid", 8), &failure);
    -    BOOST_CHECK(failure);
    -    (void)DecodeBase32(std::string("AWSX3VPP"), &failure);
    +    (void)DecodeBase32(std::string("my======", 8), &failure);
         BOOST_CHECK(!failure);
    -    (void)DecodeBase32(std::string("AWSX3VPP\0invalid", 16), &failure);
    +    (void)DecodeBase32(std::string("my======\0", 9), &failure);
    +    BOOST_CHECK(failure);
    +    (void)DecodeBase32(std::string("\0my======", 9), &failure);
         BOOST_CHECK(failure);
    -    (void)DecodeBase32(std::string("AWSX3VPPinvalid", 16), &failure);
    +    (void)DecodeBase32(std::string("my=\0=====", 9), &failure);
         BOOST_CHECK(failure);
     }
    

    do more according to what the comment on L24 implies?


    vasild commented at 7:30 PM on September 23, 2020:

    no?

    no :-)

    Before this PR DecodeBase32(std::string("invalid", 7), &failure) fails because the supplied input has invalid length (7) and is not padded to a proper length (8) with =.

    With this PR the same call fails due to the ValidAsCString() check early in DecodeBase32() (this check passes before this PR). The trailing \0 upsets the ValidAsCString() check which I think was the intent.


    vasild commented at 12:49 PM on September 24, 2020:

    Sorry, with this PR it is not "the same call", but DecodeBase32(std::string("invalid", 8), &failure).


    MarcoFalke commented at 1:21 PM on September 24, 2020:

    Because the failure reason is not returned, it might be good to use a valid Base32 string, which properly decodes and then make it an invalid cstr, which fails. Otherwise it is harder to guess which failure reason was the intent from just reading the test code.


    vasild commented at 2:09 PM on October 6, 2020:

    it might be good to use a valid Base32 string, which properly decodes and then make it an invalid cstr, which fails

    The tests below already do that with AWSX3VPP (length 8, valid) and AWSX3VPP\0invalid (length 16, invalid due to embedded \0).


    eriknylund commented at 2:36 PM on October 6, 2020:

    To my understanding AWSX3VPP is not a valid base32 encoded string, but perhaps I'm missing something obvious?


    vasild commented at 3:19 PM on October 6, 2020:

    Why not? It has proper length (8) and no embedded \0s. DecodeBase32() decodes is successfully.


    eriknylund commented at 4:00 PM on October 6, 2020:

    Thanks, I see your point and why I misunderstood the test in this case. To me the value AWXS3VPP looks like a magic string chosen on purpose and not on random, and I expected it to actually decode into something more human readable, like the other test values in the file on L14-15.


    vasild commented at 3:11 PM on October 31, 2020:

    Added some comments to make it clearer.

  10. eriknylund changes_requested
  11. in src/test/base32_tests.cpp:32 in b7f1cc6c91 outdated
      34 |      (void)DecodeBase32(std::string("AWSX3VPP\0invalid", 16), &failure);
      35 | -    BOOST_CHECK_EQUAL(failure, true);
      36 | -    (void)DecodeBase32(std::string("AWSX3VPPinvalid", 15), &failure);
      37 | -    BOOST_CHECK_EQUAL(failure, true);
      38 | +    BOOST_CHECK(failure);
      39 | +    (void)DecodeBase32(std::string("AWSX3VPPinvalid", 16), &failure);
    


    laanwj commented at 12:44 PM on September 30, 2020:

    Maybe we can wait with making this change for the C++17 switch? We'll have the s-suffix strings, so don't need to worry about specifying the length explicitly at all anymore even when there are trailing '\0' characters.

    #include <iostream>
    #include <string>
    
    int main()
    {
        using namespace std::string_literals;
    
        std::string x = "test\x00\x00\x00\x00"s;
        std::cout << x.size() << std::endl; // prints 8
    }
    

    vasild commented at 2:12 PM on October 6, 2020:

    I think this PR is good as is, but if it is still unmerged when we start allowing C++17, then I will update it to use "..."s;


    fanquake commented at 12:35 PM on October 31, 2020:

    then I will update it to use "..."s;

    Even though this has a few ACKs, I think you can go ahead and update it to use the s-suffix strings. We'll merge this post branch off when requiring C++17.


    vasild commented at 3:10 PM on October 31, 2020:

    Done

  12. eriknylund commented at 4:25 PM on October 6, 2020: contributor

    ACK b7f1cc6c911b8c18d1181e6ede8e1b7ed3c6ea14 I've reviewed the changes and I think they are an improvement. I would consider renaming the issue to better reflect that changes that are not only base32 related.

  13. eriknylund approved
  14. vasild renamed this:
    test: fix DecodeBase32 tests
    test: fix string handling with embedded \0s
    on Oct 7, 2020
  15. vasild renamed this:
    test: fix string handling with embedded \0s
    test: fix creation of "std::string"s with \0s
    on Oct 7, 2020
  16. theStack approved
  17. theStack commented at 1:41 PM on October 11, 2020: member

    Code Review ACK b7f1cc6c911b8c18d1181e6ede8e1b7ed3c6ea14 :file_cabinet:

  18. test: fix creation of std::string objects with \0s
    A string literal `"abc"` contains a terminating `\0`, so that is 4
    bytes. There is no need to write `"abc\0"` unless two terminating
    `\0`s are necessary.
    
    `std::string` objects do not internally contain a terminating `\0`, so
    `std::string("abc")` creates a string with size 3 and is the same as
    `std::string("abc", 3)`.
    
    In `"\01"` the `01` part is interpreted as one number (1) and that is
    the same as `"\1"` which is a string like `{1, 0}` whereas `"\0z"` is a
    string like `{0, 'z', 0}`. To create a string like `{0, '1', 0}` one
    must use `"\0" "1"`.
    
    Adjust the tests accordingly.
    ecc6cf1a3b
  19. vasild force-pushed on Oct 31, 2020
  20. vasild commented at 3:09 PM on October 31, 2020: member

    Changed to use s-suffix strings. Tested with --enable-c++17.

  21. in src/test/util_tests.cpp:1242 in ecc6cf1a3b
    1237 | @@ -1235,9 +1238,9 @@ BOOST_AUTO_TEST_CASE(util_ParseMoney)
    1238 |      BOOST_CHECK(!ParseMoney("-1", ret));
    1239 |  
    1240 |      // Parsing strings with embedded NUL characters should fail
    1241 | -    BOOST_CHECK(!ParseMoney(std::string("\0-1", 3), ret));
    1242 | -    BOOST_CHECK(!ParseMoney(std::string("\01", 2), ret));
    1243 | -    BOOST_CHECK(!ParseMoney(std::string("1\0", 2), ret));
    1244 | +    BOOST_CHECK(!ParseMoney("\0-1"s, ret));
    1245 | +    BOOST_CHECK(!ParseMoney("\0" "1"s, ret));
    


    eriknylund commented at 5:26 PM on October 31, 2020:

    This change strikes me as odd, shouldn't it be "\01"s or did you intentionally want the other approach here to improve the test?


    vasild commented at 1:42 PM on November 1, 2020:

    I guess the intention of std::string("\01", 2) was to test the string {0, 49} (ASCII code of '1' is 49). But this is not what it did because "\01" is the same as "\1" and so the resulting string is {1, 0} (1 from \1 and the terminating nul).

  22. laanwj commented at 10:23 AM on November 19, 2020: member

    ACK ecc6cf1a3b097b9b5b047282063a0b6779631b83

    This very much clarifies the code and works locally for me. I don't know what the travis and CI failures are about.

  23. practicalswift commented at 10:48 AM on November 19, 2020: contributor

    ACK ecc6cf1a3b097b9b5b047282063a0b6779631b83 modulo happily green CI

    Needs rebase to pull in C++17 switch?

  24. vasild closed this on Nov 19, 2020

  25. vasild reopened this on Nov 19, 2020

  26. laanwj merged this on Nov 20, 2020
  27. laanwj closed this on Nov 20, 2020

  28. vasild deleted the branch on Nov 20, 2020
  29. sidhujag referenced this in commit f76c6a968c on Nov 20, 2020
  30. random-zebra referenced this in commit b4751e10ce on Aug 11, 2021
  31. deadalnix referenced this in commit 952efc13f0 on Jan 11, 2022
  32. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-25 15:15 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me