streams: Fix read-past-the-end and integer overflows
#24231

pull MarcoFalke wants to merge 3 commits into bitcoin:master from MarcoFalke:2201-streamIgnore changing 2 files +50 −52
  1. MarcoFalke commented at 8:51 pm on February 1, 2022: member

    This is a follow-up to commit e26b62093ae21e89ed7d36a24a6b863f38ec631d with the following fixes:

    • Fix unsigned integer overflow in ignore(), when nReadPos wraps.
    • Fix unsigned integer overflow in read(), when nReadPos wraps.
    • Fix read-past-the-end in read(), when nReadPos wraps.

    This shouldn’t be remote-exploitable, because it requires a stream of more than 1GB of size. However, it might be exploitable if the attacker controls the datadir (I haven’t checked).

    A unit test for the overflow in ignore() looks like following. It is left as an excercise to the reader to replace foo.ignore(7) with the appropriate call to read() to reproduce the overflow and read-error in read().

     0diff --git a/src/test/coins_tests.cpp b/src/test/coins_tests.cpp
     1index 922fd8e513..ec6ea93919 100644
     2--- a/src/test/coins_tests.cpp
     3+++ b/src/test/coins_tests.cpp
     4@@ -534,6 +534,20 @@ BOOST_AUTO_TEST_CASE(ccoins_serialization)
     5     } catch (const std::ios_base::failure&) {
     6     }
     7 
     8+    CDataStream foo{0, 0};
     9+    auto size{std::numeric_limits<uint32_t>::max()};
    10+    foo.resize(size);
    11+    BOOST_CHECK_EQUAL(foo.size(), size);
    12+    foo.ignore(std::numeric_limits<int32_t>::max());
    13+    size -= std::numeric_limits<int32_t>::max();
    14+    BOOST_CHECK_EQUAL(foo.size(), size);
    15+    foo.ignore(std::numeric_limits<int32_t>::max());
    16+    size -= std::numeric_limits<int32_t>::max();
    17+    BOOST_CHECK_EQUAL(foo.size(), size);
    18+    BOOST_CHECK_EQUAL(foo.size(), 1);
    19+    foo.ignore(7); // Should overflow, as the size is only 1
    20+    BOOST_CHECK_EQUAL(foo.size(), uint32_t(1 - 7));
    21+
    22     // Very large scriptPubKey (3*10^9 bytes) past the end of the stream
    23     CDataStream tmp(SER_DISK, CLIENT_VERSION);
    24     uint64_t x = 3000000000ULL;
    
  2. DrahtBot added the label Utils/log/libs on Feb 1, 2022
  3. MarcoFalke force-pushed on Feb 2, 2022
  4. MarcoFalke force-pushed on Feb 2, 2022
  5. MarcoFalke commented at 5:19 pm on February 2, 2022: member

    Pushed another commit to fix a premature throw of “end of data” on 64-bit systems, when there is still data.

    Unit test:

     0diff --git a/src/test/coins_tests.cpp b/src/test/coins_tests.cpp
     1index 922fd8e513..d7a266cf1c 100644
     2--- a/src/test/coins_tests.cpp
     3+++ b/src/test/coins_tests.cpp
     4@@ -534,6 +534,21 @@ BOOST_AUTO_TEST_CASE(ccoins_serialization)
     5     } catch (const std::ios_base::failure&) {
     6     }
     7 
     8+    CDataStream foo{0, 0};
     9+    auto size{size_t{std::numeric_limits<uint32_t>::max()} + 100};
    10+    foo.resize(size);
    11+    BOOST_CHECK_EQUAL(foo.size(), size);
    12+    foo.ignore(std::numeric_limits<int32_t>::max());
    13+    size -= std::numeric_limits<int32_t>::max();
    14+    BOOST_CHECK_EQUAL(foo.size(), size);
    15+    foo.ignore(std::numeric_limits<int32_t>::max());
    16+    size -= std::numeric_limits<int32_t>::max();
    17+    BOOST_CHECK_EQUAL(foo.size(), size);
    18+    BOOST_CHECK_EQUAL(foo.size(), 101);
    19+    foo.ignore(7); // Should work fine
    20+    size -= 7;
    21+    BOOST_CHECK_EQUAL(foo.size(), size);
    22+
    23     // Very large scriptPubKey (3*10^9 bytes) past the end of the stream
    24     CDataStream tmp(SER_DISK, CLIENT_VERSION);
    25     uint64_t x = 3000000000ULL;
    
  6. MarcoFalke commented at 6:39 pm on February 3, 2022: member

    mingw can’t compile the bugfix:

     0In file included from /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++/cstring:42,
     1                 from ./serialize.h:13,
     2                 from ./streams.h:9,
     3                 from test/streams_tests.cpp:5:
     4In function void* memcpy(void*, const void*, size_t),
     5    inlined from void CDataStream::insert(CDataStream::iterator, std::vector<std::byte>::const_iterator, std::vector<std::byte>::const_iterator) at ./streams.h:256:19,
     6    inlined from void streams_tests::streams_serializedata_xor::test_method() at test/streams_tests.cpp:181:14:
     7/usr/share/mingw-w64/include/string.h:202:32: error: void* __builtin_memcpy(void*, const void*, long long unsigned int) specified size between 9223372036854775808 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
     8  202 |   return __builtin___memcpy_chk(__dst, __src, __n, __mingw_bos(__dst, 0));
     9      |          ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    10In function void* memcpy(void*, const void*, size_t),
    11    inlined from void CDataStream::insert(CDataStream::iterator, std::vector<std::byte>::const_iterator, std::vector<std::byte>::const_iterator) at ./streams.h:256:19,
    12    inlined from void streams_tests::streams_serializedata_xor::test_method() at test/streams_tests.cpp:198:14:
    13/usr/share/mingw-w64/include/string.h:202:32: error: void* __builtin_memcpy(void*, const void*, long long unsigned int) specified size between 9223372036854775808 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
    14  202 |   return __builtin___memcpy_chk(__dst, __src, __n, __mingw_bos(__dst, 0));
    15      |          ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    16cc1plus: all warnings being treated as errors
    17make[2]: *** [Makefile:16816: test/test_bitcoin-streams_tests.o] Error 1
    
  7. DrahtBot commented at 4:31 am on February 4, 2022: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #14485 (Try to use posix_fadvise with CBufferedFile by luke-jr)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  8. in src/streams.h:368 in fa0af11b2e outdated
    371+    void ignore(size_t num_ignore)
    372     {
    373         // Ignore from the beginning of the buffer
    374-        if (nSize < 0) {
    375-            throw std::ios_base::failure("CDataStream::ignore(): nSize negative");
    376+        auto next_read_pos{CheckedAdd<uint32_t>(nReadPos, num_ignore)};
    


    luke-jr commented at 9:57 pm on February 5, 2022:
    Won’t this truncate num_ignore to 32-bits before checking for overflows?

    MarcoFalke commented at 9:46 am on February 6, 2022:
    Yes, but this is already done on current master. The second commit fixes this.
  9. luke-jr approved
  10. luke-jr commented at 7:12 pm on February 6, 2022: member
    utACK
  11. MarcoFalke marked this as a draft on Feb 7, 2022
  12. MarcoFalke commented at 8:09 am on February 7, 2022: member
    Please review #24253 first, while this stays in draft. (Can’t be merged because mingw bug)
  13. luke-jr commented at 6:17 pm on February 7, 2022: member
    Could just disable the -Werror affected, but I guess #24253 is fine too…
  14. laanwj referenced this in commit 5e8e0b3d7f on Feb 9, 2022
  15. MarcoFalke marked this as ready for review on Feb 9, 2022
  16. streams: Fix read-past-the-end and integer overflows fab02f7991
  17. Make CDataStream work properly on 64-bit systems fa56c79df9
  18. scripted-diff: Rename nReadPos to m_read_pos in streams.h
    -BEGIN VERIFY SCRIPT-
     sed -i 's/nReadPos/m_read_pos/g' ./src/streams.h
    -END VERIFY SCRIPT-
    fa1b89a6bd
  19. DrahtBot added the label Needs rebase on Feb 9, 2022
  20. MarcoFalke force-pushed on Feb 9, 2022
  21. MarcoFalke commented at 4:33 pm on February 9, 2022: member
    Rebased. Should be trivial to re-ACK.
  22. DrahtBot removed the label Needs rebase on Feb 9, 2022
  23. klementtan commented at 2:53 pm on February 10, 2022: contributor

    Code Review ACK fa1b89a6bdbab50bdb0504782afd4bb3375d1b57:

    • Verified that CheckedAdd is used to prevent overflow in read and ignore
    • Verified that the test in #24231 (comment) passed on my local machine
    • Verified that an exception is thrown when there is an overflow with the following ad hoc test:
       0diff --git a/src/test/coins_tests.cpp b/src/test/coins_tests.cpp
       1index 82e4e1c90f..07758799f8 100644
       2--- a/src/test/coins_tests.cpp
       3+++ b/src/test/coins_tests.cpp
       4@@ -534,6 +534,21 @@ 
       5BOOST_AUTO_TEST_CASE(ccoins_serialization)
       6     } catch (const std::ios_base::failure&) {
       7     }
       8
       9+    try {
      10+      std::byte write_bytes[100] =  {std::byte{0x3F}};
      11+      std::byte read_bytes[100] = {std::byte{0x0}};
      12+      CDataStream foo{0, 0};
      13+      auto size{100};
      14+      foo.resize(size);
      15+      foo.write({write_bytes, write_bytes + 99});
      16+      foo.read(read_bytes);
      17+      foo.ignore(std::numeric_limits<uint64_t>::max());
      18+      BOOST_CHECK_MESSAGE(false, "We should have thrown");
      19+    } catch (const std::ios_base::failure& f) {
      20+      std::string err(f.what());
      21+      BOOST_CHECK_EQUAL(err, "CDataStream::ignore(): end of 
      22  data: unspecified iostream_category error");
      23+    }
      24+
      25     // Very large scriptPubKey (3*10^9 bytes) past the end of the 
      26stream
      27     CDataStream tmp(SER_DISK, CLIENT_VERSION);
      28     uint64_t x = 3000000000ULL;
      
  24. klementtan approved
  25. sidhujag referenced this in commit e7195b437a on Feb 12, 2022
  26. luke-jr approved
  27. luke-jr commented at 1:44 am on February 20, 2022: member
    re-utACK
  28. MarcoFalke merged this on Feb 21, 2022
  29. MarcoFalke closed this on Feb 21, 2022

  30. MarcoFalke deleted the branch on Feb 21, 2022
  31. sidhujag referenced this in commit d74b54cfab on Feb 22, 2022
  32. DrahtBot locked this on Feb 21, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-17 15:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me