p2p: fix ubsan addrman errors, make nTime truncation conversion explicit #22094

pull jonatack wants to merge 5 commits into bitcoin:master from jonatack:fix-ubsan-implicit-conversion-in-CAddrMan_Connected changing 4 files +24 −18
  1. jonatack commented at 2:04 PM on May 28, 2021: member

    This patch

    1. drops an unused nTime param from CAddrMan::Connected_() and CAddrMan::Connected()

    2. makes an nTime integer truncations explicit, as CAddress::nTime is declared in src/protocol.h as uint32 but int64 values can be written to it in CAddrMan::Connected_(). This also fixes the following non-suppressed UBSan error in addrman.cpp:

    SUMMARY: UndefinedBehaviorSanitizer
    addrman.cpp:535:22: runtime error: implicit conversion
      from type 'int64_t' (aka 'long') of value 68719478016 (64-bit, signed)
      to type 'uint32_t' (aka 'unsigned int') changed the value to 1280 (32-bit, unsigned)
    

    (According to https://en.cppreference.com/w/cpp/language/implicit_conversion, if the destination type is unsigned, the resulting value is the smallest unsigned value equal to the source value modulo (2 exponent n), where n is the number of bits used to represent the destination type. So it seems this is defined behavior and the signed integer is being truncated correctly. However, using a named cast makes the truncation conversion explicit and reduces the warnings by the UBSan integer sanitizer.)

    1. does the same for two similar UBSan errors in CAddrMan::Add_() that were suppressed

    2. fixes the following UBSan error in CAddrMan::Unserialize() that was suppressed

    SUMMARY: UndefinedBehaviorSanitizer
    addrman.h:320:43: runtime error: implicit conversion
      from type 'int' of value -32 (32-bit, signed)
      to type 'const uint8_t' (aka 'const unsigned char')
      changed the value to 224 (8-bit, unsigned)
    
    1. removes the four addrman UBSan suppressions

    2. adds TODO documentation for updates needed by the Year 2106

    To test

    $ ./autogen.sh && ./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined,integer CC=clang CXX=clang++ && make clean && make
    $ FUZZ=addrman src/test/fuzz/fuzz ../qa-assets/fuzz_seed_corpus/addrman
    
  2. fanquake added the label P2P on May 28, 2021
  3. lsilva01 approved
  4. klementtan approved
  5. klementtan commented at 8:05 AM on June 1, 2021: contributor

    ACK ffd89f1

    Tested on macOs with

     ./autogen.sh && ./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined,integer CC=clang CXX=clang++
    make
    FUZZ=addrman src/test/fuzz/fuzz ~/Desktop/codes/SSD/bitcoin_data/qa-assets`
    

    addrman.cpp:535:22: runtime error: implicit conversion error not thrown from test

  6. MarcoFalke commented at 9:12 AM on June 1, 2021: member

    What is the goal of this change? Is it to remove a suppression? If yes, why is the suppression not removed in this pull?

    Also, it seems a bit early to fuzz years larger than 2106 (80 years in the future). The network code will need to be changed anyway by then and this change doesn't seem to make it any easier (or harder).

  7. MarcoFalke commented at 9:14 AM on June 1, 2021: member
  8. jonatack commented at 9:48 AM on June 1, 2021: member

    Using a named cast makes the truncation conversion explicit, which ISTM is preferable than implicit. Happy to close if explicit isn't seen as better or sufficient justification.

  9. MarcoFalke added the label Refactoring on Jun 1, 2021
  10. jonatack renamed this:
    p2p: fix UBSan implicit conversion error in CAddrMan::Connected_()
    p2p, refactor: make nTime truncation conversion explicit in CAddrMan::Connected_()
    on Jun 1, 2021
  11. jonatack commented at 10:12 AM on June 1, 2021: member

    Updated the PR title.

  12. MarcoFalke commented at 10:13 AM on June 1, 2021: member

    I agree that absent any other points the explicit conversion is better than the implicit one. Though, this silences a warning, which might also be worthy to fix. Either by saying it is "invalid" (can be fixed in several decades) and ignoring it or by updating the fuzz test to not fuzz years past 2106.

  13. MarcoFalke commented at 10:14 AM on June 1, 2021: member

    Either way, the suppressions file should be updated as part of the change, if applicable.

  14. vasild commented at 11:28 AM on July 28, 2021: member

    Given that the second argument of Connected() is never used, maybe it is better to drop it:

    <details> <summary>[patch] remove nTime argument</summary>

    diff --git i/src/addrman.cpp w/src/addrman.cpp
    index 8192b4eba6..149d929e73 100644
    --- i/src/addrman.cpp
    +++ w/src/addrman.cpp
    @@ -570,13 +570,13 @@ void CAddrMan::GetAddr_(std::vector<CAddress>& vAddr, size_t max_addresses, size
             if (ai.IsTerrible(now)) continue;
     
             vAddr.push_back(ai);
         }
     }
     
    -void CAddrMan::Connected_(const CService& addr, int64_t nTime)
    +void CAddrMan::Connected_(const CService& addr)
     {
         AssertLockHeld(cs);
     
         CAddrInfo* pinfo = Find(addr);
     
         // if not found, bail out
    @@ -588,14 +588,16 @@ void CAddrMan::Connected_(const CService& addr, int64_t nTime)
         // check whether we are talking about the exact same CService (including same port)
         if (info != addr)
             return;
     
         // update info
         int64_t nUpdateInterval = 20 * 60;
    -    if (nTime - info.nTime > nUpdateInterval)
    -        info.nTime = nTime;
    +    const auto now = GetAdjustedTime();
    +    if (now - info.nTime > nUpdateInterval) {
    +        info.nTime = now;
    +    }
     }
     
     void CAddrMan::SetServices_(const CService& addr, ServiceFlags nServices)
     {
         AssertLockHeld(cs);
     
    diff --git i/src/addrman.h w/src/addrman.h
    index 1fc64ac07f..28f445b071 100644
    --- i/src/addrman.h
    +++ w/src/addrman.h
    @@ -605,18 +605,18 @@ public:
             GetAddr_(vAddr, max_addresses, max_pct, network);
             Check();
             return vAddr;
         }
     
         //! Outer function for Connected_()
    -    void Connected(const CService &addr, int64_t nTime = GetAdjustedTime())
    +    void Connected(const CService &addr)
             EXCLUSIVE_LOCKS_REQUIRED(!cs)
         {
             LOCK(cs);
             Check();
    -        Connected_(addr, nTime);
    +        Connected_(addr);
             Check();
         }
     
         void SetServices(const CService &addr, ServiceFlags nServices)
             EXCLUSIVE_LOCKS_REQUIRED(!cs)
         {
    @@ -760,15 +760,14 @@ private:
          *  this information doesn't leak topology information to network spies.
          *
          *  net_processing calls this function when it *disconnects* from a peer to
          *  not leak information about currently connected peers.
          *
          * [@param](/bitcoin-bitcoin/contributor/param/)[in]   addr     The address of the peer we were connected to
    -     * [@param](/bitcoin-bitcoin/contributor/param/)[in]   nTime    The time that we were last connected to this peer
          */
    -    void Connected_(const CService& addr, int64_t nTime) EXCLUSIVE_LOCKS_REQUIRED(cs);
    +    void Connected_(const CService& addr) EXCLUSIVE_LOCKS_REQUIRED(cs);
     
         //! Update an entry's service bits.
         void SetServices_(const CService &addr, ServiceFlags nServices) EXCLUSIVE_LOCKS_REQUIRED(cs);
     
         //! Remove invalid addresses.
         void RemoveInvalid() EXCLUSIVE_LOCKS_REQUIRED(cs);
    diff --git i/src/test/fuzz/addrman.cpp w/src/test/fuzz/addrman.cpp
    index 92c34e74d9..38c551e0e3 100644
    --- i/src/test/fuzz/addrman.cpp
    +++ w/src/test/fuzz/addrman.cpp
    @@ -100,13 +100,13 @@ FUZZ_TARGET_INIT(addrman, initialize_addrman)
                         addr_man.Attempt(*opt_service, fuzzed_data_provider.ConsumeBool(), ConsumeTime(fuzzed_data_provider));
                     }
                 },
                 [&] {
                     const std::optional<CService> opt_service = ConsumeDeserializable<CService>(fuzzed_data_provider);
                     if (opt_service) {
    -                    addr_man.Connected(*opt_service, ConsumeTime(fuzzed_data_provider));
    +                    addr_man.Connected(*opt_service);
                     }
                 },
                 [&] {
                     const std::optional<CService> opt_service = ConsumeDeserializable<CService>(fuzzed_data_provider);
                     if (opt_service) {
                         addr_man.SetServices(*opt_service, ConsumeWeakEnum(fuzzed_data_provider, ALL_SERVICE_FLAGS));
    

    </details>

    Either way, the suppression unsigned-integer-overflow:addrman.cpp cannot be removed because according to #21000 there is another place in addrman.cpp that has a similar problem.

  15. jonatack force-pushed on Jul 29, 2021
  16. jonatack force-pushed on Jul 30, 2021
  17. jonatack renamed this:
    p2p, refactor: make nTime truncation conversion explicit in CAddrMan::Connected_()
    p2p: fix ubsan addrman errors, make nTime truncation conversion explicit
    on Jul 30, 2021
  18. jonatack commented at 9:43 AM on July 30, 2021: member

    Given that the second argument of Connected() is never used, maybe it is better to drop it: [patch] remove nTime argument

    Either way, the suppression unsigned-integer-overflow:addrman.cpp cannot be removed because according to #21000 there is another place in addrman.cpp that has a similar problem.

    Thanks! Removing the argument seems out of scope of this change, but you're right.

    Rebased, added a second UBSan fix, and removed the addrman UBSan suppressions that these changes seem to allow. There is a third one at the top of #21000; <strike>I didn't reproduce the warning but may still look into it.</strike> -> Edit: done.

  19. in src/addrman.h:321 in 523c26d8c2 outdated
     317 | @@ -317,7 +318,7 @@ class CAddrMan
     318 |  
     319 |          uint8_t compat;
     320 |          s >> compat;
     321 | -        const uint8_t lowest_compatible = compat - INCOMPATIBILITY_BASE;
     322 | +        const uint8_t lowest_compatible = std::max(compat - INCOMPATIBILITY_BASE, 0);
    


    MarcoFalke commented at 9:55 AM on July 30, 2021:

    Can you explain why treating corrupt files as file format version 0 is the right fix?


    jonatack commented at 1:20 PM on July 30, 2021:

    Good point, it should raise.

    -        if (lowest_compatible > FILE_FORMAT) {
    +        if (lowest_compatible > FILE_FORMAT || compat < INCOMPATIBILITY_BASE) {
    
  20. vasild commented at 12:49 PM on July 30, 2021: member

    Removing the argument seems out of scope of this change

    Why? If the purpose is to silence the warning, removing the argument will achieve that (+ remove some dead code).

    With current master we have wrong behavior (e.g. treat year 2110 as year 1974) and a warning due to that. With this PR in its current form (523c26d) we have wrong behavior (same as in master), no warning now from fuzzing and no warning ever, even in year 2110.

  21. jonatack commented at 1:05 PM on July 30, 2021: member

    Why? If the purpose is to silence the warning, removing the argument will achieve that (+ remove some dead code).

    Would it not just replace one int64 value for another int64 being passed into a uint32?

  22. jonatack commented at 3:46 PM on July 30, 2021: member

    Updated per @MarcoFalke's #22094 (review) to raise if the file compat is corrupt.

  23. jonatack force-pushed on Jul 30, 2021
  24. vasild commented at 9:24 AM on August 2, 2021: member

    Would it not just replace one int64 value for another int64 being passed into a uint32?

    Correct. But that new int64 (returned by GetAdjustedTime()) will not overflow the uint32_t range before year 2106 and after that we would get a warning as deserved.

    The underlying problem is that the code is not ready for year>2106 and the fuzzer tests with such years. Given that it is difficult to fix the code, I think it would be better to make the fuzzer not test with years>2106 (removing the second argument of CAddrMan::Connected() will achieve that).

  25. jonatack force-pushed on Aug 10, 2021
  26. jonatack commented at 2:12 PM on August 10, 2021: member

    @vasild I added your proposed change at the top as a commit authored by you (though it doesn't fix the UBSan error for me) and added TODO comments regarding updates needed by the Year 2106.

    Hopefully this makes the issue more explicit in the code and documentation.

    Also fixed the remaining addrman issues listed in #21000 that were suppressed.

    A final commit removes all four of the UBSan addrman suppressions.

    Edit: hm, seeing new UBSan errors in both the CI and locally. Looking.

  27. p2p: drop unused nTime param from CAddrMan::Connected() 14119e9a59
  28. p2p: make nTime truncation explicit in CAddrMan::Connected_()
    CAddress::nTime is declared in src/protocol.h as uint32
    but in CAddrMan::Connected_() it needs to be converted to int64
    for subtraction from an int64 to avoid unsigned integer overflow,
    and int64 values can be written to it with implicit integer truncation.
    
    Fixes two UBSan errors reported by the addrman fuzzer
    
    - a currently suppressed error
    addrman.cpp:594:13: runtime error: unsigned integer overflow:
      513 - 100000000 cannot be represented in type 'unsigned int'
    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior addrman.cpp:594:13 in
    
    - a non-suppressed error
    addrman.cpp:595:22: runtime error: implicit conversion
      from type 'int64_t' (aka 'long') of value 68719478016 (64-bit, signed)
      to type 'uint32_t' (aka 'unsigned int') changed the value to 1280 (32-bit, unsigned)
    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior addrman.cpp:595:22
    f4b5064d97
  29. p2p: make nTime truncation explicit in CAddrMan::Add_()
    CAddress::nTime is declared in src/protocol.h as uint32,
    but in CAddrMan::Add_() int64 values can be written to it
    with implicit integer truncation, and to avoid unsigned
    integer overflow errors nTime needs to be converted to
    int64 for int64 values to be subtracted from it.
    
    Fixes these currently suppressed UBSan errors (addrman fuzzer)
    - unsigned integer overflow
    - implicit-signed-integer-truncation
    148b98f66d
  30. p2p: fix ubsan implicit conversion error in CAddrman::Unserialize()
    and raise if the file compat is corrupt.
    
    Found by the addrman fuzzer with UBSan:
    
    SUMMARY: UndefinedBehaviorSanitizer
    addrman.h:320:43: runtime error: implicit conversion
      from type 'int' of value -32 (32-bit, signed)
      to type 'const uint8_t' (aka 'const unsigned char')
      changed the value to 224 (8-bit, unsigned)
    ca7f8f59bc
  31. fuzz: remove addrman ubsan integer suppressions 2706a9d81b
  32. jonatack force-pushed on Aug 10, 2021
  33. DrahtBot commented at 5:14 PM on August 13, 2021: member

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #22740 ([addrman] Move serialization code to cpp by amitiuttarwar)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  34. in src/addrman.cpp:600 in 2706a9d81b
     598 | +    const int64_t nUpdateInterval{20 * 60};
     599 | +    const int64_t now{GetAdjustedTime()};
     600 | +    // TODO: change CAddress::nTime from uint32 to int64 before the Year 2106.
     601 | +    if (now - static_cast<int64_t>(info.nTime) > nUpdateInterval) {
     602 | +        info.nTime = static_cast<uint32_t>(now);
     603 | +    }
    


    vasild commented at 8:23 AM on August 23, 2021:

    This warrants an assert that now, when cast to uint32_t will not overflow, i.e. year 2106 be silently converted to year 1970:

            assert(now <= std::numeric_limits<uint32>::max());
            info.nTime = static_cast<uint32_t>(now);
    
  35. in src/addrman.cpp:336 in 2706a9d81b
     335 |          int64_t nUpdateInterval = (fCurrentlyOnline ? 60 * 60 : 24 * 60 * 60);
     336 | -        if (addr.nTime && (!pinfo->nTime || pinfo->nTime < addr.nTime - nUpdateInterval - nTimePenalty))
     337 | -            pinfo->nTime = std::max((int64_t)0, addr.nTime - nTimePenalty);
     338 | -
     339 | +        if (addr.nTime && (!pinfo->nTime || pinfo->nTime < static_cast<int64_t>(addr.nTime) - nUpdateInterval - nTimePenalty)) {
     340 | +            pinfo->nTime = std::max<uint32_t>(0, static_cast<int64_t>(addr.nTime) - nTimePenalty);
    


    vasild commented at 8:30 AM on August 23, 2021:

    This will implicitly convert the second argument of std::max from int64_t to uint32_t:

        int64_t x = 10;
        int64_t y = 14;
        std::cout << std::max<uint32_t>(0, x - y) << std::endl;
    

    produces this warning (with both gcc and clang; that particular warning switch is not turned on in Bitcoin Core):

    warning: implicit conversion loses integer precision: 'long' to 'const unsigned int' [-Wshorten-64-to-32]
        std::cout << std::max<uint32_t>(0, x - y) << std::endl;
                     ~~~                   ~~^~~
    

    and prints 4294967292 instead of the intended 0.

    I would go for KISS:

    if (addr.nTime > nTimePenalty) {
        pinfo->nTime = static_cast<uint32_t>(addr.nTime - nTimePenalty);
    } else {
        pinfo->nTime = 0;
    }
    
  36. vasild commented at 9:16 AM on August 23, 2021: member

    I see now why removing the second argument of Connected() (which comes from the fuzzer and could contain year >2106) does not fix the warning - I was thinking that GetAdjustedTime() would return "now" and since "now" is year 2021, we would not get an overflow. However the fuzzer has set mocked time which could be year >2106, so GetAdjustedTime() could overflow uint32_t too.

  37. DrahtBot commented at 3:11 AM on September 1, 2021: member

    <!--cf906140f33d8803c4a75a2196329ecb-->

    🐙 This pull request conflicts with the target branch and needs rebase.

    <sub>Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".</sub>

  38. DrahtBot added the label Needs rebase on Sep 1, 2021
  39. MarcoFalke commented at 12:04 PM on October 22, 2021: member

    Needs rebase if still relevant

  40. theStack commented at 5:58 PM on December 21, 2021: member

    Concept ACK

  41. fanquake commented at 9:55 AM on April 20, 2022: member

    What is the state of this PR? The ubsan suppressions it set out to fix/remove no-longer exist in master (#24302), so I don't know if it's still relevant.

  42. jonatack commented at 11:33 AM on April 20, 2022: member

    Will update or close.

  43. fanquake commented at 11:46 AM on April 29, 2022: member

    Will update or close.

    Feel free to reopen whenever you get a chance to look at this again, and/or determine if any of the changes are still relevant. Going to close for now, as this has needed rebase for 7 months, and is currently neither reviewable or mergeable.

  44. fanquake closed this on Apr 29, 2022

  45. DrahtBot locked this on Apr 29, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-14 21:14 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me