util: Return uint64_t from _MiB and _GiB operators #35097

pull maflcko wants to merge 3 commits into bitcoin:master from maflcko:2604-u64 changing 7 files +59 −74
  1. maflcko commented at 10:34 AM on April 17, 2026: member

    Currently, the _MiB literal operator returns a value of type size_t. This is brittle and confusing:

    • It is inherently impossible to represent larger values, like storage device usage.
    • Similarly, it is not possible to even type an upper cap on the memory, see the failure in #34692 (comment)
    • Using size_t isn't required here. The function is evaluated at compile time, and even 32-bit compilers can evaluate an uint64_t at compile time.
    • Using size_t here encourages it to be used in more places, which will likely lead to more bugs and CVEs, such as #34109, #33724, etc.

    Fix all issues, by:

    • Marking the operator consteval, to ensure it is really only called at compile-time.
    • Returning an uint64_t value.
    • Using it in the place where it was previously not possible.

    Review note:

    This should have no downside, because the C++11 narrowing checks continue to work as expected. For example, typing uint8_t{1_MiB}; will continue to fail with the correct error message error: constant expression evaluates to 1048576 which cannot be narrowed to type 'uint8_t' (aka 'unsigned char') [-Wc++11-narrowing] (like it does on current master).

  2. DrahtBot added the label Utils/log/libs on Apr 17, 2026
  3. maflcko added the label Refactoring on Apr 17, 2026
  4. DrahtBot commented at 10:34 AM on April 17, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK l0rinc, hebasto, hodlinator, achow101
    Concept ACK kevkevinpal
    Stale ACK sedited

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #34641 (node: scale default -dbcache with system RAM by l0rinc)
    • #34628 (p2p: Replace per-peer transaction rate-limiting with global rate limits by ajtowns)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  5. maflcko force-pushed on Apr 17, 2026
  6. DrahtBot added the label CI failed on Apr 17, 2026
  7. DrahtBot commented at 11:19 AM on April 17, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task No wallet: https://github.com/bitcoin/bitcoin/actions/runs/24560728284/job/71810830710</sub> <sub>LLM reason (✨ experimental): CI failed to build due to C++ template parameter mismatch errors in util_tests.cpp (invalid template-template argument/static_assert compilation error).</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  8. maflcko commented at 11:21 AM on April 17, 2026: member

    Force pushed to make the older clang happy:

    -template <template <class, auto> class Op, class Int, auto v>
    +template <template <class, auto...> class Op, class Int, auto v>
    

    Godbolt: https://godbolt.org/z/ex1KEo5a3

  9. DrahtBot removed the label CI failed on Apr 17, 2026
  10. l0rinc commented at 1:58 PM on April 17, 2026: contributor

    Concept ACK, had quite a few problems because of this before. Personally I would prefer #34435 being merged first and _GiB also adjusted accordingly - that could reveal the advantage (or disadvantage) of this conversion better. But I will adapt if this will be accepted first - will review in detail later.

  11. maflcko commented at 2:23 PM on April 17, 2026: member

    Personally I would prefer #34435 being merged first and _GiB also adjusted accordingly - that could reveal the advantage (or disadvantage) of this conversion better.

    I think the advantages should be clear from the pull description and the tests, but happy to wait for the other pull. It is just a size_t->uint64_t change in two or three lines, so should be trivial to rebase one way or the other. Anything should be fine, from my side.

  12. hodlinator commented at 2:51 PM on April 17, 2026: contributor

    Concept ACK

  13. kevkevinpal commented at 9:01 PM on April 17, 2026: contributor

    Concept ACK

    It makes sense to return uint64_t instead of size_t and this code change is pretty minimal and scoped to just a few files.

  14. sedited approved
  15. sedited commented at 1:11 PM on April 22, 2026: contributor

    ACK faeabdd21fc34819e6b3acda1e094e033813cf35

  16. DrahtBot requested review from l0rinc on Apr 22, 2026
  17. DrahtBot requested review from hodlinator on Apr 22, 2026
  18. in src/util/byte_units.h:17 in faeabdd21f outdated
      15 | +consteval uint64_t operator""_MiB(unsigned long long mebibytes)
      16 |  {
      17 |      auto bytes{CheckedLeftShift(mebibytes, 20)};
      18 | -    if (!bytes || *bytes > std::numeric_limits<size_t>::max()) {
      19 | -        throw std::overflow_error("MiB value too large for size_t byte conversion");
      20 | +    if (!bytes || *bytes > std::numeric_limits<uint64_t>::max()) {
    


    hodlinator commented at 2:13 PM on April 22, 2026:

    Seems like unsigned long long is the same size as uint64_t on both 32 and 64 bit platforms, making the second condition reduntant (~I guess even on master~ edit 2026-04-23: I was mistaken). Would prefer something like the following (so we only potentially fail on unusual/future 128-bit platforms):

        static_assert(sizeof(decltype(bytes)::value_type) == sizeof(decltype(operator""_MiB(mebibytes))));
        if (!bytes) {
    

    maflcko commented at 7:01 PM on April 22, 2026:

    I haven't tried it, but I think the static_assert can be expressed differently by just using:

    auto bytes{CheckedLeftShift(uint64_t{mebibytes}, 20)};
    

    Happy to push either version.

    However, I wonder what the risk is. I don't expect a 128 bit platform to exist, and even if it did, I don't think we have any reason to represent a size in bytes larger than can fit in u64.

    To me it seems the version in this pull request is the ideal one, and the static_assert or equivalent narrowing check would be stricter, but with unclear benefit. To me it seems it would plain reject the build on 128 bit platforms, which doesn't seem useful or desirable?


    hodlinator commented at 8:03 PM on April 22, 2026:

    Rejecting 128-bit platforms seems acceptable as neither of us expect it to exist, just have it as a sanity check in case of some weird platform.

    Nice idea to achieve something equivalent without the static_assert! Ideally I'd like to explicitly bind it to the return type instead of repeating it, something like:

        using ReturnType = decltype(operator""_MiB(mebibytes));
        std::optional<ReturnType> bytes{CheckedLeftShift(ReturnType{mebibytes}, 20)};
        if (!bytes) {
    

    Another way of expressing this concern is that setting:

    --- a/CMakeLists.txt
    +++ b/CMakeLists.txt
    @@ -478,6 +478,7 @@ else()
       try_append_cxx_flags("-Wundef" TARGET warn_interface SKIP_LINK)
       try_append_cxx_flags("-Wleading-whitespace=spaces" TARGET warn_interface SKIP_LINK)
       try_append_cxx_flags("-Wtrailing-whitespace=any" TARGET warn_interface SKIP_LINK)
    +  try_append_cxx_flags("-Wtype-limits" TARGET warn_interface SKIP_LINK)
     
       # Some compilers (gcc) ignore unknown -Wno-* options, but warn about all
       # unknown options if any other warning is produced. Test the -Wfoo case, and
    

    Does among a few other things expose this on the current version of this PR:

    ../src/util/byte_units.h:17:26: warning: result of comparison 'unsigned long long' > 18446744073709551615 is always false [-Wtautological-type-limit-compare]
       17 |     if (!bytes || *bytes > std::numeric_limits<uint64_t>::max()) {
          |                   ~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    

    maflcko commented at 9:19 PM on April 22, 2026:

    Rejecting 128-bit platforms seems acceptable

    I am not sure if this pull request is the right place to make this change.

    +  try_append_cxx_flags("-Wtype-limits" TARGET warn_interface SKIP_LINK)
    

    That seems unrelated to this pull request, because it happens on master as well:

    <source>:13:13: warning: result of comparison 'unsigned long long' > 18446744073709551615 is always false [-Wtautological-type-limit-compare]
       13 |     if (ull > std::numeric_limits<size_t>::max()) {
          |         ~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    

    hodlinator commented at 9:24 PM on April 22, 2026:

    On master it does make a difference for 32-bit platforms as we have *bytes > std::numeric_limits<size_t>::max() where bytes is ull. This PR makes it redundant there as well.


    hodlinator commented at 9:28 PM on April 22, 2026:

    (But to be clear, I'm not advocating for adding that warning in this very PR).


    maflcko commented at 6:52 AM on April 23, 2026:

    I guess I can just change the return type to ULL or auto if you want to remove the check?


    l0rinc commented at 1:07 PM on April 23, 2026:

    Given we're maxing out the value space, it seems to me we could simplify this to:

    consteval uint64_t ByteUnitsToBytes(unsigned long long units, const char* exception_msg)
    {
        if (const auto bytes{CheckedLeftShift(static_cast<uint64_t>(units), SHIFT)}) return *bytes;
        throw std::overflow_error(exception_msg);
    }
    

    maflcko commented at 6:24 AM on April 24, 2026:

    I don't mind bike-shedding this, but what is the goal here even? I've already said that I don't want to explicitly or implicitly break 128 bit archs. Certainly they won't exists in practise any time soon, but I don't see why in theory some researcher in an 128-bit env (for whatever reason) should face compile errors or silent breakages for no good reason.

    I can offer two alternatives:

    • Just leave this as-is, because it doesn't matter.
    • Return ULL/auto, because this is consteval, and any narrowing checks need to happen later anyway.

    l0rinc commented at 1:02 PM on April 24, 2026:

    I don't mind bike-shedding this, but what is the goal here even?

    My concern here is only to avoid the illusion of size validation - since we were doing it before, the method is mostly vestigial now


    maflcko commented at 1:26 PM on April 24, 2026:

    Ah, I see. I can offer two alternatives:

    • Leave the code as-is, and add a comment that the left shift can never fail for any real value.
    • Return ULL/auto and remove the numeric_limits , because this is consteval, and any narrowing checks need to happen later anyway.

    l0rinc commented at 1:32 PM on April 24, 2026:

    I'm fine with either, would just prefer something that dissolves the illusion of architectural validation.


    maflcko commented at 2:52 PM on April 24, 2026:

    Haven't added a comment, I changed the line below to say "too large". Eg.

    error: call to consteval function 'operator""_GiB' is not a constant expression
       24 | 18'123'456'789_GiB;
          | ^
    ./src/util/byte_units.h:19:9: note: subexpression not valid in a constant expression
       19 |         throw std::overflow_error("Too large");
    

    Is that fine?


    l0rinc commented at 2:56 PM on April 24, 2026:

    Absolutely, already acked, thanks!

  19. in src/test/util_tests.cpp:1852 in faeabdd21f
    1849 | +    }
    1850 | +    {
    1851 | +        using Int = uint32_t;
    1852 | +        constexpr auto max_mib{std::numeric_limits<Int>::max() >> 20};
    1853 | +        static_assert(is_valid<call_operator_MiB, Int, max_mib>);
    1854 | +        static_assert(!is_valid<call_operator_MiB, Int, max_mib + 1>);
    


    hodlinator commented at 2:13 PM on April 22, 2026:

    My first thought was "Why would _MiB ever fail for 32-bit integers?". But I guess this is kind of simulating 32-bit platforms on both 32 and 64 bit platforms? Or rather assigning to a given type using curly braces in order to have narrowing checks enabled (Int{operator""_MiB(v)}).

    Not sure what value this uint32_t path serves. IIUC the uint64_t path fails due to the exception being thrown inside the _MiB UDL function due to CheckedLeftShift() failing. The uint32_t failure is due to the output of _MiB not being narrowable to Int as described above. It's essentially asserting that std::numeric_limits<uint32_t>::max() + 1U does not fit inside uint32_t without wrapping around, which isn't specific to the _MiB function.


    maflcko commented at 6:34 PM on April 22, 2026:

    Happy to remove it, if you think it is harmful. However, there is code using uint32_t{nn_MiB}; (and friends), which is only safe when the expression does not compile on overflow, so I think it is useful to test. It is not only about size_t (platform dependent), but also about a plain u32:

    git grep 'MiB'|grep 'unsigned int'
    

    hodlinator commented at 8:34 PM on April 22, 2026:

    It's confusing and at the very least deserving of a comment justifying it. Very light but compounding form of harm. :)

    ~If we change the return type to uint64_t then uint32_t{nn_MiB} (when braces are used) should fail to compile consistently on any platform. I don't see what's special with the _MiB-function in this respect. In fact I am surprised the is_valid<call_operator_MiB, uint32_t, max_mib> call compiles, but maybe in full consteval land the anti-narrowing braces get to operate on actual return values, not just types?~

    Edit: I realized that constexpr uint32_t{nn_MiB} does benefit from knowing the value of the uint64_t at compile time. My wish for a comment explaining what exactly it's testing still stands.


    maflcko commented at 9:10 PM on April 22, 2026:

    Edit: I realized that constexpr uint32_t{nn_MiB} does benefit from knowing the value of the uint64_t at compile time.

    Yes, see https://eel.is/c++draft/dcl.init.list#7.4.2

    [except] the source is a constant expression whose value after integral promotions will fit into the target type

    My wish for a comment explaining what exactly it's testing still stands.

    It is just documenting that the rule of the standard is followed, because we rely on it and it may not be obvious for everyone. Also, it is not uncommon for older compilers to violate the standard. E.g in #34524 (comment), the prior rule was violated: https://eel.is/c++draft/dcl.init.list#7.4.1


    hodlinator commented at 9:43 PM on April 22, 2026:

    If you don't want to add a comment, maybe a more descriptive name than call_operator_MiB would help - assign_MiB_result_to_int?


    maflcko commented at 6:52 AM on April 23, 2026:

    Sure, done

  20. hodlinator commented at 6:23 PM on April 22, 2026: contributor

    Reviewed faeabdd21fc34819e6b3acda1e094e033813cf35

  21. DrahtBot requested review from hodlinator on Apr 22, 2026
  22. DrahtBot added the label Needs rebase on Apr 23, 2026
  23. maflcko force-pushed on Apr 23, 2026
  24. maflcko force-pushed on Apr 23, 2026
  25. DrahtBot added the label CI failed on Apr 23, 2026
  26. DrahtBot commented at 7:12 AM on April 23, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task i686, no IPC: https://github.com/bitcoin/bitcoin/actions/runs/24821373101/job/72646683446</sub> <sub>LLM reason (✨ experimental): CI failed due to a C++ compile error in cuckoocache_tests.cpp from a narrowing conversion (uint64_t to size_t) for size_t bytes{megabytes * 1_MiB}; treated as an error.</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  27. DrahtBot removed the label Needs rebase on Apr 23, 2026
  28. test: Use MiB operator directly in cuckoocache_tests
    Previously, they were using a pattern of defininig a constant megabytes
    symbol, and then multiplying that by 1_MiB.
    
    It is easier to use the _MiB operator directly.
    fa9ddb01c9
  29. maflcko force-pushed on Apr 23, 2026
  30. sedited commented at 11:51 AM on April 23, 2026: contributor

    Should mention the _GiB operator in the title and commit message now too :)

  31. maflcko renamed this:
    util: Return uint64_t from _MiB operator
    util: Return uint64_t from _MiB and _GiB operators
    on Apr 23, 2026
  32. maflcko force-pushed on Apr 23, 2026
  33. in src/util/byte_units.h:28 in faca929d70 outdated
      27 |  //! Overflow-safe conversion of MiB to bytes.
      28 | -constexpr size_t operator""_MiB(unsigned long long mebibytes)
      29 | +consteval uint64_t operator""_MiB(unsigned long long mebibytes)
      30 |  {
      31 | -    return util::detail::ByteUnitsToBytes<20>(mebibytes, "MiB value too large for size_t byte conversion");
      32 | +    return util::detail::ByteUnitsToBytes<20>(mebibytes, "MiB value too large for uint64_t byte conversion");
    


    l0rinc commented at 12:14 PM on April 23, 2026:

    faca929 util: Return uint64_t from _MiB and _GiB operators:

    This message reads a bit awkwardly now, maybe we don't need a message, the std::overflow_error type should already make it clear.


    maflcko commented at 6:44 AM on April 24, 2026:

    This is the same message as it was on master, I am not changing it, and I don't understand why it would be awkward now. #34435 was merged with this message, so I don't see why it should be changed right after. If you think any of the style nits are blockers, then it would have been better to not ask for the merge of #34435 and just fix them up there.

    Leaving as-is for now.


    l0rinc commented at 1:05 PM on April 24, 2026:

    No, I mean we're not really validating the size anymore, i.e. 10_GiB is valid on 32 bits now and larger than 64 bits isn't a realistic limit, so my opinion was that the fine-grained messages aren't useful anymore.

    If you think any of the style nits are blockers

    I didn't mean to imply that, please let me know what gave you that impression.


    maflcko commented at 1:33 PM on April 24, 2026:

    larger than 64 bits isn't a realistic limit

    Ok, I see the point. Let's continue this topic in a single thread in #35097 (review) ?


    maflcko commented at 2:21 PM on April 24, 2026:

    Simplified to say "Too large". E.g.

    error: call to consteval function 'operator""_GiB' is not a constant expression
       24 | 18'123'456'789_GiB;
          | ^
    ./src/util/byte_units.h:19:9: note: subexpression not valid in a constant expression
       19 |         throw std::overflow_error("Too large");
    
  34. in src/test/cuckoocache_tests.cpp:54 in fa9ddb01c9 outdated
      49 | @@ -50,16 +50,15 @@ BOOST_AUTO_TEST_CASE(test_cuckoocache_no_fakes)
      50 |  };
      51 |  
      52 |  struct HitRateTest : BasicTestingSetup {
      53 | -/** This helper returns the hit rate when megabytes*load worth of entries are
      54 | - * inserted into a megabytes sized cache
      55 | +/** This helper returns the hit rate when bytes*load worth of entries are
      56 | + * inserted into a bytes sized cache
    


    l0rinc commented at 12:15 PM on April 23, 2026:

    fa9ddb0 test: Use MiB operator directly in cuckoocache_tests:

    nit: "a bytes sized cache" reads awkwardly:

     * inserted into a cache with the given byte size
    

    maflcko commented at 6:44 AM on April 24, 2026:

    I think it is understandable. Leaving as-is for now.

  35. DrahtBot removed the label CI failed on Apr 23, 2026
  36. in src/test/util_tests.cpp:1855 in faca929d70 outdated
    1860 | @@ -1850,16 +1861,24 @@ BOOST_AUTO_TEST_CASE(mib_string_literal_test)
    1861 |      BOOST_CHECK_EQUAL(128_MiB, 0x8000000U);
    1862 |      BOOST_CHECK_EQUAL(550_MiB, 550ULL * 1024 * 1024);
    1863 |  
    1864 | -    // Overflow handling
    1865 | -    constexpr auto max_mib{std::numeric_limits<size_t>::max() >> 20};
    1866 | -    if constexpr (SIZE_MAX == UINT32_MAX) {
    


    l0rinc commented at 12:48 PM on April 23, 2026:

    faca929 util: Return uint64_t from _MiB and _GiB operators:

    Now that big values can be compiled on 32 bits maybe we could adjust:

    Something like this could work:

    diff --git a/src/test/caches_tests.cpp b/src/test/caches_tests.cpp
    --- a/src/test/caches_tests.cpp	(revision 64caf269d5ac6fc9cf9b44d64ce3b9fd85a91867)
    +++ b/src/test/caches_tests.cpp	(revision a4647738af026fe73da69ae2025bf40910da6cbc)
    @@ -22,23 +22,21 @@
         BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/1500_MiB, /*total_ram=*/2_GiB)); // Under cap
         BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/1600_MiB, /*total_ram=*/2_GiB)); // Over cap
     
    -    if constexpr (SIZE_MAX == UINT64_MAX) {
    -        // 4 GiB RAM - cap is 75%
    -        BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/2500_MiB, /*total_ram=*/4_GiB)); // Under cap
    -        BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/3500_MiB, /*total_ram=*/4_GiB)); // Over cap
    +    // 4 GiB RAM - cap is 75%
    +    BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/2500_MiB, /*total_ram=*/4_GiB)); // Under cap
    +    BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/3500_MiB, /*total_ram=*/4_GiB)); // Over cap
     
    -        // 8 GiB RAM - cap is 75%
    -        BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/6000_MiB, /*total_ram=*/8_GiB)); // Under cap
    -        BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/7000_MiB, /*total_ram=*/8_GiB)); // Over cap
    +    // 8 GiB RAM - cap is 75%
    +    BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/6000_MiB, /*total_ram=*/8_GiB)); // Under cap
    +    BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/7000_MiB, /*total_ram=*/8_GiB)); // Over cap
     
    -        // 16 GiB RAM - cap is 75%
    -        BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/10'000_MiB, /*total_ram=*/16_GiB)); // Under cap
    -        BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/15'000_MiB, /*total_ram=*/16_GiB)); // Over cap
    +    // 16 GiB RAM - cap is 75%
    +    BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/10_GiB, /*total_ram=*/16_GiB)); // Under cap
    +    BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/15_GiB, /*total_ram=*/16_GiB)); // Over cap
     
    -        // 32 GiB RAM - cap is 75%
    -        BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/20'000_MiB, /*total_ram=*/32_GiB)); // Under cap
    -        BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/30'000_MiB, /*total_ram=*/32_GiB)); // Over cap
    -    }
    +    // 32 GiB RAM - cap is 75%
    +    BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/20_GiB, /*total_ram=*/32_GiB)); // Under cap
    +    BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/30_GiB, /*total_ram=*/32_GiB)); // Over cap
     }
     
    diff --git a/src/test/system_ram_tests.cpp b/src/test/system_ram_tests.cpp
    --- a/src/test/system_ram_tests.cpp	(revision 64caf269d5ac6fc9cf9b44d64ce3b9fd85a91867)
    +++ b/src/test/system_ram_tests.cpp	(revision a4647738af026fe73da69ae2025bf40910da6cbc)
    @@ -21,12 +21,7 @@
         }
     
         BOOST_CHECK_GE(*total, 1000_MiB);
    -
    -    if constexpr (SIZE_MAX == UINT64_MAX) {
    -        // Upper bound check only on 64-bit: 32-bit systems can reasonably have max memory,
    -        // but extremely large values on 64-bit likely indicate detection errors
    -        BOOST_CHECK_LT(*total, 10'000'000_MiB); // >10 TiB memory is unlikely
    -    }
    +    BOOST_CHECK_LT(*total, 10'000_GiB); // ~10 TiB memory is unlikely
     }
     
     BOOST_AUTO_TEST_SUITE_END()
    

    maflcko commented at 6:48 AM on April 24, 2026:

    Thx, pushed the diff.

  37. in src/node/caches.cpp:33 in faca929d70 outdated
      29 | @@ -30,7 +30,7 @@ static constexpr size_t MAX_32BIT_DBCACHE{1_GiB};
      30 |  //! Larger default dbcache on 64-bit systems with enough RAM.
      31 |  static constexpr size_t HIGH_DEFAULT_DBCACHE{1_GiB};
      32 |  //! Minimum detected RAM required for HIGH_DEFAULT_DBCACHE.
      33 | -static constexpr uint64_t HIGH_DEFAULT_DBCACHE_MIN_TOTAL_RAM{uint64_t{4} * 1_GiB};
    


    l0rinc commented at 12:49 PM on April 23, 2026:

    faca929 util: Return uint64_t from _MiB and _GiB operators:

    Noticed we're casting to signed in a few places: any corner cases where this could be problematic?

    And a cast may be redundant now:


    Nit: if touching nearby casts, these could use brace-style casts for consistency:

    diff --git a/src/node/blockstorage.cpp b/src/node/blockstorage.cpp
    --- a/src/node/blockstorage.cpp	(revision fcffb924c3f94ab8d90e0c8a4e45629ba69988f3)
    +++ b/src/node/blockstorage.cpp	(revision cf15c3fd3f26928ef88ef0e71a895ed886ffadf5)
    @@ -395,7 +395,7 @@
     
         LogDebug(BCLog::PRUNE, "[%s] target=%dMiB actual=%dMiB diff=%dMiB min_height=%d max_prune_height=%d removed %d blk/rev pairs\n",
                  chain.GetRole(), target / 1_MiB, nCurrentUsage / 1_MiB,
    -             (int64_t(target) - int64_t(nCurrentUsage)) / int64_t(1_MiB),
    +             (int64_t(target) - int64_t(nCurrentUsage)) / int64_t{1_MiB},
                  min_block_to_prune, last_block_can_prune, count);
     }
     
    diff --git a/src/validation.h b/src/validation.h
    --- a/src/validation.h	(revision fcffb924c3f94ab8d90e0c8a4e45629ba69988f3)
    +++ b/src/validation.h	(revision cf15c3fd3f26928ef88ef0e71a895ed886ffadf5)
    @@ -518,7 +518,7 @@
     constexpr int64_t LargeCoinsCacheThreshold(int64_t total_space) noexcept
     {
         // No periodic flush needed if at least this much space is free
    -    constexpr int64_t MAX_BLOCK_COINSDB_USAGE_BYTES{int64_t(10_MiB)};
    +    constexpr int64_t MAX_BLOCK_COINSDB_USAGE_BYTES{int64_t{10_MiB}};
         return std::max((total_space * 9) / 10,
                         total_space - MAX_BLOCK_COINSDB_USAGE_BYTES);
     }
    

    maflcko commented at 6:43 AM on April 24, 2026:

    Yeah, those should be using narrowing checks, but this is unrelated to this pull request and narrowing checks should have been used when the code was written in #34435. If you want to change them, you can do it in the follow-up to #34435? I can also push a commit here, if you want, collecting the other nits (https://github.com/bitcoin/bitcoin/pull/35097#discussion_r3131057306 and #35097 (review)), I can push the diff or your commit here.


    maflcko commented at 1:34 PM on April 24, 2026:

    If you push a commit to your repo, I can push it here, with you as author, but up to you.


    hodlinator commented at 7:52 PM on April 24, 2026:

    Seems like constexpr int64_t MAX_BLOCK_COINSDB_USAGE_BYTES{10_MiB}; would be equivalent to constexpr int64_t MAX_BLOCK_COINSDB_USAGE_BYTES{int64_t{10_MiB}};?

  38. in src/test/util_tests.cpp:1869 in faca929d70
    1871 |          BOOST_CHECK_EQUAL(max_mib, 4095U);
    1872 | -        BOOST_CHECK_EQUAL(4095_MiB, size_t{4095} << 20);
    1873 | -        BOOST_CHECK_EXCEPTION(4096_MiB, std::overflow_error, HasReason("MiB value too large for size_t byte conversion"));
    1874 | -    } else {
    1875 | -        BOOST_CHECK_EQUAL(4096_MiB, size_t{4096} << 20);
    1876 | +        BOOST_CHECK_EQUAL(4095_MiB, Int{4095} << 20);
    


    l0rinc commented at 1:08 PM on April 23, 2026:

    faca929 util: Return uint64_t from _MiB and _GiB operators:

    Kinda' off topic, but I just noticed that we have a few leftover << 20 and >> 20 changes in the code we could migrate to * 1_MiB and / 1_MiB now (I don't mind pushing these in a follow-up instead)

  39. in src/node/caches.cpp:1 in faca929d70 outdated


    l0rinc commented at 1:17 PM on April 23, 2026:

    nit: consider applying your code comment adjustment/removal from the other PR here: https://github.com/bitcoin/bitcoin/blob/cd7865b0ce17568945c7f72db0248489cf284787/src/randomenv.cpp#L117

    I don't mind doing this myself in a follow-up

  40. in src/test/util_tests.cpp:1841 in faca929d70 outdated
    1836 | +
    1837 | +template <class Int, auto v, auto eval = Int{operator""_GiB(v)}>
    1838 | +struct assign_GiB_result_to_int {
    1839 | +};
    1840 | +
    1841 |  BOOST_AUTO_TEST_CASE(mib_string_literal_test)
    


    l0rinc commented at 2:33 PM on April 23, 2026:

    faca929 util: Return uint64_t from _MiB and _GiB operators:

    If the test is only checking brace initialization/narrowing, maybe we can test brace-initializability™️ directly and avoid the assign_*_result_to_int scaffolding:

    diff --git a/src/test/util_tests.cpp b/src/test/util_tests.cpp
    --- a/src/test/util_tests.cpp	(revision a7346d6065fb9c96330eef0bf9abb997d9245194)
    +++ b/src/test/util_tests.cpp	(revision d5b1b02d1ac72b1114698bf62859a89355dfe617)
    @@ -36,6 +36,7 @@
     #include <optional>
     #include <string>
     #include <thread>
    +#include <type_traits>
     #include <univalue.h>
     #include <utility>
     #include <vector>
    @@ -1827,19 +1828,13 @@
         TestSaturatingLeftShift<int64_t>();
     }
     
    -template <template <class, auto...> class Op, class Int, auto v>
    -concept is_valid = requires { typename Op<Int, v>; };
    -
    -template <class Int, auto v, auto eval = Int{operator""_MiB(v)}>
    -struct assign_MiB_result_to_int {
    -};
    -
    -template <class Int, auto v, auto eval = Int{operator""_GiB(v)}>
    -struct assign_GiB_result_to_int {
    -};
    +template <class Int, auto BYTES>
    +concept BraceInitializesTo = requires { Int{BYTES}; };
     
     BOOST_AUTO_TEST_CASE(mib_string_literal_test)
     {
    +    static_assert(std::is_same_v<decltype(1_MiB), uint64_t>);
    +
         // Basic equivalences and simple arithmetic operations
         BOOST_CHECK_EQUAL(0_MiB, 0);
         BOOST_CHECK_EQUAL(1_MiB, 1 << 20);
    @@ -1861,24 +1856,12 @@
         BOOST_CHECK_EQUAL(128_MiB, 0x8000000U);
         BOOST_CHECK_EQUAL(550_MiB, 550ULL * 1024 * 1024);
     
    -    // Overflow handling and narrowing checks at compile time
    -    {
    -        using Int = uint32_t;
    -        constexpr auto max_mib{std::numeric_limits<Int>::max() >> 20};
    -        BOOST_CHECK_EQUAL(max_mib, 4095U);
    -        BOOST_CHECK_EQUAL(4095_MiB, Int{4095} << 20);
    -        static_assert(is_valid<assign_MiB_result_to_int, Int, max_mib>);
    -        static_assert(is_valid<assign_MiB_result_to_int, Int, 4095U>);
    -        static_assert(!is_valid<assign_MiB_result_to_int, Int, 4096U>);
    -        static_assert(!is_valid<assign_MiB_result_to_int, Int, max_mib + 1>);
    -    }
    -    {
    -        using Int = uint64_t;
    -        BOOST_CHECK_EQUAL(4096_MiB, Int{4096} << 20);
    -        constexpr auto max_mib{std::numeric_limits<Int>::max() >> 20};
    -        static_assert(is_valid<assign_MiB_result_to_int, Int, max_mib>);
    -        static_assert(!is_valid<assign_MiB_result_to_int, Int, max_mib + 1>);
    -    }
    +    // 4095 MiB fits in uint32_t bytes. 4096 MiB requires the uint64_t return type.
    +    static_assert( BraceInitializesTo<uint32_t, 4095_MiB>);
    +    static_assert(!BraceInitializesTo<uint32_t, 4096_MiB>);
    +    static_assert( BraceInitializesTo<uint64_t, 4096_MiB>);
    +    BOOST_CHECK_EQUAL(4095_MiB, uint32_t{4095} << 20);
    +    BOOST_CHECK_EQUAL(4096_MiB, uint64_t{4096} << 20);
     }
     
     BOOST_AUTO_TEST_CASE(ceil_div_test)
    @@ -1920,6 +1903,8 @@
     
     BOOST_AUTO_TEST_CASE(gib_string_literal_test)
     {
    +    static_assert(std::is_same_v<decltype(1_GiB), uint64_t>);
    +
         // Basic equivalences and simple arithmetic operations
         BOOST_CHECK_EQUAL(0_GiB, 0);
         BOOST_CHECK_EQUAL(1_GiB, 1 << 30);
    @@ -1935,22 +1920,13 @@
         BOOST_CHECK_EQUAL(3_GiB / 1_GiB, 3U);
         BOOST_CHECK_EQUAL(3_GiB, 3U << 30);
     
    -    // Overflow handling and narrowing checks at compile time
    -    {
    -        using Int = uint32_t;
    -        constexpr auto max_gib{std::numeric_limits<Int>::max() >> 30};
    -        BOOST_CHECK_EQUAL(max_gib, 3U);
    -        BOOST_CHECK_EQUAL(3_GiB, Int{3} << 30);
    -        static_assert(is_valid<assign_GiB_result_to_int, Int, max_gib>);
    -        static_assert(!is_valid<assign_GiB_result_to_int, Int, max_gib + 1>);
    -    }
    -    {
    -        using Int = uint64_t;
    -        BOOST_CHECK_EQUAL(4_GiB, Int{4} << 30);
    -        constexpr auto max_gib{std::numeric_limits<Int>::max() >> 30};
    -        static_assert(is_valid<assign_GiB_result_to_int, Int, max_gib>);
    -        static_assert(!is_valid<assign_GiB_result_to_int, Int, max_gib + 1>);
    -    }
    +    // 3 GiB fits in uint32_t bytes. 4 GiB requires the uint64_t return type.
    +    static_assert( BraceInitializesTo<uint32_t, 3_GiB>);
    +    static_assert(!BraceInitializesTo<uint32_t, 4_GiB>);
    +    static_assert( BraceInitializesTo<uint64_t, 4_GiB>);
    +    BOOST_CHECK_EQUAL(3_GiB, uint32_t{3} << 30);
    +    BOOST_CHECK_EQUAL(4_GiB, uint64_t{4} << 30);
    +
         // Specific codebase values
         BOOST_CHECK_EQUAL(4_GiB, 4096_MiB);
         BOOST_CHECK_EQUAL(8_GiB, 8192_MiB);
    

    maflcko commented at 6:42 AM on April 24, 2026:

    Nice, pushed. Looks like it works on the older compilers as well: https://godbolt.org/z/P1EoEoMPe, so it should be fine.


    l0rinc commented at 1:07 PM on April 24, 2026:

    Thanks for checking.


    hodlinator commented at 8:08 PM on April 24, 2026:

    That is indeed nicer, cheers!

  41. in src/node/caches.cpp:33 in faca929d70 outdated
      29 | @@ -30,7 +30,7 @@ static constexpr size_t MAX_32BIT_DBCACHE{1_GiB};
      30 |  //! Larger default dbcache on 64-bit systems with enough RAM.
      31 |  static constexpr size_t HIGH_DEFAULT_DBCACHE{1_GiB};
      32 |  //! Minimum detected RAM required for HIGH_DEFAULT_DBCACHE.
      33 | -static constexpr uint64_t HIGH_DEFAULT_DBCACHE_MIN_TOTAL_RAM{uint64_t{4} * 1_GiB};
      34 | +static constexpr uint64_t HIGH_DEFAULT_DBCACHE_MIN_TOTAL_RAM{4_GiB};
    


    l0rinc commented at 2:49 PM on April 23, 2026:

    faca929 util: Return uint64_t from _MiB and _GiB operators:

    If we're removing this we could update the corresponding test case as well: https://github.com/bitcoin/bitcoin/blob/f17cd18d02b4bd6ff0aed03f04968ea52271ee70/src/test/util_tests.cpp#L1914


    maflcko commented at 6:41 AM on April 24, 2026:

    Not going to touch it, maybe it can be removed or left as-is?

  42. l0rinc changes_requested
  43. l0rinc commented at 3:28 PM on April 23, 2026: contributor

    I went over the call sites and left a few nits/suggestions where this change lets us simplify follow-up code from #34435: dropping leftover SIZE_MAX guards, expressing large _MiB values as _GiB, removing now-redundant casts/workarounds, and cleaning up a few remaining << 20 / >> 20 byte conversions. I’m also happy to do those in follow-ups.

    My main request is to make the changed validation model explicit in the code/tests and PR description: before, the literal operators also validated against size_t, while after this PR they only reject values that cannot be represented as bytes in uint64_t. Maybe that was already obvious to others, but I initially read the PR as preserving the same validation boundary while only unifying the return type.

  44. maflcko force-pushed on Apr 24, 2026
  45. maflcko force-pushed on Apr 24, 2026
  46. DrahtBot added the label CI failed on Apr 24, 2026
  47. maflcko force-pushed on Apr 24, 2026
  48. DrahtBot removed the label CI failed on Apr 24, 2026
  49. l0rinc approved
  50. l0rinc commented at 1:16 PM on April 24, 2026: contributor

    ACK faefcba8681540d92d8f8522a29bf74feb885ec9

  51. DrahtBot requested review from sedited on Apr 24, 2026
  52. maflcko force-pushed on Apr 24, 2026
  53. util: Return uint64_t from _MiB and _GiB operators fa5801762e
  54. refactor: Run ShouldWarnOversizedDbCache calculation in u64
    This follows the approach of the MiB and GiB operators.
    
    This allows to remove some `if constexpr (SIZE_MAX == UINT64_MAX)` in
    the tests.
    fa43da21f1
  55. in src/util/byte_units.h:25 in fa7fd201d6
      23 |      }
      24 |      return *bytes;
      25 |  }
      26 |  } // namespace util::detail
      27 |  
      28 |  //! Overflow-safe conversion of MiB to bytes.
    


    l0rinc commented at 2:33 PM on April 24, 2026:

    Nit: is it still accurate to call these "overflow-safe"?

  56. maflcko force-pushed on Apr 24, 2026
  57. DrahtBot added the label CI failed on Apr 24, 2026
  58. l0rinc commented at 2:45 PM on April 24, 2026: contributor

    ACK fa43da21f1979bd8ef52d895d970f19dbb5c8c9e

  59. DrahtBot removed the label CI failed on Apr 24, 2026
  60. hebasto approved
  61. hebasto commented at 4:17 PM on April 24, 2026: member

    ACK fa43da21f1979bd8ef52d895d970f19dbb5c8c9e, I have reviewed the code and it looks OK.

  62. in src/test/system_ram_tests.cpp:24 in fa43da21f1
      25 | -    if constexpr (SIZE_MAX == UINT64_MAX) {
      26 | -        // Upper bound check only on 64-bit: 32-bit systems can reasonably have max memory,
      27 | -        // but extremely large values on 64-bit likely indicate detection errors
      28 | -        BOOST_CHECK_LT(*total, 10'000'000_MiB); // >10 TiB memory is unlikely
      29 | -    }
      30 | +    BOOST_CHECK_LT(*total, 10'000_GiB); // ~10 TiB memory is unlikely
    


    hodlinator commented at 8:03 PM on April 24, 2026:

    Re https://github.com/bitcoin/bitcoin/pull/35097/changes#r3135843318:

    Certainly they won't exists in practise any time soon, but I don't see why in theory some researcher in an 128-bit env (for whatever reason) should face compile errors or silent breakages for no good reason.

    remark: I don't know if 10 TiB RAM is less likely than 128-bit archs, but we already had this here and it would only break the tests rather than the node. So it makes sense to keep it for a purer refactor. Maybe it is less likely even.


    l0rinc commented at 8:23 PM on April 24, 2026:

    The 10 TiB was just meant as an absurd upper limit as a heuristic to check if total RAM detection is working correctly.

  63. hodlinator approved
  64. hodlinator commented at 8:18 PM on April 24, 2026: contributor

    ACK fa43da21f1979bd8ef52d895d970f19dbb5c8c9e

  65. achow101 commented at 9:39 PM on April 24, 2026: member

    ACK fa43da21f1979bd8ef52d895d970f19dbb5c8c9e

  66. achow101 merged this on Apr 24, 2026
  67. achow101 closed this on Apr 24, 2026

  68. sedited referenced this in commit 75f2bf1011 on Apr 25, 2026
  69. luke-jr referenced this in commit 3b0192135d on May 3, 2026
  70. luke-jr referenced this in commit c0a634d2b4 on May 3, 2026
  71. maflcko deleted the branch on May 6, 2026

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-12 15:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me