fees: fall back to `block_policy` and minor api change #36365

pull ismaelsadeeq wants to merge 3 commits into bitcoin:master from ismaelsadeeq:09-2026-fall-back-to-block-policy-slimmed changing 13 files +185 −50
  1. ismaelsadeeq commented at 10:15 AM on September 28, 2026: member

    This is split from #36182 to allow review of the first three commits that we want to backport to 32.x.

    This is a simple PR that does three things:

    1. A minor, non-breaking estimatesmartfee API update: it renames the default estimator option from "none" to "auto". See the rationale here: #36182 (comment)

    2. Falls back to the block_policy fee rate estimator when mempool_policy is not available. This preserves the previous behavior and lets users who update their node keep using block_policy estimates until the mempool policy has finished gathering stats and is ready to serve estimates.

    3. After a restart in which mempool loading fails, we clear the data gathered by the mempool fee rate estimator, to prevent it from returning the fee rate floor just because the node lost its mempool transactions.

  2. DrahtBot added the label TX fees and policy on Sep 28, 2026
  3. DrahtBot commented at 10:15 AM on September 28, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/36365.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK willcl-ark, davidgumberg, w0xlt, achow101
    Concept ACK kevkevinpal
    Approach ACK stickies-v

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #33741 (rpc: Optionally print feerates in sat/vb by polespinasa)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. maflcko added the label Needs Backport (32.x) on Sep 28, 2026
  5. maflcko added this to the milestone 32.0 on Sep 28, 2026
  6. willcl-ark approved
  7. willcl-ark commented at 12:41 PM on September 28, 2026: member

    ACK 305f7df895d3b8eb8ede8ca2cb37ae1ccd1631b6

    I agree this split out is all that's necessary for 32.x (if we are not already too late).

    Only other thing we might like is a release note for the RPC change, but perhaps not a big deal, just update the wiki?

  8. ismaelsadeeq commented at 12:45 PM on September 28, 2026: member

    Only other thing we might like is a release note for the RPC change, but perhaps not a big deal, just update the wiki?

    I plan to update the wiki directly if it gets in.

  9. in test/functional/feature_fee_estimation.py:647 in 305f7df895 outdated
     643 | +            after = self.nodes[0].estimatesmartfee(1, "economical", {"fee_rate_estimator": "auto", "verbosity": 2})
     644 | +            assert_equal(after["mempool_health_statistics"], [])
     645 | +            assert_equal(after["estimator"], "block_policy")
     646 | +            assert_equal(self.nodes[0].estimatesmartfee(1, "economical", {"fee_rate_estimator": "mempool_policy"})["errors"],
     647 | +                         ["mempool_policy: Not enough recent block data for fee rate estimation"])
     648 | +            self.stop_node(0)
    


    kevkevinpal commented at 12:56 PM on September 28, 2026:

    test_mined_block_stats_cleared_on_failed_mempool_load checks the cleared window only in memory. The next iteration copies estimator_bak back over estimator_dat, so a shutdown flush that left the old window on disk would still pass.

    After stop_node(), restore only the mempool file and restart. A successful load will not call MempoolLoadFailed() again, so an empty mempool_health_statistics means the clear was written to disk

                self.stop_node(0)
                # Shutdown flush should have persisted the cleared window.
                # Restore a loadable mempool, but not the estimator file.
                shutil.copyfile(mempool_bak, mempool_dat)
                self.start_node(0)
                self.wait_until(lambda: self.nodes[0].getmempoolinfo()["loaded"])
                persisted = self.nodes[0].estimatesmartfee(1, "economical", {"fee_rate_estimator": "auto", "verbosity": 2})
                assert_equal(persisted["mempool_health_statistics"], [])
                self.stop_node(0)
    

    ismaelsadeeq commented at 1:25 PM on September 28, 2026:

    so a shutdown flush that left the old window on disk would still pass.

    I don't think there's a shutdown flush that can do that. FlushMinedBlockStats() serializes m_prev_mined_blocks, the same vector MempoolLoadFailed() clears. Once the window is empty in memory, any subsequent flush necessarily writes an empty window. So asserting the in-memory state after the failed load is sufficient; a disk round-trip is redundant.

  10. kevkevinpal commented at 12:58 PM on September 28, 2026: contributor

    Concept ACK 305f7df

    It makes sense to change FeeRateEstimatorType::NONE to FeeRateEstimatorType::AUTO as the wording is more accurate.

    Falling back to block_policy while mempool_policy is not available doesn't disrupt current node behavior

  11. in src/rpc/fees.cpp:63 in 305f7df895
      63 | +                     "block policy fee rate estimate is returned instead. An error is returned only if the block\n"
      64 | +                     "policy fee rate estimate is unavailable.\n"
      65 |                       "\"block_policy\" uses only the block policy fee rate estimator.\n"
      66 |                       "\"mempool_policy\" uses only the mempool fee rate estimator.\n"
      67 | -                     "Unknown values are treated as \"none\"."},
      68 | +                     "Unknown values are treated as \"auto\"."},
    


    stickies-v commented at 5:15 PM on September 28, 2026:

    Unknown values are still treated as to auto, so the previous "none" spelling works.

    I'm not sure that's a good thing? Explicitly failing when unrecognized user input is provided is generally the much saner default, and since this feature has never been shipped yet I don't think there's any backwards compatibility to maintain? Would really prefer removing the unknown value fallback here while we still can.


    w0xlt commented at 4:42 AM on September 29, 2026:

    +1, I'd also prefer rejecting unknown values here. The option hasn't been in a release yet, so there's nothing to stay compatible with, but once 32.0 ships, turning the fallback into an error would be a breaking change.

    It's also inconsistent with the rest of the RPC: a bad estimate_mode already throws RPC_INVALID_PARAMETER a few lines above, while a typo like {"fee_rate_estimator": "mempool"} silently returns the combined estimate (which may come from block_policy), with nothing telling the caller their input was ignored.

    Suggested patch: FeeRateEstimatorTypeFromString() returns std::optional, the RPC throws -8 with the same message format as estimate_mode, the "Unknown values are treated as auto" help sentence is dropped, and the tests are updated. "none" is rejected too, since it was never released.

    <details> <summary>diff</summary>

    diff --git a/src/rpc/fees.cpp b/src/rpc/fees.cpp
    index 566f9712ea..af177a2a2b 100644
    --- a/src/rpc/fees.cpp
    +++ b/src/rpc/fees.cpp
    @@ -59,8 +59,7 @@ static RPCMethod estimatesmartfee()
                          "block policy fee rate estimate is returned instead. An error is returned only if the block\n"
                          "policy fee rate estimate is unavailable.\n"
                          "\"block_policy\" uses only the block policy fee rate estimator.\n"
    -                     "\"mempool_policy\" uses only the mempool fee rate estimator.\n"
    -                     "Unknown values are treated as \"auto\"."},
    +                     "\"mempool_policy\" uses only the mempool fee rate estimator."},
                         {"verbosity", RPCArg::Type::NUM, RPCArg::Default{1},
                          "1 returns feerate or errors. 2 also returns \"mempool_health_statistics\"."},
                     },
    @@ -112,8 +111,12 @@ static RPCMethod estimatesmartfee()
                                     {"fee_rate_estimator", UniValueType(UniValue::VSTR)},
                                     {"verbosity", UniValueType(UniValue::VNUM)},
                                 }, /*fAllowNull=*/true, /*fStrict=*/true);
    -            const auto fee_rate_estimator{FeeRateEstimatorTypeFromString(
    +            const auto parsed_fee_rate_estimator{FeeRateEstimatorTypeFromString(
                     options["fee_rate_estimator"].isNull() ? "auto" : options["fee_rate_estimator"].get_str())};
    +            if (!parsed_fee_rate_estimator) {
    +                throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid fee_rate_estimator parameter, must be one of: \"auto\", \"block_policy\", \"mempool_policy\"");
    +            }
    +            const FeeRateEstimatorType fee_rate_estimator{*parsed_fee_rate_estimator};
                 bool conservative{fee_mode == FeeEstimateMode::CONSERVATIVE};
                 int verbosity{ParseVerbosity(options["verbosity"], /*default_verbosity=*/1, /*allow_bool=*/false)};
                 UniValue result(UniValue::VOBJ);
    diff --git a/src/test/fees_util_tests.cpp b/src/test/fees_util_tests.cpp
    index 70ae23fc48..4c9d37f224 100644
    --- a/src/test/fees_util_tests.cpp
    +++ b/src/test/fees_util_tests.cpp
    @@ -20,9 +20,11 @@ BOOST_AUTO_TEST_CASE(fee_rate_estimator_type_from_string)
         BOOST_CHECK(FeeRateEstimatorTypeFromString("auto") == FeeRateEstimatorType::AUTO);
         BOOST_CHECK(FeeRateEstimatorTypeFromString("block_policy") == FeeRateEstimatorType::BLOCK_POLICY);
         BOOST_CHECK(FeeRateEstimatorTypeFromString("mempool_policy") == FeeRateEstimatorType::MEMPOOL_POLICY);
    -    BOOST_CHECK(FeeRateEstimatorTypeFromString("unknown") == FeeRateEstimatorType::AUTO);
    -    // Unknown values, including the previous "none" spelling, are coerced to AUTO.
    -    BOOST_CHECK(FeeRateEstimatorTypeFromString("none") == FeeRateEstimatorType::AUTO);
    +    BOOST_CHECK(FeeRateEstimatorTypeFromString("Block_Policy") == FeeRateEstimatorType::BLOCK_POLICY);
    +    BOOST_CHECK(!FeeRateEstimatorTypeFromString("unknown"));
    +    BOOST_CHECK(!FeeRateEstimatorTypeFromString(""));
    +    // The previous "none" spelling was never released, so it is rejected too.
    +    BOOST_CHECK(!FeeRateEstimatorTypeFromString("none"));
     }
     
     BOOST_AUTO_TEST_SUITE_END()
    diff --git a/src/util/fees.cpp b/src/util/fees.cpp
    index 291c92f318..a77689d01b 100644
    --- a/src/util/fees.cpp
    +++ b/src/util/fees.cpp
    @@ -7,6 +7,7 @@
     #include <util/strencodings.h>
     
     #include <cassert>
    +#include <optional>
     #include <string_view>
     
     std::string_view FeeRateEstimatorTypeToString(FeeRateEstimatorType feerate_estimator_type)
    @@ -23,10 +24,11 @@ std::string_view FeeRateEstimatorTypeToString(FeeRateEstimatorType feerate_estim
         assert(false);
     }
     
    -FeeRateEstimatorType FeeRateEstimatorTypeFromString(std::string_view feerate_estimator_type)
    +std::optional<FeeRateEstimatorType> FeeRateEstimatorTypeFromString(std::string_view feerate_estimator_type)
     {
         const auto normalized{ToLower(feerate_estimator_type)};
    +    if (normalized == "auto") return FeeRateEstimatorType::AUTO;
         if (normalized == "block_policy") return FeeRateEstimatorType::BLOCK_POLICY;
         if (normalized == "mempool_policy") return FeeRateEstimatorType::MEMPOOL_POLICY;
    -    return FeeRateEstimatorType::AUTO;
    +    return std::nullopt;
     }
    diff --git a/src/util/fees.h b/src/util/fees.h
    index 43ffa82dcd..ce566b68ec 100644
    --- a/src/util/fees.h
    +++ b/src/util/fees.h
    @@ -9,6 +9,7 @@
     #include <util/expected.h>
     #include <util/feefrac.h>
     
    +#include <optional>
     #include <string>
     #include <string_view>
     #include <utility>
    @@ -90,6 +91,6 @@ inline const FeeRateEstimation& FeeRateEstimationRef(const util::Expected<FeeRat
     }
     
     std::string_view FeeRateEstimatorTypeToString(FeeRateEstimatorType feerate_estimator_type);
    -FeeRateEstimatorType FeeRateEstimatorTypeFromString(std::string_view feerate_estimator_type);
    +std::optional<FeeRateEstimatorType> FeeRateEstimatorTypeFromString(std::string_view feerate_estimator_type);
     
     #endif // BITCOIN_UTIL_FEES_H
    diff --git a/test/functional/rpc_estimatefee.py b/test/functional/rpc_estimatefee.py
    index abaaf86ad0..de606392a3 100755
    --- a/test/functional/rpc_estimatefee.py
    +++ b/test/functional/rpc_estimatefee.py
    @@ -36,6 +36,9 @@ class EstimateFeeTest(BitcoinTestFramework):
                 assert_raises_rpc_error(-3, "JSON value of type string is not of expected type number", self.nodes[0].estimaterawfee, 1, 'foo')
     
             assert_raises_rpc_error(-8, 'Invalid estimate_mode parameter, must be one of: "unset", "economical", "conservative"', self.nodes[0].estimatesmartfee, 1, 'foo')
    +        for fee_rate_estimator in ["foo", "none", ""]:
    +            assert_raises_rpc_error(-8, 'Invalid fee_rate_estimator parameter, must be one of: "auto", "block_policy", "mempool_policy"',
    +                                    self.nodes[0].estimatesmartfee, 1, 'ECONOMICAL', {'fee_rate_estimator': fee_rate_estimator})
             assert_raises_rpc_error(-8, "Unknown named parameter fee_rate_estimator", self.nodes[0].estimatesmartfee, 1, fee_rate_estimator=True)
             assert_raises_rpc_error(-3, "Unexpected key block_policy_only", self.nodes[0].estimatesmartfee, 1, 'ECONOMICAL', {'block_policy_only': True})
             # extra params
    @@ -54,9 +57,7 @@ class EstimateFeeTest(BitcoinTestFramework):
             self.nodes[0].estimatesmartfee(1, 'conservative')
             self.nodes[0].estimatesmartfee(1, 'ECONOMICAL', {"fee_rate_estimator": "block_policy"})
             self.nodes[0].estimatesmartfee(1, 'ECONOMICAL', {"fee_rate_estimator": "mempool_policy"})
    -        self.nodes[0].estimatesmartfee(1, 'ECONOMICAL', {"fee_rate_estimator": "foo"})
             self.nodes[0].estimatesmartfee(1, 'ECONOMICAL', {'verbosity': 1, 'fee_rate_estimator': "auto"})
    -        self.nodes[0].estimatesmartfee(1, 'ECONOMICAL', {'verbosity': 1, 'fee_rate_estimator': "none"})
     
             self.nodes[0].estimaterawfee(1)
             self.nodes[0].estimaterawfee(1, None)
    

    </details>


    ismaelsadeeq commented at 9:43 AM on September 29, 2026:

    Thanks, I have taken your diff and added you as coauthor.

  12. in src/rpc/fees.cpp:1 in f167348db5 outdated


    stickies-v commented at 6:43 PM on September 28, 2026:

    nit: commit message of f167348db504bb3d3b6d3d9252a9711981a4ac54 describes fallback behaviour that is not introduced until the next commit. This commit is a simple rename, I don't think it needs to talk about fee estimator behaviour at all?

    "none" read as though fee estimation was disabled, when it actually selects the default combined behavior: the lower of the block policy and mempool fee rate estimates, falling back to the block policy estimate when the mempool estimator can't produce one.


    ismaelsadeeq commented at 9:43 AM on September 29, 2026:

    Fixed, thanks.

  13. stickies-v commented at 7:18 PM on September 28, 2026: contributor

    Approach ACK

  14. davidgumberg commented at 9:17 PM on September 28, 2026: contributor

    crACK https://github.com/bitcoin/bitcoin/commit/305f7df895d3b8eb8ede8ca2cb37ae1ccd1631b6

    Thanks for splitting this change from #36182, agree with @stickies-v suggestion of errors for unknown estimator types, it is not blocking for me, but happy to reACK with that change here or in a separate PR.

  15. DrahtBot requested review from kevkevinpal on Sep 28, 2026
  16. DrahtBot requested review from stickies-v on Sep 28, 2026
  17. in test/functional/feature_fee_estimation.py:345 in 305f7df895
     340 | +        original_path = self.nodes[0].chain_path / "fees" / "mempool_policy_estimator.dat"
     341 | +        temp_path = self.nodes[0].chain_path / "fees" / "mempool_policy_estimator.dat.bak"
     342 | +        original_path.replace(temp_path)
     343 | +        self.start_node(0)
     344 | +        self.wait_until(lambda: self.nodes[0].getmempoolinfo()["loaded"])
     345 | +        assert "errors" in self.nodes[0].estimatesmartfee(1, "economical", {"fee_rate_estimator": "mempool_policy"})
    


    w0xlt commented at 4:51 AM on September 29, 2026:

    nit: the test here also passes if the mempool estimator fails for a different reason (e.g. the mempool not being loaded yet), so it doesn't confirm we're on the insufficient-data path the test is named after. Checking the exact error, like L646 already does, would make that explicit:

    diff --git a/test/functional/feature_fee_estimation.py b/test/functional/feature_fee_estimation.py
    index 239a444a3d..1b7deed17f 100755
    --- a/test/functional/feature_fee_estimation.py
    +++ b/test/functional/feature_fee_estimation.py
    @@ -342,7 +342,8 @@ class EstimateFeeTest(BitcoinTestFramework):
             original_path.replace(temp_path)
             self.start_node(0)
             self.wait_until(lambda: self.nodes[0].getmempoolinfo()["loaded"])
    -        assert "errors" in self.nodes[0].estimatesmartfee(1, "economical", {"fee_rate_estimator": "mempool_policy"})
    +        assert_equal(self.nodes[0].estimatesmartfee(1, "economical", {"fee_rate_estimator": "mempool_policy"})["errors"],
    +                     ["mempool_policy: Not enough recent block data for fee rate estimation"])
             assert_equal(self.nodes[0].estimatesmartfee(1, "economical", {"fee_rate_estimator": "auto"})["estimator"], "block_policy")
     
             # Restore the mempool estimator data.
    

    ismaelsadeeq commented at 9:43 AM on September 29, 2026:

    Fixed.

  18. ismaelsadeeq force-pushed on Sep 29, 2026
  19. DrahtBot added the label CI failed on Sep 29, 2026
  20. DrahtBot commented at 10:49 AM on September 29, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/36550912655/job/109348578077</sub> <sub>LLM reason (✨ experimental): CI failed because IWYU detected incorrect/changed #include directives (reported “Failure generated from IWYU” for src/rpc/fees.cpp) and exited with status 1.</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  21. fees: rename the default fee_rate_estimator value to "auto"
    "none" read as though fee estimation was disabled, so rename the public
    value (and the FeeRateEstimatorType::NONE enumerator) to "auto", which
    describes what it does.
    
    Also reject unknown fee_rate_estimator values instead of silently
    coercing them to the default, so a mistyped value surfaces an error. The
    previous "none" spelling was never released, so it is rejected too.
    
    Co-authored-by: w0xlt <94266259+w0xlt@users.noreply.github.com>
    4056908f0f
  22. fees: fall back to block_policy when the mempool estimator can't estimate
    When the mempool policy estimator cannot produce an estimate, GetFeeRateEstimate()
    now returns the block policy estimate rather than an error. The combined estimate
    starts from the block policy estimate and lowers it with the mempool estimate only
    when present, so the fallback path shares the log that reports the selected fee
    rate. An error is returned only when the block policy estimate itself is unavailable.
    c0b7ca3dbd
  23. fees: clear mined-block stats when the mempool load fails
    When the mempool fails to load at startup (persistence disabled, or a
    missing or corrupt mempool.dat), init notifies the fee_rate_estimator_man,
    which clears its tracked mined-block window. Otherwise the estimator
    would keep a window describing a mempool the node no longer has and,
    once the mempool refilled, serve an estimate built on it.
    
    Clearing makes the estimator report insufficient data until the window refills from the
    current tip, so the combined estimate falls back to the block policy
    estimate meanwhile.
    5b77288673
  24. ismaelsadeeq force-pushed on Sep 29, 2026
  25. DrahtBot removed the label CI failed on Sep 29, 2026
  26. willcl-ark approved
  27. willcl-ark commented at 12:52 PM on September 29, 2026: member

    reACK 5b77288673e2b74de1512180a8911f435311ca7a

    Reviewed the range-diff of 305f7df895d...5b77288673e

    Failing on unknown fee_rate_estimator types is nicer.

  28. DrahtBot requested review from davidgumberg on Sep 29, 2026
  29. davidgumberg commented at 5:26 PM on September 29, 2026: contributor
  30. w0xlt commented at 6:07 PM on September 29, 2026: contributor

    ACK 5b77288673e2b74de1512180a8911f435311ca7a

  31. achow101 commented at 8:26 PM on September 29, 2026: member

    ACK 5b77288673e2b74de1512180a8911f435311ca7a

  32. achow101 merged this on Sep 29, 2026
  33. achow101 closed this on Sep 29, 2026

  34. ismaelsadeeq deleted the branch on Sep 30, 2026
  35. fanquake removed the label Needs Backport (32.x) on Sep 30, 2026
  36. fanquake commented at 9:33 AM on September 30, 2026: member

    Backported to 32.x in #36300.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-10-11 08:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me