fees: rpc: `estimatesmartfee` now returns a fee rate estimate during low network activity

ismaelsadeeq commented at 3:12 pm on May 1, 2025: member

This PR fixes #32105 and related issues: #32178, #31116, and #31032.
The main motivation is outlined in this comment).

Bitcoin Core users have reported several issues where nodes are unable to provide fee rate estimates, despite having run long enough to do so for a given target. This typically occurs when there is low activity on the network.

As a result, the issue primarily affects nodes running on test networks signet and testnetn.

I think it is safe safe to return the maximum of mempoolminfee and minrelaytxfee but only after the node has run long enough to be able to provide an estimate for the target and still cannot.
(This avoids the issue of returning an unreliable estimate too early, as described by glozow).

The only potential downside is in scenarios where the user has very very poor peer connectivity and most incoming mempool transactions are not seen by the node.
Even then, with the current full RBF support, txs can safely be fee bumped if needed.

This PR includes two commits:

Bugfix for MaxUsableEstimate: It should not return 1 as a usable target, since this is effectively not usable.
Update to the estimatesmartfee RPC behavior: Now returns the maximum of minrelaytxfee and mempoolminfee when the estimator has recorded enough blocks for the requested confirmation target, but cannot produce an estimate.

Additional Context:

The fee estimator should be able of providing an estimate for a confirmation target n after observing at least n * 2 blocks from either historical or current stats. The method MaxUsableEstimate determines this.

In the estimateSmartFee RPC method, the value shown as blocks in the result corresponds to the returnedTarget, which is the minimum of the user providedconf_target and the output of MaxUsableEstimate.

Therefore, taking the maximum of mempoolminfee and minrelaytxfee when returnedTarget >= conf_target indicates that enough blocks have been recorded, but there is simply no activity on the network to provide estimate.
In such cases, the mempool might be empty, and the transaction should pay enough to be relayed to peers making it likely to confirm since their is no demand.

Note: The condition uses >= because, for example, a conf_target of 1 will result in a returnedTarget of 2 internally, so it is possible for returnedTarget to be > conf_target.

fees: bugfix: prevent returning 1 as usable estimate ffcc1c6567

DrahtBot commented at 3:12 pm on May 1, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32395.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#31664 (Fees: add Fee rate Forecaster Manager by ismaelsadeeq)
#30157 (Fee Estimation via Fee rate Forecasters by ismaelsadeeq)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

ismaelsadeeq commented at 3:08 pm on May 8, 2025: member

Friendly ping @glozow @willcl-ark @instagibbs

instagibbs commented at 3:17 pm on May 8, 2025: member

Is this something we need to support? In a sense the current mechanism is honestly saying it doesn’t have enough data. If you don’t actually know what level of fees are required to get in a block, you may miss the fact that, f.e., miners are not mining anything 2x higher than your floating minfee.

I think this circles back to what the end goal of this RPC is, A one stop shop for best guess of feerates needed?

I haven’t thought about this much in years, so I’d be curious to hear what you think.

ismaelsadeeq commented at 4:00 pm on May 8, 2025: member

Is this something we need to support? In a sense the current mechanism is honestly saying it doesn’t have enough data.

I think it is. Even based on recent mainnet network activity, it’s possible for us to see frequent empty blocks. In cases where the estimator have seen enough blocks to be able to have the data but there is just no activity, paying the floating minfee is sufficient to get transactions to miners. I think this is the sane thing to do.

In test networks, this has been the reality for a while users want to transact, but the fee estimator says there’s insufficient data and indeed there isn’t

you may miss the fact that, f.e., miners are not mining anything 2x higher than your floating minfee.

This is unlikely in cases where there’s genuinely no activity on the network, as the floating minfee is sufficient to propagate transactions to miners. The scenario where miners aren’t including transactions paying 2x the floating minfee typically occurs when there is enough activity on the network to price out low-feerate txs something the estimator, being only backward-looking, will eventually observe.

However, there is one scenario where this might be perceived as an issue: when the node has run for (n * 2 + n) blocks, where the first n * 2 blocks are mostly empty (i.e., no activity), and activity picks up only in the last n blocks. In such a case, data from just the last n blocks may not be sufficient to provide a reliable fee rate estimate for target n. As a result, we fall back to using the floating minfee as the estimate even though it may no longer reflect current network conditions. But that edge case may not be an issue, as it reflects the behavior of the block_policy_estimator, which is inherently backward-looking in order to predict future rates.

The disconnect between current conditions and recent historical data is a key limitation of the block_policy_estimator, and it’s one that appears frequently on mainnet. This is where a forward-looking fee rate forecaster would be particularly valuable.

Also, with full RBF now available, users can safely fee-bump.

I think this circles back to what the end goal of this RPC is, A one stop shop for best guess of feerates needed?

I think that’s how most users interpret it, hence the issue.

in src/rpc/fees.cpp:84 in caa9f99b92 outdated

78@@ -79,7 +79,10 @@ static RPCHelpMan estimatesmartfee()
79             UniValue errors(UniValue::VARR);
80             FeeCalculation feeCalc;
81             CFeeRate feeRate{fee_estimator.estimateSmartFee(conf_target, &feeCalc, conservative)};
82-            if (feeRate != CFeeRate(0)) {
83+            // Return the maximum of minrelaytxfee, mempoolminfee and estimateSmartFee fee rate when
84+            //  a) estimateSmartFee successfully recommended a fee rate
85+            //  b) fee rate estimator has recorded enough blocks to able to provide an estimate for conf_target.

DrahtBot commented at 9:38 am on May 21, 2025:

to able → to be able

ismaelsadeeq commented at 11:22 am on May 21, 2025:

Thanks @DrahtBot this is fixed.

fees: rpc: update `estimatesmartfee` to return an estimate during low network activity c259dbd14f

ismaelsadeeq force-pushed on May 21, 2025

fees: rpc: estimatesmartfee now returns a fee rate estimate during low network activity #32395

This PR includes two commits:

Additional Context:

Code Coverage & Benchmarks

Reviews

Conflicts

fees: rpc: `estimatesmartfee` now returns a fee rate estimate during low network activity #32395