Test persisted health requires mempool restore #35970

The block policy estimator's FeeReason enum mixed two unrelated
concerns: the threshold that produced an estimateSmartFee result
(NONE, HALF_ESTIMATE, ...) and the reason the wallet selected a fee
rate (FALLBACK, MEMPOOL_MIN, REQUIRED).

Split them so each layer owns the reasons it reports:

- Add a wallet-facing FeeReason enum with the reasons the wallet can
  select a fee rate: FEE_RATE_ESTIMATOR, MEMPOOL_MIN, USER_SPECIFIED,
  FALLBACK, and REQUIRED.

- Rename the estimator enum to BlockPolicyEstimateReason and narrow it
  to estimator reasons: NONE, HALF_ESTIMATE, FULL_ESTIMATE,
  DOUBLE_ESTIMATE, and CONSERVATIVE.

- Return wallet fee selection metadata through MinimumFeeRateResult
  instead of exposing FeeCalculation to wallet callers. The returned
  target is now optional and is only set for fee rate estimator results.

Flatten GetMinimumFeeRate() with early returns while preserving the fee
selection order: user feerate still only applies the required-fee check,
while smart-fee results keep fallback, mempool-min, and required fallbacks.
The returned target is cleared for fallback, mempool-min, and required
results.

Replace the CreateTransactionInternal log with a simpler message that
does not depend on estimateSmartFee internals. Detailed estimator
logging will be added in a follow-up commit.

Now that the wallet reports its own FeeReason, StringForBlockPolicyEstimateReason
is only used internally by the block policy estimator. Move it from
common/messages into the block policy fee rate estimator.

Also add the detailed FeeCalculation debug log to estimateSmartFee, where
the FeeCalculation data originates, and always populate feeCalc locally so
the log is available even when the caller does not pass a valid
FeeCalculation pointer.

Rename policyestimator_tests.cpp to blockpolicyestimator_tests.cpp.

Also rename the policy_estimator fuzz target to block_policy_estimator so the
test names match CBlockPolicyEstimator.

This makes the block policy fee rate estimator test files accurate and concise,
which makes adding another fee rate estimator test files straightforward.

The purpose of the test was to exercise CBlockPolicyEstimator behavior, but it
previously used a real CTxMemPool plus validation signals to track the
txs. Since TryAddToMempool does not emit TransactionAddedToMempool
callbacks, the test also had to fire those callbacks manually and sync the
validation interface queue around estimate checks.

Call processTransaction() and processBlock() directly instead. This removes
the mempool and validation-signal plumbing from the test, makes event
ordering explicit and synchronous, and avoids coupling the test to the
validation interface notifications.

This is useful because subsequent commits moved CBlockPolicyEstimator
from being validation interface client to FeeRateEstimatorManager.

Introduce that common interface:
- FeeRateEstimatorType identifies the source estimator in a result.
- FeeRateEstimation carries the feerate and returned target of a
  successful estimate; FeeRateEstimationError carries the error
  message alongside a zero-value estimation.
- EstimateFeeRate wraps estimateSmartFee and returns
  util::Expected<FeeRateEstimation, FeeRateEstimationError>.
- MaximumTarget delegates to HighestTargetTracked(LONG_HALFLIFE) so
  callers do not need to know about block policy horizons.

Update call sites in rpc/fees.cpp and node/interfaces.cpp.

A later commit introduces FeeRateEstimatorManager, which selects
between multiple fee rate estimators. To compare estimates and
report which estimator produced them, the manager needs each fee
rate estimator to expose a uniform output, whereas
estimateSmartFee's CFeeRate/FeeCalculation output is specific to
the block policy fee rate estimator.

Co-authored-by: willcl-ark <will@256k1.dev>

Introduce FeeRateEstimatorManager to wrap CBlockPolicyEstimator and
act as the single point of contact for fee rate estimation in the node.

It inherits CValidationInterface so it can register directly with the
validation signals and receive mempool/block events.

Wire it into NodeContext (fee_estimator_man), init, shutdown, the RPC
server utility helpers (EnsureAnyFeeEstimatorMan), and the wallet-facing
interfaces::Chain API.

The Chain method estimateSmartFee is renamed to getFeeRateEstimate and
now returns util::Expected<FeeRateEstimation, FeeRateEstimationError>
instead of CFeeRate, so callers get the full estimation context without
needing FeeCalculation. estimateMaxBlocks is renamed to
maximumFeeEstimationTargetBlocks (still returns the max target).

CBlockPolicyEstimator no longer inherits CValidationInterface; the
manager now receives the mempool/block validation events and forwards
them to the CBlockPolicyEstimator.

Co-authored-by: willcl-ark <will@256k1.dev>

Add a string fee_rate_estimator option (default "none") to
estimatesmartfee options. "block_policy" consults only the block
policy fee rate estimator, "none" uses the fee rate estimator
manager selected behaviour, and unknown values are treated as
"none". Unknown option keys are rejected.

Still only the block policy fee rate estimator, so the result is
unchanged; a subsequent commit will change the default behaviour.

Also adds GetFeeRateEstimate(FeeRateEstimatorType, target, conservative)
to FeeRateEstimatorManager so callers can query a single estimator by
type; NONE returns the manager-selected combined estimate.

Add MemPoolFeeRateEstimator, which calls Bitcoin Core's block
assembler with the mempool and chainstate to build a block template and
use its chunk fee rates for fee rate estimation.

Add CalculateMaxWeightPercentiles to return the 50th and 75th
percentile chunk feerates by cumulative block weight. If sparse,
EstimateFeeRate uses the higher of the minimum relay fee rate and the
current mempool minimum fee rate.

The 50th percentile is returned as the conservative estimate, and the
75th percentile as the economical estimate.

Wire MemPoolFeeRateEstimator into FeeRateEstimatorManager and add
FeeRateEstimatorType::MEMPOOL_POLICY for result attribution.

Add unit tests for the mempool fee rate estimator and fee estimator
string conversions, plus fuzz coverage for the string conversions.

Co-authored-by: willcl-ark <will@256k1.dev>

Cache previous mempool fee rate estimates.

Cached estimates are tagged with the chain tip they were computed on
(the template's hashPrevBlock). They are only served while they are
not stale and the chain tip has not changed. This avoids generating
block templates too often.

The estimator lock is not held while building a block template, so
concurrent callers may duplicate estimation work; the tip tag keeps
stale results out of the cache.

Co-authored-by: willcl-ark <will@256k1.dev>

Integrate MemPoolFeeRateEstimator into FeeRateEstimatorManager.
When both estimators succeed, select the lower of the block policy
and mempool estimates.

When either estimator fails, return its error instead of falling back
to the block policy estimate: if the mempool estimator cannot produce
an estimate (mempool still loading or unhealthy) the combined estimate
fails. Callers that want a block-policy-only estimate can request it
explicitly via fee_rate_estimator.

estimatesmartfee now emits the estimator field only for successful
manager-selected estimates.

Add a test that ensures estimatesmartfee returns the mempool fee rate
estimate when it is lower than the block policy estimate, and returns
the mempool error when the mempool estimator fails while block policy
succeeds.

Two wallet functional tests also need adjusting. When the mempool is
too sparse to fill its percentile buckets, MemPoolFeeRateEstimator
returns a relayable floor of max(min relay fee, mempool min fee), so in
regtest getFeeRateEstimate now returns the min relay fee where the
wallet previously had no estimate and fell back to a higher rate:

- wallet_taproot.py: the cleanup sendall used automatic fee estimation.
  GetMinimumFeeRate previously fell back to the wallet fallback fee
  (fallbackfee, 20 sat/vB in the test framework); it now uses the min
  relay fee floor. At that lower feerate the wallet's underestimate of
  the taproot script-path witness size drops the effective feerate
  below min relay, so the transaction is rejected. Pin fee_rate=20 to
  match the framework fallbackfee.

- wallet_bumpfee.py: GetDiscardRate() previously fell back to the
  wallet discard rate (-discardfee); it now takes the minimum of that
  and the estimate, so the min relay fee floor collapses the discard
  rate down to the dust relay feerate. The lower discard rate reduces
  the cost of change, so the ~614 sat leftover change in
  test_dust_to_fee is now retained instead of being dropped to fee.
  Rework the test to leave a sub-dust (20/270 sat) change that is
  dropped regardless of the discard rate.

Co-authored-by: willcl-ark <will@256k1.dev>

Return the removed mempool transaction info from
CTxMemPool::removeForBlock instead of dispatching the
MempoolTransactionsRemovedForBlock notification from the mempool.

Emit it from ConnectTip after mempool removal and before BlockConnected,
passing the connected block, the removed mempool transactions, and the
block height to the callback.

Because the signal now originates from ConnectTip, where the IBD state is
known, gate it on !IsInitialBlockDownload(): the notification is no longer
fired for blocks connected during initial block download or reindex, while
the mempool removal in removeForBlock still runs unconditionally. This keeps
fee rate estimators from recording blocks connected before the node is
synced.

Gate the mempool fee rate estimator on a coverage check: recent
connected blocks must be well represented by transactions removed from
our mempool.

Track per-block weight for the last MEMPOOL_HEALTH_WINDOW_BLOCKS blocks.
AddMinedBlockStats drops stats at or above a connected block's height
before appending it, and resets the window on a forward height gap so
tracked heights stay consecutive.

Only apply the coverage ratio once the window holds at least one block
of transactions; below that activity is too low for the ratio to be
meaningful, so treat the mempool as healthy.

Replace a boolean health check with a MempoolHealth enum so
EstimateFeeRate() can report whether estimation is unavailable because
too few recent blocks have been tracked (INSUFFICIENT_DATA) or because
recent blocks poorly represent the mempool (LOW_COVERAGE).

Add a verbosity option to the existing estimatesmartfee options object.
The default verbosity remains 1.

When verbosity is at least 2 and fee_rate_estimator is "none", include
mempool_health_statistics in the response. The array reports the mined
blocks tracked by the mempool fee rate estimator in most-recent-first
order, with each entry containing:

- block_height
- block_weight: total non-coinbase transaction weight in the block
- mempool_txs_weight: weight of transactions removed from our mempool
  for that block

Expose these stats through the fee rate estimator manager so RPC users
can inspect the block coverage data used by the mempool health check.

Move block policy fee estimates from fee_estimates.dat to
fees/block_policy_estimates.dat.

On startup, migrate the legacy file to the new path when only the legacy
file exists. If both files exist, keep the new file and remove the
legacy file.

Rename the block policy estimator args source files to the generic
estimator_args.{cpp,h} names and rename FeeestPath to
BlockPolicyFeeEstPath while the path helper is moved into the shared fee
estimator argument code.

Persist MemPoolFeeRateEstimator's recent mined-block statistics
to fees/mempool_policy_estimator.dat and reload them at startup.

Without this, the mempool estimator starts cold after each restart
and treats the mempool as unhealthy until MEMPOOL_HEALTH_WINDOW_BLOCKS
blocks have been observed, causing estimatesmartfee to fall back to the
block policy estimator.

Files with more stats than MEMPOOL_HEALTH_WINDOW_BLOCKS,
non-consecutive block heights, or a final block that does not match the
active chain tip are rejected on read, preserving the invariant that
loaded stats describe the current chain.

Add MempoolPolicyEstimatorPath(), pass the path through
FeeRateEstimatorManager, and flush both block-policy
and mempool-policy estimator files on interval and shutdown.