estimatesmartfee always returns "Insufficient data or no feerate found" #11500

Describe the issue

My bitcoind 0.15.0 node is fully sync-ed.

$ bitcoin-cli getinfo
{
  "deprecation-warning": "WARNING: getinfo is deprecated and will be fully removed in 0.16. Projects should transition to using getblockchaininfo, getnetworkinfo, and getwalletinfo before upgrading to 0.16",
  "version": 150000,
  "protocolversion": 70015,
  "walletversion": 60000,
  "balance": 0.00000000,
  "blocks": 489885,
  "timeoffset": 0,
  "connections": 15,
  "proxy": "",
  "difficulty": 1123863285132.967,
  "testnet": false,
  "keypoololdest": 1492149683,
  "keypoolsize": 1000,
  "paytxfee": 0.00000000,
  "relayfee": 0.00001000,
  "errors": ""
}

Now attempt estimatefee

$ bitcoin-cli estimatefee 2
0.00004999

Now attempt estimatesmartfee

$ bitcoin-cli estimatesmartfee 2
{
  "errors": [
    "Insufficient data or no feerate found"
  ],
  "blocks": 0
}                                                                                                                                                                                                                       

Can you reliably reproduce the issue?

Yes, getting this every time running the command.

Expected behaviour

estimatesmartfee to returns the estimation

Actual behaviour

Return error

What version of bitcoin-core are you using?

"version": 150000, "protocolversion": 70015, "walletversion": 60000,

Machine specs:

• OS: Ubuntu
• CPU: Amazon EC2
• RAM: 16GB
• Disk size: 1TB
• Disk Type (HD/SDD): HDD

Can you post your debug.log somewhere (pastebin, etc.)?

I enabled debug=1 so there is too much log. I have to truncate it, restart my bitcoind and leave it running for a minute then submit the log. Here it is: https://clbin.com/ckvfc

It looks like you haven't run your peer long enough to collect the required information:

Imported mempool transactions from disk: 496 successes, 0 failed, 0 expired

Keep you peer running and check your mempool content with getmempoolinfo.

Last time I recall I let it run for 30 minutes and invoke but the result is the same. How do I know when it's good to go? My last getmempoolinfo

{"size"=>434, "bytes"=>171259, "usage"=>617040, "maxmempool"=>300000000, "mempoolminfee"=>0.0}

Maybe check your connected peers. getpeerinfo. But you mempool is pretty small (looks like your not getting enough transactions to estimate fees)

Is there a way to improve that beside waiting? How long should I wait then?

Wait until you get estimates with bitcoin-cli estimatesmartfee 2. For more details, see https://github.com/bitcoin/bitcoin/pull/10199

okay thank you.

Did anyone find the solution ?

When I submit this command bitcoin-cli getmempoolinfo : I get this : { "size": 0, "bytes": 0, "usage": 0, "maxmempool": 300000000, "mempoolminfee": 0.00000000 }

@phuongnd08 Did you find the Solution ?

@RajaHassan Once the node run for long enough there will be estimation.

What does "long enough" mean?
How do you know it's reaching the "long enough" threshold? Can getmempoolinfo give you an idea?

The node runs for many days and I have had synced all the blocks . Still not returning fee. @nopara73

@RajaHassan First of all I'm using 0.16.0.
On the mainnet estimatesmartfee works right away after turn on without even syncing up the blocks.
On the testnet I was running a few hours while blocks are fully synced, it still didn't work. It doesn't work today, either. That being said previously it was working before with a few seconds/minutes of waiting. Update: Testnet estimatefee started working today within 5 minutes.

I get this result on regtest. My qt status bar is finished initializing. I have many transactions, some are confirmed. Fees have been paid on confirmed transactions.

My getpeerinfo returns empty set [].

I made sure to get enough blocks to have six confirmations on my original transaction.

Why exactly is estimatesmartfee failing to provide a feerate?

@toddmo Usually the issue tracker is used to track technical issues related to the Bitcoin Core code base. Keep in mind that general bitcoin questions and/or support requests are best directed to the Bitcoin StackExchange.

@MarcoFalke This is a bug.

It always returns this in regtest:

{
  "errors": [
    "Insufficient data or no feerate found"
  ],
  "blocks": 0
}

What it should do is return something, even if just valid test data, for completeness, and so developers won't have to write special code to handle an environment (regtest). Environments in and of themselves shouldn't introduce code complexity beyond configuration settings for the environment.

So maybe the cause is elsewhere, say peers, but whatever is missing in regtest should be mocked IMO.

+1 for mocking on RegTest. It resulted in some minor inconvenience for me during development multiple times.

You're free to simulate data yourself (by creating transactions). Perhaps it would be useful to have a script to do so.

But estimatesmartfee should not report data that isn't actually in the chain - that would defeat the ability to use regtest to test estimatesmartfee.

@sipa,

I agree that the data should be there for the function to see (whether mocked or from user input).

For me, creating transactions doesn't cause estimatesmartfee to provide a feerate. I made sure they were confirmed as well.

Maybe the missing data is something else besides just transactions?

@toddmo See ./test/functional/feature_fee_estimation.py for an example on how to generate transactions for fee estimation.

What if we have pruning turned on in our bitcon.conf file?

prune=1

Does this hinder our ability to use estimatesmartfee?

@knoxcard No, not at all.

The fee estimation is completely based on how long your node has been running, how many transactions have entered the mempool, and how many of those transactions have been confirmed. It is not (and cannot be) based on the fee information of transactions already included in blocks. So it does not matter whether you are fully synced or not. If you just started up or if there are few transactions being confirmed, then the fee estimation will not have enough data to work. (c)

For anyone interested, I had a go at the creation of transactions from ./test/functional/feature_fee_estimation.py to bootstrap the estimatesmartfee in a regtest environment using javascript. It uses the bitcoin-core package (https://www.npmjs.com/package/bitcoin-core) In the code below bitcoind is the initialized bitcoin-core client.

for (let i = 0; i < 10; i++) {
  for (let j = 0; j <= Math.floor(Math.random() * (150 - 50 + 1) + 50); j++) {
    const newAddress = await bitcoind.getNewAddress()
    const unfundedTx = await bitcoind.createRawTransaction([], { [newAddress]: '0.005' })
    const randFee = 0.00001 * Math.pow(1.1892, Math.floor(Math.random() * 29))
    const fundedTx = await bitcoind.fundRawTransaction(unfundedTx, { feeRate: randFee.toFixed(8).toString() })
    const signedTx = await bitcoind.signRawTransactionWithWallet(fundedTx.hex)
    const sentTx = await bitcoind.sendRawTransaction(signedTx.hex)
  }
  bitcoind.generateToAddress(1, 'mp76nrashrCCYLy3a8cAc5HufEas11yHbh')
}

I think the number of transactions is overkill, because while this was running, the estimatesmartfee started to return results, if queried. But the overkill doesn't hurt, so it didn't bother me.