cost too many fees?

JokerCatz commented at 4:56 am on April 21, 2017: none

Hi , I send a transaction , but it cost 5,209.456 sat/B …… & used txin all over 50 confirms

bitcoin-cli sendtoaddress 1846VsYbYJmqLteuMLdZAYxoVDsjWZyMRz 20

https://blockchain.info/tx/37f98737fe28f1f29840926edeb9cb3a85eb9496070048eb98f2523d612c3cfb

and my getinfo

{
  "version": 120100,
  "protocolversion": 70012,
  "walletversion": 60000,
  "balance": SKIP,
  "blocks": 462806,
  "timeoffset": 0,
  "connections": 8,
  "proxy": "",
  "difficulty": 520808749422.1398,
  "testnet": false,
  "keypoololdest": 1485144781,
  "keypoolsize": 301,
  "paytxfee": 0.00000000,
  "relayfee": 0.00001000,
  "errors": "Warning: Unknown block versions being mined! It's possible unknown rules are in effect"
}

& my config

upnp=0
server=1
rpcuser=SKIP
rpcpassword=SKIP
rpcthreads=8
rpckeepalive=false
rpctimeout=30
rpcallowip=127.0.0.1
rpcport=8332
keypool=300
mintxfee=0.00000001
checkblocks=16

did it just version too old & need to upgrade?

JokerCatz renamed this:
~~code too many fees?~~
cost too many fees?
on Apr 21, 2017

jonasschnelli commented at 10:56 am on April 21, 2017: contributor

Please use something like http://bitcoin.stackexchange.com for future user questions. This is the development issue tracker.

It looks like that your fee estimation spits out a relatively high value. Try to upgrade to 0.14.0. Maybe play with txconfirmtarget=<your desired target in blocks>.

jonasschnelli added the label Wallet on Apr 21, 2017

NicolasDorier commented at 1:03 pm on April 21, 2017: contributor

I think this question have its place here, it might have been a bug.

morcos commented at 3:05 pm on April 21, 2017: member

You should definitely upgrade to 0.14.0, but it is still possible although rare for such a result to happen.

Do you know what your transaction confirm target was set to? The default in 0.12 was changed to 2, but if you are running the GUI, then the GUI setting from a previous version may have persisted.

This kind of result is much more likely with a target of 1 than with a target of 2. Personally I would recommend increasing your transaction confirm target to at least 6 blocks.

JokerCatz commented at 6:00 pm on April 21, 2017: none

I use ubuntu server only , and install from tarball binary(not compiler by my self) , and I send 5 transactions same time at 3 node(different VM) , same config same amount same version , nodeA x 3 , nodeB x 1 , nodeC x 1 , and it only happen in nodeA

1.fail 73b8a20980eca8706a083e0bc511af8a8a93e408228411fafe5f05487a4d191d 2,100.84_ sat/B
2.okay f343524000eaedcbc61d68367facea6d268941ba8d464075048c2a2bf056f649   188.295 sat/B
3.fail 37f98737fe28f1f29840926edeb9cb3a85eb9496070048eb98f2523d612c3cfb 5,209.456 sat/B

hmm …… anyway I’ll upgrade my node , and add confirm target(maybe 4) , and maybe rebuild the nodeA

& thanx @morcos recommand : ) , first post is my all config file , so I think it’s default value …

morcos commented at 9:06 pm on April 21, 2017: member

Those are really very high values, so I am a bit curious how that could have happened. Had your node not been running very long?

But this is a good reminder that we should record more debugging information by default in the log when a transaction automatically uses fee estimation.

laanwj commented at 8:17 am on April 22, 2017: member

But this is a good reminder that we should record more debugging information by default in the log when a transaction automatically uses fee estimation.

Having fee estimation more auditable would be a great idea.

JokerCatz commented at 3:52 am on April 25, 2017: none

hmm @morcos it’s my production node , it run over 2 month … but I write my code to make & send rawtransaction to clear dust (very low fee) , but this time is some emergency things must immediately to send , so I send it from terminal

so it possible the fee calculate weight include another older transaction like manual to send rawtransaction ? even no relation with another non-used vouts ? & just I say , picked vouts all over 50 comfirms

morcos commented at 1:28 pm on April 25, 2017: member

@laanwj just closed #4082 (because it is now fixed) and I don’t know why I didn’t think of that before, but it seems possible that was the cause of this problem and not an error in fee estimation. Especially given that there was a normal transaction in between.

dabura667 commented at 4:03 am on April 30, 2017: none

bitcoin-cli sendtoaddress 1846VsYbYJmqLteuMLdZAYxoVDsjWZyMRz 20

I would like to point out:

the only output is exactly 20 BTC (the amount entered in the command)
3 of the lowest value inputs could have been omitted and the transaction would have still been in the 3000 sat/byte range… However, those low value inputs were not low enough to be considered unusable imo.

instagibbs commented at 1:58 pm on April 30, 2017: member

just to note: A user on IRC last night reported extremely high fees(2,000+ sat/byte) a couple times using sendmany on 0.14.1 even though settxfee was set.

MarcoFalke commented at 2:48 pm on April 30, 2017: member

Did they set subtract_fee_from_amount=True? That is still an open issue: https://github.com/bitcoin/bitcoin/blob/80c3a734298e824f9321c4efdd446086a3baad89/src/wallet/wallet.cpp#L2627

On Sun, Apr 30, 2017 at 3:58 PM, Gregory Sanders notifications@github.com wrote:

just to note: A user on IRC last night reported extremely high fees(2,000+ sat/byte) a couple times using sendmany on 0.14.1 even though settxfee was set.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bitcoin/issues/10247#issuecomment-298233960, or mute the thread https://github.com/notifications/unsubscribe-auth/AGGmv8l7hG66vIpifetrAsuNdM32kxtlks5r1JOcgaJpZM4ND3PA .

cncr04s commented at 6:55 pm on April 30, 2017: none

@MarcoFalke I did not set subtract_fee_from_amount. I’m just simply using sendmany to multiple addresses. It created a few transactions with 2000+ sat/byte, used settxfee specifically and txconfirmtarget=4. It usually creates transaction correctly with change and the correct fee, so in my case the same behavior as the OP.

It appears to me as it ate the rest of the change for the fee in all cases. My inputs vary widely in size, there are usually a few inputs less than 0.05 for it to choose from when building it, usually from existing change. 80% of the time it usually comes up with an input set that is within ~0.01 of the output sum plus fee.

Additionally, occasionally sendtoaddress would send with a fee a bit higher than I expected, decided to try sendmany to reduce fees and combine individual sends into a single transaction.

Using this in a production environment, not expecting bitcoin core to have issues like this.

MarcoFalke added the label Bug on Apr 30, 2017

MarcoFalke added the label Priority Medium on Apr 30, 2017

instagibbs commented at 7:55 pm on May 2, 2017: member

@MarcoFalke It may be the line two lines down: https://github.com/bitcoin/bitcoin/blob/80c3a734298e824f9321c4efdd446086a3baad89/src/wallet/wallet.cpp#L2630

Since he has no change in his transactions with such a high fee.

RHavar commented at 4:26 pm on May 3, 2017: contributor

@JokerCatz If you run bitcoin-cli listunspent | grep amount | wc -l what do you get? This used to be a really common issue when you had lots of unspent, but I haven’t noticed it running the latest version

JokerCatz commented at 2:53 am on May 4, 2017: none

I have 4 full node : [777 , 454 , 390 , 225]

morcos commented at 5:03 pm on May 4, 2017: member

As mentioned on IRC https://botbot.me/freenode/bitcoin-core-dev/2017-05-04/?msg=85156140&page=2 , I think its still possible for the old coin selection problem to rear it’s head. I’m not sure exactly how rare it is, but should be significantly less common than it used to be. #10333 could be one way to avoid this, i haven’t reviewed it yet, but I’m hoping to get to a more complete rewrite of the coin selection algorithm.

RHavar commented at 5:10 pm on May 4, 2017: contributor

@JokerCatz You have a lot of unspent on all your nodes, you really should upgrade them all ASAP. In total I’ve lost over a bitcoin in total due to the bug that was fixed (or at least, heavily mitigated), it’ll probably be affecting you often when you send money

instagibbs commented at 6:09 pm on May 4, 2017: member

Since this issue appears to be the “no change” situation, it’s likely not mitigated yet.

JokerCatz commented at 7:25 am on May 5, 2017: none

@RHavar my production normally use rawtransaction send and clear dust , so it not problem & thanx , I’ll upgrade my all node next week , just it look like bug & report it

cncr04s commented at 5:51 am on June 30, 2017: none

Is there any update on this issue?

instagibbs commented at 12:40 pm on June 30, 2017: member

@cncr04s #10333 should fix this particular case, and is targeted to 0.15. We’ll see if it can get enough review.

ysangkok commented at 9:59 am on September 7, 2017: none

@JokerCatz Did you test with a 0.15 release candidate? It would be interesting to see if it fixes your problem.

JokerCatz commented at 5:23 pm on September 7, 2017: none

Hi , I’m so sorry , my project is fixed , use rawtransaction to send transaction & calculate fees… so I think I can’t test this issue …

morcos commented at 5:55 pm on September 7, 2017: member

I think this can be closed

cncr04s commented at 6:32 pm on September 7, 2017: none

I am also having this problem, I may be able to test the actual release though. I havent kept up on when that would be. I run live environment, so may be a while till I can get an answer. I do however see many other incoming transactions having the same fee overpayment issue from time to time.

On Sep 7, 2017 1:00 PM, “Alex Morcos” notifications@github.com wrote:

I think this can be closed

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bitcoin/issues/10247#issuecomment-327875899, or mute the thread https://github.com/notifications/unsubscribe-auth/ABaLWkuM5WJbnQc3XTG05BDI67n0KbUDks5sgC5TgaJpZM4ND3PA .

instagibbs commented at 6:38 pm on September 7, 2017: member

@cncr04s 0.15/master should catch any of these types of vast overpayment errors.

morcos commented at 6:53 pm on September 7, 2017: member

@cncr04s If you are running 0.14, then it should be very relatively rare to have overpayment issues, but they can still exist when you have a transaction without change. That edge case should be fixed in 0.15. If you have this problem on 0.14 node on a transaction with change, please provide us with us much information as you can.

laanwj removed the label Priority Medium on Dec 6, 2017

Safari77 commented at 10:43 am on December 10, 2017: none

at least 0.15.1 is not doing overpayment, with txconfirmtarget=6 it uses 20 sat/byte, and tx still unconfirmed after two days. :expressionless:

0Fee Calculation: Fee:4520 Bytes:226 Needed:4520 Tgt:0 (requested 6) Reason:"Fallback fee" Decay 0.00000: Estimation: (-1 - -1) -nan% 0.0/(0.0 0 mem 0.0 out) Fail: (-1 - -1) -nan% 0.0/(0.0 0 mem 0.0 out)

JokerCatz commented at 2:13 pm on December 10, 2017: none

@Safari77 did your mempool size is too small ? & is sync finished ? or offline signature ? , I tried use estimatesmartfee will use mempool to fees , and you can use estimatefee / estimatesmartfee to calculate it first , but it per kb fees , so you need * 100000000 / 1024 will be per byte satoshi

morcos commented at 2:43 pm on December 10, 2017: member

@Safari77 When it says that the reason is “Fallback Fee” that means fee estimation is not working. Seems likely that this was a new node and had received less than 4 or 5 blocks since syncing. If using the GUI, there should be a red warning, but using bitcoin-cli we don’t currently have any way of warning people that it’s not possible to do fee estimation yet. There is really no solution to this other than to get your fee estimate from other sources and manually set the fee.

Safari77 commented at 2:58 pm on December 10, 2017: none

Well I definitely was not inspecting debug.log before sending the transaction in GUI after it synced and also didn’t have an idea about that 4-5 block “rule”…

So my only option is to quit bitcoin-qt, delete mempool.dat, and start bitcoin-qt again?

instagibbs commented at 3:39 pm on December 10, 2017: member

No do not delete your mempool data. Let your node run for a while and it will fix itself.

On Dec 10, 2017 9:58 AM, “Sami Farin” notifications@github.com wrote:

Well I definitely was not inspecting debug.log before sending the transaction in GUI after it synced and also didn’t have an idea about that 4-5 block “rule”…

So my only option is to quit bitcoin-qt, delete mempool.dat, and start bitcoin-qt again?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bitcoin/bitcoin/issues/10247#issuecomment-350554030, or mute the thread https://github.com/notifications/unsubscribe-auth/AFgC02g-zN9LKc-zM3dvHsfSmwMSyP5Lks5s-_GogaJpZM4ND3PA .

morcos commented at 3:47 pm on December 10, 2017: member

@Safari77 Just to clarify, i was suggesting that if you sent transaction from bitcoin-qt you should have seen something like this and there should have been no need to inspect debug.log:

But @instagibbs is right, you should not need to do anything to “fix” the problem. You should be able to see from bitcoin-qt that the estimates are more reasonable now.

Safari77 commented at 4:09 pm on December 10, 2017: none

Yes, it now suggests Transaction Fee 0.0019… So I guess I missed that warning. Maybe it could ask for extra confirmation if Smart free is not yet initialized in case user is half-asleep.

morcos commented at 4:19 pm on December 10, 2017: member

Yeah that’s not a bad idea Ping @jonasschnelli

BTW you can use bumpfee if you had RBF enabled and want to pay more fee to get it confirmed faster.

jonasschnelli commented at 6:51 pm on December 11, 2017: contributor

Thing that could be improved:

a) Don’t allow to send transactions when fee estimation is not initialised b) Show an additional popup warning c) Same a b) but force BIP125-RBF

I prefere c.

instagibbs commented at 7:30 pm on December 11, 2017: member

I like (c), or (b) at a minimum. (a) is a non-starter, if nothing else because that would make regtest hard to use

morcos commented at 8:11 pm on December 11, 2017: member

I wouldn’t force BIP125-RBF, but additional popup warning could strongly recommend it

cncr04s commented at 8:33 pm on December 11, 2017: none

Can we add a API function that estimates the fee. I looked at the code and there is such an internal function, ie it creates the transaction object then sends it to the network. I propose that the estimate fee would just return the created transaction without relaying it to the network. This way, users can decide for them selves if the fee being paid is appropriate for their needs. Thanks.

sipa commented at 8:34 pm on December 11, 2017: member

@cncr04s We have estimatefee (deprecated), estimatesmartfee, and estimaterawfee for getting feerate estimates. Furthermore, the fundrawtransaction will add inputs and change to a transaction according to the fee estimation, without broadcasting.

cncr04s commented at 8:41 pm on December 11, 2017: none

@sipa I’m not sure that applies to a sendtoaddress request. I want to know how much fee is paid in the sendtoaddress before its actually sent. I don’t know what inputs it selects/ the size of the transaction and therefore can’t calculate the fee using estimatesmartfee.

sipa commented at 8:42 pm on December 11, 2017: member

@cncr04s You can use fundrawtransaction for that.

laanwj deleted a comment on Dec 20, 2017

MarcoFalke commented at 3:28 pm on December 20, 2017: member

To the best of my knowledge this issue can be closed, as we fixed all known instances of “excessive fee-rate”, i.e. accidentally creating a transaction that pays more satoshis per byte in fees than it should (taking into account the target fee rate).

The issue that coin selection will yield a very unfortunate subset of coins, and cause an “excessive overall fee”, i.e. a transaction which spends many “uneconomical coins” is tracked in #1643.

The issue that fallbackfee silently results in “insufficient fee-rate or overall fee” is a completely separate issue and should be tracked on a separate ticket. Solutions to improve the fallbackfee situations are proposed in: #11918, #11882, #11892, #9481 (merged in 0.15)

JokerCatz commented at 7:14 pm on December 20, 2017: none

@MarcoFalke okay & why not , many thanx dev team :)

JokerCatz closed this on Dec 20, 2017

DrahtBot locked this on Sep 8, 2021

cost too many fees? #10247