wallet: CPU use proportional to wallet transaction count when idle

dooglus commented at 3:23 am on September 6, 2019: contributor

I recently noticed that the Bitcoin Core GUI (v0.18.1) was constantly using significant amounts of CPU when idle. I disconnected all peers and it continued to use the CPU.

It turns out that it is calling WalletModel::pollBalanceChanged 4 times per second for all of my wallets, which causes it to loop through all the transactions in all my wallets 4 times per second, and uses a lot of CPU.

Is this intentional? It seems like it may be unnecessary.

I tried changing the pollTimer, using 2.5 seconds instead of 250 ms and it dropped the CPU usage to much lower, so I’m pretty sure this is what’s causing the CPU load:

connect(pollTimer, &QTimer::timeout, this, &WalletModel::pollBalanceChanged);
pollTimer->start(MODEL_UPDATE_DELAY*10);

Most of the CPU is being used by LookupBlockIndex looking up the same few thousand block hashes over and over, here:

[#3](/bitcoin-bitcoin/3/)  std::unordered_map<...>::find (__x=..., this=<optimized out>) at /usr/include/c++/8/bits/unordered_map.h:920
[#4](/bitcoin-bitcoin/4/)  LookupBlockIndex (hash=...) at ./validation.h:436
[#5](/bitcoin-bitcoin/5/)  interfaces::(anonymous namespace)::LockImpl::getBlockHeight (this=<optimized out>, hash=...) at interfaces/chain.cpp:34
[#6](/bitcoin-bitcoin/6/)  interfaces::(anonymous namespace)::LockImpl::getBlockDepth (this=0x555556ebb4c0, hash=...) at interfaces/chain.cpp:43
[#7](/bitcoin-bitcoin/7/)  CMerkleTx::GetDepthInMainChain (this=this@entry=0x7fff7ce26940, locked_chain=...) at wallet/wallet.cpp:4479
[#8](/bitcoin-bitcoin/8/)  CWalletTx::IsTrusted (this=this@entry=0x7fff7ce26940, locked_chain=...) at wallet/wallet.cpp:2081
[#9](/bitcoin-bitcoin/9/)  CWallet::GetUnconfirmedBalance (this=0x7fff7dc19150) at /usr/include/c++/8/bits/unique_ptr.h:342
[#10](/bitcoin-bitcoin/10/) interfaces::(anonymous namespace)::WalletImpl::getBalances (this=0x7fff7d4e66a0) at /usr/include/c++/8/bits/shared_ptr_base.h:1307
[#11](/bitcoin-bitcoin/11/) interfaces::(anonymous namespace)::WalletImpl::tryGetBalances (this=0x7fff7d4e66a0, balances=..., num_blocks=@0x7fffffffd9cc: -1) at interfaces/wallet.cpp:388
[#12](/bitcoin-bitcoin/12/) WalletModel::pollBalanceChanged (this=0x5555569caef0) at /usr/include/c++/8/bits/unique_ptr.h:342

fanquake added the label Resource usage on Sep 6, 2019

ryanofsky commented at 4:17 pm on September 6, 2019: member

Some of this sounds like it could be broken or unintential. Having balances updated multiple times a second is by design, but could I think could easily improved by only repeating computation when contents of mapWallet changes. (I’d like to see this because the polling causes a ridculous amount of spam in #10102 IPC logs.)

The trace showing lots of calls to LookupBlockIndex is suprising to me because generally balances are cached, so I’m not sure why they aren’t being cached in this case. It could be interesting to look into. Also #15931 will prevent the repeated LookupBlockIndex calls regardless of other things.

dooglus commented at 1:24 am on September 7, 2019: contributor

generally balances are cached

I just tried running bitcoin-cli getbalance 10 times in a row quickly. It looped through all my transactions and summed them 10 times.

Do you have an example of an operation that uses a cached balance instead of recalculating it every time please? I’d like to look into this and see why the cache isn’t being used, but I’m not having any success finding the cache!

sipa commented at 1:36 am on September 7, 2019: member

The balances are cached per transaction, AFAIK, not at the wallet level.

MarcoFalke commented at 7:32 am on September 7, 2019: member

Even when the balances are cached, the block is looked up multiple times via GetDepthInMainChain, see also https://dev.visucore.com/bitcoin/doxygen/group___actions.html#ga67f3c4f8a18222d1a091ac946725b0c7

MarcoFalke renamed this:
~~qt: CPU use proportional to wallet transaction count when idle~~
wallet: CPU use proportional to wallet transaction count when idle
on Sep 7, 2019

MarcoFalke added the label Wallet on Sep 7, 2019

dooglus commented at 3:03 pm on September 8, 2019: contributor

Re. the title change. I’m not sure the wallet uses any CPU when idle. It’s the Qt application that is checking the balance every 250 ms not the wallet itself.

MarcoFalke commented at 3:32 pm on September 8, 2019: member

I think you report several issues:

GetBalance() is slow for various reasons ( repeated calls to LookupBlockIndex(), loop over the map every time instead of using a cache).
The gui polls the balance every 250ms and is thus using a lot of cpu

Probably both issues should be fixed

promag commented at 11:22 pm on September 8, 2019: member

Related #10251.

Probably both issues should be fixed

I’d say remove polling first. Is it subject to 0.19 feature freeze or accepted after that?

MarcoFalke commented at 11:17 am on September 9, 2019: member

Maybe also related: #13612?

promag commented at 3:55 pm on February 5, 2020: member

Related #16874.

FWIW when idle polling period could be increased, going to push a suggestion.

wallet: CPU use proportional to wallet transaction count when idle #16815