wallet getbalance is slow with 3K addresses and 3M transactions #24607

1. 
   sauliuspaukstys commented at 10:02 am on March 18, 2022: none

   getbalance almost blocks bitcoin deamon on first call

   I think that’s a pretty common use case (and the recommended practice) for bitcoin businesses: to generate a new bitcoin address any time your customer wants to deposit something. After a while wallet has thousands of addresses and millions of transactions. After bitcoind starts, getbalance needs to be called (in case of bitcoin-qt it done in background). Service gets stuck as it takes a lot of time (im my case 11 hours) to calculate and cache the balance.

   It would be good to cache the wallet balance in order to speed up spin-up time I have made wallet in testnet with 3K watch only addresses (with lot of transactions) In total it has ~3M transactions

   Caching would help a lot Speed-up can bee achieved by storing m_amounts next to transaction data in the wallet DB.

   Any other ideas, recommendations are more than welcome

2. sauliuspaukstys added the label Feature on Mar 18, 2022
3. MarcoFalke added the label Wallet on Mar 18, 2022
4. MarcoFalke added the label Resource usage on Mar 18, 2022
5. 
   promag commented at 5:15 pm on March 21, 2022: member

   You could partition to multiple wallets - switch to a fresh wallet once the current one reaches some address/tx count. Later you can spend/sweep those deposits.

   Do you mean persistent cache of balances in the wallet file?

6. 
   sauliuspaukstys commented at 6:24 am on March 22, 2022: none

   Yes it would work if balances would be kept in the wallet file persistent
7. 
   sauliuspaukstys commented at 8:35 am on March 22, 2022: none

   Not sure what you mean. From my point of view it would solve if wallet transaction’s m_amounts would be stored in wallet file after set/update. m_amounts can be restored on wallet transactions load.
8. 
   achow101 commented at 1:56 pm on March 22, 2022: member

   Persisting the cached balance on disk must be done with care. It could result in situations where the cache is incorrect if the balance calculation operation is interrupted before the updated balances are written to disk. This is particularly important because of imports that effect transactions already in the wallet. An import can make existing inputs and outputs change their IsMine state which will have an effect on the balance.

Contributors
sauliuspaukstys promag achow101

Labels
Feature Wallet Resource usage