slow GUI with large wallets #15015

1. 
   HashUnlimited commented at 12:51 pm on December 21, 2018: contributor

   Loading a large wallet (around 23 mio transactions here on test) is resulting in an unusable GUI. Spinning wheel of death on every mouse click for 5 seconds at least.

   branch: master OS: macOS 10.14.2 CPU: Intel i9 (8th Gen) RAM: 32 GB DDR4

   Note: this behaviour is not limited to the Overview page or CoinControl but in general as long as the wallet is loaded. A headless setup performs nicely though.

2. fanquake added the label GUI on Dec 21, 2018
3. 
   cryptozeny commented at 12:21 pm on February 7, 2019: none

   same here. on custom testnet, empty wallet works like charm, but large wallet tooooooooo slow… one click for 5 seconds even no RPC calls!

   debug is OK, but only GUI freeze

   OS: ubuntu 16.04.5-64bit CPU: i5 skylake RAM: 24GB

   v0.16.3

4. 
   promag commented at 10:09 am on February 17, 2020: member

   @HashUnlimited are you able reproduce with #18160? ty
5. 
   jonatack commented at 9:53 am on March 30, 2020: member

   I was not able to reproduce this issue, though I initially had a different one (see the bottom).

   Yesterday I tested the GUI on current master 5f9cd62f33f, and with #18160 rebased onto it, both on Debian and MacOS.

   • Debian 4.19.37amd64 (2019-08-08) x86_64 - Intel Core i7-6500U 2.50GHz × 4 - 32GB
   • MacOS 10.14.6 - MacBook Pro Retina 2012 - Intel Core i7 2.7GHz - 16GB 1600 DDR3

   I did not see a real difference in performance between the four configurations. EDIT: While scanning, or in sometimes in general, performance does seem better with #18160. The results are variable and it’s hard for me to have confidence in them. When all is snappy and well, I am not seeing a difference. When the GUI is laggy/scanning, the PR does seem better, but it’s hard to say. I built and compared both several times, back and forth.

   The test was to load several wallets, 2-3 small wallets and then one large 177MB wallet, and run various console commands like getbalances, getwalletinfo, listwallets, etc. The large wallet is a regtest one generated with @achow101’s make-big-wallet.py script.

   On both Debian and MacOS it takes the GUI 5-6 minutes to load the big wallet.

   

   Once loaded, the GUI ran fine in my tests, with an additional reaction latency of ~400-500 ms with console commands like getbalances or getwalletinfo when the commands were run on big wallet. The other commands, and clicking on and opening transactions, ran basically as fast as they do with tiny wallets.

   With one build on Debian, I did see the GUI halt, and then crash with segmentation faults, when loading any wallet at all. However, rebuilding after make clean solved that, so I suspect it was the build.

   If anyone has suggestions on how to best instrument this or for further tests to run with the big wallet, let me know.

6. 
   achow101 commented at 4:15 pm on March 30, 2020: member

   @jonatack The big wallet script does ~100,000 txs. It’s possible that this is not big enough as it seems that OP has a wallet with several million txs. You should try increasing the iteration count to make a wallet that has a few million txs.
7. 
   promag commented at 4:19 pm on March 30, 2020: member

   According to OP

   Note: this behaviour is not limited to the Overview page or CoinControl but in general as long as the wallet is loaded. A headless setup performs nicely though. @achow101 see my last post at #18160, I think it’s safe to say that #18160 will improve in this case.

8. 
   jonatack commented at 4:30 pm on March 30, 2020: member

   @achow101 @promag thanks. Both trying with a larger wallet and instrumenting as per #18160 (comment) to reproduce @promag’s result (and/or with a flamegraph) seem worthwhile.
9. 
   HashUnlimited commented at 10:56 am on April 1, 2020: contributor

   Sorry for the late reply. Just tested with the same wallet - performance is WAY better now.
10. HashUnlimited closed this on Apr 1, 2020

    ---

11. 
    promag commented at 10:58 am on April 1, 2020: member

    performance is WAY better now. @HashUnlimited I guess you tested with #18160, and if so happy to know that.

12. 
    HashUnlimited commented at 11:01 am on April 1, 2020: contributor

    Yes, tested twice. Once with master and once with 0.18 and implemented #18160 just to make sure this is the performance booster.
13. 
    promag commented at 11:17 am on April 1, 2020: member

    Great, thanks for reporting the result.
14. DrahtBot locked this on Feb 15, 2022

Contributors
HashUnlimited cryptozeny promag jonatack achow101

Labels
GUI