wallet: pre-HD HDD migratewallet #29438

issue maflcko openend this issue on February 15, 2024
  1. maflcko commented at 11:24 am on February 15, 2024: member

    Opening a brainstorming issue, if someone wants to test migratewallet on a pre-HD wallet on a slow storage device (HDD).

    Steps to reproduce:

    • Create a wallet in a folder. For example: ./releases/bitcoin-0.11.2/bin/bitcoin-qt -datadir=/tmp/rgw -regtest. Then stop bitcoin-qt.
    • Copy the wallet file to a folder residing on the intended storage device and call migratewallet with the latest version of Bitcoin Core.

    The second step can be done by a functional test (I modified an existing test). Callable with:

    0./test/functional/wallet_disable.py --tracerpc --tmpdir=/run/media/pick_the_storage/folder/temp3 --timeout-factor=0
    
    • --tracerpc will print the time it took
    • --tmpdir is used to select the location
    • --timeout-factor=0 is needed to disable the client RPC timeout

    My diff:

     0diff --git a/test/functional/wallet_disable.py b/test/functional/wallet_disable.py
     1index 9c73f7dead..7be397c630 100755
     2--- a/test/functional/wallet_disable.py
     3+++ b/test/functional/wallet_disable.py
     4@@ -12,19 +12,31 @@ from test_framework.test_framework import BitcoinTestFramework
     5 from test_framework.util import assert_raises_rpc_error
     6 
     7 class DisableWalletTest (BitcoinTestFramework):
     8+    def add_options(self, parser):
     9+        self.add_wallet_options(parser)
    10+
    11     def set_test_params(self):
    12-        self.setup_clean_chain = True
    13         self.num_nodes = 1
    14-        self.extra_args = [["-disablewallet"]]
    15-        self.wallet_names = []
    16 
    17     def run_test (self):
    18-        # Make sure wallet is really disabled
    19-        assert_raises_rpc_error(-32601, 'Method not found', self.nodes[0].getwalletinfo)
    20-        x = self.nodes[0].validateaddress('3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy')
    21-        assert x['isvalid'] == False
    22-        x = self.nodes[0].validateaddress('mneYUmWYsuk7kySiURxCi3AGxrAqZxLgPZ')
    23-        assert x['isvalid'] == True
    24+        n = self.nodes[0]
    25+        (n.wallets_path/"old").mkdir(parents=True)
    26+        import os
    27+        os.system(f"cp /tmp/rgw/regtest/wallet.dat {n.wallets_path}/old/")
    28+
    29+        n.loadwallet("old")
    30+        w = n.get_wallet_rpc("old")
    31+        w.encryptwallet("pw")
    32+        w.walletpassphrase("pw", 999999999)
    33+        w.importprivkey(privkey=n.get_deterministic_priv_key().key, label='coinbase', rescan=True)
    34+
    35+        for i in range(10):
    36+            _ = w.getnewaddress()
    37+            w.sendtoaddress(w.getnewaddress(), round(w.getbalance() / 100, 8))
    38+            if i % 10:
    39+                self.generate(n, 1)
    40+        w.getwalletinfo()
    41+        w.migratewallet("old", "pw")
    42 
    43 
    44 if __name__ == '__main__':
    
  2. maflcko added the label Brainstorming on Feb 15, 2024
  3. maflcko added the label Wallet on Feb 15, 2024
  4. maflcko added the label Resource usage on Feb 15, 2024
  5. maflcko commented at 11:48 am on February 15, 2024: member

    Putting a perf context around the RPC call (with n.profile_with_perf("migrate-wallet-perf"):) allows to generate flame graphs. However, I am not sure how useful they are, as they may not capture IO?

    SSD:

    Screenshot from 2024-02-15 12-34-33

    HDD:

    Screenshot from 2024-02-15 12-45-10

  6. maflcko commented at 11:54 am on February 15, 2024: member
    Closing for now, but feel free to leave a comment if you think this should be re-opened. Happy to provide more details, if there are questions.
  7. maflcko closed this on Feb 15, 2024

  8. achow101 commented at 2:16 pm on February 15, 2024: member

    It seems like most of the time is being spent in reading the wallet rather than writing it?

    Can you provide the raw perf data?

  9. maflcko commented at 2:33 pm on February 15, 2024: member

    Can you provide the raw perf data?

    Does it not reproduce for you (or anyone else)? If not, then the problem may be entirely different.

  10. achow101 commented at 2:37 pm on February 15, 2024: member

    Does it not reproduce for you (or anyone else)? If not, then the problem may be entirely different.

    My test wallet was not nearly as slow. I’m in the process of making a bigger one.


    Even with the larger wallet, I don’t see that big of a slowdown. On master, it takes 328 seconds. After my latest push to #26008, it’s now 252 seconds.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-28 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me