multiwallet: issues arising from copying wallet files #11429

issue dooglus openend this issue on September 30, 2017
  1. dooglus commented at 4:47 pm on September 30, 2017: contributor

    I tried loading up all my wallet files at once to see how well it worked, then ran ‘getbalance’ on each of them. The balances of some were reported as being zero when they should have been non-zero.

    I believe the root cause is that I loaded up some old backups of the funded wallets as well as the up to date versions.

    Is that behavior meant to be supported? Or should each wallet have been created separately?

    My guess is that the wallet database gets some kind of unique ID assigned to it when it is created, and loading up an old backup alongside the current version means I have two wallets with the same “unique” ID loaded at once, causing problems.

    As an example, I have two wallets: wallet.dat.finex and wallet.dat.finex2. One is an old copy of the other. I loaded up a single third wallet, let it sync 3 blocks, shut down, then loaded the two finex wallets together. I had added debug to see why rescanning was being skipped in some cases.

    wallet.cpp has this test to decide whether the rescan is necessary:

    if (chainActive.Tip() && chainActive.Tip() != pindexRescan)
    

    In the debug output I saw:

    2017-09-30 16:32:36 [wallet.dat.finex] rescan? checking tip()
    2017-09-30 16:32:36 [wallet.dat.finex] rescan? maybe: chainActive.Tip() is true
    2017-09-30 16:32:36 [wallet.dat.finex] rescan? yes: 0x7f60bf3f3db0 (487616) != 0x7f60d3d4aad0 (487613)
    
    2017-09-30 16:32:36 [wallet.dat.finex2] rescan? checking tip()
    2017-09-30 16:32:36 [wallet.dat.finex2] rescan? maybe: chainActive.Tip() is true
    2017-09-30 16:32:36 [wallet.dat.finex2] rescan? no: 0x7f60bf3f3db0 (487616) == 0x7f60bf3f3db0 (487616)
    

    so the finex wallet gets 3 blocks rescanned, but the finex2 wallet doesn’t, even though it needs a rescan. I guess they share a CWalletDB walletdb(*walletInstance->dbw);.

    I stopped looking into it here, because maybe the solution is going to be “stop loading copies of a wallet at the same time”.

  2. jonasschnelli added the label Wallet on Oct 1, 2017
  3. jonasschnelli commented at 3:14 am on October 1, 2017: contributor
    Hmm… I hope someone can verify that. But it could indeed be a BDB issue. BerkleyDB operates on a directory & a file. Maybe this is another reason to either have a directory / BDB env per wallet or just finally make some progress for a new database format.
  4. dooglus commented at 4:01 am on October 4, 2017: contributor

    Here’s a simple way to reproduce the problem:

    $ ls -l ~/.bitcoin/wallet.dat.x1
    ls: cannot access '/home/chris/.bitcoin/wallet.dat.x1': No such file or directory
    $ ls -l ~/.bitcoin/wallet.dat.x2
    ls: cannot access '/home/chris/.bitcoin/wallet.dat.x2': No such file or directory
    $ btc -wallet=wallet.dat.x1 &
    $ bc getblockcount
    488234
    $ bc stop
    Bitcoin server stopping
    $ cp wallet.dat.x1 wallet.dat.x2
    $ btc -wallet=wallet.dat.x3 &
    $ sleep 300 [or whatever]
    $ bc getblockcount
    488235
    $ bc stop
    Bitcoin server stopping
    $ btc -wallet=wallet.dat.x1 -wallet=wallet.dat.x2 -printtoconsole | grep -i rescan
    2017-10-04 03:57:31 [wallet.dat.x1] rescan? checking tip()
    2017-10-04 03:57:31 [wallet.dat.x1] rescan? maybe: chainActive.Tip() is true
    2017-10-04 03:57:31 [wallet.dat.x1] rescan? yes: 0x7f0cb341d450 (488235) != 0x7f0cca494000 (488234)
    2017-10-04 03:57:31 init message: Rescanning...
    2017-10-04 03:57:31 [wallet.dat.x1] Rescanning last 1 blocks (from block 488234)...
    2017-10-04 03:57:31 [wallet.dat.x1]  rescan                   25ms
    2017-10-04 03:57:31 [wallet.dat.x2] rescan? checking tip()
    2017-10-04 03:57:31 [wallet.dat.x2] rescan? maybe: chainActive.Tip() is true
    2017-10-04 03:57:31 [wallet.dat.x2] rescan? no: 0x7f0cb341d450 (488235) == 0x7f0cb341d450 (488235)
    $ 
    
    • show that x1 and x2 don’t exist
    • create x1 and run getblockcount
    • copy x1 to x2
    • run new wallet x3 until a new block is found
    • run wallets x1 and x2, and note that only x1 is rescanned while both x1 and x2 require a 1 block rescan
  5. ryanofsky commented at 6:50 pm on October 10, 2017: member

    I was able to reproduce this, and I think I can add better error handling to detect it, but it doesn’t appear to be fixable because Berkeley db can’t distinguish between the two copies of the wallet file. Basically your idea about wallet databases having “some kind of unique ID” is right. From https://docs.oracle.com/cd/E17275_01/html/programmer_reference/program_copy.html:

    Because system file identification information (for example, filenames, device and inode numbers, volume and file IDs, and so on) are not necessarily unique or maintained across system reboots, each Berkeley DB database file contains a unique 20-byte file identification bytestring. When multiple processes or threads open the same database file in Berkeley DB, it is this bytestring that is used to ensure the same underlying pages are updated in the database environment cache, no matter which Berkeley DB handle is used for the operation.

    What happens as a result is that after WriteBestBlock is called on the first wallet file following the rescan, the ReadBestBlock call on the second wallet picks up the not fully flushed value from the first wallet, instead of the right value inside its own wallet file.

    We can retrieve the file id’s with pdb->get_mpf()->get_fileid(), so it should be possible to add a check that will refuse to load more than one wallet with the same id. I’ll try to open a PR that does this today.

  6. ryanofsky referenced this in commit ebc7c2ff3c on Oct 10, 2017
  7. dooglus commented at 8:08 pm on October 10, 2017: contributor
    Thanks @ryanofsky. I’m glad you took the time to look into this. I’m guessing the bug could result in wallet file corruption if two wallet files are sharing a single database cache, so it’s good to fix and backport.
  8. ryanofsky referenced this in commit e32c2a51b7 on Oct 11, 2017
  9. ryanofsky referenced this in commit 4164132f6b on Oct 12, 2017
  10. ryanofsky referenced this in commit f98832222c on Oct 12, 2017
  11. ryanofsky referenced this in commit 478a89c1ef on Oct 19, 2017
  12. laanwj closed this on Oct 19, 2017

  13. laanwj referenced this in commit 99e93de6f8 on Oct 19, 2017
  14. MarcoFalke referenced this in commit dca447bb54 on Nov 1, 2017
  15. MarcoFalke referenced this in commit 736ea103df on Nov 1, 2017
  16. MarcoFalke referenced this in commit 9c8006dc33 on Nov 1, 2017
  17. HashUnlimited referenced this in commit c74a1f9835 on Mar 12, 2018
  18. codablock referenced this in commit 7d6c4d6f01 on Sep 26, 2019
  19. codablock referenced this in commit b155130224 on Sep 29, 2019
  20. charlesrocket referenced this in commit 7068977081 on Dec 15, 2019
  21. barrystyle referenced this in commit f0babdd189 on Jan 22, 2020
  22. random-zebra referenced this in commit ae70f68ed3 on Apr 28, 2021
  23. random-zebra referenced this in commit efca898706 on Apr 29, 2021
  24. random-zebra referenced this in commit 7047d3d9e0 on May 8, 2021
  25. random-zebra referenced this in commit ba56c54d3a on May 17, 2021
  26. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-05 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me