wallet: Automatically repair corrupted metadata with doubled derivation path #29124

pull achow101 wants to merge 2 commits into bitcoin:master from achow101:fix-double-keypath changing 3 files +132 −24
  1. achow101 commented at 8:47 pm on December 20, 2023: member

    A bug in 0.21.x and 22.x resulted in some wallets storing invalid derivation paths that are the concatenation of two derivation paths. This corruption follows a rigid pattern and can be easily detected and repaired.

    The bug that resulted in such wallets was fixed in #23304 but wallets that already ran into it would still have bad data stored. This PR should fix such wallets.

    Fixes #29109

  2. DrahtBot commented at 8:47 pm on December 20, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Stale ACK furszy

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #28574 (wallet: optimize migration process, batch db transactions by furszy)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Wallet on Dec 20, 2023
  4. achow101 force-pushed on Dec 20, 2023
  5. DrahtBot added the label CI failed on Dec 20, 2023
  6. achow101 force-pushed on Dec 20, 2023
  7. achow101 force-pushed on Dec 21, 2023
  8. achow101 force-pushed on Dec 21, 2023
  9. DrahtBot removed the label CI failed on Dec 21, 2023
  10. DrahtBot added the label CI failed on Jan 15, 2024
  11. DrahtBot commented at 7:42 am on April 5, 2024: contributor
    Could rebase for fresh CI, if still relevant?
  12. achow101 force-pushed on Apr 5, 2024
  13. DrahtBot removed the label CI failed on Apr 5, 2024
  14. in src/wallet/walletdb.cpp:699 in 627216a174 outdated
    694         return DBErrors::LOAD_OK;
    695     });
    696     result = std::max(result, keymeta_res.m_result);
    697 
    698+    // Update any changed metadata
    699+    if (updated_metas.size()) {
    


    furszy commented at 2:40 pm on May 22, 2024:
    nit: !updated_metas.empty()

    achow101 commented at 11:54 pm on June 4, 2024:
    Done
  15. in src/wallet/walletdb.cpp:530 in 627216a174 outdated
    526@@ -527,7 +527,7 @@ static LoadResult LoadRecords(CWallet* pwallet, DatabaseBatch& batch, const std:
    527     return LoadRecords(pwallet, batch, key, prefix, load_func);
    528 }
    529 
    530-static DBErrors LoadLegacyWalletRecords(CWallet* pwallet, DatabaseBatch& batch, int last_client) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet)
    531+static DBErrors LoadLegacyWalletRecords(CWallet* pwallet, DatabaseBatch& batch, WalletBatch& w_batch, int last_client) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet)
    


    furszy commented at 2:46 pm on May 22, 2024:
    Non-blocking because it is the only way now but it is kinda unnatural to provide batch and w_batch when w_batch contains the batch object.

    achow101 commented at 11:55 pm on June 4, 2024:
    I’ve made m_batch public so that we can just pass the WalletBatch but still use the underlying batch object. However I’m not sure if that’s something we want to do in the long term, but I think it should be okay.

    furszy commented at 1:49 pm on June 5, 2024:

    I’ve made m_batch public so that we can just pass the WalletBatch but still use the underlying batch object. However I’m not sure if that’s something we want to do in the long term, but I think it should be okay.

    Yeah.. I’m not happy reading all the the batch.m_batch usages neither but np.

  16. in src/wallet/walletdb.cpp:704 in 627216a174 outdated
    699+    if (updated_metas.size()) {
    700+        LegacyScriptPubKeyMan* spkm = pwallet->GetLegacyScriptPubKeyMan();
    701+        Assert(spkm);
    702+        LOCK(spkm->cs_KeyStore);
    703+        for (const auto& pubkey : updated_metas) {
    704+            w_batch.WriteKeyMetadata(spkm->mapKeyMetadata.at(pubkey.GetID()), pubkey, true);
    


    furszy commented at 2:46 pm on May 22, 2024:
    nit: should check WriteKeyMetadata return value.

    achow101 commented at 11:55 pm on June 4, 2024:
    Done
  17. furszy commented at 2:54 pm on May 22, 2024: member
    Code review ACK 078c2d74c2779dd459e42daf67ab450fbf223506. Nothing blocking, just tiny nits.
  18. achow101 force-pushed on Jun 4, 2024
  19. DrahtBot added the label CI failed on Jun 5, 2024
  20. DrahtBot commented at 3:21 am on June 5, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/25814613888

  21. achow101 force-pushed on Jun 5, 2024
  22. furszy commented at 1:50 pm on June 5, 2024: member
    ACK 9db2d8b
  23. DrahtBot removed the label CI failed on Jun 6, 2024
  24. DrahtBot added the label Needs rebase on Jul 11, 2024
  25. achow101 force-pushed on Jul 22, 2024
  26. DrahtBot removed the label Needs rebase on Jul 22, 2024
  27. furszy commented at 1:42 pm on July 23, 2024: member

    ACK 1c3462e908994f0d63c1f39b5908f48d083dd10e

    Could minimize the first commit diff https://github.com/furszy/bitcoin-core/commit/145737e46e71859608334ab738817f7b323df367.

  28. DrahtBot added the label CI failed on Sep 6, 2024
  29. walletdb: Repair corrupted doubled derivation paths
    A bug in 0.21.x and 22.x resulted in some wallets having invalid
    derivation paths that are the concatenation of two derivation paths.
    This corruption follows a rigid pattern and can be easily detected and
    repaired.
    36826094b4
  30. tests: Test that bad inactive chain derivation path is cleaned up 66b851d7f2
  31. achow101 force-pushed on Sep 10, 2024
  32. DrahtBot removed the label CI failed on Sep 11, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-28 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me