wallet: make corrupted transaction records fail wallet loading instead of forcing a rescan #35760

pull achow101 wants to merge 5 commits into bitcoin:master from achow101:wallet-rm-needs-rescan changing 8 files +62 −23
  1. achow101 commented at 10:11 PM on July 20, 2026: member

    In wallet loading, the NEED_RESCAN enum was only ever returned if the stored hash of a transaction did not match the calculated hash of the transaction. This would then trigger a rescan from genesis during loading, and then allow the wallet to be operational as if it were normal.

    However, it seems incorrect to be treating such corruption as acceptable. That transaction would be inserted to mapWallet, but not be added to mapTxSpends or any of the conflict tracking or to TXO caching. Furthermore, the rescan doesn't actually fix the problem, the incorrect record would persist, which means that the wallet would be rescanning the entire chain every time it is loaded. The only situation this could occur is if the wallet actually experienced corruption, so it's reasonable to return CORRUPT instead and force the user to load a backup or otherwise un-corrupt the wallet before it can be used.

    Implements the idea from #35501 (review)

  2. DrahtBot added the label Wallet on Jul 20, 2026
  3. DrahtBot commented at 10:11 PM on July 20, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35760.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    Concept ACK polespinasa, jonatack, rkrux
    Approach ACK pablomartin4btc, jeanpablojp

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #36167 ([RFC] Enable -Wunused by fanquake)
    • #36133 (wallet: store multipath descriptor by Sjors)
    • #35440 (wallet: check descriptor cache xpub length before decoding by alhudz)
    • #27865 (wallet: Track no-longer-spendable TXOs separately by achow101)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. DrahtBot added the label Needs rebase on Aug 4, 2026
  5. achow101 force-pushed on Aug 5, 2026
  6. DrahtBot removed the label Needs rebase on Aug 6, 2026
  7. pablomartin4btc commented at 1:43 AM on August 7, 2026: member

    Approach ACK at fdd1a439fbf462c5d0fbfa2c4b7ac7f9c9a33a15

    When a tx record's stored hash doesn't match the hash of its deserialized content, returning NEED_RESCAN was the wrong response: the rescan is not a reliable repair — it only overwrites the corrupted DB record as a side effect if the tx happens to be on-chain and wallet-relevant; if it isn't, the corrupted record stays in the DB permanently and the wallet rescans from genesis on every subsequent load. I think failing hard with CORRUPT makes no unverifiable assumptions about what the DB was supposed to contain, and signals unambiguous failure rather than attempting an unreliable recovery.

    Having said that, the error message "Error loading %s: Wallet corrupted" gives the user no indication of what to do next.

    Since this is a user-visible behaviour change — wallet now fails to load instead of triggering a rescan — some release note seems required.

    As a side effect, bitcoin-wallet info on a wallet with this corruption previously printed a misleading "Rescanning wallet" warning to stdout and silently exited with an error code — the tool never calls AttachChain so no rescan would ever run. It now correctly prints "Wallet corrupted" to stderr (but still no indication to the user of what to do next).

    One suggestion: a unit test that writes a tx record with a mismatched key to the DB and asserts loading fails with CORRUPT would pin this behaviour and prevent regressions. I have one ready at 5e3212293b on top of this PR if useful.

  8. pablomartin4btc commented at 2:34 PM on August 10, 2026: member

    The PR correctly surfaces the corruption at load time rather than silently rescanning forever. The missing piece for users is what to do when they see that error — opened #35944 which adds a "Recovering from a Corrupted Wallet" section to doc/managing-wallets.md to address that gap.

    Also worth noting: walletdb.cpp:1050 has an error message that still says "This can be fixed by removing transactions from wallet and rescanning." — that message covers a different corruption path but contradicts the same user-facing guidance. Could be addressed here since this PR is already touching the corruption detection code.

  9. polespinasa commented at 8:12 AM on August 19, 2026: member

    opened #35944 which adds a "Recovering from a Corrupted Wallet" section to doc/managing-wallets.md to address that gap

    That's a small doc change probably worth adding here as an extra commit, as both PRs are small and related. I don't think there's the need for be looking for reviewers and going through the ACK process twice :)

  10. polespinasa commented at 8:12 AM on August 19, 2026: member

    Aproach ACK

    How can I test this?

  11. achow101 commented at 5:06 PM on August 19, 2026: member

    How can I test this?

    Open a wallet file with sqlite, find a tx record and change the hash.

  12. jonatack commented at 5:27 PM on August 19, 2026: member

    Concept ACK. Worth a release note?

  13. pablomartin4btc commented at 5:33 PM on August 19, 2026: member

    How can I test this?

    Open a wallet file with sqlite, find a tx record and change the hash.

    I've suggested a unit test earlier that writes a tx record with a mismatched key to the DB and asserts loading fails with CORRUPT -> https://github.com/bitcoin/bitcoin/commit/5e3212293b0e149b10ddad0a4a916fde1405b2ac.

  14. in src/wallet/walletdb.cpp:1050 in 304ec378b2


    pablomartin4btc commented at 5:38 PM on August 19, 2026:

    This error message is stale now and it should be updated to match the new behaviour.


    achow101 commented at 8:37 PM on August 19, 2026:

    Removed

  15. achow101 force-pushed on Aug 19, 2026
  16. achow101 commented at 8:37 PM on August 19, 2026: member

    I've suggested a unit test earlier that writes a tx record with a mismatched key to the DB and asserts loading fails with CORRUPT -> 5e32122.

    Pulled the test

    Worth a release note?

    Added a release note

  17. polespinasa commented at 8:13 AM on August 21, 2026: member

    I just came with a concern regarding this change.

    Before this PR a corrupted wallet would be anyway loaded after a rescan and would still show be able to spend funds, maybe not all due to corruption, but the valid and non-corrupted tx entries.

    If we disable allowing this corrupted wallets, users that unconsciously did not create a backup will be unable to access any of their funds at all without switching to an old Core version.

  18. in src/wallet/walletdb.cpp:1042 in a79db308cb outdated
    1038 | @@ -1031,15 +1039,15 @@ static DBErrors LoadTxRecords(CWallet* pwallet, DatabaseBatch& batch, bool& any_
    1039 |          try {
    1040 |              CWalletTx wtx{deserialize, value, ReadWtxVariants(batch, hash)};
    1041 |              if (wtx.GetHash() != hash) {
    1042 | -                result = std::max(result, DBErrors::NEED_RESCAN);
    1043 | +                return DBErrors::CORRUPT;
    


    polespinasa commented at 8:46 AM on August 21, 2026:

    in a79db308cb9bafc6c910e64f4cf6aceee7db55c3 walletdb: LoadToWallet failure is wallet corruption

    Maybe worth setting err?


    achow101 commented at 8:28 PM on August 21, 2026:

    Doe

  19. in src/wallet/test/walletload_tests.cpp:10 in 7bae5221ba
       3 | @@ -4,8 +4,10 @@
       4 |  
       5 |  #include <wallet/test/util.h>
       6 |  #include <wallet/wallet.h>
       7 | +#include <wallet/walletdb.h>
       8 |  #include <test/util/common.h>
       9 |  #include <test/util/logging.h>
      10 | +#include <test/util/random.h>
    


    polespinasa commented at 8:47 AM on August 21, 2026:

    in 7bae5221ba69962531b1f9fc2e12a273a4df2c2d wallet/test: add test for corrupt tx hash on wallet load

    random is included but never used


    achow101 commented at 8:29 PM on August 21, 2026:

    Removed

  20. in src/wallet/test/walletload_tests.cpp:105 in 7bae5221ba outdated
     100 | +    // serialised transaction. Loading must fail with CORRUPT rather than
     101 | +    // silently accepting broken data or scheduling a rescan.
     102 | +    auto database = CreateMockableWalletDatabase();
     103 | +    {
     104 | +        CMutableTransaction mtx;
     105 | +        mtx.vout.emplace_back(COIN, CScript() << OP_TRUE);
    


    polespinasa commented at 9:18 AM on August 21, 2026:

    in 7bae5221ba69962531b1f9fc2e12a273a4df2c2d wallet/test: add test for corrupt tx hash on wallet load

    This test is wrong it is not testing the DBErrors::CORRUPT path intended to test. This can be easily tested by just running this test on master.

    There are three path for LoadTxRecords that make the load fail as DBErrors::CORRUPT:

            try {
                CWalletTx wtx{deserialize, value, ReadWtxVariants(batch, hash)};
                if (wtx.GetHash() != hash) {
                    return DBErrors::CORRUPT;
                }
    
                if (wtx.nOrderPos == -1) {
                    any_unordered = true;
                }
    
                if (!pwallet->LoadToWallet(std::move(wtx))) {
                    err = "Error: Corrupt transaction found";
                    return DBErrors::CORRUPT;
                }
            } catch (const std::exception& e) {
                err = strprintf("Error: Corrupt tx record found: %s" ,e.what());
                return DBErrors::CORRUPT;
            }
            return result;
        });
    

    The one intended to test is the Corrupt transaction found catch is the first one which where a tx has an invalid hash. However this test is passing because the third one (the one inside the catch) is hit. This happens because the transaction has no inputs and when trying to deserialize it runs out of bytes and throws with end of data.

    $ ./build/bin/test_bitcoin --run_test=walletload_tests/wallet_load_corrupt_tx_hash -- -printtoconsole=1 -logthreadnames=1 -debug=walletdb 2>&1 | grep "WalletLogPrintf"
    2026-08-21T09:11:45.625958Z [test] [wallet/wallet.h:949] [WalletLogPrintf] [default wallet] Legacy Wallet Keys: 0 plaintext, 0 encrypted, 0 w/ metadata, 0 total.
    2026-08-21T09:11:45.625989Z [test] [wallet/wallet.h:949] [WalletLogPrintf] [default wallet] Descriptors: 0, Descriptor Keys: 0 plaintext, 0 encrypted, 0 total.
    2026-08-21T09:11:45.626255Z [test] [wallet/wallet.h:949] [WalletLogPrintf] [default wallet] Error: Corrupt tx record found: DataStream::read(): end of data: iostream error
    

    The fix is simple just add a dummy input to the transaction:

    $ git diff
    diff --git a/src/wallet/test/walletload_tests.cpp b/src/wallet/test/walletload_tests.cpp
    index 6946503542..c5e07b64ee 100644
    --- a/src/wallet/test/walletload_tests.cpp
    +++ b/src/wallet/test/walletload_tests.cpp
    @@ -102,6 +102,7 @@ BOOST_FIXTURE_TEST_CASE(wallet_load_corrupt_tx_hash, TestingSetup)
         auto database = CreateMockableWalletDatabase();
         {
             CMutableTransaction mtx;
    +        mtx.vin.emplace_back();
             mtx.vout.emplace_back(COIN, CScript() << OP_TRUE);
             CWalletTx wtx{MakeTransactionRef(std::move(mtx)), TxStateInactive{}};
             auto batch = database->MakeBatch();
    
    
    $ ./build/bin/test_bitcoin --run_test=walletload_tests/wallet_load_corrupt_tx_hash -- -printtoconsole=1 -logthreadnames=1 -debug=walletdb 2>&1 | grep "WalletLogPrintf"
    2026-08-21T09:14:29.109542Z [test] [wallet/wallet.h:949] [WalletLogPrintf] [default wallet] Legacy Wallet Keys: 0 plaintext, 0 encrypted, 0 w/ metadata, 0 total.
    2026-08-21T09:14:29.109585Z [test] [wallet/wallet.h:949] [WalletLogPrintf] [default wallet] Descriptors: 0, Descriptor Keys: 0 plaintext, 0 encrypted, 0 total.
    2026-08-21T09:14:29.109849Z [test] [wallet/wallet.h:949] [WalletLogPrintf] [default wallet] 
    

    As said in https://github.com/bitcoin/bitcoin/pull/35760/changes/a79db308cb9bafc6c910e64f4cf6aceee7db55c3#r3828796073 the empty WalletLogPrintf is ugly, could be improved by adding an err.


    pablomartin4btc commented at 7:27 PM on August 21, 2026:

    Thanks for catching this and working on a fix — checked that it works fine.


    achow101 commented at 8:29 PM on August 21, 2026:

    Fixed

  21. walletdb: LoadToWallet failure is wallet corruption
    If the wallet fails to load a transaction from the database, the wallet
    is corrupted.
    79eff69c16
  22. wallet: Remove DBErrors::NEED_RESCAN and rescan_required
    DBErrors::NEED_RESCAN is no longer returned by any part of wallet
    loading, so it can be deleted. AttachChain's rescan_required parameter
    was only ever set to true when NEED_RESCAN was returned, so this can be
    removed as well.
    1ba0884da5
  23. wallet/test: add test for corrupt tx hash on wallet load
    Verifies that a tx record whose DB key does not match the hash of its
    serialised transaction causes `PopulateWalletFromDB` to return
    `DBErrors::CORRUPT`, pinning the behaviour introduced in this PR and
    preventing regressions.
    d261af4d9e
  24. doc: add wallet corruption recovery guidance to managing-wallets.md
    Adds section 1.7 "Recovering from a Corrupted Wallet" to
    doc/managing-wallets.md. The section documents that restoring from a
    known-good backup is the only supported recovery path for corrupted
    wallet databases, and that there is no safe in-place repair mechanism.
    
    Motivated by achow101's conclusion in #35605: "we should make the only
    supported corruption recovery mechanism be to restore a backup and not
    do any of this stuff with maybe being able to recover."
    c34db6bbef
  25. doc: Release note for removing needs rescan 1e45c91abb
  26. achow101 force-pushed on Aug 21, 2026
  27. achow101 commented at 8:31 PM on August 21, 2026: member

    If we disable allowing this corrupted wallets, users that unconsciously did not create a backup will be unable to access any of their funds at all without switching to an old Core version.

    The user experience of such wallets is already pretty awful and I would expect there to be issues opened if anyone is actually running into this today.

  28. bitcoin deleted a comment on Aug 21, 2026
  29. polespinasa commented at 7:01 AM on August 22, 2026: member

    The user experience of such wallets is already pretty awful and I would expect there to be issues opened if anyone is actually running into this today.

    Yes I don't disagree with this, I am only pointing out that even if the experience is awful, might be the only way to recover funds o part of the funds.

    Maybe we could think in some way we can help the user instead of prohibiting access to the funds. If the transactions are the only thing corrupted we could give the user the option to list the descriptors so he can create a blank wallet and import them to recover funds. Or something like that, it's just an idea.

  30. rkrux commented at 12:59 PM on August 26, 2026: contributor

    Concept ACK 1e45c91

    I do prefer the wallet to stop and fail fast instead of trying to fix rare corrupted scenarios automatically everytime on load, which makes the wallet do too much and bloats the codebase as well. Ideally the corrupted scenarios of write time should fail-fast while writing itself (which I see might not be possible in every such case) instead of the read flow taking on the onus of fixing - unfortunate that the wallet has been doing this already and setting such a precedent.

    There indeed is a slight concern regarding the user not being able to access the funds if the backup is not there, which is also not ideal though in the first place. But I don't think the wallet should preemptively try to fix automatically at load time as such corrupted scenarios should be very unlikely.

    force the user to load a backup or otherwise un-corrupt the wallet before it can be used.

    Is there even a way for the user to un-corrupt the wallet?

  31. achow101 commented at 5:15 PM on August 26, 2026: member

    Is there even a way for the user to un-corrupt the wallet?

    Depends on their knowledge of the wallet. You can always use sqlite to manually modify records directly, outside of the wallet.

  32. jeanpablojp commented at 12:30 PM on August 30, 2026: contributor

    Approach ACK

    Built it, ran the unit tests and the wallet functional tests, and reproduced the corruption on regtest.

    The description says the tx is inserted into mapWallet but not added to mapTxSpends. That stopped being true in #35501, where LoadToWallet started getting called on the mismatch path too. With the old code put back here, IsSpent on the prevout returns true. The rest of the argument, the record that stays behind and the rescan on every load, still holds.

  33. in src/wallet/walletdb.cpp:1042 in 1e45c91abb
    1038 | @@ -1039,15 +1039,16 @@ static DBErrors LoadTxRecords(CWallet* pwallet, DatabaseBatch& batch, bool& any_
    1039 |          try {
    1040 |              CWalletTx wtx{deserialize, value, ReadWtxVariants(batch, hash)};
    1041 |              if (wtx.GetHash() != hash) {
    1042 | -                result = std::max(result, DBErrors::NEED_RESCAN);
    1043 | +                err = strprintf("Error: Corrupt transaction. Stored hash of %s but an actual hash of %s", wtx.GetHash().ToString(), hash.ToString());
    


    jeanpablojp commented at 12:30 PM on August 30, 2026:

    The two are swapped. In the description you call the DB key the stored hash and wtx.GetHash() the calculated one, and here the strprintf passes them the other way round. The new test writes under a zeroed key, so this is what it prints.

    Stored hash of 7c2ae489...40fe but an actual hash of 0000...0000
    

    Anyone looking for that record in sqlite by the message searches for a key that isn't there.

  34. polespinasa commented at 7:16 AM on August 31, 2026: member

    Depends on their knowledge of the wallet. You can always use sqlite to manually modify records directly, outside of the wallet.

    I don't think that is an option lol.

    But I don't think the wallet should preemptively try to fix automatically at load time as such corrupted scenarios should be very unlikely.

    I was not referring to fix corrupted scenarios on load, but to just let them load, with a big warning, if I am not wrong a corrupted wallet can still work, just the corrupted entries will not. That is a partial access to the funds which is, imho, better than 0 access. As I mentioned in #35760 (comment), imagine if the wallet is corrupted, instead of throwing, just return a message saying the wallet is corrupted, here are your private descriptors (if wallet is unlocked) so the user can import them to some other place. I don't think that is too much complexity and it's not trying to fix the corrupt state. The user will fix it by importing descriptors to a new wallet.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-09-06 08:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me