The data storage layer of the wallet is fairly complicated and not well encapsulated or modularized. This makes it difficult for us to introduce new data storage methods. So this PR refactors it such that there is a clear way to introduce other storage options. This is also intended to clean up the logic of the database layer so that it is easier to follow and reason about.
In the database, there are 3 classes: BerkeleyBatch, BerkeleyDatabase, and BerkeleyEnvironment. BerkeleyDatabase is typenamed to WalletDatabase. The goal is to introduce two abstract classes: WalletDatabase and DatabaseBatch. These will be implemented by BerkeleyDatabase and BerkeleyBatch. This abstract classes can be inherited by other storage methods. All of the database specific things should be hidden in those classes.
To achieve this, we move the database handling things in BerkeleyBatch and turn it purely into a data accessor for the BerkeleyDatabase. Particularly, various static functions that operated on a BerkeleyDatabase are changed to be member functions of BerkeleyDatabase. Read, Write, Erase, and Exists are refactored to have separate functions that can be overridden by other classes. Instead of GetCursor returning a BDB specific Dbc object, new CreateCursor and CloseCursor functions are added and the cursor is handled internally within the BerkeleyBatch. We are then able to introduce the DatabaseBatch abstract class.
Functionality is further moved out of BerkeleyBatch into BerkeleyDatabase. Notably the Db handle creation is moved from BerkeleyBatch’s constructor and to a new Open function in BerkeleyDatabase. BerkeleyDatabase will also handle closing with a new Close function instead of having this be part of Flush.
Additionally, the existence of BerkeleyEnvironment is hidden inside of BerkeleyDatabase. mapFileUseCount is removed. Instead WalletDatabase will track it’s use count with m_refcount. This is incremented by newly introduced AddRef and RemoveRef functions. m_refcount is used to ensure that the database and environment are not closed while the database may still be in use.
Instead of each BerkeleyEnvironment tracking the fileids of the databases in that environment, a global map g_fileids is used. Since we were already going through every open fileid for the uniqeness check, it doesn’t matter what BerkeleyEnvironment has a database with a particular fileid. All we care about is that there is a database in any of the environments that has the same fileid of the database that is currently being opened.
Lastly, BerkeleyBatch, BerkeleyDatabase, and BerkeleyEnvironmentare moved into new fileswallet/bdb.{cpp/h}. WalletDatabaseandDatabaseBatchare inwallet/db.{cpp/h}`. This just gives better organization of the code so they aren’t all mixed together.
I’ve started breaking this down into separate PRs. Some of the simpler stuff is moved up to the front so they can be merged first.
- Mostly simple moveonly things: #19290
- walletdb: Make SpliWalletFilePath non-static
- walletdb: Add IsBDBWalletLoaded to look for BDB wallets specifically
- walletdb: move IsWalletLoaded to walletdb.cpp
- walletdb: moveonly: Move BerkeleyBatch Cursor and Txn funcs to cpp
- walletdb: Move BDB specific things into bdb.{cpp/h}
These PRs are independent of each other and can be merged out of order after #19290 is merged
- Refactor Read, Write, Erase, Exists: #19292
- walletdb: refactor Read, Write, Erase, and Exists into non-template func
- Handle cursor internally: #19308
- walletdb: Handle cursor internally
- Make Create, CreateMock, and CreateDummy standalone: #19310
- Add Create*WalletDatabase functions
- scripted-diff: Replace WalletDatabase::Create* with CreateWalletDatabase
- Remove WalletDatabase::Create, CreateMock, and CreateDummy
- Move BerkeleyBatch static functions: #19324
- walletdb: Combine VerifyDatabaseFile and VerifyEnvironment
- walletdb: Move PeriodicFlush into WalletDatabase
- walletdb: Move Rewrite into BerkeleyDatabase
These PRs require the previous listed to be merged first and need to be merged in order
- Introduce
DatabaseBatch: #19325- walletdb: Refactor DatabaseBatch abstract class from BerkeleyBatch
- walletdb: Add MakeBatch function to BerkeleyDatabase and use it
- Introduce
WalletDatabase: #19334- walletdb: Move BerkeleyDatabase::Flush(true) to Close()
- walletdb: Introduce AddRef and RemoveRef functions
- walletdb: Add BerkeleyDatabase::Open dummy function
- walletdb: Introduce WalletDatabase abstract class
- Cleanup the separation: #19335
- walletdb: track database file use as m_refcount within BerkeleyDatabase
- walletdb: Move Db->open to BerkeleyDatabase::Open
- walletdb: Use a global g_fileids instead of m_fileids for each env
- walletdb: Remove BerkeleyBatch friend class from BerkeleyDatabase
I will continue to break this down as things get merged.
