This PR separates the storage abstraction layer (e.g., leveldb layer) from the bitcoin core application layer. This has many benefits:
- Easier testing for the bitcoin core application layer without requiring an open database
- Easier replacement for the storage library in case it’s decided to do it in the future with a simple implementation of an interface
- Ensure that any other future implementations can live to the same standard we have now by testing different implementations the same way
- Better interface description to understand the expectations from the implementations of the interface
- Much easier bench-marking separate from serialization
This change has caused zero modification to the bitcoin core application source code, as I deliberately maintained everything from the outside. This change is a drop-in replacement to everything that uses leveldb.
Things I failed to do without breaking backwards compatibility, so I skipped:
- Use DBIndex to decide the prefix of the storage. The problem here is that some key/value database systems provide other mechanisms for separating between different database “tables”, so to say. In our case, we use a
charas a prefix. For example, lmdb provides an integer index. The problem here is that when keys are submitted as a Span, prepending a prefix is a relatively expensive operation as it requires copying the whole key. I don’t see how we can do this without that. This is a conflict between the serialization and the storage layer that I couldn’t resolve.
Things I’d love to achieve in the next PR(s):
- ~Create an in-memory database that can easily replace leveldb and assist in tests~ (Given that tests use in-memory db as was pointed out in the comments, we may or may not do this)
- Move the block and block-undo storage behind this interface and use DBIndex as a way to tell the implementation how to store the data (in files or in leveldb)
