Only allocate a LevelDB block cache if LevelDB will actually use it #13741

1. 
   fanquake commented at 2:28 pm on July 22, 2018: member

   Rebased/squashed version of #12825. Upstream discussion at https://github.com/google/leveldb/issues/561.

   When snappy compression is disabled, LevelDB does not use a block cache. This PR avoids wasting memory reserved for the block cache in that situation, and gives it to the write buffer instead.

   Unfortunately this behavior is not documented (I have filed a bug upstream already). You can see here that PosixMmapReadableFile takes a scratch argument that is unused. This isn’t by accident; the return value of the method is compared against the supplied buffer here to determine whether the block cache should be used.

   Giving more memory to the write buffer is good for a number of reasons. Among them: LevelDB checks reads against the mem table holding current writes, so the write buffer acts as an LRU within LevelDB.

2. fanquake added the label UTXO Db and Indexes on Jul 22, 2018
3. fanquake added the label Resource usage on Jul 22, 2018
4. in src/dbwrapper.cpp:85 in 4146b48a88 outdated

   81+    return sizeof(void*) >= 8;
   82+#endif
   83+}
   84+
   85+// Systems with mmap do not use the block cache.
   86+constexpr bool LevelDBUsesBlockCache()
   

   ---

   ---

   

   Empact commented at 4:42 am on July 23, 2018:

   nit: static

   ---

   

   fanquake commented at 2:28 pm on July 25, 2018:

   Updated both with static
5. in src/dbwrapper.cpp:76 in 4146b48a88 outdated

   71@@ -72,7 +72,23 @@ class CBitcoinLevelDBLogger : public leveldb::Logger {
   72     }
   73 };
   74 
   75-static void SetMaxOpenFiles(leveldb::Options *options) {
   76+// Mmap is only used on 64-bit Unix systems.
   77+constexpr bool LevelDBUsesMmap() {
   

   ---

   ---

   

   Empact commented at 4:42 am on July 23, 2018:

   nit: static
6. Only give LevelDB a block cache if LevelDB will actually use it.

   On systems that use mmap LevelDB does not use a block cache. On these systems we
   give the memory instead to the write buffer, where it will be used more
   effectively.

   980b93e75a
7. fanquake force-pushed on Jul 25, 2018
8. fanquake added the label Up for grabs on Dec 29, 2018
9. 
   fanquake commented at 7:11 am on December 29, 2018: member

   Given this hasn’t received any further review, closing with “Up for grabs”.
10. fanquake closed this on Dec 29, 2018

    ---

11. fanquake deleted the branch on Jan 22, 2020
12. DrahtBot locked this on Feb 15, 2022

Contributors
fanquake Empact

Labels
UTXO Db and Indexes Resource usage Up for grabs