Update leveldb subtree to latest upstream #31671

Includes:

• https://github.com/bitcoin-core/leveldb-subtree/pull/40 (used in #29852)
• https://github.com/bitcoin-core/leveldb-subtree/pull/45
• https://github.com/bitcoin-core/leveldb-subtree/pull/46

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31671.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process.

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Could you share more about how these changes were selected. I noticed there have been many (useless) refactorings in LevelDB since the last stable version we’re using. While I completely agree that cherry-picking every update may not align with our goals (especially for many of the high-risk, low-reward changes), a few targeted updates addressing critical issues might still be worth considering:

• fb644cb: Fixes #1081 - [BUG] LevelDB data loss after a crash when deployed on GlusterFS.
• eb31d19: Improves TEST_CompactRange by allowing compactions to finish.
• 8a68093: Ensures VersionEdit::Clear() also clears compact_pointers_.

Have these been considered? These seem relatively low-risk and possibly high-reward - and could help improve reliability or edge-case behavior.

Concept ACK 910a11f

I think it makes sense to update to the leveldb latest upstream, I also agree with @l0rinc there might be some other changes that might be worth considering.

I also just validated that this change only includes these 3 changes https://github.com/bitcoin-core/leveldb-subtree/pull/40, https://github.com/bitcoin-core/leveldb-subtree/pull/45, https://github.com/bitcoin-core/leveldb-subtree/pull/46

Minus the removal of LEVELDB_IS_BIG_ENDIAN in cmake/leveldb.cmake

Could you share more about how these changes were selected. might still be worth considering:

They were PR'd, with some rationale, to https://github.com/bitcoin-core/leveldb-subtree/. This PR is just updating to the current state of the subtree, but if you think there are other worthwhile changes that should be looked at, I'd suggest opening a PR upstream for discussion (that would have to be the case anyways, before they could be part of this PR).

Guix Build (aarch64):

139048cf62397ec98d6863fc1706a3c7c2afd02307873ec9a9bd707c531283a6  guix-build-910a11fa6630/output/aarch64-linux-gnu/SHA256SUMS.part
7d2de717c990cbc3f7da29291e8de257e610a5a79b6a351f00b9c9cad4e8ed65  guix-build-910a11fa6630/output/aarch64-linux-gnu/bitcoin-910a11fa6630-aarch64-linux-gnu-debug.tar.gz
c9b45a7c734d6cba6f736cbc53cea1dd207600133eaa2e82f58fa4dc96bd21c9  guix-build-910a11fa6630/output/aarch64-linux-gnu/bitcoin-910a11fa6630-aarch64-linux-gnu.tar.gz
8c11bbb0fda0e16bbf7b3f9f1aed1708f8271e827dabe0365be8a8667fe58d0f  guix-build-910a11fa6630/output/arm-linux-gnueabihf/SHA256SUMS.part
fee2d0ea352b712dfa76164b01703974e3244e4ac6f896de47d23e4fa3d2589c  guix-build-910a11fa6630/output/arm-linux-gnueabihf/bitcoin-910a11fa6630-arm-linux-gnueabihf-debug.tar.gz
71d11b361378211413c2ff4376031e86d06ce3ee480c3766a3661c0551c1a41a  guix-build-910a11fa6630/output/arm-linux-gnueabihf/bitcoin-910a11fa6630-arm-linux-gnueabihf.tar.gz
1daf51607a74af072e4bb1a7c2cc1d27015cfc342e2ffab6554d66ffcb11d4f9  guix-build-910a11fa6630/output/arm64-apple-darwin/SHA256SUMS.part
5e14455f3c05f5426fe67db83c1d63441934f4a3b1dee7b4746a8862debd16d2  guix-build-910a11fa6630/output/arm64-apple-darwin/bitcoin-910a11fa6630-arm64-apple-darwin-unsigned.tar.gz
cb9e41c27371de5c6a673c9182ae934a8d4dfe4fcb1b2989029e551a678cb20c  guix-build-910a11fa6630/output/arm64-apple-darwin/bitcoin-910a11fa6630-arm64-apple-darwin-unsigned.zip
b794699a47207330c205e6200f2b056cab042f3fab6ba7c0a7dd6a09305bfc48  guix-build-910a11fa6630/output/arm64-apple-darwin/bitcoin-910a11fa6630-arm64-apple-darwin.tar.gz
c854c3d153df11f44fbd916fa8054cd39ed5f1e41d7ed67852a66d44641677f2  guix-build-910a11fa6630/output/dist-archive/bitcoin-910a11fa6630.tar.gz
748f88b9cedae4da3838f763c045646726d066f02b32b9d0655e5464bbb5c25c  guix-build-910a11fa6630/output/powerpc64-linux-gnu/SHA256SUMS.part
3e62689fd321e1f923367781c3ee9afb7843010b57b59217691b9145b554b572  guix-build-910a11fa6630/output/powerpc64-linux-gnu/bitcoin-910a11fa6630-powerpc64-linux-gnu-debug.tar.gz
34635ecd7a77bff15bbd092e1cc17a2dfa2a6c8620ec07407c91ccfdf4d4a6cc  guix-build-910a11fa6630/output/powerpc64-linux-gnu/bitcoin-910a11fa6630-powerpc64-linux-gnu.tar.gz
bda7f58f6364d41b9615bd9f9f9a703d0b7200611dde2d43b605fae2eeaf0e52  guix-build-910a11fa6630/output/riscv64-linux-gnu/SHA256SUMS.part
1c7601d2596480f2147687d567893b503b9b046071b491adac94687d8bc48ca0  guix-build-910a11fa6630/output/riscv64-linux-gnu/bitcoin-910a11fa6630-riscv64-linux-gnu-debug.tar.gz
7a4dc52ce7bcaca40ca5ec64248caa878e5ce2afda1f7fc262ff969a79cc9174  guix-build-910a11fa6630/output/riscv64-linux-gnu/bitcoin-910a11fa6630-riscv64-linux-gnu.tar.gz
d545b852fd1237d84a79bf339561e199b404faa6060785820e96f9cef599215e  guix-build-910a11fa6630/output/x86_64-apple-darwin/SHA256SUMS.part
46204fb509b43da8feb1d271e219ca92ff08c7ca396518ec988548a3412356f1  guix-build-910a11fa6630/output/x86_64-apple-darwin/bitcoin-910a11fa6630-x86_64-apple-darwin-unsigned.tar.gz
e3d92f97e00e4849ce44070d4f07e162c1703bc5ac32fdccf24829d29c27a8b1  guix-build-910a11fa6630/output/x86_64-apple-darwin/bitcoin-910a11fa6630-x86_64-apple-darwin-unsigned.zip
b1dd7a8e50167787cb6e2f4ea193572f065da5d54db7885ad5b38139d5014ee1  guix-build-910a11fa6630/output/x86_64-apple-darwin/bitcoin-910a11fa6630-x86_64-apple-darwin.tar.gz
74ec6cd5b0ba17ef072b3af4149ef210a1129a70112069e6383ac8d93cf3b363  guix-build-910a11fa6630/output/x86_64-linux-gnu/SHA256SUMS.part
2ded19b149b7c5ffc06302d9f3a60905374928fd6f083f90d3a1a4f011073862  guix-build-910a11fa6630/output/x86_64-linux-gnu/bitcoin-910a11fa6630-x86_64-linux-gnu-debug.tar.gz
3fe7b25ef52d2054b5e2ba14dc8c9e14f487ed1f420a8780123bf6f60278927b  guix-build-910a11fa6630/output/x86_64-linux-gnu/bitcoin-910a11fa6630-x86_64-linux-gnu.tar.gz
89bb522b8fad6b8bcea84f115a6ab33654b4f801e6f94aeb2a4161a6b3a6d17b  guix-build-910a11fa6630/output/x86_64-w64-mingw32/SHA256SUMS.part
58f5b6c655a5448b1003b0e1e6720f948d56b4854ecbf172826ab4b71002852d  guix-build-910a11fa6630/output/x86_64-w64-mingw32/bitcoin-910a11fa6630-win64-debug.zip
ba63ef4e8ce6ac71a1851c87aab3ac4d408be2472f1d0505b9c845ccf2ea1b80  guix-build-910a11fa6630/output/x86_64-w64-mingw32/bitcoin-910a11fa6630-win64-setup-unsigned.exe
9bcec129fb2971cbd7c3e6e8dae46379c8a6309ca7faaf9a6415f2aeed06d3bb  guix-build-910a11fa6630/output/x86_64-w64-mingw32/bitcoin-910a11fa6630-win64-unsigned.tar.gz
140ed309a4d45de507d0e5831ef04710114f5822e9277b43c8378c7dd63e9244  guix-build-910a11fa6630/output/x86_64-w64-mingw32/bitcoin-910a11fa6630-win64.zip

ACK 910a11fa66305f90b0f3a8aa9d2055b58a2d8d80, I've performed a subtree update locally and got the same changes.

I also verified that the LEVELDB_IS_BIG_ENDIAN macro is no longer used in the code.

Reapplying the original leveldb PRs and comparing it against this PR leaves:

diff --git a/cmake/leveldb.cmake b/cmake/leveldb.cmake
--- a/cmake/leveldb.cmake	(revision 846a4dfc107d6593d0133bb8ae63edd824f7cb74)
+++ b/cmake/leveldb.cmake	(date 1737461205681)
@@ -60,6 +60,7 @@
     HAVE_FULLFSYNC=$<BOOL:${HAVE_FULLFSYNC}>
     HAVE_O_CLOEXEC=$<BOOL:${HAVE_O_CLOEXEC}>
     FALLTHROUGH_INTENDED=[[fallthrough]]
+    LEVELDB_IS_BIG_ENDIAN=$<STREQUAL:${CMAKE_CXX_BYTE_ORDER},BIG_ENDIAN>
     $<$<NOT:$<BOOL:${WIN32}>>:LEVELDB_PLATFORM_POSIX>
     $<$<BOOL:${WIN32}>:LEVELDB_PLATFORM_WINDOWS>
     $<$<BOOL:${WIN32}>:_UNICODE;UNICODE>

Which was indeed unused after the LevelDB changes.

ACK 910a11fa66305f90b0f3a8aa9d2055b58a2d8d80

~Edit: running a full reindex-chainstate to make sure it passes, let's not merge it yet~

utACK 910a11fa66305f90b0f3a8aa9d2055b58a2d8d80

Since LevelDB changes are consensus-critical, I ran -reindex-chainstate multiple times up to 880000 to check for any changes in performance or correctness.

I'm happy to report that no issues were found.

<details> <summary>Benchmark results</summary>

hyperfine \
--runs 2 \
--parameter-list COMMIT 712cab3a8f8ad76db959337ddc35cb4c34cac388,910a11fa66305f90b0f3a8aa9d2055b58a2d8d80 \
--prepare 'rm -f /mnt/my_storage/BitcoinData/debug.log && git checkout {COMMIT} && git clean -fxd && git reset --hard && cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_UTIL=OFF -DBUILD_TX=OFF -DBUILD_TESTS=OFF -DENABLE_WALLET=OFF -DINSTALL_MAN=OFF && cmake --build build -j$(nproc)' \
--cleanup 'mv /mnt/my_storage/BitcoinData/debug.log /mnt/my_storage/logs/debug-{COMMIT}.log' \
'COMMIT={COMMIT} ./build/src/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=30000 -reindex-chainstate -connect=0'

Benchmark 1: COMMIT=712cab3a8f8ad76db959337ddc35cb4c34cac388 ./build/src/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=30000 -reindex-chainstate -connect=0
  Time (mean ± σ):     23745.543 s ± 172.006 s    [User: 37663.092 s, System: 696.830 s]
  Range (min … max):   23623.917 s … 23867.170 s    2 runs
 
Benchmark 2: COMMIT=910a11fa66305f90b0f3a8aa9d2055b58a2d8d80 ./build/src/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=30000 -reindex-chainstate -connect=0
  Time (mean ± σ):     23847.875 s ± 249.798 s    [User: 37649.007 s, System: 692.565 s]
  Range (min … max):   23671.241 s … 24024.509 s    2 runs
 
Summary
  COMMIT=712cab3a8f8ad76db959337ddc35cb4c34cac388 ./build/src/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=30000 -reindex-chainstate -connect=0 ran
    1.00 ± 0.01 times faster than COMMIT=910a11fa66305f90b0f3a8aa9d2055b58a2d8d80 ./build/src/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=880000 -dbcache=30000 -reindex-chainstate -connect=0

</details>