- - Pull LevelDB 1.20 into the upstream bitcoin-core/levedb repo. Currently at 1.19
- - Update this repository with LevelDB 1.20. It is also currently at 1.19.
- - Write release notes portion outlining LevelDB performance improvements.
LevelDB 1.20 release notes
Convert documentation to markdown. Implement support for Intel crc32 instruction (SSE 4.2). Based on #309. Limit the number of read-only files the POSIX Env will have open. Add option for maximum file size.
related IRC chat
015:46:11 sipa: wumpus: btw, LevelDB .20 adds hw accelerated crc for intel
115:46:30 gmaxwell: \O/
215:48:13 wumpus: sipa: good to know
315:49:12 wumpus: yep https://github.com/google/leveldb/blob/master/port/port_posix_sse.cc
415:49:27 wumpus: that will really help our performance (on supported hw0
515:50:35 gmaxwell: should probably do that upgrade sooner rather than later in 0.15 to give time to find any compatiblity issues.
615:50:35 sipa: we may want to add arm code to do the same ourselves
715:51:20 wumpus: indeed
815:57:32 wumpus: yes would make sense to bump our leveldb in master to that soon
915:58:30 sipa: we'll need some build process integration to detect and pass down the SSE defines
1015:58:37 sipa: as we bypass their build system
1115:58:44 wumpus: would also get https://github.com/bitcoin-core/leveldb/pull/17
1215:59:05 wumpus: isn't the SSE stuff at runtime?
1315:59:12 wumpus: I'd hope so 
1415:59:39 gmaxwell: at least on intel runtime autodetection is simple-- and as I've mentioned before, recent g++ has a sensible looking function overrides based way of doing the substitution.
1515:59:56 gmaxwell: but leveldb didn't implement runtime? lameo.
1616:00:06 sipa: yes, they detect at runtime
1716:00:08 wumpus: they for fact have runtime detection, I saw HaveSSE42 in the file I linked
1816:00:20 wumpus: dunno if they use it
1916:00:26 sipa: but you still need a compiler that has support for it
2016:01:28 wumpus: "In a separate source file to allow this accelerated CRC32C function to be compiled with the appropriate compiler flags to enable x86 SSE 4.2instructions."
2116:01:31 wumpus: indeed
2216:01:42 wumpus: that specific file needs special compiler flags
2316:02:28 sipa: they use intel instrinsics in gcc, not native assembly
2416:03:30 gmaxwell: yea thats a pretty normal approach, you have one file compiled with -msse4.2 and dispatch elsewhere calls into it only on the right hardware.
2516:04:22 gmaxwell: I think some of the examples for the function overriding stuff used pragmas to push the compiler flags just for that function, so if I'm not misremembering apparently that works.
2616:47:07 wumpus: cfields: do you have time to look at integrating the leveldb sse42 stuff into our build system? if not, no problem I'll take a look at it, it doesn't seem particularly difficult though I wouldn't know the autoconf/automake incantation for compiling a file with different flags
2716:47:41 sipa: i think i can do it
2816:48:06 sipa: ah, nvm, you only want that single file to have -msse4
2916:48:25 wumpus: yep, that's the challenge 
3016:48:27 sipa: that i fon't know
3116:48:30 sipa: *don't
3216:50:55 sipa: probably needs a separate .a file?
3316:51:01 wumpus: sse4.2 isn't yet widely available enough to compile everything with it, unlike sse2 for example which every 64-bit processor has
3416:51:28 wumpus: every 64-bit *x86* processor sorry 
3516:51:50 wumpus: sipa: not sure, maybe the attributes can be added per .o file too
3617:00:23 sipa: wumpus: as leveldb upstream does not look like they'll pick up my barrier patch anytime soon, i'll submit it to our leveldb repo instead?
3717:02:07 wumpus: sipa: sounds good to me
