Extreme sync undo on "Disk space is low!" error #11724

Using 0.15.0.1 from BitcoinCore.org, running on a fast x86 system with SSD and 16 GB of RAM.

While syncing this instance my UpdateTip was around 2017-06-30 when I ran out of disk space so the daemon auto shut down. I knew about this feature so I thought to go ahead with sync even though I knew I may be few GB short.

After I restarted, resumed UpdateTip entries were from early 2015-03, so it rolled back 27 months.

Maybe that's due to the fact that the node is fast and my dbcache large so I ran out of space before bitcoind could safely shutdown, but still - I find this 27 month rollback excessive. Maybe the auto-shutdown feature could shutdown in a more timely manner.

Config: dbcache=8000 txindex=1

Debug.log:

https://pastebin.com/raw/gEZz3zDD

It's probably due to the large dbcache. If the dbcache didn't flush before the shutdown, then the database would not have updated with the state stored in the dbcache.

I think so too. I wonder if it deducts dbcache from "disk space remaining" when it determines when to shut down. It seems https://github.com/bitcoin/bitcoin/blob/0.15/src/validation.cpp#L3435 initiates shutdown at 50MB remaining. Another thing I wonder about is if there are some scheduled "write cache flush" index checkpoint events that didn't kick in on time due to my node's fast speed of syncing (30 MB/s).

Edit: I forgot to mention one important point: I had another write-dominant workload running on the system, which probably messed up the calculation. I just did a test of shutdown time with dbcache=1000 vs dbcache=8000 and it was 0m20s vs 6m00s (in both cases, there was no other workload). It's possible that the other workload combined with the long shutdown time made bitcoind unable to shutdown while there was still some free space on disk. If this is a situation that makes this issue invalid, I'll close it.

wonder if it deducts dbcache from "disk space remaining" when it determines when to shut down.

It doesn't. Although it'd be nice to take database growth into account, that heuristic wouldn't be correct in most cases, because just as likely data in the dbcache exists on disk already.

I do think the 50MB threshold is on the low side.

Another thing I wonder about is if there are some scheduled "write cache flush" index checkpoint events that didn't kick in on time due to my node's fast speed of syncing (30 MB/s).

There are no such events. To achieve the highest possible speed, it flushes to disk only when the dbcache is full or the node is shut down. To flush more often, set a lower dbcache.

I see. Considering the 50MB advance notice and a 30MB/s download speed, there's basically no time left to flush DB cache of any meaningful size (but as you say at least I can limit my losses at the cost of a slower sync speed). It'd be nice to have this value exposed as a config/runtime parameter or hardcoded to a higher value (such as dbcache + 300 MB).