The code currently assumes that once a block file is ’left’, it and its corresponding rev file can be flushed, and won’t be written to any further.
(Flushing here additionally means to get rid of pre-allocated unused space.)
This is the case for the bl(oc)k files, but it turns out to not be the case for the rev files. During an IBD up to ~200k blocks, the following excerpt (one of several) was encountered (grepping out irrelevancy):
0[...]
12020-01-07T14:19:09Z Pre-allocating up to position 0x7000000 in blk00013.dat
22020-01-07T14:19:13Z Pre-allocating up to position 0x8000000 in blk00013.dat
32020-01-07T14:19:14Z Pre-allocating up to position 0xf00000 in rev00013.dat
42020-01-07T14:19:16Z Leaving block file 13: CBlockFileInfo(blocks=1099, size=134068483, heights=183531...185655, time=2012-06-08...2012-06-22)
52020-01-07T14:19:16Z Pre-allocating up to position 0x1000000 in blk00014.dat
62020-01-07T14:19:17Z Pre-allocating up to position 0x100000 in rev00014.dat
72020-01-07T14:19:20Z Pre-allocating up to position 0x1000000 in rev00013.dat
82020-01-07T14:19:20Z Pre-allocating up to position 0x2000000 in blk00014.dat
92020-01-07T14:19:24Z Pre-allocating up to position 0x1100000 in rev00013.dat
102020-01-07T14:19:26Z Pre-allocating up to position 0x3000000 in blk00014.dat
112020-01-07T14:19:27Z Pre-allocating up to position 0x1200000 in rev00013.dat
122020-01-07T14:19:30Z Pre-allocating up to position 0x200000 in rev00014.dat
132020-01-07T14:19:33Z Pre-allocating up to position 0x300000 in rev00014.dat
142020-01-07T14:19:35Z Pre-allocating up to position 0x400000 in rev00014.dat
152020-01-07T14:19:37Z Pre-allocating up to position 0x500000 in rev00014.dat
162020-01-07T14:19:39Z Pre-allocating up to position 0x600000 in rev00014.dat
172020-01-07T14:19:42Z Pre-allocating up to position 0x700000 in rev00014.dat
182020-01-07T14:19:43Z Pre-allocating up to position 0x4000000 in blk00014.dat
192020-01-07T14:19:45Z Pre-allocating up to position 0x800000 in rev00014.dat
202020-01-07T14:19:49Z Pre-allocating up to position 0x900000 in rev00014.dat
21[...]
But the only time we flush (and truncate the files to their correct size) is when we leave a block, and we already left block 13 here, so the rev files above end up with pre-allocated space that is never used. This is related to #17827.
This seems to repeat itself for every file that ended up larger than expected in #17887 (comment). (Note: #17887 is not a fix for this issue. #17887 makes the impact much smaller by fixing a separate blow-up bug, but it does not address re-flushing re-opened rev files.)