There are a few issues referencing regtest mining speed (eg #5405, #4793), but I thought this observation made more sense as a separate issue:
GetNextWorkRequired has an overflow bug that affects the regtest difficulty calculation. The way the arithmetic works in that function is to first multiply the prior difficulty by nActualTimespan and then divide by TargetTimespan. (TargetTimespan is 14 * 24* 60 * 60 = 1,209,600; nActualTimespan is bounded to be within a factor of 4 of TargetTimespan.)
Since the regtest initial difficulty is so large, on the first retarget we end up overflowing on the multiply, and so the difficulty increases dramatically after the division, regardless of the timestamps on the regtest blocks.
I don't believe this can affect mainnet or testnet, because the initial difficulties are small enough that no overflow can occur.