Importmulti uses FindLatestBefore to find the height to rescan from, FindLatestBefore is using the block time. This results in a couple issues:
One issue is that miners are able to put times in the past on blocks: You could make an address at time T, send to it at T+1 and have that confirmed in a block with a timestamp T-1.
A bigger issue is that FindLatestBefore uses std::lower_bound which bisects to find the location, which requires that the iterator be ordered with respect to the comparison function. Block times are not ordered! I don't know what lower_bound is permitted to do when your data is not sufficiently ordered, but it's probably not good. (I could construct a plausible implementation which would infinite loop.)
Instead, we change to use GetMedianTimePast. Block times are ordered with respect to the mediantimepast. MTP advances with the consent of multiple miners so it is more likely to be sufficient for our purpose, and it is offset back a bit so it is more conservative.
The result might rescan a couple extra blocks, but that is no big deal. If the mediantime calculation is ever a performance concern we could cache it in the future.