Before, we kept all valid CBlockIndex* entries in setBlockInvalidValid, but as it is only used to select potential new tips to activate, it suffices to only keep entries that are at least as good as the current tip.
This should save some memory in the stable synced state: 13 MiB on 64 bit, 8 MiB on 32 bit.