@sipa This was the alternate approach to #9700 that I came up with.
Here I tried to create a new wrapper class around CTransaction that would hold the validation-specific information, and then made CTransactionRef refer to that wrapper. However, because blocks contain a vector of CTransactionRef's, I didn't see a way to avoid this validation-specific wrapper from being in primitives/.
Also, the last commit here is designed to make sure that CTransactionRef remains thread-safe, by using std::call_once to calculate the segwit cache on demand. I really dislike adding synchronization primitives here but felt I needed to include something to address thread-safety concerns. However it occurred to me that the solution I tried to use in #9700 was to add a deserialization flag to control whether the cache was populated on construction, and perhaps that would be a better approach here as well? I'm open to whatever suggestions people have to make caching these hashes workable...