Modifies the script interpreter to represent stack elements internally using smart pointers, making duplication memory-cheap, and giving us caching keys which are cheap and guaranteed unique by the compiler. First part of a 2-part project to improve script interpreter performance in preparation for PQ signatures.
No consensus changes - a pure performance improvement refactor.
Motivation
We presently have MAX_SCRIPT_ELEMENT_SIZE = 520, but in the near future, motivated by the need for PQ signature schemes, we may want to introduce soft fork changes which increase the max stack element size for new output types well into the multi-kilobyte range. This may be perhaps as high as 10kb or more given the current landscape of PQ signature schemes' sizes.
Ongoing related work:
- https://github.com/SHRINCS/shrincs-bip/blob/main/leaf-version-0xC2.mediawiki#user-content-Relaxing_the_Max_Stack_Element_Size
- https://github.com/SHRINCS/shrincs-bip/pull/76
This poses a new problem to today's implementations if deployed naively.
Today's Bitcoin script interpreter moves around raw mutable byte vectors to represent stack elements. When those elements are duplicated, the underlying byte vectors are deep-copied. When hash opcodes execute, they always invoke the corresponding hash function and fully ingest the top stack element. Duplication and hashing cost only a single weight unit in tapscript, with no upper limit on how many times they can be executed, aside from implicit limits due to the 10,000 byte script size maximum.
If we were to simply up the stack element size limit from 520b to 10kb, then we exacerbate an existing DoS vector on bitcoin nodes: Push a big (today 520b, maybe someday 10kb) stack element, then duplicate and hash it over and over. Worst case would be a script like this:
<big_blob> DUP DUP [DUP3 RMD160 DROP RMD160 DROP RMD160 DROP]...
...with the [...] part repeating up to the script size limit. RMD160 is used because it lacks hardware acceleration support and so performs significantly worse than even HASH256 or HASH160, making a more effective CPU attack.
This costs the attacker only ~20000 weight units (10kb for the script, 10kb for the stack element), but forces all validators to evaluate OP_RIPEMD160 about 4284 times, each on the same 10kb input. That's about 672588 RMD160 compressions in software.
I ran a benchmark (not in this PR) using this style of script with 520b and 10kb script elements.
- 520b: validation takes about 3 milliseconds
- 10,000b: validation takes about 52 milliseconds
This PR
(idea credit: EthanHeilman)
To address this risk, we would want to implement a hash cache for all the hashing opcodes. However, to implement such a cache safely in consensus, we need extremely robust cache keys. A collision in the cache key would be a consensus bug because the cache would return the wrong hash for a given input. But we can't cryptographically hash the inputs to compute a cache key... that is the same work we are trying to avoid with the cache.
Instead we can kill two birds with two PRs.
- Replace stack elements with ref-counting pointers, so OP_DUP et al duplicate pointers and not the underlying raw bytes. :arrow_backward: THIS PR
- Implement a hash cache keyed by the stack element pointers. Each cache entry holds a handle to the key pointer so that the key doesn't get dropped and the pointer reallocated for a new stack element (which would cause a consensus bug). Cache key uniqueness is guaranteed by the compiler. :arrow_backward: Future PR
Stage 1 reduces the asymmetric memory inflation attack allowed by OP_DUP et al. Stage 2 fixes the asymmetric CPU DoS attack. To be clear, neither stage fully prevents an attacker from uploading a series of large stack elements, and duplicating/hashing them. But we do force attackers to pay for those operations with fees and block space.
I have both stage 1 and 2 implemented locally and can show benchmarks to prove the effect of the cache, if desired. I could have submitted it all as one PR but both stages are atomic and could do well with separate review.