This adds a callback hook to the kernel library C header for getting the current script evaluation state on each script evaluation step. This can be used for debugging scripts and gives external applications a view into how Bitcoin Core does script evaluation.
The initial sketch for this work was prepared on rust-bitcoinkernel: https://github.com/sedited/rust-bitcoinkernel/tree/feature_script_debug . That branch also includes a very rudimentary script debugger showcasing the use case for this patch. Clankers seems to do a decent job at coding up prototypes for more featureful debuggers, which seems like a fun project to hack on if anybody else wants to give it a try.
This feature is gated behind the ENABLE_SCRIPT_TRACE flag. To compile with the feature enabled, pass -DENABLE_SCRIPT_TRACE=ON. If the feature is not manually enabled, the trace hooks are stubbed out.
The traces are divided into separate frames. RAII is used to guarantee that every script evaluation has at least its start and end state captured. Each instruction generates a unique frame as well.
The ScriptError in the VerifyScript invocation run when calling script_pubkey_verify is added in order to be able to surface the error code in the trace frames.
Another use case could also be using the frames to ensure that the script interpreter is actually doing what we think it is doing during the script tests. This might prevent unit test bugs in the future, i.e. when error conditions shadow actual behaviour, and could also be interesting for fuzzing the script interpreter.