Add output script index and corresponding script history RPC #36629

Add a new index txoscriptindex which indexes all output scripts. Introduce a new getscripthistory RPC that returns all transactions that create outputs corresponding to the provided scripts, and optionally the transactions that spend those outputs if txospenderindex is also configured.

This has been a long-requested feature, attempted in previous PRs (#2802, #3652, #6835, #10370, #14053). Some concerns raised were about the ability for an index like this to scale. I think the new techniques being used to build the indexes (https://github.com/bitcoin/bitcoin/pull/35531, #36002) alleviate this concern. This index can be synced in a few hours and the current disk usage is ~52 GB. This is less than the ~66 GB txindex disk usage in v31 and earlier. This index can also be used with a pruned node, so a full node can be supported in < 100 GB of disk space. Missing blocks are returned by hash and can be requested from peers using getblockfrompeer.

Some other previous concerns were more conceptual, that this type of index could encourage address reuse and chain analysis. IMO this ship has sailed, and the current status quo of not shipping an address index is turning off users who expect this functionality. Workarounds involve installing, configuring, and syncing other third-party software, making the UX of using Bitcoin Core in a self-sovereign way unnecessarily difficult.

The index hashes each output script with SHA256, then applies salted SipHash-1-3-UJ and stores a five-byte prefix. This is to support the popular Electrum protocol, which supplies only the script hash. The RPC can also look up by address or descriptors. Ranged descriptors are also supported, so a single RPC call can return the entire history of a receive/change xpub pair.

Because there are some scripts with millions of outputs, we introduce a new OrderedVarInt which serializes the block height after the script hash prefix. This lets LevelDB seek directly to the prefix + block height, so we can page by block height range.

To prevent DoS concerns by supplying an address with millions of outputs, the RPC tracks a byte budget of reads and throws immediately if too many reads are attempted. The error message instructs the user to narrow the block range.

Improvements to txospenderindex (https://github.com/bitcoin/bitcoin/pull/36480) will also allow us to use that pruned. The end result would allow users to run with all optional indexes (including this one) in < 200 GB of disk space. Of course a larger pruned setting would be desired if many historical block lookups were being requested.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

External sites

• Code coverage & benchmarks: https://corecheck.dev/bitcoin/bitcoin/pulls/36629
• Ralph review <img src=https://review.fish.foo/assets/bitcoin/bitcoin/36629.png alt="Ralph review status"> https://git.fish.foo/bitcoin/bitcoin/pulls/36629#author-ralph

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline and AI policy for information on the review process.

If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #36618 _{<img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36618.svg">} (test: Wait for index sync in wallet_fast_rescan.py by maflcko)
• #36603 _{<img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36603.svg">} (Improve loadtxoutset performance by ajtowns)
• #36598 _{<img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36598.svg">} ([Demo] validation: remove AssumeUTXO by l0rinc)
• #33117 _{<img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/33117.svg">} (Interfaces: Expose UTXO Snapshot Loading and Add Progress Notifications by D33r-Gee)
• #31252 _{<img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/31252.svg">} (rpc: print P2WSH and P2SH redem Script in getrawtransaction and getblock by polespinasa)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

LLM Linter (✨ experimental)

Possible places where named args for integral literals may be used (e.g. func(x, /*named_arg=*/0) in C++, and func(x, named_arg=0) in Python):

• src/test/txoscriptindex_tests.cpp: index.Find(script_hash, -1, 95, remaining_bytes)
• src/test/txoscriptindex_tests.cpp: index.Find(script_hash, 95, 90, remaining_bytes)
• src/test/txoscriptindex_tests.cpp: index.Find(script_hash, 0, 102, remaining_bytes)
• src/test/txoscriptindex_tests.cpp: index.Find(script_hash, 0, 100, remaining_bytes)
• src/test/txoscriptindex_tests.cpp: index.Find(script_hash, 1, 2, remaining_bytes)

Possible places where comparison-specific test macros should replace generic comparisons:

• src/test/ordered_varint_tests.cpp: The BOOST_CHECK_THROW checks for truncated encodings, invalid prefixes, oversized values, and uint32 overflow only verify std::ios_base::failure. Prefer BOOST_CHECK_EXCEPTION(..., std::ios_base::failure, HasReason{"..."}) to check the expected failure message.
• src/test/txoscriptindex_tests.cpp: The BOOST_CHECK_THROW checks for invalid encodings, empty values, and oversized values only verify std::ios_base::failure. Prefer BOOST_CHECK_EXCEPTION(..., std::ios_base::failure, HasReason{"..."}) to check the expected failure message.
• test/functional/rpc_getscripthistory.py: assert output["spent"] is spent → Prefer assert_equal(output["spent"], spent).
• test/functional/rpc_getscripthistory.py: assert sum(sizes[:53]) < limit < sum(sizes) → Prefer assert_greater_than(limit, sum(sizes[:53])) and assert_greater_than(sum(sizes), limit).
• test/functional/rpc_getscripthistory.py: assert sum(stripped_sizes) < limit → Prefer assert_greater_than(limit, sum(stripped_sizes)).
• test/functional/rpc_getscripthistory.py: assert sum(sizes[:43]) + mempool_bytes < limit < sum(sizes[:43]) + sum(sizes[:10]) + mempool_bytes → Prefer assert_greater_than(limit, sum(sizes[:43]) + mempool_bytes) and assert_greater_than(sum(sizes[:43]) + sum(sizes[:10]) + mempool_bytes, limit).
• test/functional/rpc_getscripthistory.py: assert sum(sizes[:53]) + 400_000 < limit < sum(sizes[:53]) + 500_000 → Prefer assert_greater_than(limit, sum(sizes[:53]) + 400_000) and assert_greater_than(sum(sizes[:53]) + 500_000, limit).

^{2026-10-25 20:15:31}

<!--85328a0da195eb286784d51f73fa0af9-->

🚧 At least one of the CI tasks failed. _{Task ASan + LSan + UBSan + integer: https://github.com/bitcoin/bitcoin/actions/runs/39994088566/job/120552982572} _{LLM reason (✨ experimental): CI failed because ordered_varint_tests triggered UBSan for an implicit uint64_t-to-uint8_t truncation in index/ordered_varint.h:55.}

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

Concept ACK

Concept ACK

Strong Concept ACK

Concept ACK.

Reviewed 614302e7d14a16fe28a2e8945fa99165693fcf94. I built it and ran txoscriptindex_tests, baseindex_tests, rpc_getscripthistory.py, rpc_misc.py, feature_init.py and feature_assumeutxo.py, all passing. I also did a manual regtest run: address, Electrum scripthash, ranged wpkh() descriptor, verbosity 0/1/2, and spender linking. Results matched expectations. My comments are mostly about structure:

1. The lookup budget does not count block and undo reads. At verbosity 2, tx_to_json reads the full block and its undo data for every block that holds a non-coinbase match (rpc/blockchain.cpp:4116). Step 10 also reads a full block whenever two or more extra spenders share one (rpc/blockchain.cpp:4064). Neither read is charged. I ran this on regtest with 150 matches, each in its own block padded to ~60 kB. Bytes read by bitcoind during one call (rchar):

On mainnet that means one full block plus its undo data per matching block. A query that fits in the 50 MiB budget can therefore read far more than 50 MiB. Either charge these reads, or say in the help text that the budget does not cover them.

2. OrderedVarInt could be BigEndianFormatter<3>. Big-endian already sorts by height, and the same key already uses it for the tx offset. Both encodings are 3 bytes for heights 16,512 through 2,113,663 (roughly until 2048). After that, BE3 stays at 3 bytes while OrderedVarInt grows to 4, and BE3 goes up to height 16,777,215. I tried it locally. It removes ordered_varint.h and its tests (+4/−210), and the PR's unit and functional tests still pass:

+    static constexpr uint32_t HEIGHT_SIZE{3};
 ...
-                  Using<OrderedVarIntFormatter>(obj.height),
+                  Using<BigEndianFormatter<HEIGHT_SIZE>>(obj.height),

3. Ordering spenders doesn't need a block read. txospenderindex keys already store a CDiskTxPos, and its nTxOffset grows with the transaction's position in the block. I added it to TxoSpender and sorted extra spenders by (height, tx_offset). That removes the grouping loop and the GetBlockChecked call (+6/−20). test_confirmed_transaction_order still passes. It fails if the offset is dropped from the comparison, so the test does cover this.

4. Find could take the whole set of script hashes. Today the RPC calls Find once per hash and merges results by (height, tx_offset). A transaction paying two scripts of one descriptor is read and charged twice; rpc_getscripthistory.py relies on this ("repeated reads of overlapping transactions exceed the work budget"). A multi-hash Find would read each candidate once and drop both ScriptHistoryMatch::tx_offset and the by_position merge. The budget would then trip later than it does now, never earlier, so that test case would need updating. Returning pruned blocks as their own set instead of null-tx placeholders would also remove the duplicated branch at index/txoscriptindex.cpp:163 and :174, and the matching filter in the RPC.

5. One budget, one error. Index lookups charge a uint64_t& and fail with "Script history index lookup limit exceeded…". Extra transaction bodies throw a std::runtime_error with "Script history exceeds the … byte budget" (rpc/blockchain.cpp:3933). Both enforce the same limit through two code paths with two messages. The index-layer message also names RPC options (index/txoscriptindex.cpp:138). A small budget type that both layers charge, with the user-facing wording kept in the RPC, would be simpler.

6. Could the RPC go in its own file? blockchain.cpp grows by ~530 lines to 4,238. The handler is one ~400-line lambda that opens with a 12-step index comment. src/rpc/scripthistory.cpp with one function per step would be easier to read and review.

7. TxToJSON is now declared in rawtransaction_util.h. That header belongs to bitcoin_common, which the wallet also links. TxToJSON takes a Chainstate and is still defined in rawtransaction.cpp in bitcoin_node. Declaring it in a node-side header would keep that boundary clean.

Nits:

• The mempool scan matches against scripts or script_hashes depending on how the query was given (rpc/blockchain.cpp:3986). Hashing everything up front would leave one set and remove the branch.

• When range is given, the descriptor is parsed twice: once at rpc/blockchain.cpp:3896 and again inside EvalDescriptorStringOrObject.

• RawTransactionDoc changes the getrawtransaction help text:

  • blockhash moves below vsize_adjusted and gains "(omitted for mempool transactions)".
  • The hex description changes.
  • At verbosity 2, fee moves to the end.

  Probably fine, but worth a line in the commit message.

Disclosure: AI (Claude Opus 5.5) was used to find these issues and to draft this review.

Concept ACK, thanks!

Concept ACK. I skimmed the code but plan to take a deeper look. Commented 2 nits.