wallet: parallel fast rescan (approx 8x speed up with 8 threads) #34400

This PR speeds up wallet fast-rescan by executing the filter checks in parallel while ensuring that the filters are updated properly so that no output scripts are missed. Benchmarks, outlined below, show considerable improvement that tapers off at around 8x speedup at 8 threads.

Prerequisite PRs

• #34667 - modify the fast-rescan test to ensure that it fails when the filter is not updated properly.
• #34681 - refactor CWallet::ScanForWalletTransactions to prepare for the work in this PR.

Benchmarks

NOTE: to reproduce, please tune your system with pyperf system tune

EDIT Set up your node to use block filters by setting blockfilterindex=1 in your bitcoin.conf file and ensure your blockfilterindex is synced to the tip before attempting to reproduce.

Using the following command on mainnet with a wallet with no scripts and hyperfine version 1.20.0:

• On master

hyperfine --show-output --export-markdown master.md --export-json master.json  \
--sort command \
--runs 3 \
--prepare 'cmake --build build -j 20 && build/bin/bitcoind -blockfilterindex=1 -networkactive=0 && sleep 10 && build/bin/bitcoin-cli loadwalllet <wallet-name>' \
--conclude 'build/bin/bitcoin-cli stop && sleep 10' \
'build/bin/bitcoin-cli rescanblockchain 700000 900000'

• On this PR:

hyperfine --show-output --export-markdown results.md --export-json results.json  \
--sort command \
--runs 3 \
-L num_threads 1,2,3,4,5,6,7,8,9,16 \
--prepare 'cmake --build build -j 20 && build/bin/bitcoind -blockfilterindex=1 -networkactive=0 -walletpar={num_threads} && sleep 10 && build/bin/bitcoin-cli loadwalllet <wallet-name>' \
--conclude 'build/bin/bitcoin-cli stop && sleep 10' \
'build/bin/bitcoin-cli rescanblockchain 700000 900000'

<details> <summary>Table 1 was obtained on a machine with the following specifications</summary>

Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          46 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   20
  On-line CPU(s) list:    0-19
Vendor ID:                GenuineIntel
  Model name:             Intel(R) Core(TM) Ultra 7 265
    CPU family:           6
    Model:                198
    Thread(s) per core:   1
    Core(s) per socket:   1
    Socket(s):            20
    Stepping:             2
    CPU(s) scaling MHz:   41%
    CPU max MHz:          4800.0000
    CPU min MHz:          800.0000
    BogoMIPS:             4761.60

</details>

Table 1. Table of results of a mainnet benchmark of scanning 200000 blocks. The improvements seem to peak at 8x speedup despite the machine having an excess number of Cores (20).

Worst Case

This parallel fast rescan checks the filters for a series of blocks in parallel. One of the following cases can occur:

• No blocks matched; the wallet can skip this series of blocks
• The last block in the series matched; the wallet scans the last block and updates the filters if the wallet scripts have changed.
• One of the blocks before the last block matched; the wallet scans this block, updates the filters if new scripts are added, and rechecks filters for the blocks after the matched block. This is the worst-case scenario.

This Python script patch python script creates custom chains designed with payments at specified intervals to observe the performance of parallel fast rescan in two scenarios:

• the payments are made to the next index in the descriptor range, the expected case.
• the payments are made to the last index in the descriptor range, the worst case.

  <details> <summary> Fig 1 shown below was produced using this patch on a machine with the following specifications</summary>

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             48 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      16
  On-line CPU(s) list:       0-15
Vendor ID:                   AuthenticAMD
  Model name:                AMD Ryzen 9 8945HS w/ Radeon 780M Graphics
    CPU family:              25
    Model:                   117
    Thread(s) per core:      2
    Core(s) per socket:      8
    Socket(s):               1
    Stepping:                2
    Frequency boost:         enabled
    CPU(s) scaling MHz:      63%
    CPU max MHz:             5263.0000
    CPU min MHz:             400.0000

</details>

<img width="4760" height="5350" alt="benchmark_comparison" src="https://github.com/user-attachments/assets/9bb8536c-ef05-454a-93f1-e2d929da840f" />

Fig 1. Time to scan a 5000-block chain on Regtest with payments at varying intervals, comparing parallel Fast Rescan against serial Fast Rescan (baseline). Parallel Fast Rescan outperforms serial Fast Rescan at longer payment intervals, in both the expected and worst case. The Slow Rescan graph is included separately to check whether this PR causes any regression in Slow Rescan performance. The gap between the worst-case and expected-case runtimes on the Slow Rescan graph comes from the worst case triggering a KEYPOOL_SIZE TopUp each time a new address is found on-chain. These benchmarks use this commit as a baseline instead of master, since master lacks the -walletpar config parameter. All materials for this custom benchmark are available here.

Although not explicitly checked with Valgrind, hyperfine reported that memory usage stayed the same across all runs. I'm not sure to what degree Hyperfine's memory usage report can be trusted, but the PR limits the number of block hashes that can be held in memory for processing to 1000 (not configurable by the user).

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/34400.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline and AI policy for information on the review process.

If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #bitcoin-core/gui/954 (Add dialog to select change output when bumping fee by pablomartin4btc)
• #35752 (wallet: make encryption state updates atomic by l0rinc)
• #35716 (wallet: Replace mapWallet and wtxOrdered with a boost::multi_index by achow101)
• #34909 (wallet, refactor: modularise wallet by extracting out legacy wallet migration by rkrux)
• #34907 (wallet, test: make wallet_fast_rescan robust by rkrux)
• #34861 (wallet: Add importdescriptors interface by polespinasa)
• #34681 (wallet: move rescan logic into ChainScanner and wallet/scan by Eunovo)
• #33392 (wallet, rpc: add UTXO set check and incremental rescan to importdescriptors by musaHaruna)
• #32857 (wallet: allow skipping script paths by Sjors)
• #30343 (wallet, logging: Replace WalletLogPrintf() with LogInfo() by ryanofsky)
• #29278 (Wallet: Add maxfeerate wallet startup option by ismaelsadeeq)
• #27865 (wallet: Track no-longer-spendable TXOs separately by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

<!--85328a0da195eb286784d51f73fa0af9-->

🚧 At least one of the CI tasks failed. _{Task test max 6 ancestor commits: https://github.com/bitcoin/bitcoin/actions/runs/21320629356/job/61369934184} _{LLM reason (✨ experimental): Compilation failed due to an unused private member (m_thread_pool) in wallet.h being treated as an error under -Werror.}

<details><summary>Hints</summary>

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

</details>

I would have expected rescanning to be I/O bound rather than CPU, in which case parallelization could make things worse (more random seeking). Have you benchmarked this on a non-SSD?

I would have expected rescanning to be I/O bound rather than CPU, in which case parallelization could make things worse (more random seeking). Have you benchmarked this on a non-SSD?

Fast rescan checks block filters, which involves considerable hashing. This PR parallelises the checking of block filters, and my benchmarks show considerable improvements in rescan speeds with block filters. Slow rescan, which is I/O bound, remains the same. I expect the speedup to be transferable to non-SSD machines, but I haven't benchmarked this.

#33689 has been merged; the cherry-picked Threadpool commit has been removed.

I like the PR conceptually but I think it would be nice to first improve the current scanning code structure, then land the parallelization feature. The current code mixes a lot responsibilities. Similar to what you did in 633531614f69de49733642fd19cc9eba830fbdea, but into a separate PR so we can first land some good building blocks for this to happen.

Some quick pseudo-code structuring how I imagine it, which is similar to yours:

Scan(wallet, start_block_hash, end_block_hash, fn_filter_block, fn_process_block, interrupt) {
     it_current_hash = start_block_hash;

     while (it_current_hash != end_block_hash || interrupt) {
         // Skip block if needed (this function contains the BlockFilterIndex check if enabled)
         if (fn_filter_block(it_current_hash)) continue;
    
         // (this is more or less how we currently do it, we fetch the block and the next block hash at the same time)
         block = chain.find_block(it_current_hash).next_block(it_current_hash);
    
         // (inside this function the wallet will digest the block update the filter and save progress if needed)
         fn_process_block(block);
      }
}

Concept ACK

I like the PR conceptually but I think it would be nice to first improve the current scanning code structure, then land the parallelization feature. The current code mixes a lot responsibilities. Similar to what you did in 6335316, but into a separate PR so we can first land some good building blocks for this to happen.

I moved the test change into #34667 and the ScanForWalletTransactions refactor into #34681. I'll be putting this PR in draft while those PRs are open.

Concept ACK

I attempted to reproduce the benchmarks for this But I did not use hyperfine, I used time

System: AMD Ryzen 7 7700, 16 cores, 64GB RAM, mainnet, blocks 500000–900000, 3 runs averaged.

Baseline (881ebc4730ad15bd26e3e32ee3c9ba9d6e05552d): single-threaded fast rescan, -walletpar has no effect — consistently ~283s regardless of thread count.

Parallel scan (48154b87e2fb303ca0f3d46da29a3fbbf8758a06): fast rescan with threadpool parallelism enabled via -walletpar.

• Speedup scales well up to 16 threads, going from ~283s down to ~63s — a 4.5x improvement.
• Results are very consistent across runs (low variance), because the machine is bare metal and no other running processes apart from bitcoind are present during the benchmark runs.

Steps to reproduce

1. Restart bitcoind with -blockfilterindex=1 till it's done.
2. create a new wallet test
3. Stop the node
4. Save the script below as bench_script.sh

<details> <summary>script</summary>

#!/usr/bin/env bash
set -euo pipefail

COMMITS=(881ebc4730ad15bd26e3e32ee3c9ba9d6e05552d 48154b87e2fb303ca0f3d46da29a3fbbf8758a06)
THREADS=(1 2 4 8 16)
RUNS=3
WALLET_NAME="test"
DATADIR="$HOME/.bitcoin"
RESULTS_CSV="results.csv"
RESULTS_MD="results.md"

echo "commit,num_threads,run,seconds" > "$RESULTS_CSV"

for commit in "${COMMITS[@]}"; do
  short="${commit:0:7}"
  git checkout "$commit"
  cmake --build build -j 20

  for num_threads in "${THREADS[@]}"; do
    echo "=== commit=$short num_threads=$num_threads ==="

    for run in $(seq 1 $RUNS); do
      echo "  run $run/$RUNS"

      # start node
      build/bin/bitcoind -blockfilterindex=1 -walletpar="$num_threads" -daemonwait
      build/bin/bitcoin-cli loadwallet "$WALLET_NAME"

      build/bin/bitcoin-cli rescanblockchain 500000 900000

      # parse timing from debug log: "Rescan completed in 284737ms"
      ms=$(grep "Rescan completed in" "$DATADIR/debug.log" | tail -1 | grep -oP '\d+(?=ms)')
      seconds=$(python3 -c "print(f'{$ms / 1000:.3f}')")

      echo "$short,$num_threads,$run,$seconds" >> "$RESULTS_CSV"
      echo "  -> ${seconds}s"

      build/bin/bitcoin-cli stop
      # wait for clean shutdown
      while build/bin/bitcoin-cli ping 2>/dev/null; do sleep 1; done
      sleep 5
    done
  done
done

# markdown table with averages
python3 - "$RESULTS_CSV" "$RESULTS_MD" <<'EOF'
import sys, csv
from collections import defaultdict

infile, outfile = sys.argv[1], sys.argv[2]

rows = list(csv.DictReader(open(infile)))
groups = defaultdict(list)
for r in rows:
    groups[(r['commit'], r['num_threads'])].append(float(r['seconds']))

with open(outfile, 'w') as f:
    f.write("| commit | num_threads | run1 | run2 | run3 | mean |\n")
    f.write("|--------|-------------|------|------|------|------|\n")
    for (commit, threads), times in sorted(groups.items()):
        mean = sum(times) / len(times)
        runs = " | ".join(f"{t:.3f}" for t in times)
        f.write(f"| {commit} | {threads} | {runs} | {mean:.3f} |\n")

print(f"Written {outfile}")
EOF

echo "Done. Results in $RESULTS_CSV and $RESULTS_MD"

</details>

6. Make the script executable chmod +x bench_script.sh
7. Execute the script ./bench_script.sh
8. You can go a step further by using a top like btop https://github.com/aristocratos/btop to monitor the resource usage and how it will be well utilized when rescanning in parallel.

The current steps to reproduce in the description are stale because the commit hashes have changed since your force pushes.

I've looked only at the 48154b8 "wallet: check blockfilters in parallel" commit partially.

I have measured it on an Intel(R) Core(TM) i7-7700 CPU with 8 cores, 64Gi RAM and a HDD:

hyperfine --show-output \     
--export-markdown pr34400-results.md \     
--export-json pr34400-results.json \     
--sort command \      
--runs 3 \     
-L commit 68f030eef7b14d5ac6372a12864a813317ae0f4f,b62e9e739113fe58a86d23a00f8e9b1b561cd29f \     
-L num_threads 1,2,3,4,5,6,7,8,9,16 \     
--prepare 'git checkout -q {commit} && cmake -S . -B build && cmake --build build --parallel "$(nproc)" && build/bin/bitcoind -daemonwait -datadir=/mnt/my_storage/BitcoinData -connect=0 -listen=0 -dnsseed=0 -blockfilterindex=1 -wallet=bench_pr34400_20260416 -walletpar={num_threads} && build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 getwalletinfo > /dev/null' \     
--conclude 'build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData stop && sleep 10' \
'rescanblockchain 500000 900000'

<img width="2084" height="740" alt="image" src="https://github.com/user-attachments/assets/968632a3-cdc4-478f-b1e3-f908a1d862b4" />

<details><summary>Hyperfine relative speed comparison</summary>

3.42 ±  0.39  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 1)
3.26 ±  0.10  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 1)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 2)
1.81 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 2)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 3)
1.33 ±  0.03  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 3)
3.19 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 4)
1.20 ±  0.02  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 4)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 5)
1.12 ±  0.03  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 5)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 6)
1.05 ±  0.02  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 6)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 7)
1.00          build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 7)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 8)
1.03 ±  0.02  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 8)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 9)
1.02 ±  0.02  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 9)
3.20 ±  0.05  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = 68f030eef7b14d5ac6372a12864a813317ae0f4f, num_threads = 16)
1.03 ±  0.02  build/bin/bitcoin-cli -rpcwait -datadir=/mnt/my_storage/BitcoinData -rpcwallet=bench_pr34400_20260416 rescanblockchain 500000 900000 (commit = b62e9e739113fe58a86d23a00f8e9b1b561cd29f, num_threads = 16)

</details>

I've made some improvements which have increased the speedup from 5x at 8 threads to 8x at 8 threads. 8x speedup seems to be the peak for now. The results have been updated in the PR description, #34400#issue-3851923967

I re-ran my script with the recent changes.

Rescan of blocks 500000-900000 (mean of 3 runs).

I got a 6× speedup relative to master.