Parallel Fast Rescan (approx 5x speed up with 16 threads)

Eunovo commented at 7:46 pm on January 24, 2026: contributor

This PR uses @furszy’s ThreadPool from #33689 to implement parallel fast rescan. The ThreadPool commit has been cherry-picked into this PR for use.

This PR:

Adds a ThreadPool to the WalletContext to be shared by all the wallets
Adds -walletpar parameter to configure the number of threads to be used for parallel scanning
Updates the wallet_fast_rescan.py test to ensure it catches cases where the FastRescan filter wasn’t properly updated. This is crucial to ensure that changes in the PR do not cause newly added output scripts to be missed.
Refactors ScanForWalletTransactions to make the implementation of parallel scanning easier.
Implements parallel scanning

Benchmarks: NOTE: to reproduce, please tune your system with pyperf system tune

Using the following command on mainnet with a wallet with no scripts:

0hyperfine --show-output --export-markdown results.md --export-json results.json  \
1--sort command \
2--runs 3 \
3-L commit ef847e8,37d356b \
4-L num_threads 1,2,4,8,16 \
5--prepare 'git checkout {commit} && cmake --build build -j 20 && build/bin/bitcoind -walletpar={num_threads} && sleep 10' \
6--conclude 'build/bin/bitcoin-cli stop && sleep 10' \
7'build/bin/bitcoin-cli rescanblockchain 500000 900000'

I obtained the following results:

Command	Mean [s]	Min [s]	Max [s]	Relative
`build/bin/bitcoin-cli rescanblockchain 500000 900000 (commit = baseline, num_threads = ..)`	536.996 ± 0.722	536.257	537.701	4.64 ± 0.01
`build/bin/bitcoin-cli rescanblockchain 500000 900000 (commit = parallel_scan, num_threads = 1)`	540.210 ± 2.696	537.172	542.320	4.67 ± 0.03
`build/bin/bitcoin-cli rescanblockchain 500000 900000 (commit = parallel_scan, num_threads = 2)`	358.190 ± 0.515	357.675	358.706	3.10 ± 0.01
`build/bin/bitcoin-cli rescanblockchain 500000 900000 (commit = parallel_scan, num_threads = 4)`	230.217 ± 2.321	228.814	232.896	1.99 ± 0.02
`build/bin/bitcoin-cli rescanblockchain 500000 900000 (commit = parallel_scan, num_threads = 8)`	151.144 ± 1.748	149.506	152.984	1.31 ± 0.02
`build/bin/bitcoin-cli rescanblockchain 500000 900000 (commit = parallel_scan, num_threads = 16)`	115.642 ± 0.305	115.390	115.982	1.00

System information:

 0Architecture:             x86_64
 1  CPU op-mode(s):         32-bit, 64-bit
 2  Address sizes:          46 bits physical, 48 bits virtual
 3  Byte Order:             Little Endian
 4CPU(s):                   20
 5  On-line CPU(s) list:    0-19
 6Vendor ID:                GenuineIntel
 7  Model name:             Intel(R) Core(TM) Ultra 7 265
 8    CPU family:           6
 9    Model:                198
10    Thread(s) per core:   1
11    Core(s) per socket:   1
12    Socket(s):            20
13    Stepping:             2
14    CPU(s) scaling MHz:   41%
15    CPU max MHz:          4800.0000
16    CPU min MHz:          800.0000
17    BogoMIPS:             4761.60

Further benchmarks were performed using a python script with custom chains designed with payments at specified intervals, and the following graph was produced:

This graph was produced from a laptop with the following CPU specifications:

 0Architecture:                x86_64
 1  CPU op-mode(s):            32-bit, 64-bit
 2  Address sizes:             48 bits physical, 48 bits virtual
 3  Byte Order:                Little Endian
 4CPU(s):                      16
 5  On-line CPU(s) list:       0-15
 6Vendor ID:                   AuthenticAMD
 7  Model name:                AMD Ryzen 9 8945HS w/ Radeon 780M Graphics
 8    CPU family:              25
 9    Model:                   117
10    Thread(s) per core:      2
11    Core(s) per socket:      8
12    Socket(s):               1
13    Stepping:                2
14    Frequency boost:         enabled
15    CPU(s) scaling MHz:      63%
16    CPU max MHz:             5263.0000
17    CPU min MHz:             400.0000

All materials for the custom benchmarks can be found here.

Although not explicitly checked with Valgrind, hyperfine reported that memory usage stayed the same across all runs. I’m not sure to what degree hyperfine’s memory usage report can be trusted, but the PR limits the amount of block hashes that can be held in memory for processing to 1000 (not configurable by the user).

All benchmarks were performed against https://github.com/bitcoin/bitcoin/pull/34400/commits/37d356bbbe3efed3c7c9e64fae1bac3f4d0ec6eb instead of master because -walletpar is implemented here, and the benchmark scripts would otherwise break, or more complicated scripts would be required to accommodate master.

util: introduce general purpose thread pool 888cfdfbb0

DrahtBot commented at 7:46 pm on January 24, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/34400.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#33008 (wallet: support bip388 policy with external signer by Sjors)
#32636 (Split CWallet::Create() into CreateNew and LoadExisting by davidgumberg)
#32489 (wallet: Add exportwatchonlywallet RPC to export a watchonly version of a wallet by achow101)
#31260 (scripted-diff: Type-safe settings retrieval by ryanofsky)
#30343 (wallet, logging: Replace WalletLogPrintf() with LogInfo() by ryanofsky)
#27865 (wallet: Track no-longer-spendable TXOs separately by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot added the label CI failed on Jan 24, 2026

DrahtBot commented at 8:44 pm on January 24, 2026: contributor

🚧 At least one of the CI tasks failed. Task test max 6 ancestor commits: https://github.com/bitcoin/bitcoin/actions/runs/21320629356/job/61369934184 LLM reason (✨ experimental): Compilation failed due to an unused private member (m_thread_pool) in wallet.h being treated as an error under -Werror.

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

wallet: setup wallet threadpool

All wallets will use the same ThreadPool owned by the WalletContext.
ThreadPool::Submit() is threadsafe so there's no need for the use of
external synchronization primitives when submiting tasks.
All threads started by the ThreadPool will be destroyed as with the
ThreadPool and WalletContext.

37d356bbbe

wallet: test fast rescan filter is updated after TopUp

The fixed non-range descriptor address ensured that the fast-rescan filter would match all Blocks even if the filter wasn't properly updated.
This commit moves the non-range descriptor tx to a different block, so that the filters must be updated after each TopUp for the test to pass.

b944a0523b

wallet: move scanning logic to wallet/scan.cpp

Move rescan logic to new class to prepare for more code that
will be added for parallel rescan in the future commits.

CWallet::ScanForWalletTransactions impl is moved to scan.cpp
to prevent circular dependency of the form
"wallet/wallet -> wallet/scan -> wallet/wallet".

ce9039619f

wallet: refactor ChainScanner::Scan

Split scanning logic into blockhash read,
filter execution and block scanning to
prepare for future parallelisation commits.

940b478906

wallet: check blockfilters in parallel

Any operation that requires cs_wallet like GetLastBlockHeight()
and SyncTransaction must be done on the main thread because
cs_wallet is a RecursiveMutex and ScanForWalletTransactions
is also called from AttachChain() which already LOCKs cs_wallet.

Block hashes are read and pushed to m_blocks while block filters
are being check for other blocks.

When more block hashes cannot be read, the main thread will use
ThreadPool::ProcessTask() to join other workers and process queued
jobs, instead of wasting CPU styles constanly checking if a future
is ready.

Benchmarks show considerable improvement (approx 5x with 16 threads).

ef847e8bce

Eunovo force-pushed on Jan 25, 2026

DrahtBot removed the label CI failed on Jan 25, 2026

Parallel Fast Rescan (approx 5x speed up with 16 threads) #34400

Code Coverage & Benchmarks

Reviews

Conflicts