When ScanForWalletTransactions processes a block containing both a transaction to a key just outside the look-ahead pool and a pool-expanding transaction that triggers TopUp, and the former appears first in vtx order, the wallet misses the earlier transaction — leaving the balance wrong with no indication a second scan is needed. The fix retries any transaction that doesn't match on a full pass, dropping it once it does, until a pass resolves nothing new — so a transaction can be recognized once something else in the block makes it relevant, regardless of why.
The first commit demonstrates the incorrect behaviour; the second contains the fix; the third adds a benchmark guarding the fix's performance on a large, mostly-foreign block with an embedded multi-level cascade.
<details> <summary>Bug details...</summary> <br>
ScanForWalletTransactions iterates over a block's transactions in the order the miner placed them (vtx order). A descriptor wallet maintains a look-ahead pool of pre-derived keys so it can recognise incoming payments. When a payment arrives at a key near the pool boundary, MarkUnusedAddresses → TopUp
extends the pool.
If within a single block:
- Tx_lookahead pays to key index
N(just outside the current pool[0, N-1]), AND - Tx_expand pays to key index
N-1(last in pool — triggers TopUp, extending pool to[0, 2N-1])
and Tx_lookahead appears at an earlier vtx position than Tx_expand, the scan misses Tx_lookahead:
- Tx_lookahead processed: pool =
[0, N-1]→IsMinereturns false → missed - Tx_expand processed: key
N-1found → TopUp fires → pool =[0, 2N-1]
FastWalletRescanFilter::UpdateIfNeeded() fires only at the start of the next block iteration — there is no mechanism to re-examine Tx_lookahead within the same scan pass.
The transaction is not permanently lost — because Tx_expand extended the pool as a side effect, a second explicit rescanblockchain call recovers Tx_lookahead. But the wallet shows a wrong balance with no indication a second scan is needed.
This bug does not affect live blockConnected when both transactions pass through the mempool first: transactionAddedToMempool(Tx_expand) pre-extends the pool before the block arrives, so vtx ordering does not matter. It only manifests in ScanForWalletTransactions (rescan paths) for descriptor wallets.
</details>
<details> <summary>Affected callers...</summary> <br>
Callers affected (all use ScanForWalletTransactions):
rescanblockchainimportdescriptors(viaRescanFromTime)restorewalletand wallet migration watchonly/solvable wallets (viaAttachChain)
</details>
<details> <summary>Fix details and performance...</summary> <br>
Fix: during the per-block vtx loop, keep a list of still-unmatched vtx positions. Retry it each pass, dropping positions once SyncTransaction() returns true for them, until a full pass resolves nothing new. A transaction becomes resolvable once something else earlier in the same block changes its relevance — either a pool expansion (TopUp from MarkUnusedAddresses) now covering a key it pays, or a previously-unknown parent transaction becoming known so a spend of it is recognized. The unmatched set never grows, so this terminates without a safety cap.
This also covers two cases a shrinking-prefix rescan (an earlier version of this fix) did not: a prefix-recovered receive that triggers a second pool expansion past where the prefix had already shrunk to (see Part 5), and a same-block spend of a look-ahead receive only recognized later in the same pass via a parent-tx lookup rather than a pool expansion (see Part 6, molnard's finding — see Notes below).
Performance: the retry loop costs at most one extra pass for any block containing a wallet-relevant transaction during a rescan, in typical shapes. The worst case is a large, mostly-foreign block with a multi-level cascade embedded in it. bench_bitcoin -filter=RescanCascadeInLargeBlock guards that scenario against regression.
</details>
<details> <summary>Test coverage...</summary> <br>
Regression test: test/functional/wallet_rescan_intrablock_ordering.py
- Part 1: loadwallet rescan — Tx_lookahead found after fix.
- Part 2: importdescriptors full rescan from genesis — Tx_lookahead found after fix.
- Part 3: second rescanblockchain recovers Tx_lookahead (pre-fix behaviour, kept as sanity check).
- Part 4: two-level cascade — two successive TopUp events fire in one block, both lookahead txs found, validating the bounded multi-pass loop.
- Part 5: a prefix-recovered receive triggers a second TopUp past where a shrinking-prefix rescan would have already stopped.
- Part 6: a same-block spend of a look-ahead receive only recognized later in the same pass (molnard's finding).
</details>
<details> <summary>Manual reproduction (<code>regtest</code>)</summary>
bitcoind -regtest -keypool=5 -fallbackfee=0.0001 -daemon
bitcoin-cli -regtest -named createwallet wallet_name=funding
bitcoin-cli -regtest generatetoaddress 102 $(bitcoin-cli -regtest -rpcwallet=funding getnewaddress)
bitcoin-cli -regtest -named createwallet wallet_name=test
RECV_DESC=$(bitcoin-cli -regtest -rpcwallet=test listdescriptors \
| jq -r '[.descriptors[] | select(.internal==false and .range!=null)][0].desc')
END_RANGE=$(bitcoin-cli -regtest -rpcwallet=test listdescriptors \
| jq '[.descriptors[] | select(.internal==false and .range!=null)][0].range[1]')
ADDR_EXPAND=$(bitcoin-cli -regtest deriveaddresses "$RECV_DESC" "[$END_RANGE,$END_RANGE]" | jq -r '.[0]')
ADDR_LOOKAHEAD=$(bitcoin-cli -regtest deriveaddresses "$RECV_DESC" "[$((END_RANGE+1)),$((END_RANGE+1))]" | jq -r '.[0]')
bitcoin-cli -regtest unloadwallet test
TXID1=$(bitcoin-cli -regtest -rpcwallet=funding sendtoaddress "$ADDR_LOOKAHEAD" 0.001)
TXID2=$(bitcoin-cli -regtest -rpcwallet=funding sendtoaddress "$ADDR_EXPAND" 0.001)
bitcoin-cli -regtest generateblock "raw(51)" "[\"$TXID1\",\"$TXID2\"]"
# First scan — TXID1 missing, wrong balance
bitcoin-cli -regtest loadwallet test
bitcoin-cli -regtest -rpcwallet=test listtransactions
# Second scan — TXID1 recovered
bitcoin-cli -regtest -rpcwallet=test rescanblockchain
bitcoin-cli -regtest -rpcwallet=test listtransactions
# With the fix (this PR's tip), both transactions are found on the first scan.
bitcoin-cli -regtest loadwallet test
bitcoin-cli -regtest -rpcwallet=test listtransactions
</details>
Notes:
Originally noted by furszy in #31629 (comment); his "case 2" describes a related but harder inter-block variant: new blocks arriving via
blockConnectedduring an active rescan with a not-yet-expanded pool. That problem remains a potential follow-up. This PR fixes the simpler intra-block sub-case where the miss happens entirely within a single block's vtx loop, with no concurrent block arrivals required.Perhaps the issue is related to the symptom was previously reported in #19808 but could not be reproduced at the time and was closed without a fix.
molnard found a related gap the shrinking-prefix rescan didn't close: if a parent tx pays a look-ahead key and a later tx in the same block spends that parent's output, and the spend sits at or after the block's last pool expansion, the prefix rescan recovers the parent but never revisits the spend. jeanpablojp separately found a second gap in the same shrinking-prefix approach — a prefix-recovered receive that triggers a further pool expansion past where the prefix had already shrunk to (Part 5) — and proposed and benchmarked the fix now in this PR: retry based on each tx's
SyncTransaction()result until a full pass causes no change, rather than tracking a shrinking prefix. Both gaps are now fixed in this PR (Parts 5 and 6).
