wallet: start descriptor top up at range_start instead of index 0 #36245

pull kriss39 wants to merge 1 commits into bitcoin:master from kriss39:fix-topup-cache-from-range-start changing 2 files +43 −3
  1. kriss39 commented at 8:34 AM on September 14, 2026: contributor

    DescriptorScriptPubKeyMan::TopUpWithDB() expands the descriptor from m_max_cached_index + 1, which is 0 for a freshly imported descriptor regardless of its range_start. Importing a descriptor with "range": [N, M] therefore derives and watches every index in [0, M), so the cost of an import is O(M) rather than O(M - N) and the wallet treats scriptPubKeys below N as its own. Load() only expands [range_start, range_end), so those scriptPubKeys stop being IsMine the next time the wallet is opened, and a transaction picked up for one of them during the import rescan is no longer credited to the wallet.

    Since ParseDescriptorRange() only bounds the size of the range and not its start, "range": [2146483648, 2147483647] is accepted and expands 2^31 indexes into m_map_script_pub_keys before the RPC returns.

    Start the top up loop at range_start and record the index that was actually cached instead of counting from -1, so that Load() and TopUpWithDB() agree on m_max_cached_index for descriptors with a non-zero range_start. Before this change Load() left m_max_cached_index at range_end - range_start - 1 for such descriptors, which made the first top up after opening the wallet re-expand the stored range.

    Importing wpkh(xpub/0/*) with "range": [600000, 600009] into a blank watch-only wallet goes from 19.5s to 0.09s on this machine.

  2. DrahtBot added the label Wallet on Sep 14, 2026
  3. DrahtBot commented at 8:34 AM on September 14, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK fametrano

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    No conflicts as of last run.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. DrahtBot added the label Needs rebase on Oct 9, 2026
  5. wallet: start descriptor top up at range_start instead of index 0
    DescriptorScriptPubKeyMan::TopUpWithDB() expands the descriptor from
    m_max_cached_index + 1, which is 0 for a freshly imported descriptor
    regardless of its range_start. Importing a descriptor with
    "range": [N, M] therefore derives and watches every index in [0, M),
    so the cost of an import is O(M) rather than O(M - N) and the wallet
    treats scriptPubKeys below N as its own. Load() only expands
    [range_start, range_end), so those scriptPubKeys stop being IsMine the
    next time the wallet is opened, and a transaction picked up for one of
    them during the import rescan is no longer credited to the wallet.
    
    Since ParseDescriptorRange() only bounds the size of the range and not
    its start, "range": [2146483648, 2147483647] is accepted and expands
    2^31 indexes into m_map_script_pub_keys before the RPC returns.
    
    Start the top up loop at range_start and record the index that was
    actually cached instead of counting from -1, so that Load() and
    TopUpWithDB() agree on m_max_cached_index for descriptors with a
    non-zero range_start. Before this change Load() left m_max_cached_index
    at range_end - range_start - 1 for such descriptors, which made the
    first top up after opening the wallet re-expand the stored range.
    
    Importing wpkh(xpub/0/*) with "range": [600000, 600009] into a blank
    watch-only wallet goes from 19.5s to 0.09s on this machine.
    20a828b321
  6. kriss39 force-pushed on Oct 9, 2026
  7. DrahtBot removed the label Needs rebase on Oct 9, 2026
  8. fametrano commented at 10:00 PM on November 3, 2026: none

    tACK 20a828b3219844c1f1b7624915254bac0c4de8f9

    I built this commit and its parent 4bacf21a13 and, on regtest, imported wpkh(tpub/0/*) with "range": [1000, 1009] and timestamp 0 into a blank watch-only wallet, after paying the addresses at index 0, 999 and 1000. The addresses for indexes 0, 999, 1000, 1009, 1010, 1999 and 2000 come from bip32-descriptors, a BIP32 and descriptor library I maintain, so the boundary is checked without relying on deriveaddresses; Core agreed on all seven and on the checksum.

    On the parent, indexes 0 and 999 are mine right after the import, with 6 BTC trusted and three transactions; after unloadwallet/loadwallet both turn false and the wallet shows 2 BTC and one transaction. With this PR it shows 2 BTC and one transaction before and after the reload, index 1000 being the first owned. A wallet written by the parent and opened by this commit also shows 2 BTC, so existing wallets end up where a reload already puts them today. Importing [600000, 600009] with timestamp "now" took 21.0 s on the parent and 0.03 s here, in line with the description. The new functional test fails on the parent at the below-start check and passes here.

    Ralph's report takes the timing as the author's own and suggests a cache-end unit test; this adds the timing reproduced, the range boundary checked against a second derivation, and the upgrade path from a wallet the old code wrote. The script and its output: https://gist.github.com/fametrano/7071a6299ad44193c059564f3fbaa00c/74e2eb2449f7f7161524f3f8869e4a3fd7078555

Labels

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-16 01:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me