txindex: report iterator errors from FindTx #36789

pull musaHaruna wants to merge 1 commits into bitcoin:master from musaHaruna:txindex-report-iterator-errors-master changing 3 files +6 −0
  1. musaHaruna commented at 9:27 AM on November 6, 2026: contributor

    This is a follow-up to #35728.

    While reviewing that PR, I noticed that FindTx() could return nullopt even when LevelDB had encountered an error. I first confirmed that a transaction could be retrieved, then deliberately damaged part of the test’s temporary txindex database. LevelDB detected the corruption, but the same lookup returned nullopt instead of throwing. Restoring the original data made the lookup work again.

    The candidate scan checks whether the iterator has a valid entry, but it does not check its error status. An invalid iterator can mean that the scan reached the end normally, or that it stopped because of a database error. This means a failed scan can be treated as an ordinary missing transaction.

    This adds CDBIterator::CheckStatus() and calls it after the candidate scan, before choosing a result. Any iterator error goes through the existing HandleError() path and throws dbwrapper_error. A normal missing transaction still returns nullopt.

    I added a reproducer on my branch. The first commit adds a test that fails before the fix. The second commit applies the fix, and the same test passes.

  2. DrahtBot commented at 9:27 AM on November 6, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process. A summary of reviews will appear here.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  3. in src/dbwrapper.h:158 in 1ea57a05a8
     154 | @@ -155,6 +155,9 @@ class CDBIterator
     155 |  
     156 |      bool Valid() const;
     157 |  
     158 | +    /** Throw dbwrapper_error if the iterator encountered an underlying database error. */
    


    maflcko commented at 9:36 AM on November 6, 2026:
        /// Throw dbwrapper_error if the iterator encountered an underlying database error.
    

    nit: Could use the shorter doxygen format, like the other functions in this file (HasKeyStartingWith, CompactFull, ...)

  4. maflcko commented at 9:38 AM on November 6, 2026: member

    this should be cosmically rare, so I am unsure about adding a test, but it could make sense to drop one based on plyvel into a comment, so that reviewers can use it if they want?

  5. musaHaruna force-pushed on Nov 8, 2026
  6. musaHaruna commented at 10:22 AM on November 8, 2026: contributor

    Thanks for the review @maflcko!

    Pushed the docs fix to use //!, matching the other comments in the file.

    I also added a reproducer on my branch, so reviewers can try it. The first commit adds a test that reproduces the failure, and the second applies the fix and makes the same test pass.

    Updated the PR description to link to the reproducer as well.

  7. musaHaruna force-pushed on Nov 8, 2026
  8. DrahtBot added the label CI failed on Nov 8, 2026
  9. DrahtBot removed the label CI failed on Nov 9, 2026
  10. txindex: report iterator errors from FindTx
    An iterator becoming invalid does not tell us whether it reached the end
    or stopped because of a database error. Add CDBIterator::CheckStatus()
    and call it after the scan, before choosing a candidate. This passes any
    iterator error to the existing HandleError() path, which throws
    dbwrapper_error. A normal missing transaction still returns nullopt.
    d267ec7786
  11. musaHaruna force-pushed on Nov 9, 2026
  12. andrewtoth commented at 4:30 PM on November 10, 2026: contributor

    The same issue exists for any consumer using iterators. Should we instead check status and throw inside Seek and Next? Would there be a noticeable performance penalty?

  13. musaHaruna commented at 7:40 PM on November 11, 2026: contributor

    The same issue exists for any consumer using iterators. Should we instead check status and throw inside Seek and Next? Would there be a noticeable performance penalty?

    If I understand the code path correctly, I think adding the check inside Seek() and Next() would do more than check a single error flag. DBIter::status() delegates to its internal iterator when its own status is OK. When that is a merging iterator, MergingIterator::status() loops over its children, which can also check their underlying iterators. On the healthy path, it checks every child.

    Putting this inside Next() would repeat that work for every entry. For a scan of one million entries, that means about one million status checks, plus the initial seek check. With the approach used in FindTx(), we check status once after the candidate scan, before using the results. LevelDB records errors encountered during iteration, so the final check can still report them. It also needs to run when Valid() is true, since an iterator can skip a damaged block and remain valid.

    I also compared both approaches using bench_bitcoin. The shared version checks inside Seek(), SeekToFirst(), and Next(). For the disk key/value scan, six paired runs showed a median 11.90% increase in time, ranging from 11.47% to 13.03%. That was roughly 144 ms versus 161 ms for one million entries.

    Pair Check after scan, ns/entry Check after each movement, ns/entry Time change
    1 144.193 162.981 +13.03%
    2 144.417 160.984 +11.47%
    3 143.856 160.444 +11.53%
    4 144.238 162.455 +12.63%
    5 143.861 161.503 +12.26%
    6 143.624 160.162 +11.51%

    I ran the benchmarks on an Apple M5 with 16 GiB of RAM.

    So I think the extra work is measurable for these scans. Checking inside the movement functions would protect callers automatically, but we could also use explicit checks for the other callers, making sure early returns check status and single-seek callers check after the seek. That would include callers such as IsEmpty(), NeedsUpgrade(), FindSpender(), and the UTXO cursor consumers. If this approach is desirable, I can extend this PR to cover them too.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-21 11:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me