intermittent issue in feature_init.py (bitcoind should have exited with expected error LevelDB error: Corruption) #32600

1. 
   maflcko commented at 6:21 am on May 23, 2025: member

   https://cirrus-ci.com/task/5185890844147712?logs=ci#L3553:

    0267/267 - feature_init.py failed, Duration: 2420 s
    1[17:16:50.450] 
    2[17:16:50.450] stdout:
    3[17:16:50.450] 2025-05-21T20:36:30.563000Z TestFramework (INFO): PRNG seed is: 630547506121562679
    4
    5...
    6
    7[17:16:50.450] 2025-05-21T20:36:49.282000Z TestFramework (INFO): Perturbing file to ensure failure /ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20250521_201845/feature_init_53/node0/regtest/indexes/coinstats/db/000033.ldb
    8[17:16:50.450] 2025-05-21T20:36:49.282000Z TestFramework (INFO): Perturbing file to ensure failure /ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20250521_201845/feature_init_53/node0/regtest/indexes/coinstats/db/000034.log
    9[17:16:50.450] 2025-05-21T20:36:50.003000Z TestFramework (INFO): Perturbing file to ensure failure /ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20250521_201845/feature_init_53/node0/regtest/indexes/txindex/000029.log
   10[17:16:50.450] 2025-05-21T21:16:50.010000Z TestFramework (ERROR): Assertion failed
   11[17:16:50.450] Traceback (most recent call last):
   12[17:16:50.450]   File "/ci_container_base/test/functional/test_framework/test_node.py", line 682, in assert_start_raises_init_error
   13[17:16:50.450]     ret = self.process.wait(timeout=self.rpc_timeout)
   14[17:16:50.450]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   15[17:16:50.450]   File "/usr/lib/python3.12/subprocess.py", line 1264, in wait
   16[17:16:50.450]     return self._wait(timeout=timeout)
   17[17:16:50.450]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
   18[17:16:50.450]   File "/usr/lib/python3.12/subprocess.py", line 2045, in _wait
   19[17:16:50.450]     raise TimeoutExpired(self.args, timeout)
   20[17:16:50.450] subprocess.TimeoutExpired: Command '['bitcoin-node', '-datadir=/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20250521_201845/feature_init_53/node0', '-logtimemicros', '-debug', '-debugexclude=libevent', '-debugexclude=leveldb', '-debugexclude=rand', '-uacomment=testnode0', '-logthreadnames', '-logsourcelocations', '-loglevel=trace', '-v2transport=1', '-txindex=1', '-blockfilterindex=1', '-coinstatsindex=1', '-checkblocks=200', '-checklevel=4']' timed out after 2400 seconds
   21[17:16:50.450] 
   22[17:16:50.450] During handling of the above exception, another exception occurred:
   23[17:16:50.450] 
   24[17:16:50.450] Traceback (most recent call last):
   25[17:16:50.450]   File "/ci_container_base/test/functional/test_framework/test_framework.py", line 184, in main
   26[17:16:50.450]     self.run_test()
   27[17:16:50.450]   File "/ci_container_base/ci/scratch/build-i686-pc-linux-gnu/test/functional/feature_init.py", line 190, in run_test
   28[17:16:50.450]     self.init_stress_test()
   29[17:16:50.450]   File "/ci_container_base/ci/scratch/build-i686-pc-linux-gnu/test/functional/feature_init.py", line 161, in init_stress_test
   30[17:16:50.450]     start_expecting_error(err_fragment)
   31[17:16:50.450]   File "/ci_container_base/ci/scratch/build-i686-pc-linux-gnu/test/functional/feature_init.py", line 51, in start_expecting_error
   32[17:16:50.450]     node.assert_start_raises_init_error(
   33[17:16:50.450]   File "/ci_container_base/test/functional/test_framework/test_node.py", line 712, in assert_start_raises_init_error
   34[17:16:50.450]     self._raise_assertion_error(assert_msg)
   35[17:16:50.450]   File "/ci_container_base/test/functional/test_framework/test_node.py", line 195, in _raise_assertion_error
   36[17:16:50.450]     raise AssertionError(self._node_msg(msg))
   37[17:16:50.450] AssertionError: [node 0] bitcoind should have exited within 2400s with expected error LevelDB error: Corruption
   

2. maflcko added the label CI failed on May 23, 2025
3. 
   Cyber-Lord commented at 8:51 pm on May 23, 2025: none

   I’ve attempted reproducing this issue locally using “test/functional/feature_init.py”. After deleting some LevelDB .ldb files manually, I was able to trigger the failure. Instead of the expected “Error opening coins database.”, the node exited with “Error opening block database.”, confirming that the error message varies depending on which part of the LevelDB data is corrupted.

   I am thinking of submitting a patch to make the test more robust by allowing multiple expected error messages (e.g., both “coins database” and “block database”). This should reduce flakiness and make the test better reflect real-world failure modes.

4. bitcoin deleted a comment on May 25, 2025

Contributors
maflcko Cyber-Lord

Labels
CI failed