core stops to run with `Failed to read block` error #27142

Expected behavior

While my node was running it get an internal failure that force it to stop the running

2023-02-21T23:29:22Z Syncing basic block filter index with block chain from height 624447
2023-02-21T23:29:34Z ERROR: SerializeFileDB: Rename-into-place failed
2023-02-21T23:29:52Z Syncing basic block filter index with block chain from height 624889
2023-02-21T23:30:22Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at FlatFilePos(nFile=2030, nPos=82255446)
2023-02-21T23:30:22Z *** ThreadSync: Failed to read block 0000000000000000000def83407e184fbc5f53b47effdaef7b15e2d6f6b8579d from disk
2023-02-21T23:30:22Z Error: A fatal internal error occurred, see debug.log for details

The code is based on the commit 94070029fb6b783833973f9fe08a3a871994492f

To reproduce

No idea how I can reproduce it, now I switch to the tagged version to see if this happens too

System information

➜  bitcoin git:(master) neofetch                                        
       _,met$$$$$gg.          vincent@x86 
    ,g$$$$$$$$$$$$$$$P.       ----------- 
  ,g$$P"     """Y$$.".        OS: Debian GNU/Linux 11 (bullseye) x86_64 
 ,$$P'              `$$$.     Host: OptiPlex 3050 
',$$P       ,ggs.     `$$b:   Kernel: 5.10.0-21-amd64 
`d$$'     ,$P"'   .    $$$    Uptime: 5 days, 16 hours, 33 mins 
 $$P      d$'     ,    $$P    Packages: 502 (dpkg) 
 $$:      $$.   -    ,d$$'    Shell: zsh 5.8 
 $$;      Y$b._   _,d$P'      Terminal: /dev/pts/1 
 Y$$.    `.`"Y$$$$P"'         CPU: Intel i5-7500T (4) @ 3.300GHz 
 `$$b      "-.__              GPU: Intel HD Graphics 630 
  `Y$$                        Memory: 2214MiB / 7840MiB 
   `Y$$.
     `$$b.                                            
       `Y$$b.                                         
          `"Y$b._
              `"""

<!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->

<!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->

<!-- GUI-related issue? What is your operating system and its version? If Linux, what is your desktop environment and graphical shell? -->

<!-- Any extra information that might be useful in the debugging process. -->

<!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

We do not currently propagate the exact error code raised by fs::rename so it might be difficult to diagnose precisely... The most usual cases are the filesystem being full or corrupt. Are you sure your filesystem has enough free space?

Seeing as it later hit the ReadCompactSize deserialize error afterwards, it does feel more likely to be a fs corruption issue...

The most usual cases are the filesystem being full or corrupt. Are you sure your filesystem has enough free space?

Yes 100%

/dev/sdb1      4883759996 2603409780 2280350216  54% /media/Matrox

Seeing as it later hit the ReadCompactSize deserialize error afterward, it does feel more likely to be an fs corruption issue...

The strage thing is that this is running for days, so I did not get any downside that can end up in something half written or just put the trash inside the .dat file.

I open the issue because this happens while I was running the master, I ran this node for years now with a tagged version, and I had no problem with it. So it may be a regression?

Hey @vincenzopalazzo just wanted to follow up here...

Was this something that you managed to fix, has it gone away on it's own, or are you still experiencing problems with it?

@willcl-ark after the downgrade to the tagged version I have not try it again, sorry!

Do you think it's something you will try again in the future? It would be good to get reproduction steps if possible...

yeah I will try asap

Checking in here, is this still an issue? We'll close for now, please comment if you can reproduce.