Random IO errors if data folder is stored on Microsoft ReFS partition #9563

Bitcoin Core (currently v13.2) randomly crashes with IO error accessing some files if the data folder is stored on ReFS partition (mirrored Storage Space on Windows 10) IO error: R:\Bitcoin\chainstate\3176876.ldb: Процесс не может получить доступ к файлу, так как этот файл занят другим процессом. transtation: The process cannot get access to the file because it is open by another process.

Probably, because ReFS does checksum calculations on all files when accessing them, probably interfering with Bitcoin Core... Maybe the fact that ReFS is a "Copy on Write" filesystem may have an effect too...

Can you reliably reproduce the issue?

The issue is random probably due to a race condition between Bitcoin Core and ReFS housekeeping process when it creates new files in the data folder. I've just moved the data folder to ReFS partition now, and trying to sync the blockchain for the last 4 weeks it crashed twice already with the same kind of error in the log...

Machine specs:

• OS: Windows 10 1607 x64
• CPU: Core i5 760
• RAM: 8GB
• Disk size: 2TB
• Disk Type (HD/SDD): Microsoft Storage Space in 2-way mirror with ReFS filesystem on it.

PS: I'm totally aware that this is a very low-priority issue, because ReFS is not used by many people yet, and probably will not be used very much in the near future...

That filesystem has no TxF support, that's probably why it can't handle it (and if that why, there's probably not much that can be done from the app side, as long as it uses a DB to store data). https://msdn.microsoft.com/en-us/library/windows/desktop/bb968806(v=vs.85).aspx

That filesystem has no TxF support, that's probably why it can't handle it

Does Bitcoin Core use TxF?.. Interestingly, in the article you linked, it says "Microsoft strongly recommends developers utilize alternative means to achieve your application’s needs. Many scenarios that TxF was developed for can be achieved through simpler and more readily available techniques. Furthermore, TxF may not be available in future versions of Microsoft Windows."

Yep, wrong link/term, sorry about that. It was late and I was looking for a support statement for transactional workloads or ReFS and couldn't find anything closer. Now I looked again and for example here there's a more recent statement related to SQL Server DB files prior to SQL Server 2014 ("Hosting databases on Windows Server 2012 R2 and Windows Server 2012 ReFS file systems is not supported for production use") , so it's likely that app has to be modified to properly handle database transactions in order to preserve consistency because there's insufficient support from the FS itself.

The crash is listed as in leveldb....does chrome work on ReFS (I assume so?). are there any special considerations in the chrome posix wrapping stuff to handle ReFS?

@TheBlueMatt Chrome ldb crash will be hard to test actually... The crash in Bitcoin Core happens randomly under heavy load, i.e. when syncing with a blockchain for a long time, when simultaneous writes to the file from several threads is possible... Maybe, limiting Bitcore Core to one thread would be a workaround, but I already moved the files back to NTFS and synced the whole blockchain...

Bitcoind only writes to the database (and I think, also the block data) from one thread.

This seems a duplicate of #8250. In my case it had something to do with Windows Defender (or another virus scanner) having opened the file at the same time. Nothing bitcoin core can help, really, though terribly annoying.