Crashes constantly on Mac OS Sonoma when using an ExFAT external SSD

drgarlic commented at 7:54 pm on September 29, 2023: none

⚠️ Fix: Use APFS instead of ExFAT disk format ⚠️

Is there an existing issue for this?

I have searched the existing issues

Current behaviour

Both bitcoind and Bitcoin Core crash in various way, sometimes it’s a corrupted block, other times there are issues with the indexes or the chain state. Tried reindexing without success.

Tried on different computers one on 14.0 and the other on 14.1.

It seem to only crash when using external storage, I didn’t run it for very long since I don’t have the space for but it worked without issue when leaving the default datadir.

Tried with Bitcoin Core 24.1, 25.0, and bitcoind 25.0 and built from source, reindexing and restarting from scratch. It seems to always fail around mid 2012

It worked perfectly on both machines with Mac OS Ventura (13.X) and Bitcoin Core 24.1 with the same external SSD

Expected behaviour

I expect the software to run without crashing

Steps to reproduce

Update a Mac to Sonoma (14.X)

Select an external SSD as storage for datadir

Run bitcoind until it crashes

Relevant log output

Here’s the most common type of errors

 0date='2012-06-05T09:58:55Z' progress=0.004282 cache=229.8MiB(1876048txo)
 12023-09-29T19:49:36Z UpdateTip: new best=00000000000007656ea41bf3b7157b4852d548dad3353aed95803230c4c371fd height=183104 version=0x00000001 log2_work=68.224759 tx=3756934 date='2012-06-05T10:22:50Z' progress=0.004283 cache=229.8MiB(1876231txo)
 22023-09-29T19:49:36Z UpdateTip: new best=00000000000007f347d5d728664c21f549494207e4b94a0b6a89c2be6e327255 height=183105 version=0x00000001 log2_work=68.224787 tx=3757624 date='2012-06-05T10:30:01Z' progress=0.004283 cache=229.8MiB(1876500txo)
 32023-09-29T19:49:36Z ERROR: ReadBlockFromDisk: Errors in block header at FlatFilePos(nFile=12, nPos=948913)
 42023-09-29T19:49:36Z *** Failed to read block
 52023-09-29T19:49:36Z Error: A fatal internal error occurred, see debug.log for details
 6Error: A fatal internal error occurred, see debug.log for details
 72023-09-29T19:49:36Z ERROR: ProcessNewBlock: ActivateBestChain failed (Failed to read block)
 82023-09-29T19:49:36Z tor: Thread interrupt
 92023-09-29T19:49:36Z Shutdown: In progress...
102023-09-29T19:49:36Z msghand thread exit
112023-09-29T19:49:36Z addcon thread exit
122023-09-29T19:49:36Z torcontrol thread exit
132023-09-29T19:49:36Z net thread exit
142023-09-29T19:49:37Z opencon thread exit
152023-09-29T19:49:38Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat started
162023-09-29T19:49:38Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat completed (0.01s)
172023-09-29T19:49:38Z scheduler thread exit
182023-09-29T19:49:38Z Writing 0 unbroadcast transactions to disk.
192023-09-29T19:49:38Z Dumped mempool: 0.00129013s to copy, 0.00299875s to dump
202023-09-29T19:49:41Z Shutdown: done

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

25.0

Operating system and version

MacOS Sonoma 14.1

Machine specifications

Mac mini / MBP Mac OS 14.X M1 External SSD (Samsung T7 Shield)

hebasto commented at 7:55 am on September 30, 2023: member

Similar to #26455?

drgarlic commented at 7:59 am on September 30, 2023: none

@hebasto Hi ! I actually also had that error at the time before it was fixed and everything ran fine until I updated to Sonoma. As far as I remember I never saw that error again. Like I said it crashes because of a corrupted block found or an error with the index or the block header

hebasto commented at 8:05 am on September 30, 2023: member

From https://developer.apple.com/documentation/macos-release-notes/macos-14-release-notes#File-System:

The implementations of the exfat and msdos file systems on macOS have changed; these file systems are now provided by services running in user-space instead of by kernel extensions. If the application has explicit checks or support for either the exfat or msdos file systems, validate the applications with those file systems and report any issues.

drgarlic commented at 8:50 am on September 30, 2023: none

So, if I understand that correctly it’s much more a macOS issue than a bitcoin core one ?

maflcko added the label Data corruption on Oct 1, 2023

maflcko added the label Block storage on Oct 1, 2023

maflcko commented at 10:19 am on October 1, 2023: member

Which file system are you using on the external drive?

drgarlic commented at 10:21 am on October 1, 2023: none

Which file system are you using on the external drive?

Hi ! My bad I totally thought I did specify but exfat

maflcko commented at 10:25 am on October 1, 2023: member

It looks like exfat has issues on macOS. At least there are many similar reports, see for example #27972 (comment)

My recommendation would be to change the file system, but I don’t have macOS to test, so I don’t know what will work.

drgarlic commented at 10:28 am on October 1, 2023: none

Very interesting, this issue didn’t show up when I looked for similar ones. I’ll try APFS then, thanks !

drgarlic commented at 3:42 pm on October 1, 2023: none

I confirm that when using the APFS format everything works fine

Maybe adding a Mac OS label would help people with the same problem to find this issue ?

drgarlic renamed this:
~~Crashes constantly on Mac OS Sonoma~~
Crashes constantly on Mac OS Sonoma when using an `exfat` external SSD
on Oct 1, 2023

drgarlic renamed this:
~~Crashes constantly on Mac OS Sonoma when using an `exfat` external SSD~~
Crashes constantly on Mac OS Sonoma when using an ExFAT external SSD
on Oct 1, 2023

maflcko added the label macOS on Oct 2, 2023

fanquake commented at 9:38 am on October 2, 2023: member

I guess we can close this for now. Maybe we can update some documentation to state that using exfat with macOS is not recommended?

maflcko commented at 11:23 am on October 5, 2023: member

An alternative would be for macOS to fix their software, but someone would have to figure out how to report bugs to them.

maflcko closed this on Oct 5, 2023

maflcko added the label Upstream on Oct 5, 2023

bitcoin deleted a comment on Oct 17, 2023

bitcoin locked this on Oct 16, 2024

Crashes constantly on Mac OS Sonoma when using an ExFAT external SSD #28552