Flushing block file to disk failed when using external drive on macOS Ventura #26455

issue aureliusltd openend this issue on November 4, 2022
  1. aureliusltd commented at 9:14 pm on November 4, 2022: none

    I kept getting an error regarding Flushing Block to disk. I’ve tried to use bitcoin core V0.23 and it doesn’t help; although V0.22 was better at loading the backup file, but I couldn’t send any of my bitcoin out. I can’t seem to load the wallet or the backup file properly. Not a coder and hoping someone can help. I don’t think it’s an issue with the hard drive. Any suggestion? I’m on MacOSx Ventura 13.0.

    Debug.log file:

    2022-11-04T20:34:28Z Using wallet /Volumes/Extreme SSD/bruceltd wallet.dat 2022-11-04T20:34:28Z BerkeleyEnvironment::Open: LogDir=/Volumes/Extreme SSD/database ErrorFile=/Volumes/Extreme SSD/db.log . . . 2022-11-04T20:58:16Z FileCommit: fcntl F_FULLFSYNC failed: 25 2022-11-04T20:58:16Z ERROR: SerializeFileDB: Failed to flush file /Volumes/Extreme SSD/peers.176f

    db.log file: fsync: Inappropriate ioctl for device PANIC: Inappropriate ioctl for device txn_checkpoint: failed to flush the buffer cache: DB_RUNRECOVERY: Fatal error, run database recovery

  2. maflcko added the label macOS on Nov 5, 2022
  3. maflcko added the label UTXO Db and Indexes on Nov 5, 2022
  4. maflcko added the label Data corruption on Nov 5, 2022
  5. maflcko added the label Questions and Help on Nov 5, 2022
  6. brl4n commented at 1:07 am on November 6, 2022: none

    also getting similar internal errors for the past few days. MacOS DiskUtility doesn’t report any problems. No other apps are having file IO issues. Tried v24.0.0rc3 and same issue. Able to check blocks for a few minutes then crashes. I recently updated to Ventura but don’t know enough to decide if they changed something which would cause this. Let me know if you need more info.

    Crashed Thread: 0 Dispatch queue: com.apple.main-thread

    Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x0100000100000012 -> 0x0000000100000012 (possible pointer authentication failure) Exception Codes: 0x0000000000000001, 0x0100000100000012

    Termination Reason: Namespace SIGNAL, Code 11 Segmentation fault: 11 Terminating Process: exc handler [37021]

    VM Region Info: 0x100000012 is not in any region. Bytes before following region: 15368174 REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL UNUSED SPACE AT START —>
    __TEXT 100ea8000-10282c000 [ 25.5M] r-x/r-x SM=COW …OS/Bitcoin-Qt

    Kernel Triage: VM - pmap_enter retried due to resource shortage VM - pmap_enter retried due to resource shortage

  7. aureliusltd commented at 0:55 am on November 12, 2022: none
    Update on the issue. The current Bitcoin Core software, V0.22 & V0.23, are not working well with the new Mac Software update 13.0 Ventura. I went back to the previous version, Monterey 12.6, and the software works just fine. How can i help to make the new bitcoin core version, V0.24, more compatible with Mac? Thank you!
  8. hebasto commented at 10:33 am on November 12, 2022: member

    Update on the issue. The current Bitcoin Core software, V0.22 & V0.23, are not working well with the new Mac Software update 13.0 Ventura.

    Looks similar to #26400.

    #26400 (comment):

    Using external drives is not a configuration which is extensively tested and guaranteed to work in all circumstances. For example, #26397 (comment).

    Please try to reproduce the issue while using local drives only. You could use pruning to limit block storage space.

  9. hebasto renamed this:
    Flushing block file to disk failed. This is likely the result of an I/O error.
    Flushing block file to disk failed when using external drive on macOS Ventura
    on Nov 18, 2022
  10. hebasto commented at 3:12 pm on November 18, 2022: member

    The current Bitcoin Core software, V0.22 & V0.23, are not working well with the new Mac Software update 13.0 Ventura. I went back to the previous version, Monterey 12.6, and the software works just fine.

    I’ve updated the issue title.

  11. hebasto commented at 3:35 pm on November 18, 2022: member

    @aureliusltd @mateusgalasso

    What are filesystems on your external drives?

  12. mateusgalasso commented at 5:20 pm on November 18, 2022: none

    @aureliusltd @mateusgalasso

    What are filesystems on your external drives?

    ExFAT

  13. john-moffett commented at 7:21 pm on November 18, 2022: contributor

    It looks like other projects (LevelDB, SQLite, etc.) have had similar issues with fcntl using F_FULLFSYNC. Apparently there are certain external drives / filesystems that don’t support it, so they return an error. I can only speculate why it seems to be only showing up on macOS Ventura.

    Those projects have all used the same strategy: fallback to fsync on error (see previous links or this helpful twitter thread). It seems like the error is specifically ENOTTY in this case, so we could check for that and try fsync.

    I’ve created a branch that does that. Happy to submit as a PR if this approach seems sound.

    https://github.com/john-moffett/bitcoin/blob/d99a8bee66656d29017b42fb2c0935de768ccd70/src/util/system.cpp#L1187-L1200

    Edit: It may not fix OP’s full issue, as BerkeleyDB falls back to fsync only on ENOTSUP. See here:

    https://github.com/observerdev/db-4.8.30/blob/3c75267f11336acb1be06bed419e8056cc689d09/os/os_fsync.c#L81-L88

  14. hebasto commented at 2:47 pm on November 19, 2022: member

    I’ve created a branch that does that. Happy to submit as a PR if this approach seems sound.

    A well motivated PR is better than a comment in an issue :)

  15. hebasto commented at 9:57 am on November 30, 2022: member

    Apparently, it is an macOS 13 issue. Upgrading to 13.1 should fix it:

    Close?

  16. fanquake closed this on Nov 30, 2022
  17. mateusgalasso commented at 3:10 pm on November 30, 2022: none

    Apparently, it is an macOS 13 issue. Upgrading to 13.1 should fix it:

    Close?

    There is no 13.1

  18. john-moffett commented at 3:15 pm on November 30, 2022: contributor
    13.1 is available as a public beta: https://beta.apple.com/sp/betaprogram/welcome
  19. asylejmani commented at 9:10 am on December 7, 2022: none
    I can confirm it works in 13.1! Had v24.0.0 running all night without any issues.
  20. fanquake referenced this in commit ac7c1772f9 on Jul 20, 2023
  21. bitcoin locked this on Dec 7, 2023


aureliusltd brl4n hebasto mateusgalasso john-moffett asylejmani

Labels
macOS UTXO Db and Indexes Data corruption Questions and Help

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me