"Error opening block database" on new installation on Windows 8 when using custom database directory #6200

I have 3 hard drives - one internal, two external. Creating a custom data directory works on 2 of my drives (the internal, and one external), but doesn't work on my third hard drive. The problem is, I don't have space to store the 50GB+ blockchain on the two drives it works on. The one it doesn't work on is a terrabyte drive - bitcoin's window acknowledges there's 600GB of free space on it.

But when I write a path on that drive, and press ok, it creates what look to be the correct file structure, but opens with an error message saying:

Error opening block database.
Do you want to rebuild the block database now?

I can abort, or press ok. If i press ok it says "Error opening block database" and quits. Dick.. How can I debug what's going on here?

This might be related: https://github.com/etotheipi/BitcoinArmory/issues/298

What file systems are involved?

Oh huh. The ones that work are NTFS, the one that doesn't work is exFAT - I guess that's too much of a coincidence to discount. I would expect, tho, that filesystems are abstracted by the OS so the program doesn't have to care about it, no?

Is there any more info in the debug.log? Maybe you can paste that, so we can take a look? Any special things in bitcoin.conf or passed to the executable via parameters?

Edit: And perhaps most important, which version it it ;)?

I was trying Bitcoin Core version v0.9.3.0-g40d2041-beta, now I'm trying v0.10.2, same issue. My debug.log contains:

2015-05-30 22:05:25 GUI: "registerShutdownBlockReason: Successfully registered: Bitcoin Core didn't yet exit safely..." 
2015-05-30 22:05:25 



















2015-05-30 22:05:25 Bitcoin version v0.10.2 (2015-05-16 10:37:27 +0200)
2015-05-30 22:05:25 Using OpenSSL version OpenSSL 1.0.1k 8 Jan 2015
2015-05-30 22:05:25 Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
2015-05-30 22:05:25 Default data directory C:\Users\fresheneesz\AppData\Roaming\Bitcoin
2015-05-30 22:05:25 Using data directory E:\bitcoinData\Bitcoin
2015-05-30 22:05:25 Using config file E:\bitcoinData\Bitcoin\bitcoin.conf
2015-05-30 22:05:25 Using at most 125 connections (2048 file descriptors available)
2015-05-30 22:05:25 Using 8 threads for script verification
2015-05-30 22:05:25 Using wallet wallet.dat
2015-05-30 22:05:25 init message: Verifying wallet...
2015-05-30 22:05:25 CDBEnv::Open : LogDir=E:\bitcoinData\Bitcoin\database ErrorFile=E:\bitcoinData\Bitcoin\db.log
2015-05-30 22:05:25 Bound to [::]:8333
2015-05-30 22:05:25 Bound to 0.0.0.0:8333
2015-05-30 22:05:25 init message: Loading block index...
2015-05-30 22:05:25 Opening LevelDB in E:\bitcoinData\Bitcoin\blocks\index
2015-05-30 22:05:25 IO error: WinMmapFile.Append::UnmapCurrentRegion or MapNewRegion: : The operation could not be completed because the volume is dirty. Please run chkdsk and try again.


2015-05-30 22:06:04 init message: Loading block index...
2015-05-30 22:06:04 Wiping LevelDB in E:\bitcoinData\Bitcoin\blocks\index
2015-05-30 22:06:04 Opening LevelDB in E:\bitcoinData\Bitcoin\blocks\index
2015-05-30 22:06:04 IO error: WinMmapFile.Append::UnmapCurrentRegion or MapNewRegion: : The operation could not be completed because the volume is dirty. Please run chkdsk and try again.


2015-05-30 22:06:05 Shutdown: In progress...
2015-05-30 22:06:05 StopNode()
2015-05-30 22:06:05 Shutdown: done

And my db.log exists but is empty. I haven't modified bitcoin.conf - I didn't do anything other than install and run the program this time round. I don't know where to find bitcoin.conf either - I don't see it in the install directory nor the data directory.

I ran chkdsk, and that solved it : ) . The bitcoin client should have printed that error in its GUI tho. Can we keep this ticket open until the error is displayed to the user via the GUI?

Great you fixed it and yeah it seems the DB init flow doesn't handle this correctly... @sipa can you have a look?

The generic message should be extended to mention that one should check the debug log for more details about the errors. This is too rare a condition to implement specific logic for, or add specific translation messages, IMO.

@laanwj How exactly do you know how rare this is? This literally came up for me twice in a row (in the same day after I ran a chkdsk, I had to do it again later that day). Also it shouldn't matter how rare this specific case is, because I'm sure there are a wide variety of cases where a message is written to the debug log that should find its way to a user-facing error message in the GUI.

Also, please don't simply close tickets without allowing people to discuss your opinion.

@fresheneesz I'm going to guess that this drive is using write-caching and isn't being unmounted properly (pulled the plug/power without safely removing). Are either of those correct?

Dunno about write caching, but yes indeed the issue is likely the power being turned off without safely removing the drive. Assuming most people are safely removing their drives isn't a safe assumption. (See what I did there ; )

Sorry, but that logic just doesn't follow. If write-caching is enabled and you're unplugging the drive suddenly, all bets are off. You have essentially just unplugged a mounted (and busy) internal hard-drive. In fact, last I used Windows, it would even pop up and say "drive not ejected cleanly, corruption may have occured" or something. If that's the case, there's little that can be done here.

If write-caching is disabled, unplugging "should be" safe, but I should think it would also be horrendously slow.

You mean the logic of "most people don't give a shit what their computer wants them to do" doesn't sound like truth to you? Why is it unreasonable to expect Armory to tell me "Hey, run a chkdsk on drive E please" ?

And then, suddenly, they care what the computer wants them to do?

Computer: Hey, don't unplug that, it won't end well. User: I'll unplug it anyway. Computer: Hey, you shouldn't have unplugged that.

Computer: Error opening block database. Hey, run a chkdsk on drive E please. User: Dammit computer, what do you want from me?!

The fact that leveldb spits out an error that says "run chkdsk" might mean that we can detect that specific case and present it to the user. But in nearly every case I can think of where we would show that, the more correct response would be "something's dangerously wrong with your setup".

@theuni Most people only care when it immediately affects them. Plus your computer isn't very vocal about its desire for you to disconnect things safely. But if your program says "hey you can't use me until you do this" people will do it. So yes, "suddenly" they care. I'd appreciate less sarcasm.

But in nearly every case I can think of where we would show that, the more correct response would be "something's dangerously wrong with your setup".

And what should users do if they run into some "database can't be loaded error" and their setup is "dangerously wrong"? Just give up and never use Armory again? I don't understand why you think a more understandable error message is unreasonable.

What are these cases you can think of where the correct response would be "your setup sucks and we won't help you fix it"?

Ok, the bickering is getting us nowhere. On to the code:

We don't get any useful info out of leveldb other than a failed open and an error message.. nothing to test against. We don't know what failed or why. Advice to run chkdsk could possibly end up being wrong (and harmful) more often than not.

So the only solution here, other than "do nothing" is to print the leveldb error verbatim to the user as well as to the log. In this case, it'd be a popup saying "IO error: WinMmapFile.Append::UnmapCurrentRegion or MapNewRegion: : The operation could not be completed because the volume is dirty. Please run chkdsk and try again"

As @laanwj said, this is a rare edge-case. The question to ask is "would showing the user that info directly result in more or less confusion overall".

I've seen no evidence that this edge case is at all rare. That really sounds like an unsubstantiated claim. We do want your average non-techie to be able to use Armory, right?

Anyways, printing out the leveldb error verbatim sounds like a perfectly fine solution to me. As long as errors aren't pages long, people read em, and if the solution to the problem is written right there in the error (like it seems to be in this case) then great!