Broken wallet.dat sym link is overwritten

EhVedadoOAnonimato commented at 8:39 pm on November 23, 2011: none

My wallet.dat is a symbolic link to a directory which is not automatically mounted. If I forget to mount this directory before starting bitcoin, it will overwrite the broken link by a new wallet.dat.

I would expect it to display an error message, like “corrupt wallet” or something, that would remember me of mounting the target directory.

thiloplanz commented at 7:37 am on November 26, 2011: none

I have the same problem. My wallet is also on an encrypted volume that is usually not mounted. Fortunately, it does not overwrite any real wallet in this case (just the symlink which can be reconstructed easily), so no data is lost. However, I am worried that if the wallet file was for some reason unreadable, that it would also overwrite it. If there is any problem in reading an existing “wallet.dat”, the program should not touch it, complain about a “corrupt wallet” and either quit or continue without a wallet (if there is such a mode). To recover, the user would have to manually remove or fix the “corrupt” file, which I think is a reasonable thing to have him do at this point.

laanwj commented at 5:36 pm on December 3, 2011: member

I agree. However this logic is not build into bitcoin itself, afaik, but into berkelydb. You’ll probably have to see if berkelydb supports a flag to fail in this case.

TheBlueMatt commented at 1:57 am on December 4, 2011: member

We could still do manual testing before we open the wallet dir, though I would agree that its not an entirely supported case that we should employ extra logic to complain about - do other programs fail in similar situations?

laanwj commented at 8:19 am on December 4, 2011: member

As I understand it he’s symlinking individual files, not the directory the wallet is in (which AFAIK already does fail if the target doesn’t exist). So in his case to be correct we’d have to check every file bdb potenialtially opens, including logs, not just the wallet.dat. It’s very ugly to tack that on on our side.

thiloplanz commented at 8:24 am on December 4, 2011: none

Well, the wallet.dat is the only file that you might want to symlink away individually to an encrypted and usually offline disk (the other files are not security-critical), so some special treatment for just wallet.dat would be sufficient here, I think.

laanwj commented at 8:26 am on December 4, 2011: member

It wouldn’t be enough, because the bdb binary log files can also contain sensitive data such as private keys, so it would ’leak’ into that filesystem.

thiloplanz commented at 8:33 am on December 4, 2011: none

Oh, that’s good to know. So the whole approach of only encrypting the wallet.dat is flawed? Is there a way to remove the logs?

But I agree that this issue has a low priority (especially if there are plans for a new wallet file format that is more portable, less binary, and less leaky, are there?). The clobbered symlink does not cause any real problems, only confusion. If you are manually symlinking you should be able to be careful enough not to run into too much trouble here.

However, I am worried that if the wallet file was for some reason unreadable, that it would also overwrite it.

How about that. What happens if wallet.dat is a broken file? Does BDB preserve it?

laanwj commented at 8:48 am on December 4, 2011: member

AFAIK the application crashes hard if the wallet.dat is broken, it won’t simply overwrite it. But to test that you’ll probably have to try with various levels of corruption.

The best idea I’ve heard with regard to re-engineering wallet files is an append-only wallet for the private keys. All the private-key action would happen there (and in mlocked main memory), keys could be referred to by number in other places. All the less sensitive mutable metadata such as labels/settings/address book would remain in the old wallet database.

EhVedadoOAnonimato commented at 10:38 pm on December 5, 2011: none

@laanwj, I was not aware about these other sensitive files. Don’t you think this deserves an issue of its own? I mean.. I never treated any other file with the same care that I treat wallet.dat, and I guess I’m not the only one. There’s either a bug there (leaking sensitive data to logs) or a great need of more clarification for most users, as I don’t think I’m the only one that’s not aware of such thing…

thiloplanz commented at 0:49 am on December 6, 2011: none

I never treated any other file with the same care that I treat wallet.dat, and I guess I’m not the only one.

You are not…

I have opened a thread on StackExchange about this. http://bitcoin.stackexchange.com/questions/2140/

One answer seems to suggest that encrypting the wallet using new built-in encryption feature is safe, but before you turn that on, your private keys may leak.

laanwj commented at 5:50 am on December 6, 2011: member

Safe options are:

Symlink the entire walllet directory to the encrypted fs. Bitcoin will not write any files outside this directory so this will prevent leaks. If you want to store blk*.dat and addr.dat somewhere else, symlink those.
Use the built-in wallet encryption

Or both. I did not know about the log “leak” until a discussion a few weeks ago on IRC either. But such behaviour comes with using an OTS database that is not geared toward secure key storage. This is not a problem restricted to bitcoin only, hackers are having a field day with all the hidden stores,caches,logs, and their interaction with journaling filesystems. Though of course, bitcoin is especially sensitive…

jgarzik commented at 9:55 pm on July 5, 2012: contributor

The discussion in this issue seems to have resolved several questions. Possibly reference issue #68 here, as it is a common request.

jgarzik closed this on Jul 5, 2012

ptschip referenced this in commit 15d970c9b5 on Jun 7, 2017

kallewoof referenced this in commit d54d02bfe1 on Oct 4, 2019

rajarshimaitra referenced this in commit 695a1ebd1d on Aug 5, 2021

DrahtBot locked this on Sep 8, 2021

Broken wallet.dat sym link is overwritten #660