rfc: Wallets should be in their own directory

laanwj commented at 6:17 am on September 16, 2017: member

With multi-wallet (both “parallel” RPC multiwallet and “serial” -wallet), the situation has become quite confusing. There is no way to list the wallets - the obvious ls ~/.bitcoin/*.dat will yield a combination of transient state files as well as wallets. This is quite annoying when doing backups, for example.

So at least for new installs I’d propose to set the wallet bdb root to ~/.bitcoin/wallets. If this directory does not exist, it should fall back to just using the data directory. Another option would be a one-time move, but that doesn’t need to be in the first iteration.

Also it would be useful to override the wallet directory with an option, say -walletdir, for people that want to store the wallets on a separate file system without potentially hazardous symlink tricks. Implementing just this for a start, and not changing the default, would be fine too.

(Another idea would be to introduce a random component into the wallet-dir name, like firefox does with profile directories ~/.mozilla/firefox/1naf6a1d.default, this makes it slightly harder to steal a wallet file if all you can do is guess paths - but I digress)

(see also discussion in #10885 and #11343)

laanwj added the label Wallet on Sep 16, 2017

esotericnonsense commented at 4:39 pm on September 16, 2017: contributor

What might be useful along with this is some documentation on files within the .bitcoin directory that are affected by the wallet (not only those with potential for corruption of the wallet itself).

For example - debug.log can contain information on ‘interesting’ txids (AddToWallet messages, as I recall), and so if you’re storing the wallet on another filesystem for the purpose of encrypting pubkeys at rest, even if you’ve included all BDB state, a lot of that information will ’leak’ into debug.log. This isn’t a concern for me in particular, but there are probably many different reasons users would like to store their wallets seperately (it would be useful if those use cases were posted here, I think?)

TheBlueMatt commented at 4:19 pm on September 17, 2017: member

Yea, not a fan of the one-time-move thing, but defaulting to creating a wallets folder and having a -walletdir option is definitly where we should be headed.

laanwj commented at 4:34 pm on September 17, 2017: member

What might be useful along with this is some documentation on files within the .bitcoin directory that are affected by the wallet (not only those with potential for corruption of the wallet itself).

Everything bdb logs and scribbles will move along to the walletdir.

For example - debug.log can contain information on ‘interesting’ txids (AddToWallet messages, as I recall), and so if you’re storing the wallet on another filesystem for the purpose of encrypting pubkeys at rest, even if you’ve included all BDB state, a lot of that information will ’leak’ into debug.log.

It’s really difficult to contain metadata within a single computer. If the goal is to hide existence of a certain wallet, my recommendation would be to run the software in a VM with the disk fully encrypted, and having it only communicating over Tor/VPN. There’s various ways that information can leak and trying to enumerate them is brittle.

It might be an interesting project for someone but this is certainly not part of the scope of this issue.

esotericnonsense commented at 4:37 pm on September 17, 2017: contributor

Can’t argue with that.

jnewbery commented at 2:09 pm on September 18, 2017: member

There is no way to list the wallets

This is something I’ve thought about before. I think the listwallets RPC that we introduced in v0.15 as experimental should be changed to return an object with two key-values: one for loaded wallets, and one for ‘available’ wallets in the .bitcoin/wallets directory

gmaxwell commented at 6:50 am on September 20, 2017: contributor

sounds good to me in general

rodentrabies commented at 6:51 am on October 4, 2017: contributor

I’d like to take this, if everyone agrees.

jnewbery commented at 9:39 pm on October 9, 2017: member

I’d like to take this, if everyone agrees. @mrwhythat: no need to ask for permission to start working on something :)

(although in this case it looks like @MeshCollider has now opened a PR for this at #11466. If you’re still interested in this, then reviewing/testing that PR would be a really good way to contribute)

rodentrabies commented at 12:01 pm on October 10, 2017: contributor

@jnewbery, sorry, actually I planned to work on that while in Prague for HCPP17 but as soon as I got there, I completely forgot about it :)

DarthJahus commented at 11:20 pm on November 4, 2017: none

Also it would be useful to override the wallet directory with an option, say -walletdir.

Amazing, it would allow creating of standalone wallets (on removable hard drives, for example).

meshcollider commented at 2:31 am on November 5, 2017: contributor

@DarthJahus Indeed, but keep in mind that it could still be dangerous to use them from removal devices, if the device got removed while core was running you could end up corrupting the wallet

laanwj commented at 12:02 pm on November 17, 2017: member

#11466 could use some testing!

laanwj closed this on Nov 18, 2017

laanwj referenced this in commit d080a7d503 on Nov 18, 2017

PastaPastaPasta referenced this in commit c68dc2bf5e on Jan 31, 2020

PastaPastaPasta referenced this in commit e2edefa22e on Feb 4, 2020

PastaPastaPasta referenced this in commit 2c7b29bac5 on Feb 9, 2020

ckti referenced this in commit d26cd3fea8 on Mar 28, 2021

gades referenced this in commit 2ca7534241 on Jun 30, 2021

DrahtBot locked this on Sep 8, 2021

gades referenced this in commit 5c0603a06b on Feb 12, 2022

rfc: Wallets should be in their own directory #11348