Document about wallet backup and restoration

lsilva01 commented at 3:30 am on July 22, 2021: contributor

This PR adds a document about backing up and restoring the Bitcoin Core wallet as suggested in the issue #20149 .

fanquake added the label Docs on Jul 22, 2021

fanquake added the label Wallet on Jul 22, 2021

in doc/managing_wallet.md:22 in 0a5494d2f6 outdated

17+
18+The `descriptors` parameter can be omitted if the intention is to create a legacy wallet.
19+
20+For now, the default type is the legacy wallet, but that should change in the near future.
21+
22+By default, wallets are created in the `~/.bitcoin/wallets/wallet_name` folder. If the node is running on testnet, the wallets are created in `~/.bitcoin/testnet3/wallets/wallet_name` or if signet, `~/.bitcoin/signet/wallets/wallet_name`.

jarolrod commented at 3:45 am on July 22, 2021:

This is not the case for macOS, they are in /Users/<user>/Library/Application Support/Bitcoin

Perhaps you can reword somehow to just say the default bitcoin directory for your system.

lsilva01 commented at 3:05 am on July 27, 2021:

This content has been changed. A table with some default directories was added.

in doc/managing_wallet.md:36 in 0a5494d2f6 outdated

31+
32+The `encryptwallet` command is used only when the wallet has not been encrypted yet. Otherwise, the `walletpassphrasechange` command should be used.
33+
34+`$ bitcoin-cli -rpcwallet="wallet-01" walletpassphrasechange "oldpassphrase" "newpassphrase"`
35+
36+The `-rpcwallet` is the name of the wallet that will be encrypted.

jarolrod commented at 3:46 am on July 22, 2021:

0The argument passed to `-rpcwallet` is the name of the wallet to be encrypted.

in doc/managing_wallet.md:38 in 0a5494d2f6 outdated

33+
34+`$ bitcoin-cli -rpcwallet="wallet-01" walletpassphrasechange "oldpassphrase" "newpassphrase"`
35+
36+The `-rpcwallet` is the name of the wallet that will be encrypted.
37+
38+The term "encrypt the wallet" used here is not very accurate. This command only encrypts only the private key. All other wallet information, such as transactions, is still visible.

jarolrod commented at 3:47 am on July 22, 2021:

0The term "encrypt the wallet" used here is not very accurate. This command encrypts only the private key. All other wallet information, such as transactions, is still visible.

in doc/managing_wallet.md:14 in 0a5494d2f6 outdated

 9+
10+The following command, for example, creates a descriptor wallet:
11+
12+`$ bitcoin-cli -named createwallet wallet_name="wallet01" descriptors=true`
13+
14+If the node is running on testnet network, `-testnet` parameter should be added. Or `-signet` if on signet.

jarolrod commented at 3:55 am on July 22, 2021:

I think a better way to phrase this is to just try to get across that network arguments can be passed as usual. Not to mention testnet and signet, what about regtest (don’t forget about it 😔)

0If the node is running on the testnet network, then the `-testnet` parameter should be added. Or `-signet` if on signet.

lsilva01 commented at 3:08 am on July 27, 2021:

Replaced with a table with default directories.

in doc/managing_wallet.md:112 in 0a5494d2f6 outdated

107+`$ bitcoin-cli createwallet "from_dump_file"`
108+
109+Then this command can be called.
110+
111+`$ bitcoin-cli -rpcwallet="from_dump_file" importwallet /home/node01/Backups/dump01.txt`
112+

unknown commented at 3:57 am on July 22, 2021:

0
1> :information_source: **NOTE:** Importing wallets is disabled when blocks are pruned

lsilva01 commented at 3:00 am on July 27, 2021:

Removed content about dump file as suggested in #22523 (review)

in doc/managing_wallet.md:103 in 0a5494d2f6 outdated

 98+
 99+`$ bitcoin-cli -rpcwallet="wallet-01" dumpwallet /home/node01/Backups/dump01.txt`
100+
101+### 1.8 Importing The Wallet From a Dump File
102+
103+The command `importwallet` imports keys from a wallet dump file.

unknown commented at 3:59 am on July 22, 2021:

0The command `importwallet` imports keys from a wallet dump file. Importing wallets is disabled when blocks are pruned.

OR

Suggestion for L112

luke-jr commented at 5:29 am on July 25, 2021:

It’s not and shouldn’t be, though.

unknown commented at 8:20 am on July 25, 2021:

Yes importing should be possible with pruning. But I get this error when I try to import.

lsilva01 commented at 3:00 am on July 27, 2021:

Removed content about dump file as suggested in #22523 (review)

in doc/managing_wallet.md:44 in 0a5494d2f6 outdated

39+
40+Note that if the passphrase is lost, all the coins in the wallet will also be lost forever.
41+
42+### 1.3 Backing Up The Wallet
43+
44+Wallets can be safely copied to another destination. This backup file should be stored offline, such as on a USB drive, another computer, or an external hard drive.

jarolrod commented at 4:04 am on July 22, 2021:

There is a way to safely copy a wallet to another destination, but most methods are not safe.

I think what you want to say is that the files are portable and there is A way to safely copy them. I think the following encapsulates what you want to say.

0Wallets files are portable and can be copied. For safety, one should store backup copies on offline devices only.

in doc/managing_wallet.md:58 in 0a5494d2f6 outdated

53+
54+The Bitcoin Core wallet was originally a collection of unrelated private keys with their associated addresses. If a non-HD wallet had generated a key/address, had given that address out and then had restored a backup from before that key's generation, then any funds sent to that address would have been lost definitively.
55+
56+However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced HD wallets. Users no longer lose funds when restoring old backups, as long as the addresses used were from the wallet's HD seed (since all private keys can be derived from the seed).
57+
58+So, theoretically, a single backup is enough. But it is recommended to make regular backups (1 time a day or a week) or when there is a relevant amount of new transactions in the wallet.

luke-jr commented at 4:08 am on July 22, 2021:

Failure to do so will still lose metadata.

jarolrod commented at 4:23 am on July 22, 2021:

0Theoretically, this means a single backup is enough. It is still recommended to make regular backups (once a week) or after a significant amount of new transactions.

lsilva01 commented at 3:02 am on July 27, 2021:

Information added.

in doc/managing_wallet.md:56 in 0a5494d2f6 outdated

51+
52+### 1.4 Backup Frequency
53+
54+The Bitcoin Core wallet was originally a collection of unrelated private keys with their associated addresses. If a non-HD wallet had generated a key/address, had given that address out and then had restored a backup from before that key's generation, then any funds sent to that address would have been lost definitively.
55+
56+However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced HD wallets. Users no longer lose funds when restoring old backups, as long as the addresses used were from the wallet's HD seed (since all private keys can be derived from the seed).

luke-jr commented at 4:08 am on July 22, 2021:

It should be noted that pre-0.13 wallets will still be non-HD

jarolrod commented at 4:22 am on July 22, 2021:

0However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced a deterministic mechanism from which to derive keys; HD wallets. Users no longer lose funds when restoring old backups because all addresses are derived from the HD wallet seed.

lsilva01 commented at 3:01 am on July 27, 2021:

Note added.

in doc/managing_wallet.md:62 in 0a5494d2f6 outdated

57+
58+So, theoretically, a single backup is enough. But it is recommended to make regular backups (1 time a day or a week) or when there is a relevant amount of new transactions in the wallet.
59+
60+### 1.5 Restoring The Wallet From a Backup
61+
62+To restore a wallet, an empty one must be created  first.  The backup file then rewrites the `wallet.dat` of this new wallet.

luke-jr commented at 4:09 am on July 22, 2021:

This is kind of ugly. Do we not support restoring without overwriting an existing wallet??

jarolrod commented at 4:24 am on July 22, 2021:

0To restore a wallet, an empty one must be created first. The backup file then rewrites the `wallet.dat` of this new wallet.

lsilva01 commented at 4:20 am on July 25, 2021:

I think not. I opened the PR #22541 that implements a new RPC command to restore a wallet without replacing an existing one.

in doc/managing_wallet.md:83 in 0a5494d2f6 outdated

78+
79+The `-rescan` argument rescans the blockchain for missing wallet transactions during startup.
80+
81+`$ bitcoind -rescan`
82+
83+This is only necessary if there are missing transactions after restoring the wallet.

luke-jr commented at 4:09 am on July 22, 2021:

It should never be necessary (and probably not documented for end-users)

lsilva01 commented at 3:03 am on July 27, 2021:

Section removed.

in doc/managing_wallet.md:93 in 0a5494d2f6 outdated

88+
89+`$ bitcoind -reindex`
90+
91+### 1.7 Dumping The Wallet
92+
93+Alternatively, the `dumpwallet` command can be used to backup the wallet. It dumps all wallet keys (including the master private key) in a human-readable format into a file.

luke-jr commented at 4:10 am on July 22, 2021:

Might be better not to suggest this as long as it’s buggy. (eg, #11803)

lsilva01 commented at 3:04 am on July 27, 2021:

Removed content about dump file.

luke-jr changes_requested

in doc/managing_wallet.md:54 in 0a5494d2f6 outdated

49+
50+`$ bitcoin-cli -rpcwallet="wallet-01" backupwallet /home/node01/Backups/backup01.dat`
51+
52+### 1.4 Backup Frequency
53+
54+The Bitcoin Core wallet was originally a collection of unrelated private keys with their associated addresses. If a non-HD wallet had generated a key/address, had given that address out and then had restored a backup from before that key's generation, then any funds sent to that address would have been lost definitively.

jarolrod commented at 4:17 am on July 22, 2021:

I think there’s some conceptual cleanup to be done here and with the next paragraph on HD wallets.

0The original Bitcoin Core wallet was a bag of unrelated private keys. If a non-HD wallet had received funds to an address and then restored from a backup made before the address was generated, then any funds sent to said address are lost forever because there's no deterministic mechanism to derive that address again.

in doc/managing_wallet.md:85 in 0a5494d2f6 outdated

80+
81+`$ bitcoind -rescan`
82+
83+This is only necessary if there are missing transactions after restoring the wallet.
84+
85+A pruned node, however, is incompatible with the `-rescan` option, since it does not have the data to check for relevant transactions.

jarolrod commented at 4:24 am on July 22, 2021:

0A pruned node, however, is incompatible with the `-rescan` option since it does not have the data to check for relevant transactions.

in doc/managing_wallet.md:95 in 0a5494d2f6 outdated

90+
91+### 1.7 Dumping The Wallet
92+
93+Alternatively, the `dumpwallet` command can be used to backup the wallet. It dumps all wallet keys (including the master private key) in a human-readable format into a file.
94+
95+Note that this file is not encrypted and the keys are exposed. An attacker in possession of the file could recreate the wallet and gain access to the keys.

jarolrod commented at 4:25 am on July 22, 2021:

0Note that this file is not encrypted, and the keys are exposed. An attacker in possession of the file could recreate the wallet and gain access to the keys.

in doc/managing_wallet.md:97 in 0a5494d2f6 outdated

92+
93+Alternatively, the `dumpwallet` command can be used to backup the wallet. It dumps all wallet keys (including the master private key) in a human-readable format into a file.
94+
95+Note that this file is not encrypted and the keys are exposed. An attacker in possession of the file could recreate the wallet and gain access to the keys.
96+
97+The file is generated on the server-side and so the user must have access to the server folders.

jarolrod commented at 4:25 am on July 22, 2021:

0The file is generated on the server-side, and so the user must have access to the server folders.

jarolrod commented at 4:26 am on July 22, 2021: member

Strong Concept ACK

Thanks for working on this. I think there’s still some conceptual cleanups to be made here. For now here are some notes. I will take some time to go through this deeper at some point.

in doc/managing_wallet.md:32 in 0a5494d2f6 outdated

25+
26+The `wallet.dat` file is not encrypted by default and is, therefore, vulnerable if an attacker gains access to the device where the wallet or the backups are stored.
27+
28+The wallet must be encrypted with the following command:
29+
30+`$ bitcoin-cli -rpcwallet="wallet-01" encryptwallet "passphrase"`

S3RK commented at 7:07 am on July 22, 2021:

nit: better to use consistent wallet name throughout the doc. wallet-01 vs wallet01

lsilva01 commented at 2:59 am on July 27, 2021:

Fixed.

S3RK commented at 7:39 am on July 22, 2021: member

Concept ACK. Like that!

in doc/managing_wallet.md:44 in 5c2434adcf outdated

39+
40+Note that if the passphrase is lost, all the coins in the wallet will also be lost forever.
41+
42+### 1.3 Backing Up The Wallet
43+
44+Wallets files are portable and can be copied. For safety, one should store backup copies on offline devices only.

luke-jr commented at 5:28 am on July 25, 2021:

Not directly. The backupwallet RPC or “Backup Wallet” GUI menu item must be used to ensure the file is in a safe state when the copy is made.

lsilva01 commented at 3:08 am on July 27, 2021:

Comment added.

in doc/managing_wallet.md:58 in 5c2434adcf outdated

53+
54+The original Bitcoin Core wallet was a bag of unrelated private keys. If a non-HD wallet had received funds to an address and then restored from a backup made before the address was generated, then any funds sent to said address are lost forever because there's no deterministic mechanism to derive that address again.
55+
56+However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced a deterministic mechanism from which to derive keys; HD wallets. Users no longer lose funds when restoring old backups because all addresses are derived from the HD wallet seed. 
57+
58+Theoretically, this means a single backup is enough. It is still recommended to make regular backups (once a week) or after a significant amount of new transactions.

luke-jr commented at 5:28 am on July 25, 2021:

Old backups will lose metadata, which may still cause financial loss due to not delivering goods, lacking info to pay minimal taxes, etc

lsilva01 commented at 3:09 am on July 27, 2021:

Information about metadata added.

jarolrod commented at 6:38 am on July 26, 2021: member

@lsilva01 please squash your commits and you should address these comments: 1, 2

For some guidance on squashing, see: CONTRIBUTING.md#squashing-commits

lsilva01 force-pushed on Jul 27, 2021

in doc/managing_wallet.md:66 in 777d26cae6 outdated

61+
62+However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced a deterministic mechanism from which to derive keys; HD wallets. Users no longer lose funds when restoring old backups because all addresses are derived from the HD wallet seed.
63+
64+This means that a single backup is enough to recover the coins at any time. It is still recommended to make regular backups (once a week) or after a significant amount of new transactions to keep the metadata such as labels.
65+
66+Metadata cannot be retrieved from a blockchain rescan, so if the backup is too old, it will be lost forever.

NelsonGaldeman commented at 9:12 am on July 27, 2021:

nit: Although it’s clear the statement is talking about metadata, I would add the world metadata before the will be lost forever to avoid panic thinking the backup will be lost forever.

0Metadata cannot be retrieved from a blockchain rescan, so if the backup is too old, metadata will be lost forever.

lsilva01 commented at 9:35 pm on July 27, 2021:

Done.

NelsonGaldeman approved

NelsonGaldeman commented at 9:12 am on July 27, 2021: contributor

Concept ACK 777d26cae620f2922166f6004c94750bd72b137d

Well done, documentation is always welcome!

lsilva01 force-pushed on Jul 27, 2021

in doc/managing_wallet.md:30 in f496823803 outdated

25+
26+### 1.2 Encrypting the Wallet
27+
28+The `wallet.dat` file is not encrypted by default and is, therefore, vulnerable if an attacker gains access to the device where the wallet or the backups are stored.
29+
30+The wallet must be encrypted with the following command:

bpcamac commented at 11:13 pm on July 27, 2021:

The sentence should use MAY not MUST because it is at the user’s discretion whether to encrypt the wallet or not.

Rspigler commented at 2:25 am on July 28, 2021:

Re: encryption.

This changes past policy. See @harding here “no released version of Bitcoin Core has ever created encrypted wallets by default…there’s an open question between experts about whether or not the use of wallet encryption in typical user wallets saves more money than it loses” and @laanwj here: “I’m partial to not encrypting by default. In any case the kind of scenario where encryption works is ‘other people have physical access to my PC but won’t use it to install a keylogger/backdoor’”

Different users will have different private key management and backup schemes, so I don’t think it is appropriate to say that they must encrypt their keys.

S3RK commented at 6:45 am on July 28, 2021:

Is there a difference argument to be made for encrypting backups compared to hot wallets?

Rspigler commented at 3:00 pm on July 28, 2021:

Maybe, but I don’t believe so. Again, it depends what the private key management policy is. For multisig for example, I believe it is best to keep the backups unencrypted. You get both security and redundancy with the m-of-n scheme. With encryption, you run into issues of relying on memory or how to store the passphrases.

Rspigler commented at 3:01 pm on July 28, 2021:

This satisfies the issue I brought up here

lsilva01 commented at 8:59 pm on July 28, 2021:

Right. Better to make it clear that the decision is up to the user.

lsilva01 commented at 9:00 pm on July 28, 2021:

Done.

in doc/managing_wallet.md:18 in f496823803 outdated

13+
14+The `descriptors` parameter can be omitted if the intention is to create a legacy wallet.
15+
16+For now, the default type is the legacy wallet, but that should change in the near future.
17+
18+Wallets are created in the `wallets` folder of the data directory. The default folder varies by operating systems, as shown below. The user can also change it using `-datadir` or `-walletdir` initialization parameters.

Rspigler commented at 2:19 am on July 28, 2021:

0Wallets are created in the `wallets` folder of the data directory. The default folder varies by operating systems, as shown below. The user can also change the default by using the `-datadir` or `-walletdir` initialization parameters.

Grammar nit

lsilva01 commented at 9:00 pm on July 28, 2021:

Fixed.

bpcamac changes_requested

bpcamac commented at 10:13 am on July 28, 2021: none

This is a minor point, but I think the reader/user might find the use of MAY to be more informative than MUST in this sentence. i.e. The user MAY use the command to encrypt the wallet. (Indeed, the user might even use alternative means of encrypting/decrypting a wallet). But the user shouldn’t think that the wallet must be encrypted.

lsilva01 force-pushed on Jul 28, 2021

Rspigler commented at 0:12 am on July 29, 2021: contributor

ACK bd7bb52b79a5b6e272fcb178b43f1845c56de797

Great documentation!

in doc/managing-wallet.md:50 in bd7bb52b79 outdated

45+
46+Note that if the passphrase is lost, all the coins in the wallet will also be lost forever.
47+
48+### 1.3 Backing Up the Wallet
49+
50+Wallets can be safely copied to another destination. This backup file should be stored on offline devices only, such as on a USB drive or an external hard drive.

unknown commented at 1:15 am on July 29, 2021:

Not sure about this. Offline device is better than Online device but still not safe.

Rspigler commented at 5:52 am on July 29, 2021:

If not offline or online, where?

NelsonGaldeman commented at 10:50 am on July 29, 2021:

I think he meant that not any offline device is safe. But I would say the lack of security @prayank23 pointed out on some offline devices may be out of the scope of this documentation. I don’t think we can do anything in terms of malware in the device the user is tranfering the backup to or the one he is using to do the backup.

What I understand of Wallets can be safely copied is it refers to the fact that the funds will be recoverable just backuing up that file. Then he is emphasizing the fact that online storage is a bad idea, which I think we all agree.

unknown commented at 11:41 am on July 29, 2021:

If not offline or online, where?

Offline is okay but USB drives cannot be trusted and in some cases external hard drive as well. So if I had to rephrase the above thing and add few more options, it would be something like this:

Wallets should be safely copied to another destination. This backup file can be stored on multiple offline devices that you trust will work in emergency, never connected to online devices and no malware. Test your backups regularly to avoid any issues in future. Also printing the output for dumpwallet on a sheet of paper could be helpful in some cases.

I think he meant that not any offline device is safe. But I would say the lack of security @prayank23 pointed out on some offline devices may be out of the scope of this documentation.

Agree. Can’t go in to details but this repository is used by lot of users with different levels of technical expertise so nobody should trust one file on some USB.

Rspigler commented at 5:35 pm on July 29, 2021:

Sure, there are security issues with USB devices (propriety firmware, etc) - but I think it is out of scope for this doc (for example, we don’t tell our users to only use Linux/Power ISA devices).

I do like the idea of including dumpwallet as an option - especially with this PR hopefully being merged (although we should inform that this option would lose metadata)

jonatack commented at 9:26 am on August 1, 2021:

0Wallets can be copied to another destination. This backup file should be stored on offline devices only, such as on a USB drive or an external hard drive.

It may be clearer for this paragraph and the backupwallet one to be together so the user doesn’t interpret “copy” differently.

lsilva01 commented at 3:10 am on August 5, 2021:

The document originally had two sections on dumpwallet and importwallet. But it was removed for the reason cited in #22523 (review). From the reviews, it seems that there is a preference for a shorter and less verbose document. But I agree that dumpwallet backup and security notes are good follow-ups.

Rspigler commented at 8:28 am on August 5, 2021:

Makes sense. Maybe a listdescriptors follow up too https://github.com/bitcoin/bitcoin/pull/21500

lsilva01 commented at 9:33 pm on August 7, 2021:

Added the suggested security considerations.

in doc/managing-wallet.md:22 in bd7bb52b79 outdated

17+
18+Wallets are created in the `wallets` folder of the data directory. The default folder varies by operating systems, as shown below. The user can also change the default by using the `-datadir` or `-walletdir` initialization parameters.
19+
20+| Operating System | Default wallet directory                                    |
21+| -----------------|:------------------------------------------------------------|
22+| Linux            | `home/<user>/.bitcoin/wallets`                              |

jonatack commented at 9:11 am on August 1, 2021:

0| Linux            | `/home/<user>/.bitcoin/wallets`                             |

in doc/managing-wallet.md:8 in bd7bb52b79 outdated

0@@ -0,0 +1,85 @@
1+# Managing the Wallet
2+
3+## 1. Backing Up and Restoring The Wallet
4+
5+### 1.1 Creating the Wallet
6+
7+Since version 0.21, Bitcoin Core no longer has a default wallet.
8+Wallets can be created with the RPC command `createwallet`. More information about this command can be found running `bitcoin-cli help createwallet`.

jonatack commented at 9:11 am on August 1, 2021:

“RPC command” is redundant (the “C” in RPC means “call”)

0Wallets can be created with the `createwallet` RPC. More information about this command may be found by running `bitcoin-cli help createwallet`.

in doc/managing-wallet.md:18 in bd7bb52b79 outdated

13+
14+The `descriptors` parameter can be omitted if the intention is to create a legacy wallet.
15+
16+For now, the default type is the legacy wallet, but that should change in the near future.
17+
18+Wallets are created in the `wallets` folder of the data directory. The default folder varies by operating systems, as shown below. The user can also change the default by using the `-datadir` or `-walletdir` initialization parameters.

jonatack commented at 9:14 am on August 1, 2021:

0By default, wallets are created in the `wallets` folder of the data directory, which varies by operating system, as shown below. The user can change the default by using the `-datadir` or `-walletdir` initialization parameters.

in doc/managing-wallet.md:34 in bd7bb52b79 outdated

29+
30+The wallet may be encrypted with the following command:
31+
32+`$ bitcoin-cli -rpcwallet="wallet-01" encryptwallet "passphrase"`
33+
34+The `encryptwallet` command is used only when the wallet has not been encrypted yet. Otherwise, the `walletpassphrasechange` command should be used.

jonatack commented at 9:17 am on August 1, 2021:

0Once encrypted, the passphrase may be changed with the `walletpassphrasechange` command.

in doc/managing-wallet.md:30 in bd7bb52b79 outdated

25+
26+### 1.2 Encrypting the Wallet
27+
28+The `wallet.dat` file is not encrypted by default and is, therefore, vulnerable if an attacker gains access to the device where the wallet or the backups are stored.
29+
30+The wallet may be encrypted with the following command:

jonatack commented at 9:21 am on August 1, 2021:

0The wallet's private key may be encrypted with the following command:

in doc/managing-wallet.md:40 in bd7bb52b79 outdated

35+
36+`$ bitcoin-cli -rpcwallet="wallet-01" walletpassphrasechange "oldpassphrase" "newpassphrase"`
37+
38+The argument passed to `-rpcwallet` is the name of the wallet to be encrypted.
39+
40+The term "encrypt the wallet" used here is not very accurate. This command encrypts only the private key. All other wallet information, such as transactions, is still visible.

jonatack commented at 9:24 am on August 1, 2021:

0Only the wallet's private key is encrypted. All other wallet information, such as transactions, is still visible.

in doc/managing-wallet.md:42 in bd7bb52b79 outdated

37+
38+The argument passed to `-rpcwallet` is the name of the wallet to be encrypted.
39+
40+The term "encrypt the wallet" used here is not very accurate. This command encrypts only the private key. All other wallet information, such as transactions, is still visible.
41+
42+The wallet can also be encrypted in the `createwallet` command via the `passphrase` argument:

jonatack commented at 9:24 am on August 1, 2021:

0The wallet's private key can also be encrypted in the `createwallet` command via the `passphrase` argument:

in doc/managing-wallet.md:54 in bd7bb52b79 outdated

49+
50+Wallets can be safely copied to another destination. This backup file should be stored on offline devices only, such as on a USB drive or an external hard drive.
51+
52+If the wallet and backup are lost for any reason, the bitcoins related to this wallet will become permanently inaccessible.
53+
54+The `bitcoin-cli backupwallet "destination"` RPC command or `Backup Wallet` GUI menu item must be used to ensure the file is in a safe state when the copy is made. The destination parameter must include the name of the file. Otherwise, the command will return an error message like "Error: Wallet backup failed!" for descriptor wallets. If it is a legacy wallet, it will be copied and a file will be created with the default file name `wallet.dat`.

jonatack commented at 9:29 am on August 1, 2021:

0The `backupwallet` RPC or the `Backup Wallet` GUI menu item must be used to ensure the file is in a safe state when the copy is made. The destination parameter must include the name of the file. Otherwise, the command will return an error message like "Error: Wallet backup failed!" for descriptor wallets. If it is a legacy wallet, it will be copied and a file will be created with the default file name `wallet.dat`.

in doc/managing-wallet.md:60 in bd7bb52b79 outdated

55+
56+`$ bitcoin-cli -rpcwallet="wallet-01" backupwallet /home/node01/Backups/backup-01.dat`
57+
58+### 1.4 Backup Frequency
59+
60+The original Bitcoin Core wallet was a collection of unrelated private keys. If a non-HD wallet had received funds to an address and then restored from a backup made before the address was generated, then any funds sent to said address are lost forever because there's no deterministic mechanism to derive that address again.

jonatack commented at 9:35 am on August 1, 2021:

0The original Bitcoin Core wallet was a collection of unrelated private keys. If a non-HD wallet had received funds to an address and then was restored from a backup made before the address was generated, then any funds sent to that address would have been lost because there was no deterministic mechanism to derive the address again.

in doc/managing-wallet.md:62 in bd7bb52b79 outdated

57+
58+### 1.4 Backup Frequency
59+
60+The original Bitcoin Core wallet was a collection of unrelated private keys. If a non-HD wallet had received funds to an address and then restored from a backup made before the address was generated, then any funds sent to said address are lost forever because there's no deterministic mechanism to derive that address again.
61+
62+However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced a deterministic mechanism from which to derive keys; HD wallets. Users no longer lose funds when restoring old backups because all addresses are derived from the HD wallet seed.

jonatack commented at 9:37 am on August 1, 2021:

0Bitcoin Core [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced HD wallets with deterministic key derivation. With HD wallets, users no longer lose funds when restoring old backups because all addresses are derived from the HD wallet seed.

in doc/managing-wallet.md:64 in bd7bb52b79 outdated

59+
60+The original Bitcoin Core wallet was a collection of unrelated private keys. If a non-HD wallet had received funds to an address and then restored from a backup made before the address was generated, then any funds sent to said address are lost forever because there's no deterministic mechanism to derive that address again.
61+
62+However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced a deterministic mechanism from which to derive keys; HD wallets. Users no longer lose funds when restoring old backups because all addresses are derived from the HD wallet seed.
63+
64+This means that a single backup is enough to recover the coins at any time. It is still recommended to make regular backups (once a week) or after a significant amount of new transactions to maintain the metadata such as labels.

jonatack commented at 9:39 am on August 1, 2021:

This sentence should probably be appended to the preceding paragraph.

0This means that a single backup is enough to recover the coins at any time. It is still recommended to make regular backups (once a week) or after a significant number of new transactions to maintain the metadata, such as labels.

in doc/managing-wallet.md:66 in bd7bb52b79 outdated

61+
62+However, [version 0.13](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.13.0.md) introduced a deterministic mechanism from which to derive keys; HD wallets. Users no longer lose funds when restoring old backups because all addresses are derived from the HD wallet seed.
63+
64+This means that a single backup is enough to recover the coins at any time. It is still recommended to make regular backups (once a week) or after a significant amount of new transactions to maintain the metadata such as labels.
65+
66+Metadata cannot be retrieved from a blockchain rescan, so if the backup is too old, the metadata will be lost forever.

jonatack commented at 9:41 am on August 1, 2021:

It seems this sentence should be appended to the previous paragraph.

lsilva01 commented at 1:05 am on August 5, 2021:

Done.

jonatack commented at 9:46 am on August 1, 2021: member

Concept ACK.

I wonder if the RPC usage examples aren’t redundant with the RPC helps. If an RPC and its help are updated, this document would be out of date. Perhaps it would be better to just point the user to the RPC help.

In some places this is a bit verbose and it may be good to tighten it up where possible.

Some suggestions below.

lsilva01 force-pushed on Aug 5, 2021

Zero-1729 approved

Zero-1729 commented at 9:12 am on August 5, 2021: contributor

ACK 2f4be46

An invaluable addition to the docs!

jonatack commented at 3:02 pm on August 5, 2021: member

utACK 2f4be46627acfa2cea98563e5a5363b2a7cdb60c

Possibly consider naming the file doc/managing-wallets.md.

Rspigler commented at 9:09 pm on August 5, 2021: contributor

ACK 2f4be46627acfa2cea98563e5a5363b2a7cdb60c

unknown approved

unknown commented at 11:09 am on August 6, 2021: none

ACK https://github.com/bitcoin/bitcoin/pull/22523/commits/2f4be46627acfa2cea98563e5a5363b2a7cdb60c

in doc/managing-wallet.md:30 in 2f4be46627 outdated

25+
26+### 1.2 Encrypting the Wallet
27+
28+The `wallet.dat` file is not encrypted by default and is, therefore, vulnerable if an attacker gains access to the device where the wallet or the backups are stored.
29+
30+The wallet's private key may be encrypted with the following command:

harding commented at 1:45 pm on August 6, 2021:

I think this can be addressed in a follow up, but I believe we generate a new HD seed when the wallet goes from unencrypted to encrypted, and likewise (on the lines below) when the passphrase is changed. That means the user should create a new backup immediately after encrypting or changing the passphrase, otherwise any bitcoins received to the new seed cannot be recovered from the previous backups.

Rspigler commented at 9:08 pm on August 6, 2021:

Yes, I mentioned that here (actually quoted you) :) And the text was changed from “…private key must be encrypted…” to “…private key may be encrypted….” but it is probably a good idea to include more information on why this can be dangerous.

lsilva01 commented at 9:32 pm on August 7, 2021:

Added information about tradeoff between encrypted wallet and risk of forgotten passphrases. Also added information about the new HD seed that is created when the wallet is encrypted or the passphrase changes. These are really important considerations.

harding approved

harding commented at 1:50 pm on August 6, 2021: contributor

ACK 2f4be46627acfa2cea98563e5a5363b2a7cdb60c reviewed in GitHub web UI. Text looks both accurate and informative to me. One non-blocking note below.

I do think there are some developers of the opinion that encryption does not add a meaningful amount of security in normal usecases and significantly increases the risk of coin loss due to forgotten passphrases. It might be worth noting something about those tradeoffs in followup PRs.

Thanks for writing this @lsilva01!

in doc/managing-wallet.md:8 in 2f4be46627 outdated

0@@ -0,0 +1,83 @@
1+# Managing the Wallet
2+
3+## 1. Backing Up and Restoring The Wallet
4+
5+### 1.1 Creating the Wallet
6+
7+Since version 0.21, Bitcoin Core no longer has a default wallet.
8+Wallets can be created with the `createwallet` RPC. More information about this command may be found by running `bitcoin-cli help createwallet`.

achow101 commented at 5:32 pm on August 6, 2021:

I think it would be useful to provide instructions for creating a wallet via the GUI.

lsilva01 commented at 9:28 pm on August 7, 2021:

Done.

in doc/managing-wallet.md:50 in 2f4be46627 outdated

45+
46+Note that if the passphrase is lost, all the coins in the wallet will also be lost forever.
47+
48+### 1.3 Backing Up the Wallet
49+
50+Wallets can be copied to another destination. This backup file should be stored on offline devices only, such as on a USB drive or an external hard drive.

achow101 commented at 5:32 pm on August 6, 2021:

It is potentially dangerous to just copy the wallet file. I don’t think we should encourage users to just copy wallet files. Rather they should use the built in backup functionality.

jonatack commented at 5:48 pm on August 6, 2021:

Yes. I was trying to get at that with #22523 (review).

lsilva01 commented at 9:28 pm on August 7, 2021:

It has been changed to make it clearer that the built in backup should be used.

in doc/managing-wallet.md:70 in 2f4be46627 outdated

65+
66+Wallets created before version 0.13 are not HD and must be backed up every 100 keys used since the previous backup, or even more often to maintain the metadata.
67+
68+### 1.5 Restoring the Wallet From a Backup
69+
70+To restore a wallet, an empty one must be created first. The backup file then rewrites the `wallet.dat` of this new wallet.

achow101 commented at 5:34 pm on August 6, 2021:

No, this is potentially dangerous. This entire section encourages a potentially dangerous restore process.

The correct process is to create a directory in the walletdir with a name that the user wants the wallet to be named, and then copy the wallet.dat file into that directory. There should not be any overwriting of anything.

lsilva01 commented at 9:26 pm on August 7, 2021:

Creating a new folder is actually safer than overwriting it. Updated the section.

in doc/managing-wallet.md:54 in 2f4be46627 outdated

49+
50+Wallets can be copied to another destination. This backup file should be stored on offline devices only, such as on a USB drive or an external hard drive.
51+
52+If the wallet and backup are lost for any reason, the bitcoins related to this wallet will become permanently inaccessible.
53+
54+The `backupwallet` RPC or the `Backup Wallet` GUI menu item must be used to ensure the file is in a safe state when the copy is made. The destination parameter must include the name of the file. Otherwise, the command will return an error message like "Error: Wallet backup failed!" for descriptor wallets. If it is a legacy wallet, it will be copied and a file will be created with the default file name `wallet.dat`.

achow101 commented at 5:35 pm on August 6, 2021:

It would be useful to also mention how backups can be performed via the GUI.

lsilva01 commented at 9:23 pm on August 7, 2021:

Done.

achow101 changes_requested

lsilva01 force-pushed on Aug 7, 2021

ghost commented at 10:00 pm on August 7, 2021: none

reACK https://github.com/bitcoin/bitcoin/pull/22523/commits/58c6486305d0187b776fde4f941e619378e626eb

Changes made based on the suggestions by achow101 look good. Suggestion about restoring wallet is important and something that I missed in my last review.

in doc/managing-wallets.md:8 in 58c6486305 outdated

0@@ -0,0 +1,92 @@
1+# Managing the Wallet
2+
3+## 1. Backing Up and Restoring The Wallet
4+
5+### 1.1 Creating the Wallet
6+
7+Since version 0.21, Bitcoin Core no longer has a default wallet.
8+Wallets can be created with the `createwallet` RPC. More information about this command may be found by running `bitcoin-cli help createwallet`.

Rspigler commented at 11:03 pm on August 8, 2021:

Include GUI option here, as mentioned by @achow101 ?

lsilva01 commented at 1:17 pm on August 9, 2021:

Done.

in doc/managing-wallets.md:62 in 58c6486305 outdated

57+
58+`$ bitcoin-cli -rpcwallet="wallet-01" backupwallet /home/node01/Backups/backup-01.dat`
59+
60+In the GUI, the wallet is selected in the `Wallet` drop-down list in the upper right corner. If this list is not present, the wallet can be loaded in `File` ->`Open wallet` if necessary. Then, the backup can be done in `File` -> `Backup Wallet...`.
61+
62+This backup file can be stored on one or multiple offline devices, which must be reliable enough to work in an emergency, never have connected to online devices, and be malware free. Backup files can be regularly tested to avoid problems in the future.

Rspigler commented at 11:06 pm on August 8, 2021:

0This backup file can be stored on one or multiple offline devices, which must be reliable enough to work in an emergency, never be connected to online devices, and be malware free. Backup files can be regularly tested to avoid problems in the future.

lsilva01 commented at 1:18 pm on August 9, 2021:

Done.

in doc/managing-wallets.md:80 in 58c6486305 outdated

75+
76+### 1.5 Restoring the Wallet From a Backup
77+
78+To restore a wallet, first a directory must be created in the `walletdir` with a name that the user wants the wallet to be named, and then copy the backup file into that directory. The copied file must be named `wallet.dat`.
79+
80+Then the user can load the wallet passing the name of the  newly created directory as parameter.

Rspigler commented at 11:08 pm on August 8, 2021:

0Then the user can load the wallet passing the name of the newly created directory as parameter.

nit: extra space

lsilva01 commented at 1:18 pm on August 9, 2021:

Done.

lsilva01 force-pushed on Aug 9, 2021

in doc/managing-wallets.md:28 in 0e0603235f outdated

23+| -----------------|:------------------------------------------------------------|
24+| Linux            | `/home/<user>/.bitcoin/wallets`                             |
25+| Windows          | `C:\Users\<user>\AppData\Roaming\Bitcoin\wallets`           |
26+| macOS            | `/Users/<user>/Library/Application Support/Bitcoin/wallets` |
27+
28+### 1.2 Encrypting the Wallet

achow101 commented at 7:16 pm on August 9, 2021:

Perhaps a section on unlocking the wallet? There’s the walletpassphrase RPC, and the GUI will prompt to unlock when needed.

Rspigler commented at 7:42 pm on August 9, 2021:

Will re-ACK after this!

lsilva01 commented at 5:06 am on August 11, 2021:

Section added.

meshcollider commented at 0:21 am on August 11, 2021: contributor

Looks good, ACK 0e0603235f81c6208298ebaa0b0db8158be57d51

Happy to re-ACK if you add some information about unlocking as achow101 suggested.

lsilva01 force-pushed on Aug 11, 2021

meshcollider commented at 11:44 pm on August 12, 2021: contributor

re-ACK 3c511e80b4cefc861fed7698c938ad15945eb10d

achow101 commented at 0:08 am on August 13, 2021: member

ACK 3c511e80b4cefc861fed7698c938ad15945eb10d

unknown approved

unknown commented at 0:12 am on August 13, 2021: none

reACK https://github.com/bitcoin/bitcoin/pull/22523/commits/3c511e80b4cefc861fed7698c938ad15945eb10d

Changes made since last review:

Add GUI in createwallet
Section for unlocking wallet

nit: Some commands could look better if ``` is used instead of `

Example: wallet_management.md

Rspigler commented at 0:24 am on August 13, 2021: contributor

re-ACK 3c511e80b4cefc861fed7698c938ad15945eb10d

in doc/managing-wallets.md:32 in 3c511e80b4 outdated

27+
28+### 1.2 Encrypting the Wallet
29+
30+The `wallet.dat` file is not encrypted by default and is, therefore, vulnerable if an attacker gains access to the device where the wallet or the backups are stored.
31+
32+One way to mitigate this problem is to encrypt the wallet. However, this solution significantly increases the risk of losing coins due to forgotten passphrases. There is no way to recover a passphrase. This tradeoff should be well thought out by the user.

luke-jr commented at 0:27 am on August 13, 2021:

This isn’t really solved by wallet encryption. If the attacker has access to the device, they can probably install a keylogger as well.

Wallet encryption really only handles the “little brother” threat.

Rspigler commented at 12:40 pm on August 13, 2021:

Agreed, tried to point that out here

lsilva01 commented at 12:47 pm on August 17, 2021:

Added.

in doc/managing-wallets.md:10 in 3c511e80b4 outdated

 9+
10+The following command, for example, creates a descriptor wallet:
11+
12+`$ bitcoin-cli -named createwallet wallet_name="wallet-01" descriptors=true`
13+
14+In the GUI, the `Create a new wallet` button is displayed on the main screen when there is no wallet loaded. Alternatively, there is the option `File` ->`Create wallet`.

luke-jr commented at 0:27 am on August 13, 2021:

Nit: GUI instructions first would be better

lsilva01 commented at 12:43 pm on August 17, 2021:

Done.

in doc/managing-wallets.md:17 in 3c511e80b4 outdated

12+`$ bitcoin-cli -named createwallet wallet_name="wallet-01" descriptors=true`
13+
14+In the GUI, the `Create a new wallet` button is displayed on the main screen when there is no wallet loaded. Alternatively, there is the option `File` ->`Create wallet`.
15+
16+The `descriptors` parameter can be omitted if the intention is to create a legacy wallet.
17+

luke-jr commented at 0:28 am on August 13, 2021:

Nit: Drop blank line, these two sentences are too closely related.

lsilva01 commented at 12:43 pm on August 17, 2021:

Done.

in doc/managing-wallets.md:18 in 3c511e80b4 outdated

13+
14+In the GUI, the `Create a new wallet` button is displayed on the main screen when there is no wallet loaded. Alternatively, there is the option `File` ->`Create wallet`.
15+
16+The `descriptors` parameter can be omitted if the intention is to create a legacy wallet.
17+
18+For now, the default type is the legacy wallet, but that should change in the near future.

luke-jr commented at 0:28 am on August 13, 2021:

nit

0For now, the default type is the legacy wallet, but that is expected to change in a future release.

lsilva01 commented at 12:42 pm on August 17, 2021:

Done.

in doc/managing-wallets.md:83 in 3c511e80b4 outdated

78+
79+`$ bitcoin-cli -rpcwallet="wallet-01" backupwallet /home/node01/Backups/backup-01.dat`
80+
81+In the GUI, the wallet is selected in the `Wallet` drop-down list in the upper right corner. If this list is not present, the wallet can be loaded in `File` ->`Open wallet` if necessary. Then, the backup can be done in `File` -> `Backup Wallet...`.
82+
83+This backup file can be stored on one or multiple offline devices, which must be reliable enough to work in an emergency, never be connected to online devices, and be malware free. Backup files can be regularly tested to avoid problems in the future.

luke-jr commented at 0:31 am on August 13, 2021:

“never be connected to online devices” sounds a bit strong. Maybe just express that getting the file can compromise the wallet?

Rspigler commented at 12:46 pm on August 13, 2021:

Maybe something like: “retrieving the backup file can compromise the wallet if your computer has malware. Ways to minimize this is to not connect your backup to an online device”?

lsilva01 commented at 12:48 pm on August 17, 2021:

Added this suggestion.

in doc/managing-wallets.md:85 in 3c511e80b4 outdated

80+
81+In the GUI, the wallet is selected in the `Wallet` drop-down list in the upper right corner. If this list is not present, the wallet can be loaded in `File` ->`Open wallet` if necessary. Then, the backup can be done in `File` -> `Backup Wallet...`.
82+
83+This backup file can be stored on one or multiple offline devices, which must be reliable enough to work in an emergency, never be connected to online devices, and be malware free. Backup files can be regularly tested to avoid problems in the future.
84+
85+If the wallet and backup are lost for any reason, the bitcoins related to this wallet will become permanently inaccessible.

luke-jr commented at 0:31 am on August 13, 2021:

nit

0If both the wallet and all backups are lost for any reason, the bitcoins related to this wallet will become permanently inaccessible.

lsilva01 commented at 12:42 pm on August 17, 2021:

Done.

luke-jr changes_requested

achow101 commented at 7:20 pm on August 15, 2021: member

With #22541 now merged, this should be updated to recommend using restorewallet to restore.

Document about wallet backup and restoration

This PR adds a document about backing up and restoring the Bitcoin Core wallet as suggested in the issue https://github.com/bitcoin/bitcoin/issues/20149 .

ce4e90629e

lsilva01 force-pushed on Aug 17, 2021

achow101 commented at 11:23 pm on August 17, 2021: member

re-ACK ce4e90629ed70f4d45906d7e79563b2f7ad6493e

ghost commented at 3:04 am on August 18, 2021: none

reACK https://github.com/bitcoin/bitcoin/commit/ce4e90629ed70f4d45906d7e79563b2f7ad6493e

Major changes made since last review:

Add restorewallet as suggested in #22523 (comment) L113-L125
Rephrase sentence about malware and online devices based on #22523 (review) L97
Change order and mention GUI instructions first as suggested in #22523 (review) L10
Add information about keylogger in encryption section based on #22523 (review) L34
Use ``` instead of ` for some commands as suggested in #22523#pullrequestreview-729169759

meshcollider commented at 5:27 am on August 18, 2021: contributor

re-ACK ce4e90629ed70f4d45906d7e79563b2f7ad6493e

Going to merge this now 🎉

meshcollider merged this on Aug 18, 2021

meshcollider closed this on Aug 18, 2021

Rspigler commented at 1:06 pm on August 18, 2021: contributor

Post-merge re-ACK ce4e90629ed70f4d45906d7e79563b2f7ad6493e!

MarcoFalke commented at 7:17 pm on August 18, 2021: member

Shouldn’t the doc be linked from the main doc?

fanquake referenced this in commit 1ea11e10ac on Aug 19, 2021

fanquake commented at 0:54 am on August 19, 2021: member

Shouldn’t the doc be linked from the main doc?

Done in #22739.

fanquake referenced this in commit b784ab10f5 on Aug 19, 2021

sidhujag referenced this in commit ae44d24379 on Aug 20, 2021

sidhujag referenced this in commit c84ee7261b on Aug 20, 2021

lsilva01 deleted the branch on Aug 23, 2021

DrahtBot locked this on Aug 23, 2022

Document about wallet backup and restoration #22523