creating a backup of blockchain and index

3ntranced commented at 3:21 am on July 11, 2020: none

Is your feature request related to a problem? Please describe.

I changed my external hdd enclosure for an hdd which was storing bitcoin data directory and bitcoind said chainstate was corrupted upon restart. I had rsynced bitcoin data directory to try and back it up in the past while bitcoind was running and restoring that copy didnt work either. So I had to -reindex-chainstate which took 3 days.

Describe the solution you’d like

Some way to backup the bitcoin data directory “live”, without having to stop bitcoind.

Describe alternatives you’ve considered

rsynced the directory while its running and starting bitcoind from that copy didnt work.

Additional context

3ntranced added the label Feature on Jul 11, 2020

MarcoFalke added the label Block storage on Jul 11, 2020

ghost commented at 3:06 pm on July 13, 2020: none

@3ntranced It’s an important issue but will take a while to implement. In the meantime: I backup the Bitcoin blockchain (on Ubuntu Linux) without stopping bitcoind every month on my server (basically) like this:

0bitcoin-0.20.0/src/bitcoin-cli savemempool
1rsync -L --delete -av ".bitcoin/" /media/external/.bitcoin
2rsync -L --delete -av ".bitcoin/" /media/external/.bitcoin
3rsync -L --delete -av ".bitcoin/" /media/external/.bitcoin
4rsync -L --delete -av ".bitcoin/" /media/external/.bitcoin

Usually rsync doesn’t do anything anymore for the last 2 executions. I then run bitcoin-qt on a different computer (but you could run it on your server too) like this (connecting only to my own bitcoind): bitcoin-qt -connect=192.168.178.20 -blocksonly=1 -listen=0 -datadir=/media/external/.bitcoin

Bitcoin-qt always repairs the blockchain without problems and this way you’ll know if the backup was successful. You can even stop bitcoin-qt cleanly and run it again.

3ntranced commented at 3:55 pm on July 13, 2020: none

Ok thank you, I ll try to understand and give this a go!

3ntranced commented at 4:16 am on July 15, 2020: none

@sumBTC bitcoin-qt -connect=192.168.178.20 -blocksonly=1 -listen=0 -datadir=/media/external/.bitcoin this step is purely to verify the backup correct? if the backup is ok, of course you dont know until you verify, then even if you dont verify, you can rsync the directory back and it will work, right?

ghost commented at 5:52 am on July 15, 2020: none

this step is purely to verify the backup correct?

To verify and “repair” the blockchain. Before syncing the blockchain, bitcoin-qt will see that bitcoind wasn’t shutdown correctly and will first “repair” the blockchain.

In my case, bitcoin-qt never had a problem “repairing” the blockchain so you could skip this step if you want to but it’s more risky.

You can always rsync again.

3ntranced commented at 2:59 am on August 10, 2020: none

rsync a bunch of times works really well.

3ntranced closed this on Aug 10, 2020

DrahtBot locked this on Feb 15, 2022

creating a backup of blockchain and index #19487