Pasting untrusted blocks/ and chainstate/ to new pruned node is not safe

watashikokoto commented at 1:40 pm on February 15, 2016: none

It is never safe to use untrusted copies of those directories

But how do we prevent lazy people from doing this?

watashikokoto commented at 1:40 pm on February 15, 2016: none

https://bitcointalk.org/index.php?topic=1364723.msg13888022#msg13888022

watashikokoto commented at 1:54 pm on February 15, 2016: none

Actual result: Core trusts index and chainstate blindly

Expected results: Core loads the pruned index and chainstate Core catches up Core compares the utxo - hash with other network nodes If some other node has different database hash the node will fail to start EDIT: or ban the liar node

Alternative expected result: Core loads the pruned index and chainstate Core catches up User is notified to add configuration option that the user is aware the chain might be untrusted The value of the option could be dependent on some kind of hash derived from latest block or machine specific random uuid This will prevent user to download the configuration file from untrusted source because attacker cannot predict this value

laanwj added the label Brainstorming on Feb 15, 2016

laanwj commented at 1:56 pm on February 15, 2016: member

There are legit reasons for doing this. E.g. between two nodes that you control yourself.

Lazy people that copy the chainstate from someone untrusted indeed shoot themselves in the feet.

watashikokoto commented at 2:01 pm on February 15, 2016: none

The unpredictable option could fix the obvious copy blocks/ chainstate/ and bitcoin.conf scenario with prune option set

watashikokoto commented at 2:09 pm on February 15, 2016: none

Alternative fix: Release the signed zip of both directories and bitcoin.conf with prune= on a trusted release channel Problem: it’s 1.6GB

watashikokoto commented at 2:15 pm on February 15, 2016: none

I’m leaving, If anything needed message me on bitcointalk.org

laanwj added the label UTXO Db and Indexes on Feb 16, 2016

laanwj added the label Block storage on Feb 16, 2016

unsystemizer commented at 1:32 pm on February 16, 2016: contributor

It is never safe to use untrusted copies of those directories But how do we prevent lazy people from doing this?

Why should we? How do we prevent people from using the password 1234?

Your proposed enhancements don’t prevent anyone from doing that either (the saving grace). They do, however, impose a new and additional workload (“The Stupid Tax”) on all nodes who use this convenient method of replicating blockchain data, so I hope they won’t be implemented.

laanwj commented at 1:42 pm on February 16, 2016: member

Yes, this is more of an awareness issue.

The outcome of this should probably not be a code change, but adding of a warning to documentation and guides involving pruning.

sipa commented at 6:28 pm on February 16, 2016: member

You can’t just ask other nodes to tell you what they think the UTXO set is, because they could lie, and there is no way to verify it (unless we have UTXO commitments, and even then it would mean trusting miners).

You can’t detect that the chainstate was copied from elsewhere, because copies are indistinguishable from the original (unless you include some host-specific information, which is prone to cause problems when e.g. operating system or hardware changes).

Frankly, I don’t think this is a problem beyond documenting it.

watashikokoto commented at 7:26 pm on February 16, 2016: none

Again there are two different issues

when user copies whole virtual machine image, for this there’s no solution since program itself could be compromised
when user installs the program correctly, yet pastes corrupted chainstate. The result is an empty node, the opposite of a full node.

1 is out of scope of this issue I argue that 2 is solvable and a fully validating software should have the capability to eventually discover the corruption either manually or automatically over time

solving it manually is not difficult, the user can click some button and get the utxo hash or click some button and compare the hash from other nodes, if there’s some liar show some pop warn about the suspicious case or let user to click some button to ban the liar

automatic solution is really really complex and I don’t know any that is guaranteed to succeed perhaps the good utxo hash can be circulated using an alert key

I’m going to sleep so again contact me on forum

PRabahy commented at 1:55 am on February 17, 2016: contributor

Why would a user ever do 2? That has never been a documented process and I don’t know of anyone suggesting informally. The closest supported thing I have ever seen is bootstrap.dat, but that was trust free and has since been deprecated.

If you are copying the chainstate from one trusted node, then you shouldn’t run into any issues because they are both trusted. If you a worried about them getting corrupted during the copy, then that sounds like an issue with your copy utility not bitcoin.

laanwj commented at 10:27 am on February 24, 2016: member

Agree with @PRabahy and @sipa. Not a software issue, closing this.

And as always, better documentation is welcome. If you see anyone recommending to install an untrusted chainstate, especially on a large scale, please let us know.

laanwj closed this on Feb 24, 2016

andronoob commented at 12:53 pm on February 18, 2019: none

Maybe some warning should be put on the syncing screen, so that every new user of Bitcoin Core could see it?

I think even if some people may insist to use 12345 as their password, a reminder for them is still good though.

By the way, I have some questions:

Is it safe to copy & paste blocks/ subdirectory only?
Is it possible to import blkXXXXX.dat directly? (without concatenating them into a single huge bootstrap.dat)
Can importing from bootstrap.dat work well with pruning enabled?

I’m curious about a fork-coin known as Bitcoin Clashic. I once tried a trick to avoid creating bootstrap.dat, but it didn’t seem to work well, with Bitcoin Core or Bitcoin Clashic (Bitcoin ABC):

Spin up a VM which is running Ubuntu 18.04.1 LTS x64, while the host OS is Windows 10 1809 x64;
Use SMB to share the blocks/ subdirectory on host OS;
Install nbd-server and nbd-client on VM guest OS;
Mount the SMB share in the guest OS;
Create symlinks for each blkXXXXX.dat, for example: block.0 -> blk00000.dat, block.1 -> blk00001.dat…
Configure smb-server and smb-client to create /dev/nbd0 which “virtually concatenate every blkXXXXX.dat together”;
Create a “bootstrap.dat” symlink pointing to /dev/nbd0;
Set prune=550 in ~/.bitcoin.conf
Start (Qt GUI wallet version of) Bitcoin ABC (or Bitcoin Core) to let it import blocks from the “virtual bootstrap.dat”.

As far as I could remember, after a long-lasting HDD buzzing, only a small portion (~140) of blocks were successfully imported.

I once struggled to bypass this problem, like delete&re-create symlinks for a little number (like 3) of blkXXXXX.dat file only, but it didn’t work, either, or even worse - Bitcoin ABC (or Bitcoin Core) would straightforwardly crash due to insufficient storage (tmpfs) space.

fanquake commented at 12:59 pm on February 18, 2019: member

I’m curious about a fork-coin known as Bitcoin Clashic. I once tried a trick to avoid creating bootstrap.dat, but it didn’t seem to work well, with Bitcoin Core or Bitcoin Clashic (Bitcoin ABC):

Alt coins are off topic here. If you have an issue with Bitcoin Core, that isn’t solved in an existing issue, please open a new one (try not to bump year old issues). If you have a general technical question, the bitcoin stackexchange is the place to ask it.

fanquake locked this on Feb 18, 2019

Pasting untrusted blocks/ and chainstate/ to new pruned node is not safe #7535