Relay blocks when pruning

sdaftuar commented at 0:06 am on May 16, 2015: member

This is an alternate to #6122, built off of #6130. With #6130, it should be safe to always inv the tip regardless of what height our peer is at, because we’ll only inv blocks that we actually have.

As noted in the discussion of #6122, 0.10 and later peers would still not be able to download a reorg from a pruning peer; they would only be able to receive new blocks that build on their tip (0.9 and earlier peers using getblocks would be able to download reorgs from pruning nodes).

ping @cozz

laanwj added the label P2P on May 16, 2015

cozz commented at 11:30 am on May 16, 2015: contributor

Fine with me. Reorg is still a problem, but relaying the tip is at least better than relaying nothing.

To solve the reorg-problem, I would want to also call FindNextBlocksToDownload on pruned-nodes, if it is ensured that we can not ask them for pruned blocks. @sipa Would this be possible, to just call FindNextBlocksToDownload also for pruned nodes, when we are close to being synced?

morcos commented at 9:00 pm on May 17, 2015: member

ACK. Tested for tip relay for master and reorg relay (up to a certain depth) for 0.9.

jonasschnelli commented at 11:04 am on May 20, 2015: contributor

I just sopped my pruned node, added this PR on top of the current master, ran again and had the following issue:

Stopping:

 0Prune: target=550MiB actual=467MiB diff=82MiB min_must_keep=356959 removed 0 blk/rev pairs
 1receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=***:8334, peer=18138
 2receive version message: /Snoopy:0.1/: version 60001, blocks=0, us=***:8334, peer=18139
 3receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=***:8334, peer=18140
 4receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=***:8334, peer=18141
 5receive version message: /getaddr.bitnodes.io:0.1/: version 70002, blocks=357246, us=***:8334, peer=18142
 6receive version message: /getaddr.bitnodes.io:0.1/: version 70002, blocks=357246, us=***:8334, peer=18143
 7receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=***:8334, peer=18144
 8ERROR: AcceptToMemoryPool: nonstandard transaction: dust
 9receive version message: /breadwallet:0.5.1/: version 70002, blocks=0, us=***:8334, peer=18145
10Added time data, samples 200, offset -1 (+0 minutes)
11receive version message: /Snoopy:0.1/: version 60001, blocks=0, us=***:8334, peer=18146
12receive version message: /getaddr.bitnodes.io:0.1/: version 70002, blocks=357247, us=****:8334, peer=18147
13receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=***:8334, peer=18148
14^Cnet thread interrupt
15msghand thread interrupt
16addcon thread interrupt
17opencon thread interrupt
18dumpaddr thread stop
19Shutdown: In progress...
20RPCAcceptHandler: Error: Operation canceled
21RPCAcceptHandler: Error: Operation canceled
22StopNode()
23Prune: target=550MiB actual=467MiB diff=82MiB min_must_keep=356959 removed 0 blk/rev pairs
24Shutdown: done

Restarting (after serval seconds/minutes):

 0AppInit2 : parameter interaction: -prune -> setting -disablewallet=1
 1Prune configured to target 550MiB on disk for block and undo files.
 2---snip---
 3Bitcoin version v0.10.99.0-4b30ade (2015-05-20 11:45:07 +0200)
 4Using OpenSSL version OpenSSL 1.0.1e 11 Feb 2013
 5Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
 6Default data directory ***
 7Using data directory ***
 8Using config file ***/bitcoin.conf
 9Using at most 125 connections (1024 file descriptors available)
10Using 8 threads for script verification
11scheduler thread start
12Allowing RPC connections from: 127.0.0.0/255.0.0.0 ::1/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
13Binding RPC on address ::1 port *** (IPv4+IPv6 bind any: 0)
14Binding RPC on address 127.0.0.1 port *** (IPv4+IPv6 bind any: 0)
15Bound to [::]:8334
16Bound to 0.0.0.0:8334
17Cache configuration:
18* Using 2.0MiB for block index database
19* Using 32.5MiB for chain state database
20* Using 65.5MiB for in-memory UTXO set
21init message: Loading block index...  
22Opening LevelDB in ***/blocks/index
23Opened LevelDB successfully
24Opening LevelDB in ***/chainstate
25Opened LevelDB successfully
26LoadBlockIndexDB: last block file = 271
27LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=70, size=30105947, heights=357178...357247, time=2015-05-19...2015-05-20)
28Checking all blk files are present... 
29Unable to open file ***/blocks/blk00264.dat
30: Error loading block database.
31
32Do you want to rebuild the block database now?
33: Error loading block database.
34
35Do you want to rebuild the block database now?
36Aborted block database rebuild. Exiting.
37Shutdown: In progress...

I doubt that it has something to do with this PR but it could lead to another existing issue.

jonasschnelli commented at 11:06 am on May 20, 2015: contributor

My /blocks dir contains the following:

0blk00268.dat  blk00269.dat  blk00270.dat  blk00271.dat  index  rev00268.dat  rev00269.dat  rev00270.dat  rev00271.dat

jonasschnelli commented at 11:14 am on May 20, 2015: contributor

Further up i can see in my log that blk00264.dat has been deleted because of pruning (around 9 days ago).

0receive version message: /btccrawler.ch:v0.1/: version 70002, blocks=355967, us=***:8334, peer=729
1ERROR: AcceptToMemoryPool: nonstandard transaction: dust
2Prune: target=550MiB actual=532MiB diff=17MiB min_must_keep=355679 removed 0 blk/rev pairs
3Prune: target=550MiB actual=388MiB diff=161MiB min_must_keep=355679 removed 1 blk/rev pairs
4Prune: UnlinkPrunedFiles deleted blk/rev (00264)
5UpdateTip: new best=00000000000000000cbfa2566c81e0bc78f37b66e35ff3657d552154ca9436ca  height=355968  log2_work=82.765851  tx=68384633  date=2015-05-11 20:16:36 progress=1.
600000  cache=0

sdaftuar commented at 12:15 pm on May 20, 2015: member

@jonasschnelli any chance you had been running with #6118 and then switched to running without? This looks exactly like what happens when trying to downgrade from lazy updating of mapBlockIndex. If that is the issue you can workaround by restarting your node with #6118 and then quickly stopping; the block index is refreshed on startup. You could then restart without the lazy updating code and things should work fine.

jonasschnelli commented at 12:19 pm on May 20, 2015: contributor

@sdaftuar pretty sure i did this. I’ll now try to run a master-full-node, sync up, add this PR on top and continue to confirm the missing-lazy-update issue.

jonasschnelli commented at 12:59 pm on May 20, 2015: contributor

Stopped a full-node, copied datadir, ran a new master-full-node with the just copied datadir and pruning target 550, stopped after a while, added this PR on top and it looks good.

Now testing block relying.

jonasschnelli commented at 9:24 am on May 21, 2015: contributor

Did some testing. Connected a fresh mainnet node to a pruned node. I was expecting that my fresh node can load at least the whole headers-first-chain but encountered:

0--- snip
1sending: pong (8 bytes) peer=1
2received: getheaders (997 bytes) peer=1
3getheaders -1 to 0000000000000000000000000000000000000000000000000000000000000000 from peer=1
4sending: headers (1 bytes) peer=1
5received: addr (30003 bytes) peer=1
6received: pong (8 bytes) peer=1
7received: inv (37 bytes) peer=1
8got inv: tx f6bfd86ac1d0a4a981fdb950da97c630254e716e3b05ef872a2ca94ebf892e7c  new peer=1
9--- snip

(no headers response to headers request)

Connecting a fresh node to a non-pruned full node results in:

0--- snip
1received: getheaders (997 bytes) peer=1
2getheaders -1 to 0000000000000000000000000000000000000000000000000000000000000000 from peer=1
3sending: headers (1 bytes) peer=1
4received: pong (8 bytes) peer=1
5received: headers (162003 bytes) peer=1
6more getheaders (2000) to end to peer=1 (startheight:357399)
7sending: getheaders (741 bytes) peer=1
8Requesting block 00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048 (1) peer=1
9--- snip

Will also try to connect a “half-synced” full node to a pruned node where the half-synced node has just some blocks over the prune level of the pruned node. But this needs a little bit time to set up.

sdaftuar commented at 1:32 pm on May 28, 2015: member

@jonasschnelli The fresh node won’t load the whole headers from the pruning node on startup, because we only choose full NODE_NETWORK nodes to sync the headers chain. However, if you were to wait long enough for the pruning node to inv a block, then at that point at getheaders message should be sent to the pruning node, and headers should then sync.

juscamarena commented at 4:05 am on June 10, 2015: none

Nice, I’m running 0.11.0rc1 and bitcoin in %appdata% is only taking up 1.33 GB of space.. Would love to be able to relay blocks. I’d actually leave this on with this small of a memory footprint.

in src/main.cpp: in f3b92daaf8 outdated

4150@@ -4153,6 +4151,14 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv,
4151                 LogPrint("net", "  getblocks stopping at %d %s\n", pindex->nHeight, pindex->GetBlockHash().ToString());
4152                 break;
4153             }
4154+            // If pruning, don't inv blocks unless we have on disk and are likely to still have
4155+            // for some reasonable time window that block relay might require.
4156+            const int nPrunedBlocksLikelyToHave = MIN_BLOCKS_TO_KEEP - 6;

sipa commented at 1:47 pm on June 14, 2015:

Can you use a symbolic constant here? Or some calculation based on consensus params?

sdaftuar commented at 7:44 pm on June 17, 2015:

@sipa Fixed to use a calculation based on consensus params.

sipa commented at 1:49 pm on June 14, 2015: member

I haven’t followed up on all proposed changes related to this. Is there a plan to also make peers use the inventories we send out? With a change like this, the immediate fetching logic may react to an fClient peer sending a block inv, but the asynchronous fetching won’t, making this (for now) useless for reorgs, for example.

sipa commented at 2:14 pm on June 14, 2015: member

I’m confused by the several related and seemingly-overlapping PRs here.

sdaftuar commented at 3:11 pm on June 16, 2015: member

@sipa Sorry for the PR confusion. #6130 fixes a bug where pruning nodes would respond badly to a getblocks message by inv’ing blocks they don’t actually have. This pull adds block relaying for pruning nodes, which would not work without the fix in #6130, so I built it on top.

Initially I thought the bug fix in #6130 should stand on its own, but on further thought I think it could only be triggered once we actually implement block relay. I’ll go ahead and close that pull in favor of this one.

As for next steps, yes this pull is insufficient for relaying reorgs to 0.10 and later peers. I was thinking that we’d hopefully be able to put together a sharding implementation and at that point we’d also implement the more intelligent block requesting at the same time. I guess if that work seems like it’s taking too long, we could (I guess before the 0.12 release) instead do some other change to the parallel-fetch code to try to request recent blocks from pruning peers. Would you rather see a solution like that as part of this pull?

sipa commented at 3:19 pm on June 16, 2015: member

No, just querying what the current state/plan is :)

Do not inv old or missing blocks when pruning

When responding to a getblocks message, only return inv's as
long as we HAVE_DATA for blocks in the chain, and only for blocks
that we aren't likely to delete in the near future.

0da6ae2dc3

Enable block relay when pruning ae6f957a62

sdaftuar force-pushed on Jun 17, 2015

sipa commented at 1:22 pm on June 21, 2015: member

Been running on bitcoin.sipa.be for a week, no problems.

sdaftuar commented at 1:31 pm on July 30, 2015: member

@jonasschnelli I saw your comment in #6460 about this PR – I think this could be merged as-is without any new service bits. Adding a service bit would be needed to implement a system for selectively downloading historical blocks from less-than-full nodes, but I don’t think it’s needed to enable relaying at the tip.

jonasschnelli commented at 1:33 pm on July 30, 2015: contributor

@sdaftuar: Agreed. Will test again soon and sry for the delay.

sipa commented at 1:40 pm on July 30, 2015: member

ACK (have done a “it works” test before). @laanwj Acceptable for 0.11?

jonasschnelli commented at 11:38 am on July 31, 2015: contributor

Running since >18h on a pruned synced full node (master). Served ~291887 blocks since than.

0jonasschnelli@server6:~$ cat ~/.bitcoin/debug.log | grep "sending: block" | wc -l
1291887

tested ACK

gmaxwell commented at 7:29 am on September 7, 2015: contributor

I’ve been testing this against master and it’s happily relaying at the tip among my peers without issue. Appears to work great.

Has someone tried the case where the peer is on a fork and needs to fetch futher back than the pruning window to complete the reorg?

gmaxwell commented at 11:09 pm on September 8, 2015: contributor

To be clear: ACK.

dcousens commented at 2:13 am on September 9, 2015: contributor

concept ACK

sdaftuar commented at 5:35 pm on September 21, 2015: member

@laanwj I think this is ready to be merged; are there any outstanding concerns here?

laanwj merged this on Sep 23, 2015

laanwj closed this on Sep 23, 2015

laanwj referenced this in commit 999c8be81a on Sep 23, 2015

luke-jr referenced this in commit b5753f1ddd on Dec 28, 2015

zkbot referenced this in commit 245f5ddf12 on Nov 14, 2018

zkbot referenced this in commit 3b6487d451 on Nov 15, 2018

zkbot referenced this in commit 570e09ea59 on Nov 15, 2018

MarcoFalke locked this on Sep 8, 2021

Relay blocks when pruning #6148