Manual-pruning cursor rewind #19807

Is your feature request related to a problem? Please describe.

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

Setting a lower-than-current prune height via pruneblockchain currently requires a full resync. On slower machines, this can take a very long time. It is also unnecessary.

Describe the solution you'd like

<!-- A clear and concise description of what you want to happen. -->

Since all block headers are retained for PoW and chain reorg purposes, bitcoind could feasibly re-download the raw blocks that are lower than the old cursor, and above the new cursor, and check their validity against the retained headers.

Downloading arbitrary blocks is certainly possible (this is how neutrino clients work) however, downloading old blocks would not include their UNDO data. This means you might be able to download old blocks to (for example) rescan a wallet but you would not be able to handle a deeper chain reorg than the original prune depth.

What might be possible is to increase the prune depth by 1 with every new incoming block, until a new desired prune depth is reached. The mechanism there would be to simply NOT delete the oldest block from disk until the new prune depth is reached. This would require no extra historical block downloads or weird half-validation.

Would that be sufficient for you use-case, @ProofOfKeags ?

Slightly related: #21267.

See #24286 (comment) I think these two issues can be closed and a feature we could work on is a neutrino mode for a wallet attached to a pruning node... ?

See #24286 (comment) I think these two issues can be closed and a feature we could work on is a neutrino mode for a wallet attached to a pruning node... ?

yeah, I have been working on it @pinheadmz. Just answered in #21267. Got sidetracked with some other useful features to make the mainnet tests that have been running less painful.

would not be able to handle a deeper chain reorg than the original prune depth.

The smallest prune size is 550MB. If we have a 140 block reorg, we have a pretty apocalyptic scenario. afaiu there are numerous assumptions within bitcoin core that a 100 block reorg is impossible. Under that assumption, your described limitation isn't actually a limitation.

Under that assumption, your described limitation isn't actually a limitation.

Sure, but then if you're not re-downloading old blocks for reorg protection I can think of two other use cases:

1. serving old blocks to bootstrapping nodes
2. wallet rescans

Since we are talking about pruned nodes anyway I don't think (1) matters, does it? That's why I think we should just focus on neutrino wallet + pruned node as the feature, which is being discussed in the other linked issue.

serving old blocks to bootstrapping nodes

a node should just serve the blocks it has. The cursor rewind should cause it to download earlier blocks as if it was doing IBD only it doesn't have to do any consensus validation except that the block data matches the expected blockhash

wallet rescans

A rescan has to go back to the wallet birthday irrespective of the pruning cursor.

One other thing to note. I no longer personally need this feature, but I'm simply describing the need as I understand it should people want to be using manual pruning in the future.

I no longer personally need this feature, but I'm simply describing the need as I understand it should people want to be using manual pruning in the future.

Ok thanks for engaging me anyway. I still don't see why anyone would want this feature except for re-scanning an old wallet on a pruned node. If you can provide a reasonable use case we can keep this issue open, but otherwise I think we should close and follow #21267

a node should just serve the blocks it has.

Actually pruned nodes are never supposed to serve any blocks deeper than tip-288:

https://github.com/bitcoin/bips/blob/master/bip-0159.mediawiki#counter-measures-for-peer-fingerprinting

https://github.com/bitcoin/bitcoin/blob/5c2bb2b54cc7fceef4b9db631536ee6ed8129864/src/net_processing.cpp#L2165-L2173

I still don't see why anyone would want this feature except for re-scanning an old wallet on a pruned node

We ran into this issue as a result of having different dependent services that wanted access to different amounts of block history. If you prune to height N, and you install a different service that wants height N-M, the only solution is to do a full resync and download all the way back to N-N (aka 0). When M is small, this is needlessly expensive.

It is probably a rare use case, these days as many bitcoiners are now opting for full archive nodes or ditching running a node altogether, making pruning of any kind a declining use case at this point. I certainly wouldn't contest closing it. I think it was more of an artifact of the time when it was opened, and pruning was more necessary for home node viability.

the only solution is to do a full resync and download all the way back to N-N (aka 0).

There is also getblockfrompeer #20295 I just tried this on my pruned node to inspect blocks below the prune depth, pretty neat!

This issue is unlikely to be fixed in Bitcoin Core. We'll close for now, but feel free to open another issue or pull request with a fix.