[WIP] [FEATURE] RPC: Add command for pruning past ChainTip #16827

This feature allows for the pruning of blocks from the disk, when their blockHeight is > to the current Tip height.

Why is this needed? Well let's say you want to create a backup of the blocks & chainstate at a specific block height, you can easily do this by running invalidateblock and making a copy of the blocks/ and chainstate folders. The issue with creating a backup this way is that when installing a node based on it, you would run into this issue. #16353 This feature solves this by simply removing most of the blocks that are invalidated from the disk.

If this feature gets Concept ACK, I will build the tests needed and update the PR.

let's say you want to create a backup of the blocks & chainstate at a specific block height,

Couldn't this simply be done by syncing a node from scratch to the destination backup dir? The "destination" node is only connected to the source node, which is pinned to a block height via invalidateblock.

Couldn't this simply be done by syncing a node from scratch to the destination backup dir?

Maybe, but this wouldn't work for my use case at all. I'm basically building a website for helping peoples sync their nodes, based on who they trust in the community (or the aggregate). Basically people with already existing node would run invalidateblock & gettxoutsetinfo at a specific height and sign the hash_serialized_2 with their PGP key. If you want to sync you full node, you would look for a specific signer you trust, or choose to trust the backup with the most aggregated signatures, and then check if the hash_serialized_2 is matching (through a script obv).

Maybe, but this wouldn't work for my use case at all

I didn't get why from your reply.

You need a "clean" backup of the chain and the chainstate and my solution provides just that without any need to patch Bitcoin Core.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #16365 (Log RPC parameters (arguments) if -debug=rpcparams by LarryRuane)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Synching from scratch will take a lot more time than invalidating a few hundred/thousands blocks, especially if the entire process needs to be repeated multiple times.

In my opinion, the real question is does that make sense to be included in core? Could this be useful to other peoples? Because for my exact use case I don't really need it merged since I'm the only one that needs to run a patched version.

I'm basically building a website for helping peoples sync their nodes, based on who they trust in the community

You might be interested in #15605, which is a more formalized and less error-prone way to achieve a similar goal.

I'm aware of the assumeutxo and various utxo commitment proposal, but I think this is a different/better approach:

• no reliance on merging into core (assumeutxo seems stuck in the PR process)
• require less reliance/trust in core developers since this opens up the signing to any node owner not just devs.

no reliance on merging into core (assumeutxo seems stuck in the PR process)

assumeutxo is not stuck. Progress is ongoing and a lot has already landed in Core, such as #15948 #15976 #16194 #16355 #16443 #16728. The next chunk seems to be #16743 and you can help move it forward by reviewing that pull request.

require less reliance/trust in core developers since this opens up the signing to any node owner not just devs.

You already have to review the source code to not trust the core developers. If you don't want to (or can not) read the code, it might be easier to ask someone you trust for a signature or use the gitian sigs (https://github.com/bitcoin-core/gitian.sigs) as a proxy. Gitian sigs can be done by anyone, not just devs.

The next chunk seems to be #16743 and you can help move it forward by reviewing that pull request.

I don't think I'm familiar enough with the overall codebase to be of any use in the review process, unfortunately.

You already have to review the source code to not trust the core developers.

Of course, and I never wanted to imply I didn't trust you guys (or the process), it's hard to get a nuanced point accross especially when it's not your main language, because that's clearly not the case, but in a decentralized network, the less overall reliance, the better it is in my opinion.

it might be easier to ask someone you trust for a signature or use the gitian sigs (https://github.com/bitcoin-core/gitian.sigs) as a proxy. Gitian sigs can be done by anyone, not just devs.

The project in question is about 95% ready for deployment, this was the biggest roadblock. I guess it's just gonna be one more option for users to choose from, at the very least until assumeutxo is completely integrated; but I'm not sure what's this pushback is all about, if you don't like the idea of this service, that's fine, you certainly don't have to use it. I'm not sure of why we are discussing the merits of the service I'm building instead of focussing on the content of this PR: can it be useful to have in Core or not?

Concept NACK. Although it's useful for your specific, narrow use-case. Which is good for you, but I don't think it warrants merging and maintaining this code.

FWIW what I always do to create a pre-synced node (for testing, or fast deployment) is to use fastcopy-chaindata to quickly create a clone of an existing blockchain state, then run bitcoind with prune option on the cloned state once.

Concept NACK, seems like a hack, and invalidateblock isn't something that should be normalised (just used in specific consensus scenarios).

Thanks for the contribution, however I agree with the other commenters here, so I'm going to close this PR. I'd still suggest looking at the assumeutxo project.