validation: assumeutxo params mainnet

Sjors commented at 12:53 pm on September 30, 2023: member

This adds snapshot parameters for mainnet block 840,000.

You can generate the snapshot yourself using ./contrib/devtools/utxo_snapshot.sh or download my torrent:

torrent: magnet:?xt=urn:btih:55cb2ea9b6c48aa11d676de871f9639e3fa3dd7e&dn=utxo-840000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969

It would be a good idea to test:

That you can produce the same snapshot file, sha256 sum:

09db649785fcddd8b1fe83a5009fe5c9be1901e8c322298cacc8a2d96ace4807e utxo-840000.dat

That the snapshot works

DrahtBot commented at 12:53 pm on September 30, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Stale ACK	pablomartin4btc

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

DrahtBot added the label Needs rebase on Oct 2, 2023

Sjors force-pushed on Oct 3, 2023

DrahtBot removed the label Needs rebase on Oct 3, 2023

in src/kernel/chainparams.cpp:180 in 27a8739795 outdated

172@@ -173,7 +173,12 @@ class CMainParams : public CChainParams {
173         };
174 
175         m_assumeutxo_data = {
176-            // TODO to be specified in a future patch.
177+            {
178+                .height = 800'000,
179+                .hash_serialized = AssumeutxoHash{uint256S("0x7d69b87512db3d13b9758ea32b93ce468d18cf7456fb5d250c9e1bed9339e4d2")},
180+                .nChainTx = 868965226,
181+                .blockhash = uint256S("0x00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054")

maflcko commented at 12:28 pm on October 3, 2023:

0                .blockhash = uint256S("0x00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054"),

style-nit, if you re-touch.

Sjors force-pushed on Oct 9, 2023

DrahtBot added the label CI failed on Oct 9, 2023

Sjors force-pushed on Oct 25, 2023

Sjors commented at 9:02 am on October 25, 2023: member

Updated hash_serialized after #28685.

I only tested that the loadtxoutset command finishes. @pablomartin4btc I found a different hash than you… https://github.com/Sjors/bitcoin/commit/24deb2022b822f22fba9fcbee201e37a83225eb2#commitcomment-130847775

DrahtBot removed the label CI failed on Oct 25, 2023

pablomartin4btc approved

pablomartin4btc commented at 2:02 am on November 1, 2023: member

tACK 0800be24bc6de4154653be75b8fa9902a4a8c3a7

 0./contrib/devtools/utxo_snapshot.sh 800000 - ./src/bitcoin-cli -datadir=${AU_DATADIR}
 1Rewinding chain back to height 800000 (by invalidating 00000000000000000000e26b239cf19ec7ace5edd9694d51a3f6933247720947); this may take a while
 2Generating txoutset info...
 3{
 4  "height": 800000,
 5  "bestblock": "00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054",
 6  "txouts": 111535121,
 7  "bogosize": 8443325590,
 8  "hash_serialized_3": "6e63799cac0ab45f94789f97692caf691ee66625e920cbf2525fceb24509adda",
 9  "total_amount": 19437286.83643360,
10  "transactions": 72936026,
11  "disk_size": 7264997650
12}
13Restoring chain to original height; this may take a while

I incorrectly took the muhash value from gettxoutsetinfo (without running the utxo_snapshot.sh which also performs invalidateblock right before).

pablomartin4btc referenced this in commit 24deb2022b on Nov 1, 2023

Sjors commented at 2:08 am on November 1, 2023: member

@pablomartin4btc thanks for checking that you get the same snapshot. In addition it would be good to test if you’re able to load the snapshot and complete the background sync.

pablomartin4btc commented at 2:24 am on November 1, 2023: member

In addition it would be good to test if you’re able to load the snapshot and complete the background sync.

I’ve already checked IBD completed fine with[background validation]finished, but with the previoushash_serialized, as I wanted to verify that #28698 was only happening before the IBD completion.

I’ll run all the process again with the newhash_serializedvalue.

pablomartin4btc commented at 7:49 pm on November 9, 2023: member

Tested loadtxoutset a few times (pending IBD to finish) with coinstatsindex & blockfilterindex, and without them. I’ve also validated that the sha256sum corresponds to the one obtained from the downloaded file.

Now, having a pruned node with IBD completed, from which I got the m_assumeutxo_data mentioned in my previous review, when I run ./contrib/devtools/utxo_snapshot.sh with a filename in order to perform the actual dumptxoutsetwithin the script, I got a node crash with ERROR: ReadBlockFromDisk: OpenBlockFile failed for FlatFilePos(nFile=-1, nPos=0), which instance I had to kill since it was unresponsive. I’m not sure if this situation was due to the invalidateblock performed twice by runnning the utxo_snapshot script (one to get the info from gettxoutsetinfo and the second for the actual dump), the pruned node or a combination of all three. Starting the node again without blockfilterindex (option given by the node before doing the suggested -reindex or -reindex-chainstate which doesn’t work on pruned nodes), made it worse till getting !setBlockIndexCandidates.empty(). I see there are a few open issues regarding invalidateblock & setBlockIndexCandidates so before investigating further I’d like to know first if the dumptxoutset is expected to work from a fully synced pruned node?

Sjors commented at 1:39 am on November 10, 2023: member

As discussed offline, the utxo_snapshot.sh script should have a sanity check to make the node isn’t pruned below the snapshot height. On a pruned node you can only make a snapshot for a block that hasn’t been pruned yet.

pablomartin4btc commented at 1:20 pm on November 10, 2023: member

As discussed offline, the utxo_snapshot.sh script should have a sanity check to make the node isn’t pruned below the snapshot height. On a pruned node you can only make a snapshot for a block that hasn’t been pruned yet.

I agree with it, mentioned on a #27845 that’s touching the script ATM.

In the testing I performed and mentioned above the pruned node was above the snapshot height (800'000):

0  "pruned": true,
1  "pruneheight": 600510,
2  "automatic_pruning": true,
3  "prune_target_size": 20971520000,

ryanofsky referenced this in commit 1c8893bd1c on Dec 4, 2023

DrahtBot added the label CI failed on Jan 17, 2024

Sjors commented at 2:16 pm on February 13, 2024: member

I plan to make a fresh snapshot after #26045 lands.

fjahr commented at 5:20 pm on March 4, 2024: contributor

Didn’t see this one until the conflict popped up. I think we can move to 830,000 for v27.0 and I am unsure if we need to wait for #26045. I will bring it up in the next meeting.

Sjors force-pushed on Apr 29, 2024

Sjors commented at 11:38 am on April 29, 2024: member

I switched to the halving block! Also dropped all prerequisites but #26045 from the description, but see conceptual discussion in #29616.

Sjors renamed this:
~~[do not merge] validation: assumeutxo params mainnet~~
validation: assumeutxo params mainnet
on Apr 29, 2024

DrahtBot removed the label CI failed on Apr 29, 2024

DrahtBot added the label Validation on Apr 29, 2024

chainparams: add mainnet assumeutxo param at height 840_000 f193b0fd85

Sjors force-pushed on May 28, 2024

Sjors commented at 11:09 am on May 28, 2024: member

Rebased now that #29612 landed. Will do some final testing before marking this ready to review.

Sjors marked this as ready for review on May 28, 2024

validation: assumeutxo params mainnet #28553

Code Coverage

Reviews

Conflicts