Add mainnet assumeutxo param at height 880,000 #31969

pull Sjors wants to merge 1 commits into bitcoin:master from Sjors:2025/03/utxo-880000 changing 1 files +6 −0
  1. Sjors commented at 4:02 pm on March 1, 2025: member

    #31940 suggests adding a snapshot at every major release.

    This snapshot should be suitable for v29. I picked the most recent multiple of 10K blocks.

    You can either download this torrent:

    0magnet:?xt=urn:btih:559bd78170502971e15e97d7572e4c824f033492&dn=utxo-880000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969

    Or generate the snapshot yourself:

    0bitcoin-cli -rpcclienttimeout=0 -named dumptxoutset utxo-880000.dat rollback=880000

    The SHA-256 hash should be:

    0shasum -a 256 utxo-880000.dat
143b3b1ad6e1005ffc0ff49514d0ffcc3e3ce671cc8d02da7fa7bac5405f89de4

    And then load it on a fresh node during IBD with:

    0bitcoin-cli -rpcclienttimeout=0 loadtxoutset utxo-880000.dat

    Note that it’s more performant to turn off networking while the snapshot is loading, see #29993:

    0bitcoin-cli setnetworkactive false

    Once the snapshot is loaded:

    0bitcoin-cli setnetworkactive true

    And enjoy a speedy ride to the tip.

  2. chainparams: add mainnet assumeutxo param at height 880_000 14f1674855
  3. DrahtBot commented at 4:02 pm on March 1, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31969.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK hodlinator, polespinasa, achow101, fjahr
    Concept ACK rkrux, 1440000bytes, jonatack

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

  4. hodlinator approved
  5. hodlinator commented at 11:44 pm on March 1, 2025: contributor

    ACK 14f16748557faf57cf4b0f4c91c162592557434c

    Thanks for doing this.

    (Testing performed at commit 8b5a50cb353d9524db38a3cfd537bac7a770e1a6, one markdown change away from recent master).

    Generated my own snapshot (confirmed that recommended command roughly matches doc/assumeutxo.md (and dumptxoutset-help for specifying height):

    0₿ build/src/bitcoin-cli -rpcclienttimeout=0 -named dumptxoutset utxo-880000.dat rollback=880000
1{
2  "coins_written": 184821030,
3  "base_hash": "000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880",
4  "base_height": 880000,
5  "path": "/home/hodlinator/.bitcoin/utxo-880000.dat",
6  "txoutset_hash": "dbd190983eaf433ef7c15f78a278ae42c00ef52e0fd2a54953782175fbadcea9",
7  "nchaintx": 1145604538
8}

    Copied those values into chainparams.cpp just as this PR does and recompiled (we refuse to load unrecognized snapshots).

    Loaded generated snapshot (after having moved utxo-880000.dat out from my .bitcoin dir):

    0₿ build/src/bitcoin-cli -noconf -datadir=new_node -rpcclienttimeout=0 loadtxoutset /home/hodlinator/utxo-880000.dat

    bitcoind restarted under new datadir output:

    0...
12025-03-01T22:47:21Z [snapshot] loaded 184821030 (282 MB) coins from snapshot 000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880
22025-03-01T23:05:24Z [snapshot] validated snapshot (0 MB)
32025-03-01T23:05:24Z [snapshot] successfully activated snapshot 000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880
42025-03-01T23:05:24Z [snapshot] (0 MB)
52025-03-01T23:05:25Z [Chainstate [snapshot] @ height 880000 (000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880)] resized coinsdb cache to 7.6 MiB
62025-03-01T23:05:25Z [Chainstate [snapshot] @ height 880000 (000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880)] resized coinstip cache to 418.0 MiB

    bitcoin-cli loadtxoutset output upon completion:

    0{
1  "coins_loaded": 184821030,
2  "tip_hash": "000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880",
3  "base_height": 880000,
4  "path": "/home/hodlinator/utxo-880000.dat"
5}

    Output when restarting bitcoind:

    02025-03-01T23:10:33Z Loaded best chain: hashBestChain=000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880 height=880000 date=2025-01-20T01:42:23Z progress=0.979692

    Snapshot appears to have made the new node successfully fast forward to 880000 from observing UpdateTip-log messages. Calling getblockhash 880000-RPC gives the expected hash despite the node having synced to height 882600+, making it very unlikely that we are on some invalid fork. Also double-checked with mempool.space.

    Confirmed my change to chainparams.cpp is the same as 14f16748557faf57cf4b0f4c91c162592557434c using diff-tool, and that it is a direct child of the upstream master branch.

  6. rkrux approved
  7. rkrux commented at 9:33 am on March 3, 2025: contributor

    Concept ACK 14f16748557faf57cf4b0f4c91c162592557434c

    I have not tested this because I run a pruned node and my block’s prune height is more than the 880k and there doesn’t seem to be a way at the moment to decrease the prune height without triggering IBD.

    In the PR description, the last command should pass true instead.

    Once the snapshot is loaded:

    0bitcoin-cli setnetworkactive false

    I’ll upload a torrent later.

    Any idea why don’t we upload snapshots in bitcoincore.org/bin as well?

  8. Sjors commented at 12:20 pm on March 3, 2025: member
    Added torrent to the description (and fixed the setnetworkactive instruction). @achow101, if you agree we should host it, can you add it to bitcoincore.org and give me the link? I’ll add it as a binary seed.
  9. dergoegge commented at 1:10 pm on March 3, 2025: member

    Discussions to host the actual chainstate as the project itself (i.e. on bitcoincore.org) should not be had on some unrelated pull request. I think this deserves a full discussion in a separate issue or at the irc meeting.

    Iirc there were discussions in the past about this and the sentiment (as I remember it) was not to host the actual snapshot.

  10. rkrux commented at 1:34 pm on March 3, 2025: contributor
    Well, this pull request is not completely unrelated to this topic but in order to have a more concrete discussion, I will create a Github issue for it & we can move the discussion there.
  11. Sjors commented at 1:38 pm on March 3, 2025: member
    @rkrux already done in #31972
  12. laanwj added the label UTXO Db and Indexes on Mar 3, 2025
  13. glozow added this to the milestone 29.0 on Mar 3, 2025
  14. 1440000bytes commented at 4:07 pm on March 3, 2025: none
    Concept ACK
  15. polespinasa commented at 5:16 pm on March 3, 2025: none

    ACK https://github.com/bitcoin/bitcoin/commit/14f16748557faf57cf4b0f4c91c162592557434c

    I like the idea to update the param on each release as per assumevalid

    Tested and got same values when dumping the utxo set

    0bitcoin-cli -rpcclienttimeout=0 -named dumptxoutset utxoset_880000.dat rollback=880000
1{
2  "coins_written": 184821030,
3  "base_hash": "000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880",
4  "base_height": 880000,
5  "path": "/home/sliv3r/.bitcoin/utxoset_880000.dat",
6  "txoutset_hash": "dbd190983eaf433ef7c15f78a278ae42c00ef52e0fd2a54953782175fbadcea9",
7  "nchaintx": 1145604538
8}

    Also got same sha256 for the .dat file.

    0sha256sum utxoset_880000.dat 
143b3b1ad6e1005ffc0ff49514d0ffcc3e3ce671cc8d02da7fa7bac5405f89de4  utxoset_880000.dat

    Did not download the torrent.

  16. DrahtBot requested review from rkrux on Mar 3, 2025
  17. fjahr commented at 8:04 pm on March 3, 2025: contributor
    Concept ACK
  18. achow101 added the label Has ACKs on Mar 4, 2025
  19. achow101 removed the label Has ACKs on Mar 4, 2025
  20. achow101 commented at 3:17 am on March 4, 2025: member

    ACK 14f16748557faf57cf4b0f4c91c162592557434c

    if you agree we should host it, can you add it to bitcoincore.org and give me the link? I’ll add it as a binary seed.

    That will require more discussion with the group.

  21. DrahtBot requested review from fjahr on Mar 4, 2025
  22. fjahr commented at 4:27 pm on March 4, 2025: contributor

    tACK 14f16748557faf57cf4b0f4c91c162592557434c

    I tested all of the steps outlined in the PR description and did indeed enjoy the speedy ride 🏎️

  23. jonatack commented at 6:18 pm on March 4, 2025: member

    Concept ACK, shasum of the torrent file matches up and will re-test soon after further reindexing. For now am seeing this:

    0error code: -32603
1error message:
2Unable to load UTXO snapshot: The base block header (000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880) must appear in the headers chain. Make sure all headers are syncing, and call loadtxoutset again. (utxo-880000.dat)
  24. bitcoin deleted a comment on Mar 4, 2025
  25. glozow merged this on Mar 4, 2025
  26. glozow closed this on Mar 4, 2025
  27. in src/kernel/chainparams.cpp:199 in 14f1674855 
    190@@ -191,6 +191,12 @@ class CMainParams : public CChainParams {
191                 .hash_serialized = AssumeutxoHash{uint256{"a2a5521b1b5ab65f67818e5e8eccabb7171a517f9e2382208f77687310768f96"}},
192                 .m_chain_tx_count = 991032194,
193                 .blockhash = consteval_ctor(uint256{"0000000000000000000320283a032748cef8227873ff4872689bf23f1cda83a5"}),
194+            },
195+            {
196+                .height = 880'000,
197+                .hash_serialized = AssumeutxoHash{uint256{"dbd190983eaf433ef7c15f78a278ae42c00ef52e0fd2a54953782175fbadcea9"}},
198+                .m_chain_tx_count = 1145604538,
199+                .blockhash = consteval_ctor(uint256{"000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880"}),
    l0rinc commented at 8:32 pm on March 4, 2025:

    Now that https://developercommunity.visualstudio.com/t/consteval-conversion-function-fails/1579014#T-N10721450 was released, I’m wondering if we can simplify this to:

    0                .blockhash = uint256{"000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880"},

    or even add an implicit constexpr uint256(const std::array<std::byte, 32>& hex) : base_blob(hex) {} and do:

    0                .blockhash = "000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880"_hex,

    I’ll investigate and open a follow-up PR if there are no strong objections.

  28. in src/kernel/chainparams.cpp:196 in 14f1674855 
    190@@ -191,6 +191,12 @@ class CMainParams : public CChainParams {
191                 .hash_serialized = AssumeutxoHash{uint256{"a2a5521b1b5ab65f67818e5e8eccabb7171a517f9e2382208f77687310768f96"}},
192                 .m_chain_tx_count = 991032194,
193                 .blockhash = consteval_ctor(uint256{"0000000000000000000320283a032748cef8227873ff4872689bf23f1cda83a5"}),
194+            },
195+            {
196+                .height = 880'000,
    l0rinc commented at 8:34 pm on March 4, 2025:
    At the release we’ll roughly be at 890'000, right? A bit tight, I guess, but no strong objections.
    Sjors commented at 12:22 pm on March 5, 2025:
    We want this to be updated in master, so we have to pick something before the v29 branch-off.
  29. l0rinc commented at 8:38 pm on March 4, 2025: contributor

    edit: post-merge ACK, verified that:

    • can load the snapshot ✓
    0[snapshot] successfully activated snapshot 000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880
    • can’t load the snapshot without the new entry ✓
    0Error: A fatal internal error occurred, see debug.log for details: Assumeutxo data not found for the given blockhash '000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880'.
    • we can reach the current tip after background check ✓
    02025-03-05T12:07:12Z UpdateTip: new best=000000000000000000004ad34f6b0afc08a5ea8b8385172bf4bb967033720d73 height=886412 version=0x2c630000 log2_work=95.478513 tx=1162442395 date='2025-03-05T11:44:49Z' progress=0.999992 cache=180.9MiB(1331788txo)
12025-03-05T12:07:12Z [background validation] UpdateTip: new best=0000000011d1d9f1af3e1d038cebba251f933102dbe181d46a7966191b3299ee height=12000 version=0x00000001 log2_work=45.550889 tx=12105 date='2009-04-23T22:50:39Z' progress=0.000010 cache=2.2MiB(9428txo)
    • the background check succeeds ✓
    02025-03-08T10:13:25Z [background validation] UpdateTip: new best=000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880 height=880000 version=0x268be000 log2_work=95.396383 tx=1145604538 date='2025-01-20T01:42:23Z' progress=0.976559 cache=5145.5MiB(35241631txo)
12025-03-08T10:13:25Z [warning] Flushing large (5 GiB) UTXO set to disk, it may take several minutes
22025-03-08T10:17:26Z [snapshot] computing UTXO stats for background chainstate to validate snapshot - this could take a few minutes
32025-03-08T10:32:23Z [snapshot] snapshot beginning at 000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880 has been fully validated
42025-03-08T10:32:23Z [snapshot] allocating all cache to the snapshot chainstate
5[...]
62025-03-08T10:49:53Z UpdateTip: new best=0000000000000000000069aed813954edef3972f9c8ee36632f3113a72dad42b height=886849 version=0x20400000 log2_work=95.483928 tx=1163418786 date='2025-03-08T10:49:44Z' progress=1.000000 cache=140.5MiB(1002553txo)
  30. Sjors commented at 12:20 pm on March 5, 2025: member

    @jonatack wrote:

    must appear in the headers chain.

    That’s expected behavior. The snapshot will refuse to load until we have its header.

    Though one thing that could be improved is to have it wait for headers to reach the given height instead of fail.

  31. Sjors deleted the branch on Mar 5, 2025
  32. luke-jr referenced this in commit 3882729b08 on Mar 5, 2025
  33. jonatack commented at 4:02 pm on March 7, 2025: member

    Concept ACK, shasum of the torrent file matches up and will re-test soon after further reindexing. For now am seeing this:

    0error code: -32603
1error message:
2Unable to load UTXO snapshot: The base block header (000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880) must appear in the headers chain. Make sure all headers are syncing, and call loadtxoutset again. (utxo-880000.dat)

    Successfully activated the torrent file snapshot, once the node had the headers.

    `` 2025-03-07T15:53:09.082688Z [httpworker.4] [validation.cpp:6091] [PopulateAndValidateSnapshot] [snapshot] validated snapshot (0 MB) 2025-03-07T15:53:09.085190Z [httpworker.4] [validation.cpp:5860] [ActivateSnapshot] [snapshot] successfully activated snapshot 000000000000000000010b17283c3c400507969a9c2afd1dcf2082ec5cca2880

  34. l0rinc commented at 11:21 pm on March 10, 2025: contributor
    I can confirm that full background validation succeeded, but restarting the node seems to restart validation, too - opened an issue to clarify https://github.com/bitcoin/bitcoin/issues/32029
  35. bitcoin deleted a comment on Mar 11, 2025


Sjors DrahtBot hodlinator rkrux dergoegge 1440000bytes polespinasa fjahr achow101 jonatack l0rinc


fjahr rkrux

Labels
UTXO Db and Indexes

Milestone
29.0

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-03-31 09:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me