Always show 100% verification progress when done

hMsats commented at 10:32 am on June 16, 2024: none

External software/scripts may monitor Bitcoin Core’s verification progress in debug.log. Now these have to scan for “No coin database inconsistencies” to see if the verification progress has finished. It’s cleaner to always show 100% when the verification progress has finished (without encountering any problems). If I remember correctly this was the case in older versions of Bitcoin Core.

NOW:

02024-06-16T09:12:03Z Verification progress: 0%
12024-06-16T09:12:55Z Verification progress: 16%
22024-06-16T09:13:11Z Verification progress: 33%
32024-06-16T09:13:19Z Verification progress: 50%
42024-06-16T09:13:23Z Verification progress: 66%
52024-06-16T09:13:25Z Verification progress: 83%
62024-06-16T09:13:27Z Verification progress: 99%
72024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)

NEW:

02024-06-16T09:12:03Z Verification progress: 0%
12024-06-16T09:12:55Z Verification progress: 16%
22024-06-16T09:13:11Z Verification progress: 33%
32024-06-16T09:13:19Z Verification progress: 50%
42024-06-16T09:13:23Z Verification progress: 66%
52024-06-16T09:13:25Z Verification progress: 83%
62024-06-16T09:13:27Z Verification progress: 99%
72024-06-16T09:13:27Z Verification progress: 100%
82024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)

Always show 100% verification progress when done d13dfc59e3

DrahtBot commented at 10:32 am on June 16, 2024: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

pinheadmz commented at 12:32 pm on June 16, 2024: member

External software/scripts may monitor Bitcoin Core’s verification progress in debug.log.

Do you run a script like this? I’m curious what the motivation is, since I’m pretty sure if verification fails the program quits.

hMsats commented at 1:12 pm on June 16, 2024: none

@pinheadmz yes, I make a system backup every month like this: pre-backup with rsync, turn off my bitcoin node, quick final rsync, restart bitcoin node and wait for Bitcoin Core to finish (while showing verification progress) before starting my public electrum server, lightning node and pruning node (only connected to my main node).

pinheadmz commented at 1:22 pm on June 16, 2024: member

Ok and do you understand that the verification progress you are touching here is in the startup sequence where Bitcoin ensures the database is consistent before even connecting to the network?

It’s possible you may be referring to sync progress which is the process of downloading and verifying new blocks until the most work chain has been processed. That progress will never reach 100% by design. There is some relevant discussion here:

https://github.com/bitcoin/bitcoin/issues/28847

tdb3 commented at 1:26 pm on June 16, 2024: contributor

Thanks for taking a look and presenting the idea.

With bitcoin providing an RPC interface (and the OS providing general process run state), sifting through the debug.log seems like a non-ideal method for status gathering. Are there some examples from out in the wild where this is relied upon?

A cool feature of the RPC interface is that it provides early feedback (e.g. that blocks are being verified). Here are some screenshots of what is occurring in the debug.log and what the RPC caller sees:

DebugLog

VerifyingBlocksMessage

There might be value in providing a definitive 100% message, but more for general accuracy to bitcoin itself rather than to machine observers of the debug.log.

Please check out https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#logging, which has helpful information on deprecated log functions.

hMsats commented at 1:47 pm on June 16, 2024: none

@tdb3

There might be value in providing a definitive 100% message, but more for general accuracy to bitcoin itself rather than to machine observers of the debug.log.

That was the real reason for this pull request. My example was just a side note. Seeing the 100% is useful also for users looking at the debug.log.

hMsats commented at 2:02 pm on June 16, 2024: none

@pinheadmz Ah, maybe we need different strings here. Instead of “Verification progress” maybe “Validation progress” in order not to confuse the two processes.

Linux command: ag "Verification progress" gives (including the 100%):

 0bitcoin/test/functional/interface_bitcoin_cli.py
 1184:        assert_equal(cli_get_info['Verification progress'], "%.4f%%" % (blockchain_info['verificationprogress'] * 100))
 2
 3bitcoin/src/validation.cpp
 44613:    LogPrintf("Verification progress: 0%%\n");
 54621:            LogPrintf("Verification progress: %d%%\n", percentageDone);
 64697:                LogPrintf("Verification progress: %d%%\n", percentageDone);
 74715:    LogPrintf("Verification progress: 100%%\n");
 8
 9bitcoin/src/bitcoin-cli.cpp
101057:    result_string += strprintf("Verification progress: %s%.4f%%\n", ibd_progress_bar, ibd_progress * 100);

pinheadmz commented at 3:37 pm on June 16, 2024: member

wait for Bitcoin Core to finish

At this point in your script do you mean

“wait for Bitcoin Core to be ready for RPC commands” or
“wait for Bitcoin Core to sync to the most work chain it can find on the network so my UTXO state is most likely to be globally accurate”?

hMsats commented at 3:51 pm on June 16, 2024: none

@pinheadmz Irrespective of my (simple) shell script, in my opinion Bitcoin Core should always print “Verification progress=100%” or even better “Validation progress=100%” in debug.log.

achow101 commented at 5:36 pm on June 16, 2024: member

If you want to know when bitcoind is ready to start receiving RPCs and generally finished its startup sequence, you should use -startupnotify and have it notify you, rather than parsing the debug.log.

ISTM the log line saying that no inconsistencies were found is a good enough indicator that verification was finished.

kevkevinpal commented at 6:04 pm on June 16, 2024: contributor

That progress will never reach 100% by design

I agree with @pinheadmz

why not just look for this line instead for your script? Verification: No coin database inconsistencies

hMsats commented at 6:36 pm on June 16, 2024: none

@kevkevinpal (and @achow101) I do in my script but I still believe that the two lines:

02024-06-16T09:13:27Z Verification progress: 99%
12024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)

are ugly and

02024-06-16T09:13:27Z Verification progress: 99%
12024-06-16T09:13:27Z Verification progress: 100%
22024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)

is much better in debug.log

hMsats commented at 6:52 pm on June 16, 2024: none

Of course I can live without this (trivial) addition, so if you developers don’t want to include it, it’s fine with me

hMsats commented at 7:09 pm on June 16, 2024: none

Found it. It used to be like this (note the [DONE]):

02018-02-15 07:39:04 init message: Rewinding blocks...
12018-02-15 07:39:07 init message: Verifying blocks...
22018-02-15 07:39:07 Verifying last 144 blocks at level 3
32018-02-15 07:39:07 [0%]...[10%]...[20%]...[30%]...[40%]...[50%]...[60%]...[70%]...[80%]...[90%]...[DONE].

hMsats closed this on Jun 16, 2024

bitcoin locked this on Jun 16, 2025

Always show 100% verification progress when done #30293

Code Coverage

Reviews