Related: https://github.com/bitcoin-core/bitcoincore.org/issues/793
This changeset improves the contrib/verifybinaries/verify.py script to account for the new binary verification procedure introduced in the 22.0 release; for details on this new process, see the note in #22634.
In short, instead of relying on a single signature from the lead maintainer attesting to the expected hashes of binary releases, this verify script now supports validating that a minimum threshold of trusted identities have signed the checksum file.
Both the threshold and identities to trust are configurable by the end user, but sensible defaults are provided: identities are inferred from local GPG trust and the builder-keys file and a minimum threshold of 4 trusted signatures is configured by default. There are options for overriding these.
Various improvements have been made to the script for allow for easier programmatic use; a --json option is introduced and logging output is now directed to stderr.
Automatic pubkey import
Pubkeys that are referenced in checksum signature files can now be automatically downloaded based on a user prompt. This behavior can be disabled with the --noninteractive flag to support CI use.
I have built in functionality for elevating local GPG trust of the imported keys, but it isn’t quite working yet; I think instead of manually modifying the GPG trust database I have to use the --sign-key command. But this is optional functionality and fixing the broken process is the priority here, so I can enable this nicety in a follow-up.
Builder-key diffing
Builder keys (as listed in ./contrib/builder-keys/keys.txt) are used by default to establish trust in a pubkey signature. This can be disabled with --no-builder-keys.
If builder keys are locally supplied (by running this command from the root of the repository), they are diffed with the remote version obtained over HTTPS from Github. A diff is reported if it exists, e.g.
Granular JSON output
Because binary verification is now a gradient on the basis of an end user’s trust in a set of pubkeys, users of this script may want a granular report on which signatures were used. Stderr logging can be parsed for this information, but a convenient JSON blob is also accessible using the --json flag:
0% ./contrib/verifybinaries/verify.py 22.0-x86 --noninteractive --json 2>/dev/null
1
2{
3 "good_trusted_sigs": [
4 "SigData('9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C', 'Aaron Clauson (sipsorcery) <aaron@sipsorcery.com>', trusted=False, status='unknown')",
5 "SigData('637DB1E23370F84AFF88CCE03152347D07DA627C', 'Stephan Oeste (it) <it@oeste.de>', trusted=True, status='unknown')",
6 "SigData('9DEAE0DC7063249FB05474681E4AED62986CD25D', 'Wladimir J. van der Laan <laanwj@visucore.com>', trusted=True, status='unknown')",
7 "SigData('0AD83877C1F0CD1EE9BD660AD7CC770B81FD22A8', 'Ben Carman <benthecarman@live.com>', trusted=False, status='unknown')",
8 "SigData('152812300785C96444D3334D17565732E08E5E41', 'Andrew Chow (Official New Key) <achow101@gmail.com>', trusted=True, status='expired')",
9 "SigData('D1DBF2C4B96F2DEBF4C16654410108112E7EA81F', 'Hennadii Stepanov (GitHub key) <32963518+hebasto@users.noreply.github.com>', trusted=True, status='unknown')",
10 "SigData('590B7292695AFFA5B672CBB2E13FC145CD3F4304', 'Antoine Poinsot <darosior@protonmail.com>', trusted=True, status='unknown')"
11 ],
12 "good_untrusted_sigs": [
13 "SigData('0CCBAAFD76A2ECE2CCD3141DE2FFD5B1D88CA97D', '.0xB10C <0xb10c@gmail.com>', trusted=False, status='unknown')",
14 "SigData('28F5900B1BB5D1A4B6B6D1A9ED357015286A333D', 'Duncan Dean <duncangleeddean@gmail.com>', trusted=False, status='unknown')",
15 "SigData('CFB16E21C950F67FA95E558F2EEB9F5CC09526C1', 'Michael Ford (bitcoin-otc) <fanquake@gmail.com>', trusted=False, status='unknown')",
16 "SigData('6E01EEC9656903B0542B8F1003DB6322267C373B', 'Oliver Gugger <gugger@gmail.com>', trusted=False, status='unknown')",
17 "SigData('74E2DEF5D77260B98BC19438099BAD163C70FBFA', 'Will Clark <will8clark@gmail.com>', trusted=False, status='unknown')"
18 ],
19 "unknown_sigs": [
20 "SigData('82921A4B88FD454B7EB8CE3C796C4109063D4EAF', '', trusted=False, status='')"
21 ],
22 "bad_sigs": [],
23 "verified_binaries": [
24 "bitcoin-22.0-x86_64-linux-gnu.tar.gz"
25 ]
26}
Backwards incompatibility
Note that I have broken the interface; the script is no longer invoked in the same way and output differs. I initially tried to retain the old format, but I found it made less and less sense for the new signature scheme, and was not easily parsed sensibly. If there are users that rely on the old output structure, they can copy the script from an old version of the source tree.
Examples
Validate releases with default settings:
0./contrib/verifybinaries/verify.py 22.0
1./contrib/verifybinaries/verify.py 22.0-rc2-x86_64
2./contrib/verifybinaries/verify.py bitcoin-core-0.13.0-rc3
Get JSON output and don’t prompt for user input (no auto key import):
0./contrib/verifybinaries/verify.py 22.0-x86 --json --noninteractive
Don’t trust builder-keys by default, and rely only on local GPG state and manually specified keys, while requiring a threshold of at least 10 trusted signatures:
0./contrib/verifybinaries/verify.py 22.0 \
1 --no-builder-keys \
2 --trusted-keys 74E2DEF5D77260B98BC19438099BAD163C70FBFA,9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C \
3 --min-trusted-sigs 10
Followups
- update https://github.com/bitcoin-core/bitcoincore.org with new verification instructions
- fix local GPG trust elevation during pubkey import
- use
loggingfacility instead ofprint_*functions - write small
test.shexercising basic functionality with--noninteractive
New CLI interface
0% ./contrib/verifybinaries/verify.py --help
1
2usage: verify.py [-h] [--verbose] [--cleanup] [--noninteractive]
3 [--require-all-hosts] [--bitcoin-src-path [BITCOIN_SRC_PATH]]
4 [--skip-import-builders]
5 [--min-trusted-sigs [MIN_TRUSTED_SIGS]]
6 [--keyserver [KEYSERVER]] [--trusted-keys [TRUSTED_KEYS]]
7 [--no-builder-keys] [--json]
8 version
9
10Script for verifying Bitcoin Core release binaries. This script attempts to
11download the sum file SHA256SUMS and corresponding signature file
12SHA256SUMS.asc from bitcoincore.org and bitcoin.org and compares them. The
13sum-signature file is signed by a number of builder keys. This script ensures
14that there is a minimum threshold of signatures from pubkeys that we trust.
15This trust is articulated on the basis of configuration options here, but by
16default is based upon a unionof (i) local GPG trust settings, and (ii) keys
17which appear in the builder-keys/keys.txt file. If a minimum good, trusted
18signature threshold is met on the sum file, we then download the files
19specified in SHA256SUMS, and check if the hashes of these files match those
20that are specified. The script returns 0 if everything passes the checks. It
21returns 1 if either the signature check or the hash check doesn't pass. If an
22error occurs the return value is >= 2. Logging output goes to stderr and final
23binary verification data goes to stdout. JSON output can by obtained by
24setting env BINVERIFY_JSON=1.
25
26positional arguments:
27 version version of the bitcoin release to download; of the
28 format <major>.<minor>[.<patch>][-rc[0-9]][-platform].
29 Example: 22.0-x86_64 or 0.21.0-rc2-osx
30
31options:
32 -h, --help show this help message and exit
33 --verbose
34 --cleanup if specified, clean up files afterwards
35 --noninteractive if specified, do not block for user input
36 --require-all-hosts If set, require all hosts (https://bitcoincore.org,
37 https://bitcoin.org) to provide signatures. (Sometimes
38 bitcoin.org lags behind bitcoincore.org.)
39 --bitcoin-src-path [BITCOIN_SRC_PATH]
40 specify path to bitcoin repository. Used to find
41 builder keys.
42 --skip-import-builders
43 If set, do not prompt to import builder pubkeys
44 --min-trusted-sigs [MIN_TRUSTED_SIGS]
45 The minimum number of good signatures from recognized
46 keys to require successful termination.
47 --keyserver [KEYSERVER]
48 which keyserver to use
49 --trusted-keys [TRUSTED_KEYS]
50 A list of trusted builder GPG keys, specified as CSV
51 --no-builder-keys If set, do not trust the builder-keys from the bitcoin
52 repo by default
53 --json If set, output the result as JSON
