Related: https://github.com/bitcoin-core/bitcoincore.org/issues/793
This changeset improves the contrib/verifybinaries/verify.py script to account for the new binary verification procedure introduced in the 22.0 release; for details on this new process, see the note in #22634.
In short, instead of relying on a single signature from the lead maintainer attesting to the expected hashes of binary releases, this verify script now supports validating that a minimum threshold of trusted identities have signed the checksum file.
Both the threshold and identities to trust are configurable by the end user, but sensible defaults are provided: identities are inferred from local GPG trust and the builder-keys file and a minimum threshold of 4 trusted signatures is configured by default. There are options for overriding these.
Various improvements have been made to the script for allow for easier programmatic use; a --json option is introduced and logging output is now directed to stderr.
Automatic pubkey import
Pubkeys that are referenced in checksum signature files can now be automatically downloaded based on a user prompt. This behavior can be disabled with the --noninteractive flag to support CI use.
I have built in functionality for elevating local GPG trust of the imported keys, but it isn't quite working yet; I think instead of manually modifying the GPG trust database I have to use the --sign-key command. But this is optional functionality and fixing the broken process is the priority here, so I can enable this nicety in a follow-up.
Builder-key diffing
Builder keys (as listed in ./contrib/builder-keys/keys.txt) are used by default to establish trust in a pubkey signature. This can be disabled with --no-builder-keys.
If builder keys are locally supplied (by running this command from the root of the repository), they are diffed with the remote version obtained over HTTPS from Github. A diff is reported if it exists, e.g.
Granular JSON output
Because binary verification is now a gradient on the basis of an end user's trust in a set of pubkeys, users of this script may want a granular report on which signatures were used. Stderr logging can be parsed for this information, but a convenient JSON blob is also accessible using the --json flag:
% ./contrib/verifybinaries/verify.py 22.0-x86 --noninteractive --json 2>/dev/null
{
"good_trusted_sigs": [
"SigData('9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C', 'Aaron Clauson (sipsorcery) <aaron@sipsorcery.com>', trusted=False, status='unknown')",
"SigData('637DB1E23370F84AFF88CCE03152347D07DA627C', 'Stephan Oeste (it) <it@oeste.de>', trusted=True, status='unknown')",
"SigData('9DEAE0DC7063249FB05474681E4AED62986CD25D', 'Wladimir J. van der Laan <laanwj@visucore.com>', trusted=True, status='unknown')",
"SigData('0AD83877C1F0CD1EE9BD660AD7CC770B81FD22A8', 'Ben Carman <benthecarman@live.com>', trusted=False, status='unknown')",
"SigData('152812300785C96444D3334D17565732E08E5E41', 'Andrew Chow (Official New Key) <achow101@gmail.com>', trusted=True, status='expired')",
"SigData('D1DBF2C4B96F2DEBF4C16654410108112E7EA81F', 'Hennadii Stepanov (GitHub key) <32963518+hebasto@users.noreply.github.com>', trusted=True, status='unknown')",
"SigData('590B7292695AFFA5B672CBB2E13FC145CD3F4304', 'Antoine Poinsot <darosior@protonmail.com>', trusted=True, status='unknown')"
],
"good_untrusted_sigs": [
"SigData('0CCBAAFD76A2ECE2CCD3141DE2FFD5B1D88CA97D', '.0xB10C <0xb10c@gmail.com>', trusted=False, status='unknown')",
"SigData('28F5900B1BB5D1A4B6B6D1A9ED357015286A333D', 'Duncan Dean <duncangleeddean@gmail.com>', trusted=False, status='unknown')",
"SigData('CFB16E21C950F67FA95E558F2EEB9F5CC09526C1', 'Michael Ford (bitcoin-otc) <fanquake@gmail.com>', trusted=False, status='unknown')",
"SigData('6E01EEC9656903B0542B8F1003DB6322267C373B', 'Oliver Gugger <gugger@gmail.com>', trusted=False, status='unknown')",
"SigData('74E2DEF5D77260B98BC19438099BAD163C70FBFA', 'Will Clark <will8clark@gmail.com>', trusted=False, status='unknown')"
],
"unknown_sigs": [
"SigData('82921A4B88FD454B7EB8CE3C796C4109063D4EAF', '', trusted=False, status='')"
],
"bad_sigs": [],
"verified_binaries": [
"bitcoin-22.0-x86_64-linux-gnu.tar.gz"
]
}
Backwards incompatibility
Note that I have broken the interface; the script is no longer invoked in the same way and output differs. I initially tried to retain the old format, but I found it made less and less sense for the new signature scheme, and was not easily parsed sensibly. If there are users that rely on the old output structure, they can copy the script from an old version of the source tree.
Examples
Validate releases with default settings:
./contrib/verifybinaries/verify.py 22.0
./contrib/verifybinaries/verify.py 22.0-rc2-x86_64
./contrib/verifybinaries/verify.py bitcoin-core-0.13.0-rc3
Get JSON output and don't prompt for user input (no auto key import):
./contrib/verifybinaries/verify.py 22.0-x86 --json --noninteractive
Don't trust builder-keys by default, and rely only on local GPG state and manually specified keys, while requiring a threshold of at least 10 trusted signatures:
./contrib/verifybinaries/verify.py 22.0 \
--no-builder-keys \
--trusted-keys 74E2DEF5D77260B98BC19438099BAD163C70FBFA,9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C \
--min-trusted-sigs 10
Followups
- update https://github.com/bitcoin-core/bitcoincore.org with new verification instructions
- fix local GPG trust elevation during pubkey import
- use
loggingfacility instead ofprint_*functions - write small
test.shexercising basic functionality with--noninteractive
New CLI interface
% ./contrib/verifybinaries/verify.py --help
usage: verify.py [-h] [--verbose] [--cleanup] [--noninteractive]
[--require-all-hosts] [--bitcoin-src-path [BITCOIN_SRC_PATH]]
[--skip-import-builders]
[--min-trusted-sigs [MIN_TRUSTED_SIGS]]
[--keyserver [KEYSERVER]] [--trusted-keys [TRUSTED_KEYS]]
[--no-builder-keys] [--json]
version
Script for verifying Bitcoin Core release binaries. This script attempts to
download the sum file SHA256SUMS and corresponding signature file
SHA256SUMS.asc from bitcoincore.org and bitcoin.org and compares them. The
sum-signature file is signed by a number of builder keys. This script ensures
that there is a minimum threshold of signatures from pubkeys that we trust.
This trust is articulated on the basis of configuration options here, but by
default is based upon a unionof (i) local GPG trust settings, and (ii) keys
which appear in the builder-keys/keys.txt file. If a minimum good, trusted
signature threshold is met on the sum file, we then download the files
specified in SHA256SUMS, and check if the hashes of these files match those
that are specified. The script returns 0 if everything passes the checks. It
returns 1 if either the signature check or the hash check doesn't pass. If an
error occurs the return value is >= 2. Logging output goes to stderr and final
binary verification data goes to stdout. JSON output can by obtained by
setting env BINVERIFY_JSON=1.
positional arguments:
version version of the bitcoin release to download; of the
format <major>.<minor>[.<patch>][-rc[0-9]][-platform].
Example: 22.0-x86_64 or 0.21.0-rc2-osx
options:
-h, --help show this help message and exit
--verbose
--cleanup if specified, clean up files afterwards
--noninteractive if specified, do not block for user input
--require-all-hosts If set, require all hosts (https://bitcoincore.org,
https://bitcoin.org) to provide signatures. (Sometimes
bitcoin.org lags behind bitcoincore.org.)
--bitcoin-src-path [BITCOIN_SRC_PATH]
specify path to bitcoin repository. Used to find
builder keys.
--skip-import-builders
If set, do not prompt to import builder pubkeys
--min-trusted-sigs [MIN_TRUSTED_SIGS]
The minimum number of good signatures from recognized
keys to require successful termination.
--keyserver [KEYSERVER]
which keyserver to use
--trusted-keys [TRUSTED_KEYS]
A list of trusted builder GPG keys, specified as CSV
--no-builder-keys If set, do not trust the builder-keys from the bitcoin
repo by default
--json If set, output the result as JSON
