Stop the GPG verification madness #25395

Is your feature request related to a problem? Please describe. It's becoming increasingly difficult to automate verification of the releases. For "gpg --verify SHA256SUMS.asc SHA256SUMS" to succeed, all the keys have to be imported. Some keys are not present on public keyservers and keyservers are anyway commonly considered as unreliable. Currently for v23, after importing 27 (!) keys (others are not available) the SHA256SUMS.asc verification still fails.

Describe the solution you'd like Reduce the signer list to a set of well known, trusted signers and have their keys optionally signed by whoever verified their identity and is willing to sign. Alternatively, provide one signature file PER signer, not a global file that always fails to pass all checks.

Reduce the signer list to a set of well known, trusted signers

The whole point of the transition was to remove a single trusted party being responsible to sign the releases. For obvious reasons, it is problematic if everyone that verifies releases with gpg blindly trust a single fixed party.

Picking a fixed set of trusted parties is minimally better, but this would open a whole can of worms about adding and removing parties from the "trusted" set. Who would be eligible? What is the minimum set size? What is the maximum set size? Who would be ineligible?

I think the answers to that are inherently subjective, so everyone may reach a different conclusion. It is really up to you to decide who you want to trust and who you don't want to trust. Put differently: No one is holding you back to trust as many or as few people as you want. However, no one can force you to trust a specific set of parties (this is what you are asking here).

I understand the point but it leads only to less than ideal solutions ( such as https://github.com/rootzoll/raspiblitz/blob/320015dc6086fb2bf238527d67581eaa574ac638/home.admin/config.scripts/bitcoin.install.sh#L42 ).

Out of the 56 keys here https://github.com/bitcoin/bitcoin/blob/master/contrib/builder-keys/keys.txt only 27 import successfully from public keyservers with the scripts proposed on https://github.com/bitcoin/bitcoin/tree/master/contrib/builder-keys

At the very least, keys not publicly available should, in my opinion, be removed from the list and from the signature file.

For clarity and user friendliness, a 0 result from gpg --verify SHA256SUMS.asc SHA256SUMS should be equal to "you successfully verified the signatures" which is not the case now.

The current situation leads to users bypassing the verification because it fails anyway.

I understand the point but it leads only to less than ideal solutions ( such as https://github.com/rootzoll/raspiblitz/blob/320015dc6086fb2bf238527d67581eaa574ac638/home.admin/config.scripts/bitcoin.install.sh#L42 ).

Can you explain how this is less than ideal? If you are happy solely trusting the key with fingerprint 71A3 B167 3540 5025 D447 E8F2 7481 0B01 2346 C9A6, then your code should be fine.

At the very least, keys not publicly available should, in my opinion, be removed from the list and from the signature file.

The list also includes keys of gitian builders, some of which have long stopped being active. So I think it is fine to remove them. However, it will never be possible to ensure all keys in the list will be active at all times. Any key might expire at any time or disappear from a the server.

Maybe we can improve the documentation (see for example https://github.com/bitcoin-core/bitcoincore.org/pull/807) or the tooling, but I don't think there is anything about the release process that makes sense to change.

If you don't have some keys, the rest should still verify correctly, right? Is something preventing you from importing only the keys of signers you trust?

Getting this merged would help a lot https://github.com/bitcoin/bitcoin/pull/23020

Reduce the signer list to a set of well known, trusted signers

This is up to you to determine for yourself. The idea is that the project no longer picks who is a trusted signer.

You only need the public keys of the signers you trust. You should ignore the other PGP signatures. Getting all keys isn't helpful, even if they are available. If you don't know who the person is then their signature has no additional value to you (also, doing this creates incentive to sybil attack).

Edit: heck, if you want the same security as before you could ignore all keys except mine. I don't recommend this, though, as a) putting 100% trust in one person is a bad idea b) I may not be signiing every release in the future.

Closing for now. Let us know if you have any other questions or suggestions.

You only need the public keys of the signers you trust. You should ignore the other PGP signatures.

The thing is, there's no one line gnupg command that allows you to do that. This is what I ended up implementing, but I still don't really like it:

result=$(gpg --verify SHA256SUMS.asc 2>&1)
goodsigs=$(echo $result | grep -o 'Good signature' | wc -l)

if [ $goodsigs -le 10 ]; then
  exit 2
fi

That's why separating each signature into a single file like some other projects do makes more sense, it allows you to check only the signatures you want easily.

Can you explain how this is less than ideal? If you are happy solely trusting the key with fingerprint 71A3 B167 3540 5025 D447 E8F2 7481 0B01 2346 C9A6, then your code should be fine.

One problem I encountered is that using grep to look for "Good signature" fails on non-English interfaces.

One problem I encountered is that using grep to look for "Good signature" fails on non-English interfaces.

It might be possible to set the language for this call with LC_ALL or similar.

It might be possible to set the language for this call with LC_ALL or similar.

LC_ALL only works when setlocale is used, but it is sufficient to set only the LANG

Instead of:

result=$(gpg --verify SHA256SUMS.asc 2>&1)

Would recommend using:

result=$(LANG=en_US.utf8; gpg --verify SHA256SUMS.asc 2>&1)

but need to make sure that the locale en_US.utf8 is present on all systems.

That's why separating each signature into a single file like some other projects do makes more sense, it allows you to check only the signatures you want easily.

Yes, this is much more straightforward. See LND for example https://github.com/lightningnetwork/lnd/releases/tag/v0.14.3-beta where every signer provides a separate signature file for the manifest.

You can find individual guix build sigs in the guix.sigs repo filed per release, eg for 23.0: https://github.com/bitcoin-core/guix.sigs/tree/main/23.0

With this -- if you only required trusting a subset of developers for whom you are sure you know the keys of -- you could manually grep their manifest file for the hash of your downloaded archive, then verify the signature of the manifest.

good idea @willcl-ark, we have the individual signatures after all! Can confirm it works and exits with 0 :

wget -O all.SHA256SUMS https://raw.githubusercontent.com/bitcoin-core/guix.sigs/main/23.0/laanwj/all.SHA256SUMS
wget -O all.SHA256SUMS.asc https://raw.githubusercontent.com/bitcoin-core/guix.sigs/main/23.0/laanwj/all.SHA256SUMS.asc
gpg --verify all.SHA256SUMS.asc && echo $?

gpg: assuming signed data in 'all.SHA256SUMS'
gpg: Signature made Fri 22 Apr 2022 17:18:09 BST
gpg:                using RSA key 9DEAE0DC7063249FB05474681E4AED62986CD25D
gpg: Good signature from "Wladimir J. van der Laan <laanwj@protonmail.com>" [unknown]
gpg:                 aka "Wladimir J. van der Laan <laanwj@gmail.com>" [unknown]
gpg:                 aka "Wladimir J. van der Laan <laanwj@visucore.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 71A3 B167 3540 5025 D447  E8F2 7481 0B01 2346 C9A6
     Subkey fingerprint: 9DEA E0DC 7063 249F B054  7468 1E4A ED62 986C D25D
0

That's why separating each signature into a single file like some other projects do makes more sense, it allows you to check only the signatures you want easily.

True, not concatenating the detached sigs would even make the release process simpler (could just copy them), but this would give a flurry of files in the distribution archive, given the number of signers we have nowadays. Could separate them into a directory ofc. Or a tar/zip file (easier when downloading from the web).

What if someone use "Good signature" as his name?

but need to make sure that the locale en_US.utf8 is present on all systems.

You cannot assume that. In Gentoo I can specify what locales I want to be built. But C or POSIX locale should work instead, they are required by standards and will be always present.

good idea @willcl-ark, we have the individual signatures after all! Can confirm it works and exits with 0 :

wget -O all.SHA256SUMS https://raw.githubusercontent.com/bitcoin-core/guix.sigs/main/23.0/laanwj/all.SHA256SUMS
wget -O all.SHA256SUMS.asc https://raw.githubusercontent.com/bitcoin-core/guix.sigs/main/23.0/laanwj/all.SHA256SUMS.asc
gpg --verify all.SHA256SUMS.asc && echo $?

gpg: assuming signed data in 'all.SHA256SUMS'
gpg: Signature made Fri 22 Apr 2022 17:18:09 BST
gpg:                using RSA key 9DEAE0DC7063249FB05474681E4AED62986CD25D
gpg: Good signature from "Wladimir J. van der Laan <laanwj@protonmail.com>" [unknown]
gpg:                 aka "Wladimir J. van der Laan <laanwj@gmail.com>" [unknown]
gpg:                 aka "Wladimir J. van der Laan <laanwj@visucore.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 71A3 B167 3540 5025 D447  E8F2 7481 0B01 2346 C9A6
    Subkey fingerprint: 9DEA E0DC 7063 249F B054  7468 1E4A ED62 986C D25D
0

This is great! It will of course complicate a little the automatic verification code but I can work with this!

I still encourage review of https://github.com/bitcoin/bitcoin/pull/23020

This is great! It will of course complicate a little the automatic verification code but I can work with this! @ketominer Do you consider this issue resolved? If so perhaps it could be closed.

As an average user - not a dev - I am completely lost just finding a valid key to sign my Bitcoin Core. All I want is to run a Full Bitcoin Node, but my iMac won't let my even open the downloaded program as it is not signed. Any help out there?

@AdmiralNeo did you already try following the instructions found on https://bitcoincore.org/en/download/ if so, where abouts did you get stuck?

If your question is specifically about how to find a key to import into GPG (to use for trust), then the answer is a little less clear, but there is still some guidance on the website:

You can find many developer keys listed in the bitcoin/bitcoin repository, which you can then load into your GPG key database.

For example, given the builder-keys/keys.txt line E777299FC265DD04793070EB944D35F9AC3DB76A Michael Ford (fanquake) you could load that key using this command:

gpg --keyserver hkps://keys.openpgp.org --recv-keys E777299FC265DD04793070EB944D35F9AC3DB76A

THX for your answer. I could load that key and somehow I guess he signed my downloaded Bitcoin Core program. But I am not able to start the program with the name "bitcoin-24.0.1-x86_64-apple-darwin.dmg". Must be a basic error of me as I am not a dev and using a terminal is already a big achievement for me, lol <img width="908" alt="Bildschirmfoto 2023-02-14 um 22 01 41" src="https://user-images.githubusercontent.com/125405780/218862325-81145c83-21f9-4d67-a712-1a18ce5f3537.png">

Hmm, I wonder if this is the same issue as #26176 where the .dmg is being silently mounted?

After you double click the dmg file, is it mounted to the left pane of your Finder window for you to click on to open?

My God - Apple says "can't open cause it can not check for malware. Software has to be updated."

<img width="911" alt="Bildschirmfoto 2023-02-14 um 22 11 09" src="https://user-images.githubusercontent.com/125405780/218863910-407471e4-bd8c-4973-aef7-3221716382aa.png">

And I guess it is mounted?

Yes it's mounted. @AdmiralNeo Perhaps you could try this workaround: #24140 (comment)

edit: To install the application properly, before opening using the context menu workaround, you can still drag Bitcoin.app into the Applications folder from the Finder window, so that you can run without mounting the dmg each time.

Wow! Somehow I started a process of Bitcoin Core loading. THX a lot and let's see if a Dummy like me is really able to run a node. Ordinals brought me to this. THX bro a lot! Let's see if it works.

<img width="815" alt="Bildschirmfoto 2023-02-14 um 22 37 12" src="https://user-images.githubusercontent.com/125405780/218868760-4e6dd03d-c849-4b41-85d2-e66524802876.png">

Not sure what is left to be done here in this thread. Lets continue discussion in the appropriate other threads:

• https://github.com/bitcoin-core/bitcoincore.org/issues/945 for the website
• #15774, #26176, #25834 for mac issues
• #23020 about the verify bin script

What would be the appropriate other threads as BTC Core has stopped loading and I don't have any idea how to start it again? <img width="812" alt="Bildschirmfoto 2023-02-15 um 19 29 00" src="https://user-images.githubusercontent.com/125405780/219120373-6cd0d7c0-a88a-4761-8555-dbe74943ef56.png">

THX for your proposed fixes - you could add that the official download-page for Apple-version contains now already for several months an outdated signing key of the departed Wladimir van der Laan - no valid key is there to be found.

If you have questions about usage, see https://bitcoin.stackexchange.com.

If you have comments about the website, see https://github.com/bitcoin-core/bitcoincore.org

Please don't keep commenting in a closed issue about an unrelated topic.