guix: Notarize MacOS app bundle and codesign all MacOS and Windows binaries

achow101 commented at 10:32 pm on December 2, 2024: member

I have updated signapple to notarize MacOS app bundles without adding any additional dependencies. Further, it can also sign and apply detached signatures to standalone binaries.

As such, we can use signapple to perform the notarization and stapling steps so that MacOS will run the app bundle after it is installed. detached-sig-create.sh is updated to have a notarization step and to download the ticket which will be included in the detached signatures. The workflow is largely unchanged for the MacOS codesigners except for the additional requirement of having an App Store Connect API key and Team UUID, instructions for which can be found at https://github.com/achow101/signapple/blob/master/docs/notarization.md. For guix builders, the workflow is unchanged.

Additionally, the standalone binaries packaged in the MacOS .tar.gz and Windows .zip will now be codesigned. detached-sig-create.sh was updated to handle these, so the workflow for both MacOS and Windows codesigners remains unchanged. For guix builders, the workflow is also unchanged.

Because those binaries will how have codesigned and unsigned versions, the build command is modified to output -unsigned.{tar.gz,zip} archives containing the binaries. Since this happens to conflict with the tarball used for codesigning, the codesigning tarball was renamed to -codesigning.tar.gz. Both MacOS and Windows codesigners will need to adjust their workflows to account for the new name.

Fixes #15774 and #29749

guix: Rename unsigned.tar.gz to codesigning.tar.gz

The tarballs used for codesigning are more than merely unsigned, they
also contain scripts and other data for codesigning. Rename them to
codesigning.tar.gz to distinguish from tarballs containing actually just
the unsigned binaries.

912377ac49

guix: Rename MacOS binaries to unsigned.tar.gz

The MacOS binaries are unsigned and therefore also unusable on MacOS.
Indicate as such by naming the tarball "unsigned".

69ffb78d70

guix: Rename Windows unsigned binaries to unsigned.zip

As codesigned binaries will be published, the unsigned ones should be
clearly marked as such.

708fc182df

build: Include all MacOS binaries for codesigning 764f9d176d

DrahtBot commented at 10:32 pm on December 2, 2024: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31407.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

achow101 added this to the milestone 29.0 on Dec 2, 2024

DrahtBot added the label Build system on Dec 2, 2024

build: Include all Windows binaries for codesigning 14efedc992

guix: Update signapple 480e8bb3b5

contrib: Sign all MacOS binaries and notarize MacOS app bundle

Signapple has been updated to notarize bundles, and to sign individual
binaries. The app bundle is now notarized, and the individual binaries
are codesigned.

edc18462d8

contrib: Sign all Windows binaries too cc533421e4

guix: Apply codesignatures to all MacOS binaries 51237e4418

guix: Apply all codesignatures to Windows binaries ba67aa681e

achow101 force-pushed on Dec 2, 2024

DrahtBot commented at 10:47 pm on December 2, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33817027293

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

DrahtBot added the label CI failed on Dec 2, 2024

DrahtBot removed the label CI failed on Dec 2, 2024

achow101 commented at 11:36 pm on December 2, 2024: member

It’s unclear to me whether the standalone binaries need to be notarized too. This is currently not implemented, but should not be that much more complicated to do.

edilmedeiros commented at 2:12 am on December 3, 2024: contributor

It’s unclear to me whether the standalone binaries need to be notarized too. This is currently not implemented, but should not be that much more complicated to do.

Are they being codesigned already?

I was getting the v28 binaries from bitcoincore.org instantly killed in Sonoma 14.6.1 when trying to run them in the terminal today. Took me a while to understand what was happening because the processes are killed without a security message or anything like the “nice” gatekeeper popup. Codesigning each on my own solved the issue.

 0❯ tar xvfz bitcoin-28.0-arm64-apple-darwin.tar.gz
 1x bitcoin-28.0/
 2x bitcoin-28.0/bin/
 3x bitcoin-28.0/bin/bitcoin-cli
 4x bitcoin-28.0/bin/bitcoin-qt
 5x bitcoin-28.0/bin/bitcoin-tx
 6x bitcoin-28.0/bin/bitcoin-util
 7x bitcoin-28.0/bin/bitcoin-wallet
 8x bitcoin-28.0/bin/bitcoind
 9x bitcoin-28.0/bin/test_bitcoin
10x bitcoin-28.0/bitcoin.conf
11x bitcoin-28.0/share/
12x bitcoin-28.0/share/man/
13x bitcoin-28.0/share/man/man1/
14x bitcoin-28.0/share/man/man1/bitcoin-cli.1
15x bitcoin-28.0/share/man/man1/bitcoin-qt.1
16x bitcoin-28.0/share/man/man1/bitcoin-tx.1
17x bitcoin-28.0/share/man/man1/bitcoin-util.1
18x bitcoin-28.0/share/man/man1/bitcoin-wallet.1
19x bitcoin-28.0/share/man/man1/bitcoind.1
20x bitcoin-28.0/share/rpcauth/
21x bitcoin-28.0/share/rpcauth/README.md
22x bitcoin-28.0/share/rpcauth/rpcauth.py
23
24❯ bitcoin-28.0/bin/bitcoind --version
25[1]    36572 killed     bitcoin-28.0/bin/bitcoind --version
26
27❯ codesign --sign - bitcoin-28.0/bin/bitcoind
28
29❯ bitcoin-28.0/bin/bitcoind --version
30Bitcoin Core version v28.0.0
31Copyright (C) 2009-2024 The Bitcoin Core developers
32
33Please contribute if you find Bitcoin Core useful. Visit
34<https://bitcoincore.org/> for further information about the software.
35The source code is available from <https://github.com/bitcoin/bitcoin>.
36
37This is experimental software.
38Distributed under the MIT software license, see the accompanying file COPYING
39or <https://opensource.org/licenses/MIT>

achow101 commented at 2:15 am on December 3, 2024: member

Are they being codesigned already?

This PR codesigns them.

Sjors commented at 12:20 pm on December 3, 2024: member

It’s unclear to me whether the standalone binaries need to be notarized too.

Do you mean the binaries in unsigned.{zip,tar.gz} archives? I think it’s fine not to.

in contrib/guix/guix-codesign:158 in ba67aa681e

156-            echo "$(outdir_for_host "$1")/${DISTNAME}-win64-unsigned.tar.gz"
157+            echo "$(outdir_for_host "$1")/${DISTNAME}-win64-codesigning.tar.gz"
158             ;;
159         *darwin*)
160-            echo "$(outdir_for_host "$1")/${DISTNAME}-${1}-unsigned.tar.gz"
161+            echo "$(outdir_for_host "$1")/${DISTNAME}-${1}-codesigning.tar.gz"

Sjors commented at 12:29 pm on December 3, 2024:

912377ac4999467be7dfd51481c38972fb1475dd: maybe call it -maintainers to reduce confusion with -unsigned?

A more generic term might also be handy in the future if e.g. we want to include an OTS timestamp that commits to the (pre codesigning, pgp signed?) guix hashes (with all architectures).

achow101 commented at 4:52 pm on December 3, 2024:

The codesigners are not necessarily maintainers. I don’t think it’s useful to make this broad when it has been extremely specific for codesigning for more than a decade.

Sjors commented at 12:50 pm on December 3, 2024: member

In order to properly test this, you would have to provide the detached signatures and staple for this PR.

Reviewers then need to download it from some website. If you obtain the file via SSH from your own guix machine, macOS tries to be smart about it (at least my Intel mac used to do that).

0HOSTS="x86_64-w64-mingw32 x86_64-apple-darwin arm64-apple-darwin" ./contrib/guix/guix-build 
1...
2find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum

Guix hashes for macOS and Windows, before code-sign:

 0arm_64
 1e1d50cc856902e31e966f0e0d50ac732a6d8c7c080e9f9d50ab229d7503bd601  guix-build-ba67aa681e9b/output/arm64-apple-darwin/SHA256SUMS.part
 2462617a5476e946e10be8624d73b531041c8abf4628a35038d65b80534de768a  guix-build-ba67aa681e9b/output/arm64-apple-darwin/bitcoin-ba67aa681e9b-arm64-apple-darwin-codesigning.tar.gz
 32583965a7ecbcf48946bcb7676cf6b39a04cee44b34c6292d81cbb825f0d35df  guix-build-ba67aa681e9b/output/arm64-apple-darwin/bitcoin-ba67aa681e9b-arm64-apple-darwin-unsigned.tar.gz
 496f25391136c5e3f77b6dfb4e027aa1f43ccc91edf0237b9ec0c7b01a64b32f3  guix-build-ba67aa681e9b/output/arm64-apple-darwin/bitcoin-ba67aa681e9b-arm64-apple-darwin-unsigned.zip
 574d95312c8a33c65a1ea55c1d8392c58d77f78251af551926d4e913f12b45cac  guix-build-ba67aa681e9b/output/dist-archive/bitcoin-ba67aa681e9b.tar.gz
 61f457247d24466bc7f8faeb7f8faec80cadfed66ef68354c77a36ceba1127799  guix-build-ba67aa681e9b/output/x86_64-apple-darwin/SHA256SUMS.part
 7eafb67c89ed27e1dd216e2d300829c5bb22fbe6999839189441e3d64080b65f5  guix-build-ba67aa681e9b/output/x86_64-apple-darwin/bitcoin-ba67aa681e9b-x86_64-apple-darwin-codesigning.tar.gz
 80d9f7d7ef1f71b4c3072a0af44b374caf0a4646f4ba143f5440bbcf52c8fe614  guix-build-ba67aa681e9b/output/x86_64-apple-darwin/bitcoin-ba67aa681e9b-x86_64-apple-darwin-unsigned.tar.gz
 9a0337d08f8145ed4c8548086752b8c32766db1a0483573b44f95134cba9f3913  guix-build-ba67aa681e9b/output/x86_64-apple-darwin/bitcoin-ba67aa681e9b-x86_64-apple-darwin-unsigned.zip
10635b1e00cd6792faae60085f4d0ec71bb010756caa40f7241b05e1e3ef6aa18a  guix-build-ba67aa681e9b/output/x86_64-w64-mingw32/SHA256SUMS.part
1196326f5755ee14d74b74addc740e8e89c49ee65e4f56746a018e3aba39025642  guix-build-ba67aa681e9b/output/x86_64-w64-mingw32/bitcoin-ba67aa681e9b-win64-codesigning.tar.gz
120d1f7d39c4fcee02d763d85c7f14fe28233f4fe357192e54a227cb54843a8a0b  guix-build-ba67aa681e9b/output/x86_64-w64-mingw32/bitcoin-ba67aa681e9b-win64-debug.zip
13b3531bbeccd65d52cb73be3edc38f3aa4b1abe6373ad4e153e4c036a78f29b18  guix-build-ba67aa681e9b/output/x86_64-w64-mingw32/bitcoin-ba67aa681e9b-win64-setup-unsigned.exe
14e0a39aad96ac6fa7bf0957119bfca2a527e66f9991a71e84cc5c90864a9c833d  guix-build-ba67aa681e9b/output/x86_64-w64-mingw32/bitcoin-ba67aa681e9b-win64-unsigned.zip

guix: Notarize MacOS app bundle and codesign all MacOS and Windows binaries #31407

Code Coverage & Benchmarks

Reviews