[28.x] Backport #31407 #32563

fanquake commented at 12:46 PM on May 19, 2025: member

Backports #31407 + #32003.

fanquake added this to the milestone 28.2 on May 19, 2025

DrahtBot commented at 12:46 PM on May 19, 2025: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32563.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	pinheadmz

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

DrahtBot added the label Backport on May 19, 2025

pinheadmz commented at 1:18 PM on May 19, 2025: member

Concept ACK, starting guix build of this branch and will try to codesign with certificate

DrahtBot added the label CI failed on May 19, 2025

pinheadmz commented at 2:50 PM on May 19, 2025: member

codesigning hung forever at one point. I SIGINT it and got a possibly helpful error:

--> ./detached-sig-create.sh <redacted>
WARNING: Part of the file was not parsed: 37803 bytes
Enter the passphrase for <redacted>:
Enter the passphrase for <redacted>:
WARNING: Part of the file was not parsed: 37803 bytes
Code signature created
WARNING: Part of the file was not parsed: 37803 bytes
WARNING: Part of the file was not parsed: 37803 bytes
Code signature applied
WARNING: Part of the file was not parsed: 37803 bytes
Code signature is valid
Notarization ID: 3d941711-8e4b-473c-b504-02f5348a0176
Uploading...
Polling notarization status
Polling notarization status
Polling notarization status
Polling notarization status
Polling notarization status
WARNING: Part of the file was not parsed: 37803 bytes
Stapling
Notarization stapled to bundle

^C

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/zipfile.py", line 1815, in write
    shutil.copyfileobj(src, dest, 1024*8)
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py", line 200, in copyfileobj
    fdst_write(buf)
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/zipfile.py", line 1178, in write
    data = self._compressor.compress(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/bin/signapple", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/matthewzipkin/Desktop/work/signapple/signapple/__init__.py", line 192, in main
    args.func(args)
  File "/Users/matthewzipkin/Desktop/work/signapple/signapple/__init__.py", line 52, in do_notarize
    notarize(
  File "/Users/matthewzipkin/Desktop/work/signapple/signapple/notarize.py", line 345, in notarize
    _submit_for_notarization(
  File "/Users/matthewzipkin/Desktop/work/signapple/signapple/notarize.py", line 292, in _submit_for_notarization
    zipped = shutil.make_archive(
             ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py", line 1165, in make_archive
    filename = func(base_name, base_dir, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py", line 1046, in _make_zipfile
    zf.write(path, arcname)
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/zipfile.py", line 1814, in write
    with open(filename, "rb") as src, self.open(zinfo, 'w') as dest:
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/zipfile.py", line 1201, in close
    raise RuntimeError("File size too large, try using force_zip64")
RuntimeError: File size too large, try using force_zip64

guix: Rename unsigned.tar.gz to codesigning.tar.gz

The tarballs used for codesigning are more than merely unsigned, they
also contain scripts and other data for codesigning. Rename them to
codesigning.tar.gz to distinguish from tarballs containing actually just
the unsigned binaries.

Github-Pull: #31407
Rebased-From: c214e5268fa9322a83cbba6d47d33f830efdd89e

00b401c648

guix: Rename MacOS binaries to unsigned.tar.gz

The MacOS binaries are unsigned and therefore also unusable on MacOS.
Indicate as such by naming the tarball "unsigned".

Github-Pull: #31407
Rebased-From: d9d49cd533bd430776c0cbe2fd666ffec3e6637b

9f0ee1cc9b

guix: Rename Windows unsigned binaries to unsigned.zip

As codesigned binaries will be published, the unsigned ones should be
clearly marked as such.

Github-Pull: #31407
Rebased-From: 4e5c9ceb9dd5a6ad8eea689d916a632e4d482812

2c21db657f

fanquake force-pushed on May 30, 2025

pinheadmz commented at 3:04 PM on May 31, 2025: member

Should I try to build and sign again? Recent just looks like a repository change.

fanquake commented at 3:08 PM on May 31, 2025: member

Yea, just a rebase on the Guix repo change. If you don't mind building again, that'd be great. Can debug.

pinheadmz commented at 4:46 PM on May 31, 2025: member

same issue, RuntimeError: File size too large, try using force_zip64

build: Include all MacOS binaries for codesigning

Github-Pull: #31407
Rebased-From: dd4ec840eeb468e94cfc9e3c72cfbfd6704dc0da

ac2b6083ba

build: Include all Windows binaries for codesigning

Github-Pull: #31407
Rebased-From: e8b3c44da6e060464970717bbd0a5bf84867b82c

2b279a2138

guix: Update signapple

Github-Pull: #31407
Rebased-From: 710d5b5149d0bc36d2643281d81f8f9b0c51b480

0bd5cb7ac4

contrib: Sign and notarize all MacOS binaries

Signapple has been updated to sign individual binaries, and notarize app
bundles and binaries. When codesigning, all individual binaries will be
codesigned, and both the app bundle and individual binaries will be
notarized.

Github-Pull: #31407
Rebased-From: 31d325464d0cf2d06888e0c543ae26a944f2ec6b

c60055c637

guix: Apply codesignatures to all MacOS binaries

Github-Pull: #31407
Rebased-From: aafbd23fd97ac242f7f83e5f0fff20044176e126

812cadefa2

guix: Apply all codesignatures to Windows binaries

Github-Pull: #31407
Rebased-From: e181bda061ca63021511be6e286fdf6a5818df49

744b1c8581

doc: remove note about macOS self-signing

Followup to #31407.

Github-Pull: #32003
Rebased-From: c873ab6f23e027af1c5837256ce3c9eccaf409cb

52f09633d0

doc: update release-notes.md b1f694fce2

in contrib/macdeploy/detached-sig-create.sh:47 in 0ce4a9d443 outdated

  51 | +${SIGNAPPLE} sign -f --hardened-runtime --detach "${OUTROOT}/${BUNDLE_ROOT}" --passphrase "${cs_key_pass}" "$1" "${UNSIGNED_BUNDLE}"
  52 | +${SIGNAPPLE} apply "${UNSIGNED_BUNDLE}" "${OUTROOT}/${BUNDLE_ROOT}/${BUNDLE_NAME}"
  53 | +${SIGNAPPLE} notarize --detach "${OUTROOT}/${BUNDLE_ROOT}" --passphrase "${api_key_pass}" "$2" "$3" "${UNSIGNED_BUNDLE}"
  54 | +
  55 | +# Sign each binary
  56 | +find . -maxdepth 3 -wholename "*/bin/*" -type f -exec realpath --relative-to=. {} \; | while read -r bin

pinheadmz commented at 11:07 PM on June 1, 2025:

This is the issue maybe? Is #31161 merged into 28.x at this point? There isn't a bin/ directory codesigning.tar.gz archive, just a dist/

pinheadmz commented at 11:41 PM on June 1, 2025:

By contrast, the 29.0 codesigning tar had ./bitcoin-29.0/bin/

fanquake commented at 9:21 AM on June 2, 2025:

Thanks, is should be fixed now. It's unrelated to #31161, was a bad cherry-pick fixup by me.

fanquake force-pushed on Jun 2, 2025

fanquake marked this as ready for review on Jun 2, 2025

pinheadmz commented at 3:17 PM on June 2, 2025: member

I think all macos signing is working now.

Detached sigs for this commit: https://github.com/pinheadmz/bitcoin-detached-sigs/tree/fanquake-backport_codesigning-b1f694fce2

Tested signed binaries on macos/arm64:

--> ./bitcoind --version
Bitcoin Core version v28.2.0rc1
Copyright (C) 2009-2025 The Bitcoin Core developers

codesigned guix builds SHASUMS:

30af3e1bbfa4a3f891c2e62887aa8ef5be3cf1bf7d0029f2b1b2ba30886791c6  arm64-apple-darwin-codesigned/bitcoin-b1f694fce276-arm64-apple-darwin.tar.gz
0120313d6bc0e7a93a9df1507a2c5838183cd1d8ec7f0ac5e82e2ad5260770f8  arm64-apple-darwin-codesigned/bitcoin-b1f694fce276-arm64-apple-darwin.zip
3c3612cb3419940be1b19adad354169c913d2cb8fd3431addce6bbeab0e35892  arm64-apple-darwin/bitcoin-b1f694fce276-arm64-apple-darwin-codesigning.tar.gz
39125b89de8903cf111df70abe5f2d9874fcc98f3dd234f4c202e9d59057ca48  dist-archive/bitcoin-b1f694fce276-codesignatures-bd42dd2a53b0.tar.gz
39125b89de8903cf111df70abe5f2d9874fcc98f3dd234f4c202e9d59057ca48  dist-archive/bitcoin-b1f694fce276-codesignatures-bd42dd2a53b0.tar.gz
6ec5b4badd4c4f64c09762626a4b4c727b5c90a8d74699a3af5a21cbbda4bba9  x86_64-apple-darwin-codesigned/bitcoin-b1f694fce276-x86_64-apple-darwin.tar.gz
fbdfc915ef1a77eaab181667f09481081a3fd7131d53279044999744fdfd0d87  x86_64-apple-darwin-codesigned/bitcoin-b1f694fce276-x86_64-apple-darwin.zip
a0e546b23fdef1466ae3c5d856967a34067665d8daa75a4621b1ec0b8f8185ab  x86_64-apple-darwin/bitcoin-b1f694fce276-x86_64-apple-darwin-codesigning.tar.gz

pinheadmz approved

pinheadmz commented at 3:17 PM on June 2, 2025: member

ACK b1f694fce276d68a5b983c187a4efbb231d83f79

<details><summary>Show Signature</summary>

pinheadmz's public key is on openpgp.org

</details>

fanquake requested review from achow101 on Jun 2, 2025

fanquake merged this on Jun 5, 2025

fanquake closed this on Jun 5, 2025

fanquake deleted the branch on Jun 5, 2025

glozow referenced this in commit cb13264169 on Jun 9, 2025

achow101 commented at 6:30 PM on June 10, 2025: member

Looks like 3656b828dc2204418974e94928cc8d915b10ed95 wasn't cherry-picked? I think that's necessary for 744b1c8581a88cdb2c3a1f7730b5c7caae86a702 to work.

fanquake commented at 9:45 AM on June 12, 2025: member

Thanks, addressed in #32735.