build: further consolidate macOS deployment

fanquake commented at 10:33 am on March 25, 2022: member

Rather than maintaining 2 different versions of the same code (.tiff copying and symlink generation), consolidate to just the Python code, and use it on macOS and Linux. Previously Linux would perform the 2 actions in the makefile, and then would still be running the macdeployqtplus script, so it makes sense to further consolidate deployment operations into the script.

Guix Build (on x86_64):

023343f04c426c7ff078afae4e600a7028970d4d86eed8b7834696d9e4d684151  guix-build-3d415215699e/output/arm64-apple-darwin/SHA256SUMS.part
1c28b2a2e4888bf84369aa25804e2576347d5ab09416354ec8b95c76a9d38ff96  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.dmg
29a57077b2bd722a7d85d26b66cbce5abdb791985fe9d9d37e884c79ba8751e24  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.tar.gz
3d2b06dc5b86541798ace41dab569849f7403e7ff9ec329bda671ec84e6fad549  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin.tar.gz
4608e7d51a44ab9c5b28eb3703a0f4fe98b4adff22c77a5502786b84bd96cc188  guix-build-3d415215699e/output/dist-archive/bitcoin-3d415215699e.tar.gz
53e483705b1f9f1fb8f6afedc8ad0214a6cb00e77f766c0b03c42d56f410d4362  guix-build-3d415215699e/output/x86_64-apple-darwin/SHA256SUMS.part
69370e3e3b7d47b5a44e64554cf3b6d7e0671b072c08cd251eacc7ec72ce2b53f  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.dmg
7ad0f68682d78c311497669fc3d627138be37510215d259b5f0b686d93e7d83b7  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.tar.gz
8e09dce4ff692ef66d1f4818083c1880bcf3a79c53112561d9e929bb6e5ffc011  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin.tar.gz

fanquake added the label macOS on Mar 25, 2022

fanquake added the label Build system on Mar 25, 2022

fanquake added the label DrahtBot Guix build requested on Mar 25, 2022

prusnak commented at 12:23 pm on March 25, 2022: contributor

Nit: typo in commit message 471491f471530360edfc5d523c7c3b9d1d096c3d: symlilnk

fanquake force-pushed on Mar 25, 2022

fanquake commented at 3:21 pm on March 25, 2022: member

Nit: typo in commit message https://github.com/bitcoin/bitcoin/commit/471491f471530360edfc5d523c7c3b9d1d096c3d: symlilnk

Fixed.

DrahtBot commented at 9:57 pm on March 25, 2022: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#24031 (build: don’t compress macOS DMG by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot commented at 9:14 am on March 29, 2022: member

Guix builds

File	commit 2f0f056e08cd5a1435120592a9ecd212fcdb915b(master)	commit 4e07ab79e4601a3d0a031048836d8039baf1c230(master and this pull)
SHA256SUMS.part	`2725812900a4d8ea...`	`3ae7c0e116da0901...`
*-aarch64-linux-gnu-debug.tar.gz	`3d08fb464dcd2dd1...`	`dd32df7c886c6a22...`
*-aarch64-linux-gnu.tar.gz	`f1cdab2e00b63b03...`	`e8fab32db4368cc5...`
*-arm-linux-gnueabihf-debug.tar.gz	`71aa34f8002f0ba0...`	`92d887d63cbc3f73...`
*-arm-linux-gnueabihf.tar.gz	`be89d1bb02f92f0b...`	`ff4a8567767786dd...`
*-arm64-apple-darwin-unsigned.dmg	`441ec72b81572603...`	`62b4492a78b1ce91...`
*-arm64-apple-darwin-unsigned.tar.gz	`1f629b7062a63825...`	`f99d0a57244e2ed8...`
*-arm64-apple-darwin.tar.gz	`91559ba43ddb34c4...`	`5b8eb6a829a4b740...`
*-powerpc64-linux-gnu-debug.tar.gz	`8bf65582bf3d15d7...`	`50c7cf4e3b37dd20...`
*-powerpc64-linux-gnu.tar.gz	`6f6d6f7a602e7adf...`	`08cbdc5e6e8348ab...`
*-powerpc64le-linux-gnu-debug.tar.gz	`824819c6482a91ce...`	`ab29c51ae43d4b7a...`
*-powerpc64le-linux-gnu.tar.gz	`848dd7597176fa28...`	`ca114ec2928640ef...`
*-riscv64-linux-gnu-debug.tar.gz	`73901d339b07769e...`	`b1c1919de5b01de9...`
*-riscv64-linux-gnu.tar.gz	`8cf99b122120c5cd...`	`3f1c0318d06a4496...`
*-win64-debug.zip	`7ef5105bb5652f7c...`	`edd03e3dcdba55b8...`
*-win64-setup-unsigned.exe	`f066318ff13e5cff...`	`be7acdddbb1cf871...`
*-win64-unsigned.tar.gz	`87cfc87633d69a73...`	`79b82766c8b6120f...`
*-win64.zip	`165340e43bb07aea...`	`c13a112dc490cd80...`
*-x86_64-apple-darwin-unsigned.dmg	`13221345b00c765f...`	`8543f7494b83cd85...`
*-x86_64-apple-darwin-unsigned.tar.gz	`f788e55dde11a00b...`	`025889534818d4bc...`
*-x86_64-apple-darwin.tar.gz	`67fad04929c64e74...`	`b7d746c2cfa81b2b...`
*-x86_64-linux-gnu-debug.tar.gz	`606a6685ee83a2a2...`	`035a73963716c133...`
*-x86_64-linux-gnu.tar.gz	`b4d505664d026c54...`	`24642fc075bbff0e...`
*.tar.gz	`bb766c7a1dc2e1b7...`	`2ede3bec3a786c84...`
guix_build.log	`241f6fb904efeba1...`	`c1d0f5da41569ef4...`
guix_build.log.diff		`db2809d9a608a2a3...`

DrahtBot removed the label DrahtBot Guix build requested on Mar 29, 2022

laanwj commented at 1:26 pm on April 5, 2022: member

Python code review ACK 6eacd88d7c68a55de4e43946987a968316200ed2

in contrib/macdeploy/macdeployqtplus:549 in 6eacd88d7c outdated

543@@ -544,6 +544,20 @@ ds.close()
544 if platform.system() == "Darwin":
545     subprocess.check_call(f"codesign --deep --force --sign - {target}", shell=True)
546 
547+print("+ Installing background.tiff +")
548+
549+bg_path = os.path.join('dist', '.background', os.path.basename('background.tiff'))

laanwj commented at 1:27 pm on April 5, 2022:

Nit: what’s the point of os.path.basename here? Pretty sure it will always return what is passed in.

fanquake commented at 1:52 pm on April 5, 2022:

Right, this was copied, but os.path.basename is unnecessary, have dropped it.

build: perform all .tiff copying in macdeployqtplus

By copying the .tiff earlier in the macdeploy process, we can unify the logic in
the deploy script.

dac6936719

build: perform /Applications symlink generation in macdeployqtplus

By generating the symlink earlier in the macdeploy process, we can unify the
logic in the deploy script.

3d41521569

in contrib/macdeploy/macdeployqtplus:551 in 6eacd88d7c outdated

543@@ -544,6 +544,20 @@ ds.close()
544 if platform.system() == "Darwin":
545     subprocess.check_call(f"codesign --deep --force --sign - {target}", shell=True)
546 
547+print("+ Installing background.tiff +")
548+
549+bg_path = os.path.join('dist', '.background', os.path.basename('background.tiff'))
550+os.mkdir(os.path.dirname(bg_path))
551+shutil.copy2('contrib/macdeploy/background.tiff', bg_path)

laanwj commented at 1:30 pm on April 5, 2022:

Should probably also use os.path.join here to be consistent instead of assuming / seperator?

fanquake commented at 1:53 pm on April 5, 2022:

Done.

fanquake force-pushed on Apr 5, 2022

fanquake commented at 2:09 pm on April 5, 2022: member

Rebased, addressed nits, and added updated Guix build (macOS) to PR description.

laanwj commented at 2:59 pm on April 5, 2022: member

Re-ACK 3d415215699e718b3f6eea6e3c9fb2948476f930

Sjors commented at 9:30 pm on April 5, 2022: member

Tested that macOS 12.3.1 as well as Ubuntu cross-compile still produces sane images with 3d415215699e718b3f6eea6e3c9fb2948476f930. @laanwj did you mean 3d415215699e718b3f6eea6e3c9fb2948476f930?

laanwj commented at 10:03 am on April 6, 2022: member

@laanwj did you mean https://github.com/bitcoin/bitcoin/commit/3d415215699e718b3f6eea6e3c9fb2948476f930?

Eh, yes.

laanwj merged this on Apr 6, 2022

laanwj closed this on Apr 6, 2022

fanquake deleted the branch on Apr 6, 2022

sidhujag referenced this in commit 3a7cb19f37 on Apr 6, 2022

jarolrod commented at 7:08 pm on April 6, 2022: member

Post-merge ack, and confirming same hashes as fanquake

 0find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
 1
 223343f04c426c7ff078afae4e600a7028970d4d86eed8b7834696d9e4d684151  guix-build-3d415215699e/output/arm64-apple-darwin/SHA256SUMS.part
 3c28b2a2e4888bf84369aa25804e2576347d5ab09416354ec8b95c76a9d38ff96  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.dmg
 49a57077b2bd722a7d85d26b66cbce5abdb791985fe9d9d37e884c79ba8751e24  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.tar.gz
 5d2b06dc5b86541798ace41dab569849f7403e7ff9ec329bda671ec84e6fad549  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin.tar.gz
 6608e7d51a44ab9c5b28eb3703a0f4fe98b4adff22c77a5502786b84bd96cc188  guix-build-3d415215699e/output/dist-archive/bitcoin-3d415215699e.tar.gz
 73e483705b1f9f1fb8f6afedc8ad0214a6cb00e77f766c0b03c42d56f410d4362  guix-build-3d415215699e/output/x86_64-apple-darwin/SHA256SUMS.part
 89370e3e3b7d47b5a44e64554cf3b6d7e0671b072c08cd251eacc7ec72ce2b53f  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.dmg
 9ad0f68682d78c311497669fc3d627138be37510215d259b5f0b686d93e7d83b7  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.tar.gz
10e09dce4ff692ef66d1f4818083c1880bcf3a79c53112561d9e929bb6e5ffc011  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin.tar.gz

DrahtBot locked this on Apr 6, 2023

build: further consolidate macOS deployment #24669

Conflicts

Guix builds