build: Use a zip instead of dmg for macOS releases

fanquake commented at 7:52 am on May 26, 2020: member

Implements #18128 (mostly). From discussion on that issue we’ve got 3 ACKs (to move from .dmg to .zip), 1 ~0 and 1 NACK. So I’ve opened this PR to give a better overview of what the changes might actually look like.

Gitian builds currently work, however I haven’t tested the signer, and that likely needs some additional changes. As mentioned we can drop quite a few packages from depends. There’s some commits here that can cleaned up, squashed or dropped.

If you’re compiling macOS, make appbundle will (as before) build a dynamically linked Bitcoin Core.app. make deploy will still build a statically linked dist/Bitcoin Core.app and will now also create a dist/Bitcoin-Core.zip.

Compiling on Linux there has never been a make appbundle rule, and with these changes, make deploy now behaves the same as on macOS. The make deploydir rule has been removed.

Initially I was unconvinced (of a swap from .dmg to .zip), however I have come around after looking at these changes. If there is not agreement that “this is the way”, then I’ll revive #18151 and fix it up to include xorriso in depends etc.

fanquake added the label macOS on May 26, 2020

fanquake added the label Build system on May 26, 2020

fanquake added the label Needs Conceptual Review on May 26, 2020

bitcoin deleted a comment on May 26, 2020

jonasschnelli commented at 11:56 am on May 26, 2020: contributor

Gitian Build seems to work: https://bitcoin.jonasschnelli.ch/gitian/build/142 Maybe the ZIP filename could still contain the phrase “macOS” or “osx”.

Conceptual, I’m currently ~0 (I commented on the concept before). I think it is nice to simplify the build system (probably makes it much easier for guix). Though, I’m currently unsure if this slightly worsens the UX for macOS users. If it does, should we reduce the UX to simplify the build-system (is that an acceptable tradeoff)?

However, if there is a difference in user experience between a zip and a dmg, it is probably marginal.

MarcoFalke commented at 4:25 pm on May 26, 2020: member

Is there any “official” mac documention or user surveys that suggest which one is the recommended or preferred way?

dongcarl commented at 1:03 am on May 27, 2020: contributor

Concept ACK! Great to see the amount of code we can remove.

(probably makes it much easier for guix).

Just to make myself clear, I am for this change because it is a great simplification for a marginal difference in user experience and not because of anything specific to Guix (my current implementation of the Guix macOS build is already able to build the dmg). :relaxed:

Sjors commented at 4:40 pm on May 27, 2020: member

Concept ACK

You can add a symlink to /Applications to the archive and call it “Drag Here” (so it appears on the right).

With that it’s close enough to what macOS users are used to, with a nice reduction in dependencies and complexity.

You could also keep the .DS_Store and .background folder around for full branding and a nicer layout, but that’s pretty marginal.

Another nice thing about zip files is that Safari automatically unzips them after download.

DrahtBot commented at 7:58 pm on May 27, 2020: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#18434 (tests: add a test-security target and run it in CI by fanquake)
#17919 (depends: Allow building with system clang by dongcarl)
#17311 (build: macOS fix background.svg by RandyMcMillan)
#14066 (gitian-linux: Build binaries for 64-bit POWER by luke-jr)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

fanquake commented at 8:44 am on May 28, 2020: member

You could also keep the .DS_Store and .background folder around for full branding and a nicer layout, but that’s pretty marginal.

“Keeping that around” would mostly defeat the point of these changes.

Sjors commented at 8:53 am on May 28, 2020: member

“Keeping that around” would mostly defeat the point of these changes.

It still gets rid of genisoimage and libdmg-hfsplus, but indeed .DS_Store and .background require a lof of tools themselves. Just the symlink should get us close enough to the original UX.

build: add -zip option to macdeployqtplus

This simply zips the staticly built app bundle that has been placed
into /dist.

8c56653652

build: remove native_libdmg-hfsplus from depends 9f230b4ab1

build: remove native_cdrit from depends d2cf0e40ad

build: remove macOS .dmg background image and generation

A make deploy won't currently work because -fancy is still passed to macdeployqtplus

f5b7b76bee

build: remove macOS .DS_Store and .plist generation c158baf190

build: note why we set CFBundle* f6c5282135

build: update gitian descriptors to use zip method 8ac6b34f4f

build: remove -dmg and -fancy argument from macdeployqtplus ea055ea008

build: remove -sign argument from macdeployqtplus 2c562a7a8f

build: merge -zip and -volname arguments in macdeployqtplus 83f873bd09

build: move qt_conf to where it's actually used in macdeployqtplus 0a46f886d5

build: remove -add-resources argument from macdeployqtplus 255a9de55d

docs: update macOS docs for new zip deployment method 88fd25d0be

fanquake force-pushed on May 29, 2020

fanquake commented at 2:37 am on May 29, 2020: member

Pushed a few more removals, updated the descriptors and fixed whitespace.

Is there any “official” mac documention or user surveys that suggest which one is the recommended or preferred way?

The Apple documentation is mainly focused on using Xcode and App Store distribution: https://help.apple.com/xcode/mac/current/#/devac02c5ab8. There is a section on distribution outside the App Store: https://help.apple.com/xcode/mac/current/#/dev033e997ca, however I couldn’t quickly find any mention of using .dmgs. There are some instructions on “macOS installer packages” (.pkg): https://help.apple.com/xcode/mac/current/#/deve51ce7c3d.

There seem to be at least some companies that are using a combination of .dmg and .zip for distribution. i.e: Sublime Merge is distributed as a .zip, while Sublime Text is a .dmg.

Just the symlink should get us close enough to the original UX.

I’m not aware of any projects that are using a symlink inside a .zip approach. The Bitcoin Core GUI is mostly designed for power users of bitcoin, and I think the intersection of power users, and people who don’t know how to handle a .zip file would be very small.

Also, as far as I’m aware, you can open and use an app bundle from any directory on macOS, and it makes little to no difference to how the app actually runs?

Sjors commented at 9:02 am on May 29, 2020: member

I’m not aware of any projects that are using a symlink inside a .zip approach

That’s because most projects use a DMG with a symlink and background image. We have a good reason to avoid that, namely to make deterministic builds easier. Even as an experienced macOS user, I find the symlink useful; it saves me from opening a new Finder window, navigating to Applications and then dragging between two windows. And it’s a trivial thing to add.

Also, as far as I’m aware, you can open and use an app bundle from any directory on macOS, and it makes little to no difference to how the app actually runs?

Whenever I deal with applications that don’t have an installer or a symlink, I very often make the mistake of running it from the Download folder. That works, until I accidentally clean up the download folder. There’s no obvious alternative folder to drag the program to.

When downloading an upgrade there’s an additional risk that a user doesn’t replace the original version. The first time they might double click on the download and run the new version. Second time they might use command + space and launch the old version.

in Makefile.am:119 in 88fd25d0be

110@@ -116,50 +111,18 @@ osx_volname:
111 	echo $(OSX_VOLNAME) >$@
112 
113 if BUILD_DARWIN
114-$(OSX_DMG): $(OSX_APP_BUILT) $(OSX_PACKAGING) $(OSX_BACKGROUND_IMAGE)

fanquake commented at 2:58 am on May 31, 2020:

One possible simplification after this change, would be to make the otool and installnametool checks no longer cross-compiling dependent. Then we should be able to have a single OSX_ZIP rule, and drop the need for BUILD_DARWIN.

luke-jr commented at 3:25 pm on June 3, 2020: member

Can we add the ZIP and remove the DMG in two steps?

IMO it’d be nice to build both for 0.21, but only distribute the ZIP. That way if it causes too much problems/confusion we can just change it back without rebuilds/etc.

fanquake commented at 3:50 am on November 13, 2020: member

I might come back to this in future, but closing for now as I’ve made some changes to the existing macdeploy process. Any discussion should continue in #18128.

fanquake closed this on Nov 13, 2020

bitcoin locked this on Feb 15, 2022

fanquake deleted the branch on Sep 14, 2023

build: Use a zip instead of dmg for macOS releases #19068

Conflicts