iOS Deployment Target for RPC

Sjors commented at 3:29 pm on November 18, 2017: member

The iPhone 8 and X come with 256 GB variants. Plenty to store the current blockchain, certainly enough when you prune it. Most people who can afford such a phone also have pretty liberal data plans. This means there’s at least some demographic out there that can practically run a full node on their phone.

IDB and syncing more than a day still need wifi (can be done in the background when the phone is charging). Connections to other nodes would need some throttling on mobile. Maybe a smart app could even use SPV or trust a server when it’s not fully synced.

I wonder how hard it would be to create an iOs target (for bitcoind) based on the OSX target.

Building a full fledged iOs app is out of scope here, so it would just be the RPC server. A native wallet app would be an RPC client. Leaving out QT should also make that a lot easier.

Supported configuration:

0./configure --without-gui --with-incompatible-bdb

--without-gui: this removes the dependency on QT (although QT is cross-platform, that’s beyond the scope of this issue)
--with-incompatible-bdb: no need to be backwards compatible on a new OS

Pre-requisites:

#12557 or #12530

jonasschnelli commented at 7:11 pm on November 18, 2017: contributor

+1 for iOS support for compiling bitcoind (not Qt as you mentioned). Can also contribute… have iOS experience.

I haven’t tried. But I would expect something like this is a starting point (haven’t tested):

0  export CFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=7.0"
1  export LDFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk"
2  ./configure  --without-gui --with-incompatible-bdb --host="armv7s-apple-darwin"

ghost commented at 7:19 pm on November 18, 2017: none

The problem with iOS is the app can’t run in the background unless it’s a specific type of app. It has to be either GPS or a Music app to be allowed to run in the background and Apple check this while verifying the app for the App Store.

2.5.4 Multitasking apps may only use background services for their intended purposes: VoIP, audio playback, location, task completion, local notifications, etc. If your app uses location background mode, include a reminder that doing so may dramatically decrease battery life.

This would mean the wallet would only be able to sync when the app is open.

Sjors commented at 9:08 pm on November 18, 2017: member

@jonasschnelli thanks, I’ll try that. @3ullShark App Store rules come and go and can even be lobbied for (although this one has been around for a long time). Syncing in the foreground is a pretty bad user experience. Despite that rule, it does seem like a use case as described here.

There might be two workarounds:

tell users to build from source and install using Xcode (this is safer too, obviously not n00b friendly, but I’d use that)
leverage Cloudkit

For (2), the blockchain would be stored in Cloudkit, e.g. one block per CKAsset. The app would subscribe to these blocks, receives push notifications and downloads each block in the background (which you can make conditional on wifi, etc). Most of that is handled at the OS level, meaning the app just need to wake up to handle each push notification. Processing can be done after each download or when the app is in the foreground.

See also this [explainer)(https://medium.com/@guilhermerambo/synchronizing-data-with-cloudkit-94c6246a3fda). It might still run afoul of that App Store rule; I’m not sure how to interpret it. It also requires significantly more complex interaction with bitcoind than the RPC currently allows. And using the P2P network when in the foreground vs. using Apple’s servers when in the background might have all sorts of security trade-offs.

All that is for iOs app developers to worry about though; we can just make the compiler work.

ghost commented at 11:20 pm on November 18, 2017: none

@Sjors It’s been awhile since I’ve done any iOS development but from what I remember, the SDK is quite picky and can still suspend the app if it detects that the app is not doing something that should make it run in the background.

For example, you would have to trick the system, by having a music track play silently on a loop. Not sure if that’s still the case though.

fanquake renamed this:
~~iOs Deployment Target for RPC~~
iOS Deployment Target for RPC
on Nov 19, 2017

fanquake added the label Brainstorming on Nov 19, 2017

jonasschnelli commented at 6:13 am on November 19, 2017: contributor

I guess there are no plans to deploy through an App Store. It’s utterly insecure and with auto-update, you partially hand over the private keys to apple.

Without planing to go for the app store, you can do all sorts of things like running in the background (VOIP app permissions) and the use of “private” APIs.

AFAIK since a couple of years, self-compile and install is now possible without paying the 99$/year.

Also, there is the enterprise OTA deployment (though it would be illegal by Apples evil TOC to deploy it to the wide mass over that).

Sjors commented at 9:46 am on November 19, 2017: member

@jonasschnelli I tried your commands, but no luck:

 0checking for armv7s-apple-darwin-g++... no
 1checking for armv7s-apple-darwin-c++... no
 2checking for armv7s-apple-darwin-gpp... no
 3checking for armv7s-apple-darwin-aCC... no
 4checking for armv7s-apple-darwin-CC... no
 5checking for armv7s-apple-darwin-cxx... no
 6checking for armv7s-apple-darwin-cc++... no
 7checking for armv7s-apple-darwin-cl.exe... no
 8checking for armv7s-apple-darwin-FCC... no
 9checking for armv7s-apple-darwin-KCC... no
10checking for armv7s-apple-darwin-RCC... no
11checking for armv7s-apple-darwin-xlC_r... no
12checking for armv7s-apple-darwin-xlC... no

The /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk directory exists on my system.

config.log (slightly edited for privacy)

I would probably also want to bump to armv8 / arm64 (?) and iOs 11.

I guess there are no plans to deploy through an App Store. It’s utterly insecure and with auto-update, you partially hand over the private keys to apple.

Not by Core for those reasons. Others might, but it’s up to them to figure out the App Store rules and security vs. usability trade-offs. All other things equal (which is non-trivial) it should be more secure than any existing bitcoin app for iOs.

Something tells me Apple would find a Gitian like process useful as well once they start getting too many subpoenas to send custom app updates to specific users. The least the App Store software updater should do is check the hash of the binary against a widely published commitment for that version. That way, everyone in the world gets the same update and special code for individual users would be easier to detect.

Regarding enterprise OTA: that’s what CoG did with Pheeva in 2014. Arguably an even worse trust problem and I agree Apple might not like it.

jonasschnelli commented at 11:10 pm on November 19, 2017: contributor

@Sjors: I can “start” compiling with

0CFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=7.0" LDFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk" ./configure  --without-gui --disable-wallet --with-incompatible-bdb --host="armv7s-apple-darwin

But it misses dependencies. What you first would have to achieve is to add iOS to the depends compile system (ask @theuni for help).

maflcko added the label MacOSX on Nov 22, 2017

maflcko added the label Build system on Nov 22, 2017

laanwj commented at 11:29 am on December 7, 2017: member

Building a full fledged iOs app is out of scope here, so it would just be the RPC client.

Don’t you mean “RPC server”? The RPC client would be the native app, at least if you want to follow the same route that ABcore for android took (which seems very sensible to me).

Sjors commented at 11:30 am on December 7, 2017: member

@laanwj thanks, I updated the description to clarify that.

Sjors commented at 3:34 pm on February 24, 2018: member

@practicalswift any thoughts on this?

Sjors commented at 5:21 pm on February 24, 2018: member

Removing OpenSSL from the NO_QT=1 depends would probably make this easier (see #12530).

When I remove OpenSSL manually, I seem to be able to build the depends:

0cd depends
1make NO_QT=1
2make HOST=aarch64-apple-darwin14 NO_QT=1

The latter command doesn’t throw any errors, but I’m not sure how to go about testing it. I also don’t know if they were at all optimized for this architecture.

Sjors commented at 7:07 pm on February 24, 2018: member

I create an experimental branch which gets rid of OpenSSL. So now I can create the depends using the above and try to build:

0export CFLAGS="-arch aarch64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=11.0"
1export LDFLAGS="-arch aarch64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk"
2./configure --prefix=`pwd`/depends/aarch64-apple-darwin14 --with-miniupnpc=no --with-incompatible-bdb --without-gui --disable-bench --disable-tests --host="aarch64-apple-darwin14"

Unfortunately this fails with:

0checking whether the C++ compiler works... no
1configure: error: in `/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin':
2configure: error: C++ compiler cannot create executables

Some clues from the log:

0configure:3895: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -mmacosx-version-min=10.8 -stdlib=libc++ -pipe -O2  -I/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/aarch64-apple-darwin14/share/../include/  -L/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/aarch64-apple-darwin14/share/../lib -arch aarch64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk conftest.cpp  >&5
1clang: error: invalid arch name '-arch aarch64'
2clang: warning: using sysroot for 'iPhoneOS' but targeting 'MacOSX' [-Wincompatible-sysroot]
3configure:3899: $? = 1

jonasschnelli commented at 1:24 am on February 25, 2018: contributor

Can you elaborate why openSSL is an issue for the iOS target? AFAIK openSSL compiles perfectly fine for iOS (there are even helper-scripts https://github.com/sinofool/build-openssl-ios)?

Sjors commented at 10:56 am on February 25, 2018: member

@jonasschnelli it throws an error for me on the latest master. I’m sure it can be fixed, but I thought it might be easier to not use it.

 0make HOST=aarch64-apple-darwin14 NO_QT=1
 1Fetching openssl-1.0.1k.tar.gz from https://www.openssl.org/source
 2  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
 3                                 Dload  Upload   Total   Spent    Left  Speed
 4100 4330k  100 4330k    0     0  5716k      0 --:--:-- --:--:-- --:--:-- 5713k
 5/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/work/download/openssl-1.0.1k/openssl-1.0.1k.tar.gz.temp: OK
 6Extracting openssl...
 7/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/sources/openssl-1.0.1k.tar.gz: OK
 8Preprocessing openssl...
 9Configuring openssl...
10Configuring for 
11Usage: Configure [no-<cipher> ...] [enable-<cipher> ...] [experimental-<cipher> ...] [-Dxxx] [-lxxx] [-Lxxx] [-fxxx] [-Kxxx] [no-hw-xxx|no-hw] [[no-]threads] [[no-]shared] [[no-]zlib|zlib-dynamic] [no-asm] [no-dso] [no-krb5] [sctp] [386] [--prefix=DIR] [--openssldir=OPENSSLDIR] [--with-xxx[=vvv]] [--test-sanity] os/compiler[:flags]
12
13pick os/compiler from:
14BC-32 BS2000-OSD BSD-generic32 BSD-generic64 BSD-ia64 BSD-sparc64 BSD-sparcv8 
15BSD-x86 BSD-x86-elf BSD-x86_64 Cygwin Cygwin-pre1.3 DJGPP MPE/iX-gcc OS2-EMX 
16OS390-Unix QNX6 QNX6-i386 ReliantUNIX SINIX SINIX-N UWIN VC-CE VC-WIN32 
17VC-WIN64A VC-WIN64I aix-cc aix-gcc aix3-cc aix64-cc aix64-gcc android 
18android-armv7 android-x86 aux3-gcc beos-x86-bone beos-x86-r5 bsdi-elf-gcc cc 
19cray-j90 cray-t3e darwin-i386-cc darwin-ppc-cc darwin64-ppc-cc 
20darwin64-x86_64-cc dgux-R3-gcc dgux-R4-gcc dgux-R4-x86-gcc dist gcc hpux-cc 
21hpux-gcc hpux-ia64-cc hpux-ia64-gcc hpux-parisc-cc hpux-parisc-cc-o4 
22hpux-parisc-gcc hpux-parisc1_1-cc hpux-parisc1_1-gcc hpux-parisc2-cc 
23hpux-parisc2-gcc hpux64-ia64-cc hpux64-ia64-gcc hpux64-parisc2-cc 
24hpux64-parisc2-gcc hurd-x86 iphoneos-cross irix-cc irix-gcc irix-mips3-cc 
25irix-mips3-gcc irix64-mips4-cc irix64-mips4-gcc linux-alpha+bwx-ccc 
26linux-alpha+bwx-gcc linux-alpha-ccc linux-alpha-gcc linux-aout linux-armv4 
27linux-elf linux-generic32 linux-generic64 linux-ia32-icc linux-ia64 
28linux-ia64-ecc linux-ia64-icc linux-ppc linux-ppc64 linux-sparcv8 
29linux-sparcv9 linux-x86_64 linux32-s390x linux64-s390x linux64-sparcv9 mingw 
30mingw64 ncr-scde netware-clib netware-clib-bsdsock netware-clib-bsdsock-gcc 
31netware-clib-gcc netware-libc netware-libc-bsdsock netware-libc-bsdsock-gcc 
32netware-libc-gcc newsos4-gcc nextstep nextstep3.3 osf1-alpha-cc osf1-alpha-gcc 
33purify qnx4 rhapsody-ppc-cc sco5-cc sco5-gcc solaris-sparcv7-cc 
34solaris-sparcv7-gcc solaris-sparcv8-cc solaris-sparcv8-gcc solaris-sparcv9-cc 
35solaris-sparcv9-gcc solaris-x86-cc solaris-x86-gcc solaris64-sparcv9-cc 
36solaris64-sparcv9-gcc solaris64-x86_64-cc solaris64-x86_64-gcc sunos-gcc 
37tandem-c89 tru64-alpha-cc uClinux-dist uClinux-dist64 ultrix-cc ultrix-gcc 
38unixware-2.0 unixware-2.1 unixware-7 unixware-7-gcc vos-gcc vxworks-mips 
39vxworks-ppc405 vxworks-ppc60x vxworks-ppc750 vxworks-ppc750-debug 
40vxworks-ppc860 vxworks-ppcgen vxworks-simlinux debug debug-BSD-x86-elf 
41debug-Cygwin debug-VC-WIN32 debug-VC-WIN64A debug-VC-WIN64I debug-ben 
42debug-ben-darwin64 debug-ben-debug debug-ben-debug-64 debug-ben-macos 
43debug-ben-macos-gcc46 debug-ben-no-opt debug-ben-openbsd 
44debug-ben-openbsd-debug debug-ben-strict debug-bodo debug-darwin-i386-cc 
45debug-darwin-ppc-cc debug-geoff32 debug-geoff64 debug-levitte-linux-elf 
46debug-levitte-linux-elf-extreme debug-levitte-linux-noasm 
47debug-levitte-linux-noasm-extreme debug-linux-elf debug-linux-elf-noefence 
48debug-linux-generic32 debug-linux-generic64 debug-linux-ia32-aes 
49debug-linux-pentium debug-linux-ppro debug-linux-x86_64 debug-rse 
50debug-solaris-sparcv8-cc debug-solaris-sparcv8-gcc debug-solaris-sparcv9-cc 
51debug-solaris-sparcv9-gcc debug-steve-opt debug-steve32 debug-steve64 
52debug-ulf debug-vos-gcc 
53
54NOTE: If in doubt, on Unix-ish systems use './config'.
55make: *** [/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/work/build/aarch64-apple-darwin14/openssl/1.0.1k-03f35b62b9a/./.stamp_configured] Error 1

theuni commented at 11:45 pm on February 26, 2018: member

@Sjors You just need to define the target for openssl’s brain-dead configure script. You can see our ever-growing list here: https://github.com/bitcoin/bitcoin/blob/master/depends/packages/openssl.mk#L47

You probably just need something like:

0--- a/depends/packages/openssl.mk
1+++ b/depends/packages/openssl.mk
2@@ -52,6 +52,7 @@ $(package)_config_opts_mipsel_linux=linux-generic32
3 $(package)_config_opts_mips_linux=linux-generic32
4 $(package)_config_opts_powerpc_linux=linux-generic32
5 $(package)_config_opts_x86_64_darwin=darwin64-x86_64-cc
6+$(package)_config_opts_aarch64_darwin=iphoneos-cross
7 $(package)_config_opts_x86_64_mingw32=mingw64
8 $(package)_config_opts_i686_mingw32=mingw
9 endef

Sjors commented at 11:36 am on February 27, 2018: member

@theuni thanks, I’ll try to get that to work in #12557

dr-orlovsky commented at 7:50 pm on February 5, 2019: none

@Sjors, do you have any updates on the status of the project? I’m an experienced iOS developer (obj-C, Swift from day 1), and like to contribute to this effort.

Sjors commented at 1:58 pm on February 6, 2019: member

@dr-orlovsky I haven’t worked on this for a while. The latest version, plus more details of what’s needed, is in #12557.

Sjors commented at 7:57 pm on July 5, 2019: member

So far cross-compiling to iOs works with fairly small changes (see #12557). However, actually running something on a device is a bit more involved, as it requires linking the library from Swift in XCode. I attempted that in a proof-of-concept, but I’m stuck in Xcode hell.

dr-orlovsky commented at 5:44 pm on July 10, 2019: none

That is cool. In a few days I will look into the Xcode issue and check can I do something about it. Have some experience of linking C libs to Swift…

Sent with GitHawk

danielmcclure commented at 1:13 am on August 9, 2019: none

In regards to background app update concerns mentioned above, I’ve seen some approved apps use location changes as a trigger for background syncing. Would be interested in helping to test this if you get it going.

congtung10t2 commented at 2:06 pm on February 21, 2021: none

Anyone can help me on how to build the iOS wallet from core? Is this possible? with the tutorial from @Sjors , it seems we can build without wallet.

fanquake commented at 2:56 pm on August 15, 2022: member

It’s not clear what the state of this is, or if there is an achievable goal, but I put together a depends branch for a aarch64-apple-ios HOST, which would seem to be a more correct target: https://github.com/fanquake/bitcoin/tree/12557_followup_arm_ios.

Sjors commented at 12:43 pm on September 6, 2022: member

@fanquake when trying to build depends using your commit, cherry-picked on master, and with NO_QT=1 NO_ZMQ=1 NO_QR=1 NO_UPNP=1 NO_NATPMP=1 NO_BDB=1 NO_SQLITE=1, it failed at libevent:

I compiled on an Intel mac.

 0Fetching libevent-2.1.12-stable.tar.gz from https://github.com/libevent/libevent/releases/download/release-2.1.12-stable/
 1  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
 2                                 Dload  Upload   Total   Spent    Left  Speed
 3  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 4100 1075k  100 1075k    0     0  2220k      0 --:--:-- --:--:-- --:--:-- 2220k
 5/Users/sjors/dev/bitcoin/depends/work/download/libevent-2.1.12-stable/libevent-2.1.12-stable.tar.gz.temp: OK
 6Extracting libevent...
 7/Users/sjors/dev/bitcoin/depends/sources/libevent-2.1.12-stable.tar.gz: OK
 8Preprocessing libevent...
 9Configuring libevent...
10checking for a BSD-compatible install... /usr/local/bin/ginstall -c
11checking whether build environment is sane... yes
12checking for arm64-apple-ios-strip... no
13checking for strip... strip
14configure: WARNING: using cross tools not prefixed with host triplet
15checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p
16checking for gawk... no
17checking for mawk... no
18checking for nawk... no
19checking for awk... awk
20checking whether make sets $(MAKE)... yes
21checking whether make supports nested variables... yes
22checking whether make supports nested variables... (cached) yes
23checking whether make supports the include directive... yes (GNU style)
24checking for arm64-apple-ios-gcc... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mios-version-min=14.5 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk
25checking whether the C compiler works... no
26configure: error: in `/Users/sjors/dev/bitcoin/depends/work/build/arm64-apple-ios/libevent/2.1.12-stable-d397599fd92':
27configure: error: C compiler cannot create executables
28See `config.log' for more details
29make: *** [/Users/sjors/dev/bitcoin/depends/work/build/arm64-apple-ios/libevent/2.1.12-stable-d397599fd92/./.stamp_configured] Error 77

The config.log

The goal I had in mind is to be able to run bitcoind on an iOs device and sync to the tip. The reason I put “for RPC” is to limit the scope and not worry about QT or the wallet.

I would also be happy with an even more narrow goal of getting libbitcoinkernel somehow included in an iOs proof-of-concept app. In that case the iOs app would take care of feeding it blocks, holding on the mempool, etc.

The proof-of-concept app wouldn’t be part of Bitcoin Core. This issue can be closed once it’s clear such an app won’t need more build system changes (or we have the remaining problems in separate issues). @congtung10t2 what I wrote is probably outdated by now.

fanquake commented at 1:04 pm on September 6, 2022: member

The config.log

0configure:3787: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mios-version-min=14.5 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk    -I/Users/sjors/dev/bitcoin/depends/arm64-apple-ios/include       -L/Users/sjors/dev/bitcoin/depends/arm64-apple-ios/lib     conftest.c  >&5
1ld: warning: ignoring file /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk/usr/lib/libSystem.tbd, missing required architecture x86_64 in file /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk/usr/lib/libSystem.tbd (4 slices)
2ld: dynamic main executables must link with libSystem.dylib for architecture x86_64
3clang: error: linker command failed with exit code 1 (use -v to see invocation)

pinheadmz commented at 2:25 pm on April 27, 2023: member

Superceded by kernel

pinheadmz closed this on Apr 27, 2023

Sjors commented at 5:24 pm on May 3, 2023: member

To elaborate a bit: the idea is to have Rust (or even Swift??) bindings for the kernel. That’s a much easier platform to build on than the full daemon, especially for iOs.

jeffreyjackson commented at 6:51 pm on February 19, 2024: none

This is interesting. I have a spare 1tb iPhone I’d love to get this running on.

iOS Deployment Target for RPC #11720