iOS Deployment Target for RPC #11720

The iPhone 8 and X come with 256 GB variants. Plenty to store the current blockchain, certainly enough when you prune it. Most people who can afford such a phone also have pretty liberal data plans. This means there's at least some demographic out there that can practically run a full node on their phone.

IDB and syncing more than a day still need wifi (can be done in the background when the phone is charging). Connections to other nodes would need some throttling on mobile. Maybe a smart app could even use SPV or trust a server when it's not fully synced.

I wonder how hard it would be to create an iOs target (for bitcoind) based on the OSX target.

Building a full fledged iOs app is out of scope here, so it would just be the RPC server. A native wallet app would be an RPC client. Leaving out QT should also make that a lot easier.

Supported configuration:

./configure --without-gui --with-incompatible-bdb

• --without-gui: this removes the dependency on QT (although QT is cross-platform, that's beyond the scope of this issue)
• --with-incompatible-bdb: no need to be backwards compatible on a new OS

Pre-requisites:

• #12557 or #12530

+1 for iOS support for compiling bitcoind (not Qt as you mentioned). Can also contribute... have iOS experience.

I haven't tried. But I would expect something like this is a starting point (haven't tested):

  export CFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=7.0"
  export LDFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk"
  ./configure  --without-gui --with-incompatible-bdb --host="armv7s-apple-darwin"

The problem with iOS is the app can't run in the background unless it's a specific type of app. It has to be either GPS or a Music app to be allowed to run in the background and Apple check this while verifying the app for the App Store.

2.5.4 Multitasking apps may only use background services for their intended purposes: VoIP, audio playback, location, task completion, local notifications, etc. If your app uses location background mode, include a reminder that doing so may dramatically decrease battery life.

This would mean the wallet would only be able to sync when the app is open.

@jonasschnelli thanks, I'll try that. @3ullShark App Store rules come and go and can even be lobbied for (although this one has been around for a long time). Syncing in the foreground is a pretty bad user experience. Despite that rule, it does seem like a use case as described here.

There might be two workarounds:

1. tell users to build from source and install using Xcode (this is safer too, obviously not n00b friendly, but I'd use that)
2. leverage Cloudkit

For (2), the blockchain would be stored in Cloudkit, e.g. one block per CKAsset. The app would subscribe to these blocks, receives push notifications and downloads each block in the background (which you can make conditional on wifi, etc). Most of that is handled at the OS level, meaning the app just need to wake up to handle each push notification. Processing can be done after each download or when the app is in the foreground.

See also this [explainer)(https://medium.com/@guilhermerambo/synchronizing-data-with-cloudkit-94c6246a3fda). It might still run afoul of that App Store rule; I'm not sure how to interpret it. It also requires significantly more complex interaction with bitcoind than the RPC currently allows. And using the P2P network when in the foreground vs. using Apple's servers when in the background might have all sorts of security trade-offs.

All that is for iOs app developers to worry about though; we can just make the compiler work.

@Sjors It's been awhile since I've done any iOS development but from what I remember, the SDK is quite picky and can still suspend the app if it detects that the app is not doing something that should make it run in the background.

For example, you would have to trick the system, by having a music track play silently on a loop. Not sure if that's still the case though.

I guess there are no plans to deploy through an App Store. It's utterly insecure and with auto-update, you partially hand over the private keys to apple.

Without planing to go for the app store, you can do all sorts of things like running in the background (VOIP app permissions) and the use of "private" APIs.

AFAIK since a couple of years, self-compile and install is now possible without paying the 99$/year.

Also, there is the enterprise OTA deployment (though it would be illegal by Apples evil TOC to deploy it to the wide mass over that).

@jonasschnelli I tried your commands, but no luck:

checking for armv7s-apple-darwin-g++... no
checking for armv7s-apple-darwin-c++... no
checking for armv7s-apple-darwin-gpp... no
checking for armv7s-apple-darwin-aCC... no
checking for armv7s-apple-darwin-CC... no
checking for armv7s-apple-darwin-cxx... no
checking for armv7s-apple-darwin-cc++... no
checking for armv7s-apple-darwin-cl.exe... no
checking for armv7s-apple-darwin-FCC... no
checking for armv7s-apple-darwin-KCC... no
checking for armv7s-apple-darwin-RCC... no
checking for armv7s-apple-darwin-xlC_r... no
checking for armv7s-apple-darwin-xlC... no

The /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk directory exists on my system.

config.log (slightly edited for privacy)

I would probably also want to bump to armv8 / arm64 (?) and iOs 11.

I guess there are no plans to deploy through an App Store. It's utterly insecure and with auto-update, you partially hand over the private keys to apple.

Not by Core for those reasons. Others might, but it's up to them to figure out the App Store rules and security vs. usability trade-offs. All other things equal (which is non-trivial) it should be more secure than any existing bitcoin app for iOs.

Something tells me Apple would find a Gitian like process useful as well once they start getting too many subpoenas to send custom app updates to specific users. The least the App Store software updater should do is check the hash of the binary against a widely published commitment for that version. That way, everyone in the world gets the same update and special code for individual users would be easier to detect.

Regarding enterprise OTA: that's what CoG did with Pheeva in 2014. Arguably an even worse trust problem and I agree Apple might not like it.

@Sjors: I can "start" compiling with

CFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=7.0" LDFLAGS="-arch armv7s -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk" ./configure  --without-gui --disable-wallet --with-incompatible-bdb --host="armv7s-apple-darwin

But it misses dependencies. What you first would have to achieve is to add iOS to the depends compile system (ask @theuni for help).

Building a full fledged iOs app is out of scope here, so it would just be the RPC client.

Don't you mean "RPC server"? The RPC client would be the native app, at least if you want to follow the same route that ABcore for android took (which seems very sensible to me).

@laanwj thanks, I updated the description to clarify that.

@practicalswift any thoughts on this?

Removing OpenSSL from the NO_QT=1 depends would probably make this easier (see #12530).

When I remove OpenSSL manually, I seem to be able to build the depends:

cd depends
make NO_QT=1
make HOST=aarch64-apple-darwin14 NO_QT=1

The latter command doesn't throw any errors, but I'm not sure how to go about testing it. I also don't know if they were at all optimized for this architecture.

I create an experimental branch which gets rid of OpenSSL. So now I can create the depends using the above and try to build:

export CFLAGS="-arch aarch64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=11.0"
export LDFLAGS="-arch aarch64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk"
./configure --prefix=`pwd`/depends/aarch64-apple-darwin14 --with-miniupnpc=no --with-incompatible-bdb --without-gui --disable-bench --disable-tests --host="aarch64-apple-darwin14"

Unfortunately this fails with:

checking whether the C++ compiler works... no
configure: error: in `/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin':
configure: error: C++ compiler cannot create executables

Some clues from the log:

configure:3895: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -mmacosx-version-min=10.8 -stdlib=libc++ -pipe -O2  -I/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/aarch64-apple-darwin14/share/../include/  -L/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/aarch64-apple-darwin14/share/../lib -arch aarch64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk conftest.cpp  >&5
clang: error: invalid arch name '-arch aarch64'
clang: warning: using sysroot for 'iPhoneOS' but targeting 'MacOSX' [-Wincompatible-sysroot]
configure:3899: $? = 1

Can you elaborate why openSSL is an issue for the iOS target? AFAIK openSSL compiles perfectly fine for iOS (there are even helper-scripts https://github.com/sinofool/build-openssl-ios)?

@jonasschnelli it throws an error for me on the latest master. I'm sure it can be fixed, but I thought it might be easier to not use it.

make HOST=aarch64-apple-darwin14 NO_QT=1
Fetching openssl-1.0.1k.tar.gz from https://www.openssl.org/source
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4330k  100 4330k    0     0  5716k      0 --:--:-- --:--:-- --:--:-- 5713k
/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/work/download/openssl-1.0.1k/openssl-1.0.1k.tar.gz.temp: OK
Extracting openssl...
/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/sources/openssl-1.0.1k.tar.gz: OK
Preprocessing openssl...
Configuring openssl...
Configuring for 
Usage: Configure [no-<cipher> ...] [enable-<cipher> ...] [experimental-<cipher> ...] [-Dxxx] [-lxxx] [-Lxxx] [-fxxx] [-Kxxx] [no-hw-xxx|no-hw] [[no-]threads] [[no-]shared] [[no-]zlib|zlib-dynamic] [no-asm] [no-dso] [no-krb5] [sctp] [386] [--prefix=DIR] [--openssldir=OPENSSLDIR] [--with-xxx[=vvv]] [--test-sanity] os/compiler[:flags]

pick os/compiler from:
BC-32 BS2000-OSD BSD-generic32 BSD-generic64 BSD-ia64 BSD-sparc64 BSD-sparcv8 
BSD-x86 BSD-x86-elf BSD-x86_64 Cygwin Cygwin-pre1.3 DJGPP MPE/iX-gcc OS2-EMX 
OS390-Unix QNX6 QNX6-i386 ReliantUNIX SINIX SINIX-N UWIN VC-CE VC-WIN32 
VC-WIN64A VC-WIN64I aix-cc aix-gcc aix3-cc aix64-cc aix64-gcc android 
android-armv7 android-x86 aux3-gcc beos-x86-bone beos-x86-r5 bsdi-elf-gcc cc 
cray-j90 cray-t3e darwin-i386-cc darwin-ppc-cc darwin64-ppc-cc 
darwin64-x86_64-cc dgux-R3-gcc dgux-R4-gcc dgux-R4-x86-gcc dist gcc hpux-cc 
hpux-gcc hpux-ia64-cc hpux-ia64-gcc hpux-parisc-cc hpux-parisc-cc-o4 
hpux-parisc-gcc hpux-parisc1_1-cc hpux-parisc1_1-gcc hpux-parisc2-cc 
hpux-parisc2-gcc hpux64-ia64-cc hpux64-ia64-gcc hpux64-parisc2-cc 
hpux64-parisc2-gcc hurd-x86 iphoneos-cross irix-cc irix-gcc irix-mips3-cc 
irix-mips3-gcc irix64-mips4-cc irix64-mips4-gcc linux-alpha+bwx-ccc 
linux-alpha+bwx-gcc linux-alpha-ccc linux-alpha-gcc linux-aout linux-armv4 
linux-elf linux-generic32 linux-generic64 linux-ia32-icc linux-ia64 
linux-ia64-ecc linux-ia64-icc linux-ppc linux-ppc64 linux-sparcv8 
linux-sparcv9 linux-x86_64 linux32-s390x linux64-s390x linux64-sparcv9 mingw 
mingw64 ncr-scde netware-clib netware-clib-bsdsock netware-clib-bsdsock-gcc 
netware-clib-gcc netware-libc netware-libc-bsdsock netware-libc-bsdsock-gcc 
netware-libc-gcc newsos4-gcc nextstep nextstep3.3 osf1-alpha-cc osf1-alpha-gcc 
purify qnx4 rhapsody-ppc-cc sco5-cc sco5-gcc solaris-sparcv7-cc 
solaris-sparcv7-gcc solaris-sparcv8-cc solaris-sparcv8-gcc solaris-sparcv9-cc 
solaris-sparcv9-gcc solaris-x86-cc solaris-x86-gcc solaris64-sparcv9-cc 
solaris64-sparcv9-gcc solaris64-x86_64-cc solaris64-x86_64-gcc sunos-gcc 
tandem-c89 tru64-alpha-cc uClinux-dist uClinux-dist64 ultrix-cc ultrix-gcc 
unixware-2.0 unixware-2.1 unixware-7 unixware-7-gcc vos-gcc vxworks-mips 
vxworks-ppc405 vxworks-ppc60x vxworks-ppc750 vxworks-ppc750-debug 
vxworks-ppc860 vxworks-ppcgen vxworks-simlinux debug debug-BSD-x86-elf 
debug-Cygwin debug-VC-WIN32 debug-VC-WIN64A debug-VC-WIN64I debug-ben 
debug-ben-darwin64 debug-ben-debug debug-ben-debug-64 debug-ben-macos 
debug-ben-macos-gcc46 debug-ben-no-opt debug-ben-openbsd 
debug-ben-openbsd-debug debug-ben-strict debug-bodo debug-darwin-i386-cc 
debug-darwin-ppc-cc debug-geoff32 debug-geoff64 debug-levitte-linux-elf 
debug-levitte-linux-elf-extreme debug-levitte-linux-noasm 
debug-levitte-linux-noasm-extreme debug-linux-elf debug-linux-elf-noefence 
debug-linux-generic32 debug-linux-generic64 debug-linux-ia32-aes 
debug-linux-pentium debug-linux-ppro debug-linux-x86_64 debug-rse 
debug-solaris-sparcv8-cc debug-solaris-sparcv8-gcc debug-solaris-sparcv9-cc 
debug-solaris-sparcv9-gcc debug-steve-opt debug-steve32 debug-steve64 
debug-ulf debug-vos-gcc 

NOTE: If in doubt, on Unix-ish systems use './config'.
make: *** [/Users/bitcoin/dev/iOS-bitcoin-full-node/depends/bitcoin/depends/work/build/aarch64-apple-darwin14/openssl/1.0.1k-03f35b62b9a/./.stamp_configured] Error 1

@Sjors You just need to define the target for openssl's brain-dead configure script. You can see our ever-growing list here: https://github.com/bitcoin/bitcoin/blob/master/depends/packages/openssl.mk#L47

You probably just need something like:

--- a/depends/packages/openssl.mk
+++ b/depends/packages/openssl.mk
@@ -52,6 +52,7 @@ $(package)_config_opts_mipsel_linux=linux-generic32
 $(package)_config_opts_mips_linux=linux-generic32
 $(package)_config_opts_powerpc_linux=linux-generic32
 $(package)_config_opts_x86_64_darwin=darwin64-x86_64-cc
+$(package)_config_opts_aarch64_darwin=iphoneos-cross
 $(package)_config_opts_x86_64_mingw32=mingw64
 $(package)_config_opts_i686_mingw32=mingw
 endef

@theuni thanks, I'll try to get that to work in #12557

@Sjors, do you have any updates on the status of the project? I'm an experienced iOS developer (obj-C, Swift from day 1), and like to contribute to this effort.

@dr-orlovsky I haven't worked on this for a while. The latest version, plus more details of what's needed, is in #12557.

So far cross-compiling to iOs works with fairly small changes (see #12557). However, actually running something on a device is a bit more involved, as it requires linking the library from Swift in XCode. I attempted that in a proof-of-concept, but I'm stuck in Xcode hell.

That is cool. In a few days I will look into the Xcode issue and check can I do something about it. Have some experience of linking C libs to Swift...

_{Sent with <a href="http://githawk.com">GitHawk</a>}

In regards to background app update concerns mentioned above, I've seen some approved apps use location changes as a trigger for background syncing. Would be interested in helping to test this if you get it going.

Anyone can help me on how to build the iOS wallet from core? Is this possible? with the tutorial from @Sjors , it seems we can build without wallet.

It's not clear what the state of this is, or if there is an achievable goal, but I put together a depends branch for a aarch64-apple-ios HOST, which would seem to be a more correct target: https://github.com/fanquake/bitcoin/tree/12557_followup_arm_ios.

@fanquake when trying to build depends using your commit, cherry-picked on master, and with NO_QT=1 NO_ZMQ=1 NO_QR=1 NO_UPNP=1 NO_NATPMP=1 NO_BDB=1 NO_SQLITE=1, it failed at libevent:

I compiled on an Intel mac.

Fetching libevent-2.1.12-stable.tar.gz from https://github.com/libevent/libevent/releases/download/release-2.1.12-stable/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 1075k  100 1075k    0     0  2220k      0 --:--:-- --:--:-- --:--:-- 2220k
/Users/sjors/dev/bitcoin/depends/work/download/libevent-2.1.12-stable/libevent-2.1.12-stable.tar.gz.temp: OK
Extracting libevent...
/Users/sjors/dev/bitcoin/depends/sources/libevent-2.1.12-stable.tar.gz: OK
Preprocessing libevent...
Configuring libevent...
checking for a BSD-compatible install... /usr/local/bin/ginstall -c
checking whether build environment is sane... yes
checking for arm64-apple-ios-strip... no
checking for strip... strip
configure: WARNING: using cross tools not prefixed with host triplet
checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether make supports the include directive... yes (GNU style)
checking for arm64-apple-ios-gcc... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mios-version-min=14.5 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk
checking whether the C compiler works... no
configure: error: in `/Users/sjors/dev/bitcoin/depends/work/build/arm64-apple-ios/libevent/2.1.12-stable-d397599fd92':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** [/Users/sjors/dev/bitcoin/depends/work/build/arm64-apple-ios/libevent/2.1.12-stable-d397599fd92/./.stamp_configured] Error 77

The config.log

The goal I had in mind is to be able to run bitcoind on an iOs device and sync to the tip. The reason I put "for RPC" is to limit the scope and not worry about QT or the wallet.

I would also be happy with an even more narrow goal of getting libbitcoinkernel somehow included in an iOs proof-of-concept app. In that case the iOs app would take care of feeding it blocks, holding on the mempool, etc.

The proof-of-concept app wouldn't be part of Bitcoin Core. This issue can be closed once it's clear such an app won't need more build system changes (or we have the remaining problems in separate issues). @congtung10t2 what I wrote is probably outdated by now.

The config.log

configure:3787: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mios-version-min=14.5 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk    -I/Users/sjors/dev/bitcoin/depends/arm64-apple-ios/include       -L/Users/sjors/dev/bitcoin/depends/arm64-apple-ios/lib     conftest.c  >&5
ld: warning: ignoring file /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk/usr/lib/libSystem.tbd, missing required architecture x86_64 in file /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk/usr/lib/libSystem.tbd (4 slices)
ld: dynamic main executables must link with libSystem.dylib for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Superceded by kernel

To elaborate a bit: the idea is to have Rust (or even Swift??) bindings for the kernel. That's a much easier platform to build on than the full daemon, especially for iOs.

This is interesting. I have a spare 1tb iPhone I'd love to get this running on.