This should make it possible to build for iPhone 8, iPhone X and other iOs devices with that or new chips.
Fixes #11720
It seems I need to make some additional change…
0make HOST=aarch64-apple-darwin14 NO_QT=1
1heartbeat_test.c => ../test/heartbeat_test.c
2making links in engines...
3making links in tools...
4make[2]: Nothing to be done for `links'.
5generating dummy tests (if needed)...
6make[2]: Nothing to be done for `generate'.
7
8Since you've disabled or enabled at least one algorithm, you need to do
9the following before building:
10
11 make depend
12
13Configured for iphoneos-cross.
14Building openssl...
15making all in crypto...
16/usr/bin/perl ../util/mkbuildinf.pl "-I. -I.. -I../include -DOPENSSL_THREADS -D_REENTRANT -pipe -O2 -I/Volumes/SSD/src/bitcoin/depends/aarch64-apple-darwin14/include -O3 -isysroot /SDKs/ -fomit-frame-pointer -fno-common" "iphoneos-cross" >buildinf.h
17/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mmacosx-version-min=10.8 -I. -I.. -I../include -DOPENSSL_THREADS -D_REENTRANT -pipe -O2 -I/Volumes/SSD/src/bitcoin/depends/aarch64-apple-darwin14/include -O3 -isysroot /SDKs/ -fomit-frame-pointer -fno-common -c -o cryptlib.o cryptlib.c
18clang: warning: no such sysroot directory: '/SDKs/' [-Wmissing-sysroot]
19In file included from cryptlib.c:117:
20./cryptlib.h:62:10: fatal error: 'stdlib.h' file not found
21#include <stdlib.h>
22 ^~~~~~~~~~
231 error generated.
24make[2]: *** [cryptlib.o] Error 1
25make[1]: *** [build_crypto] Error 1
26make: *** [/Volumes/SSD/src/bitcoin/depends/work/build/aarch64-apple-darwin14/openssl/1.0.1k-df65369e03c/./.stamp_built] Error 2
Full log.
See here for enough to get cross builds from Linux working: https://github.com/theuni/bitcoin/commit/3907e526b35b5e5f752f3f521ca21df51ea9c020
I can confirm that I get valid aarch64/armv7 binaries using that. Though I have no idea if they work :) Edit: That’s building depends with NO_QT=1. I assume the Qt build is horribly broken.
Native builds are more painful, as for cross builds we can assume sdk paths, versions, compilers, etc. I’ll save those for a next step.
How about change make command to
0$(MAKE) -j1 depend build_libs libcrypto.pc libssl.pc openssl.pc
Rebased and incorporated https://github.com/theuni/bitcoin/commit/3907e526b35b5e5f752f3f521ca21df51ea9c020 (some of it has already been merged). I left out armv7 because Apple dropped support for that in iOs 11. I renamed to PR to cover all depends, rather than just OpenSSL.
make
will look in depends/SDKs
by default, so you have to set SDK_PATH
or add a symlink: ln -s /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs SDKs
Note that if you use SDK_PATH and then change it, that won’t take effect until you do a make clean HOST=aarch64-apple-darwin14
.
I’m ignoring UPNP for now, so the following command is what I’d like to see working:
0make HOST=aarch64-apple-darwin14 NO_QT=1 NO_UPNP=1
Boost 1.64 seems to build 32 bit binaries, which iOs 11 doesn’t like. I bumped it from 1.64 to 1.66, as suggested here. I can move that to a separate PR once everything works.
For OpenSSL, the following SED command leads to a “sed: RE error: illegal byte sequence” error on macOS; I probably need to adjust the syntax.
0 sed -i.old "s/-isysroot.*(CROSS_SDK)//" Configure
BDB isn’t happy, though I can avoid that by dropping wallet support (hopefully adding it in later).:
0checking for mutexes... UNIX/fcntl
1configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
2configure: error: Unable to find a mutex implementation
3make: *** [/Users/bitcoin/dev/bitcoind/depends/work/build/aarch64-apple-darwin14/bdb/4.8.30-0f0d7f59f89/build_unix/.stamp_configured] Error 1
Once OpenSSL works, the next step would be:
0./configure --prefix=`pwd`/depends/aarch64-apple-darwin14`
Testing this on a real device would require building the rest of the application, so I agree with @jonasschnelli that this PR ultimately needs to be combined with whatever solves #11720.
As for cross-compilation from Ubuntu:
The Gitian build instructions use an old version of Xcode. Nowadays it seems you need to use the App Store to download it. To extract the iOs SDK on macOS machine:
0tar -C /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/ -czf iPhoneOS.sdk.tar.gz iPhoneOS.sdk iPhoneOS11.4.sdk
Then copy it to the Ubuntu machine and extract into depends/SDKs
.
0make NO_QT=1 NO_UPNP=1 HOST=aarch64-apple-darwin14
Unfortunately this fails for me with:
0 "clang++" "-target" "aarch64-apple-darwin14" "-arch" "arm64" "-miphoneos-version-min=11.4" "--sysroot" "/home/gitianuser/bitcoin/depends/SDKs/iPhoneOS11.4.sdk" "-mlinker-version=253.9" "-stdlib=libc++" -std=c++11 -fvisibility=hidden -I/home/gitianuser/bitcoin/depends/aarch64-apple-darwin14/include -O3 -Wall -pedantic -gdwarf-2 -fexceptions -Wno-long-long -pthread -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -fpermissive -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_SYSTEM_STATIC_LINK=1 -DBOOST_THREAD_BUILD_LIB=1 -DBOOST_THREAD_DONT_USE_CHRONO -DBOOST_THREAD_POSIX -DNDEBUG -I"." -c -o "bin.v2/libs/thread/build/darwin-gnu-4.2.1/release/link-static/threadapi-pthread/threading-multi/pthread/once.o" "libs/thread/src/pthread/once.cpp"
1
2clang: error: invalid version number in '-miphoneos-version-min=11.4'
3...failed darwin.compile.c++ bin.v2/libs/thread/build/darwin-gnu-4.2.1/release/link-static/threadapi-pthread/threading-multi/pthread/once.o...
Maybe I did something wrong with copying the SDK? The directory depends/SDKs/iPhoneOS11.4.sdk
contains Entitlements.plist Library SDKSettings.plist System usr
.
Prepending LC_ALL=C
before the sed
did the tick.
Next problem, when running make
:
0 CXX libbitcoin_util_a-utiltime.o
1util.cpp:1138:18: error: 'system' is unavailable: not available on iOS
2 int nErr = ::system(strCommand.c_str());
3 ^
4/Users/bitcoin/dev/bitcoind/depends/SDKs/iPhoneOS11.4.sdk/usr/include/stdlib.h:195:6: note: 'system' has been explicitly marked unavailable here
5int system(const char *) __DARWIN_ALIAS_C(system);
That’s used for -alertnotify
, -blocknotify
and -walletnotify
, which are probably fine to disable on iOs.
Hopefully we don’t need too many other workarounds in the code to handle iOs.
I also need to figure out how to actually test this on a device. The simulator is also an issue, but can wait, since it needs x86 architecture while still using the iOs SDK.
0@@ -1,4 +1,4 @@
1-SDKs/
2+SDKs
0@@ -1,8 +1,8 @@
1 package=boost
2-$(package)_version=1_64_0
3-$(package)_download_path=https://dl.bintray.com/boostorg/release/1.64.0/source/
4+$(package)_version=1_66_0
5+$(package)_download_path=https://dl.bintray.com/boostorg/release/1.66.0/source/
6 $(package)_file_name=$(package)_$($(package)_version).tar.bz2
7-$(package)_sha256_hash=7bcc5caace97baa948931d712ea5f37038dbb1c5d89b43ad4def4ed7cb683332
8+$(package)_sha256_hash=5721818253e6a0989583192f96782c4a98eb6204965316df9f5ad75819225ca9
Is bumping the boost version required?
Edit: Ah, it is only for depends as workaround.
Disabling those notify commands seems to be the only workaround needed, although it’s hard to tell without running a real device. The latter is not that easy, because iOs doesn’t allow launching arbitrary executables, which is the approach ABcore uses.
I do get a flood of the following warnings:
0ld: warning: direct access in function 'boost::unit_test::utils::print_escaped_cdata(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, boost::unit_test::basic_cstring<char const>)'
1from file '/Users/bitcoin/dev/bitcoind/depends/aarch64-apple-darwin14/lib/libboost_unit_test_framework-mt.a(xml_log_formatter.o)' to
2global weak symbol 'boost::unit_test::basic_cstring<char const>::null_str()::null' from file 'test/test_bitcoin-addrman_tests.o' means the weak symbol cannot be overridden at runtime.
3This was likely caused by different translation units being compiled with
4different visibility settings.
998@@ -999,6 +999,7 @@ bool CWallet::AddToWallet(const CWalletTx& wtxIn, bool fFlushOnClose)
999 // Notify UI of new or updated transaction
1000 NotifyTransactionChanged(this, hash, fInsertedNew ? CT_NEW : CT_UPDATED);
1001
1002+#if !(TARGET_OS_IPHONE || TARGET_IPHONE_SIMULATOR)
configure
check for system
usability.
It’s probably easiest to work with headers and compiled .a
files: http://www.swiftprogrammer.info/swift_call_cpp.html#step1
Although that’s beyond the scope of this project, I’d like to try to get this to work so that we know if this PR actually works. A minimum PoC would just print IBD progress in a console.
Given the following files:
What do I use for the corresponding header files? And is libbitcoin_server.a
the entry point for an RPC server running on the device?
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
Reviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
99@@ -88,7 +100,9 @@ void CreatePidFile(const fs::path &path, pid_t pid);
100 #ifdef WIN32
101 fs::path GetSpecialFolderPath(int nFolder, bool fCreate = true);
102 #endif
103+#if !(TARGET_OS_IPHONE || TARGET_IPHONE_SIMULATOR)
TARGET_OS_IPHONE
is defined before using. Please fix throughout: applies to all reading of TARGET_*
in util.h
, init.cpp
, validation.cpp
, util.cpp
, wallet/init.cpp
and wallet/wallet.cpp
.
#if defined(HAVE_STD__SYSTEM) || defined(WIN32)
, which is now set in configure.ac.
32@@ -33,6 +33,18 @@
33
34 #include <boost/thread/condition_variable.hpp> // for boost::thread_interrupted
35
36+// Check for Apple OSX and iOS (Darwin)
37+#if defined(__APPLE__) && defined(__MACH__)
38+#include <TargetConditionals.h>
39+#if TARGET_IPHONE_SIMULATOR == 1
TARGET_*
throughout this PR :-)
bitcoind
and other binaries/libraries for macOS, iOS and iOSSim at the moment.
@kozyilmaz nice! I was able to run make all
for environment.ios64
. How do I run bitrootfs/ios64/bin/bitcoind
on a device or simulator? Once that works I could connect to it remotely via bitcoin-cli
, which is a good enough start for me.
Ideally I would want to be able to include this into an app project, which then has some control over the node (could be via RPC on the same device, or the Unix sockets like in #9919, or some other means?).
Ideally this PR should contain whichever changes are needed to make your script run without any patches. Perhaps it can even take over some of your Makefile; I’m not sure where the scope of Bitcoin Core ends and that of application developers start.
Your 001-bitcoin-disable-run-for-ios.patch patch should already be covered by this PR.
I’ll try the 000-bitcoin-detect-ios.patch. Build pipelines are not my strong suit: can you explain a little bit what it’s doing?
The patches for dependencies seem useful too. Why is your OpenSSL patch different than what I did in this PR?
Note that if it can’t find the right SDK version you’ll get a bunch of “fatal error: ‘unistd.h’ file not found” messages.
I haven’t made progress other than that, but I thought of an easier initial goal, rather than trying to run the full bitcoind on a phone.
We could create a new build target for a simple utility. #14671 utilities that don’t need node or wallet context seems ideal for that. That could be used for a simple iOs app which does something trivial like deriving addresses from a descriptor. Such a proof of concept app (outside this repo) might be sufficient testing to at least be able to merge this.
It could even be something useful like a minimalist wallet, with essentially the same workflow as a hardware wallet. The app generates private keys that only exist on your phone, but otherwise has no knowledge of the chain or your balance. You would then import the public key descriptor into a watch-only wallet into Bitcoin Core on your computer, and use PSBT to sign transactions on your phone.
@kozyilmaz nice! I was able to run
make all
forenvironment.ios64
. How do I runbitrootfs/ios64/bin/bitcoind
on a device or simulator? Once that works I could connect to it remotely viabitcoin-cli
, which is a good enough start for me.Ideally I would want to be able to include this into an app project, which then has some control over the node (could be via RPC on the same device, or the Unix sockets like in #9919, or some other means?).
Ideally this PR should contain whichever changes are needed to make your script run without any patches. Perhaps it can even take over some of your Makefile; I’m not sure where the scope of Bitcoin Core ends and that of application developers start.
Your 001-bitcoin-disable-run-for-ios.patch patch should already be covered by this PR.
I’ll try the 000-bitcoin-detect-ios.patch. Build pipelines are not my strong suit: can you explain a little bit what it’s doing?
The patches for dependencies seem useful too. Why is your OpenSSL patch different than what I did in this PR?
Hi @Sjors I am terribly sorry my very very late comment, just missed it. I was relocating repositories and wanted to at least answer your question:
The patch you mentioned:
https://github.com/kozyilmaz/depends/blob/master/bitcoin/patches/000-bitcoin-detect-ios.patch
it compiles a simple program at ./configure
stage and if TARGET_OS_IPHONE
or TARGET_IPHONE_SIMULATOR
is defined, it adds #define APPLE_TARGET_IOS 1
to src/config/bitcoin-config.h
. Of course this works if compilation is done on macOS.
OpenSSL patch is different because I get warnings about the SDK version being too old.
Btw I am able to use Bitcoin classes on iOS 64 and able to create private/public key pairs, generate addresses (BIP32 too), export WIF-formatted private keys as you mentioned (as an example https://github.com/kozyilmaz/depends/blob/abelian/abelian/abelian-test/address.cpp)
22@@ -23,6 +23,7 @@ Common `host-platform-triplets` for cross compilation are:
23 - `i686-w64-mingw32` for Win32
24 - `x86_64-w64-mingw32` for Win64
25 - `x86_64-apple-darwin14` for macOS
26+- `aarch64-apple-darwin14` for iOs
1@@ -2,9 +2,17 @@ OSX_MIN_VERSION=10.10
2 OSX_SDK_VERSION=10.11
3 OSX_SDK=$(SDK_PATH)/MacOSX$(OSX_SDK_VERSION).sdk
4 LD64_VERSION=253.9
5+
6+IOS_MIN_VERSION=12.1
0@@ -1,8 +1,8 @@
1 package=boost
2-$(package)_version=1_64_0
60 $(package)_config_opts_i686_mingw32=mingw
61 endef
62
63 define $(package)_preprocess_cmds
64 sed -i.old "/define DATE/d" util/mkbuildinf.pl && \
65- sed -i.old "s|engines apps test|engines|" Makefile.org
903@@ -884,6 +904,15 @@ if test x$use_reduce_exports = xyes; then
904 [AC_MSG_ERROR([Cannot set default symbol visibility. Use --disable-reduce-exports.])])
905 fi
906
907+AC_MSG_CHECKING([for std::system])
908+AC_TRY_LINK( [ #include <cstdlib> ],
When I run ./autogen.sh
:
0configure.ac:914: warning: The macro `AC_TRY_LINK' is obsolete.
1configure.ac:914: You should run autoupdate.
2../../lib/autoconf/general.m4:2688: AC_TRY_LINK is expanded from...
3configure.ac:914: the top level
4glibtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
5glibtoolize: copying file 'build-aux/ltmain.sh'
Tested that building depends for aarch64-apple-darwin14
completes (using a symlink to Xcode SDKs).
If I do ./autogen.sh
and plug --prefix=`pwd`/depends/aarch64-apple-darwin14
into ./configure
I’m seeing:
0configure:3105: checking build system type
1configure:3119: result: x86_64-apple-darwin18.2.0
2configure:3139: checking host system type
3configure:3152: result: aarch64-apple-darwin14
4configure:3195: checking for a BSD-compatible install
5configure:3263: result: /usr/local/bin/ginstall -c
6configure:3274: checking whether build environment is sane
7configure:3329: result: yes
8configure:3388: checking for aarch64-apple-darwin14-strip
9configure:3418: result: no
10configure:3428: checking for strip
11configure:3444: found /usr/bin/strip
12configure:3455: result: strip
13configure:3480: checking for a thread-safe mkdir -p
14configure:3519: result: /usr/local/bin/gmkdir -p
15configure:3526: checking for gawk
16configure:3556: result: no
17configure:3526: checking for mawk
18configure:3556: result: no
19configure:3526: checking for nawk
20configure:3556: result: no
21configure:3526: checking for awk
22configure:3542: found /usr/bin/awk
23configure:3553: result: awk
24configure:3564: checking whether make sets $(MAKE)
25configure:3586: result: yes
26configure:3615: checking whether make supports nested variables
27configure:3632: result: yes
28configure:3751: checking whether to enable maintainer-specific portions of Makefiles
29configure:3760: result: yes
30configure:3785: checking whether make supports nested variables
31configure:3802: result: yes
32configure:3931: checking for C++ compiler version
33configure:3940: clang++ -target aarch64-apple-darwin14 -arch arm64 -miphoneos-version-min=12.1 --sysroot /Users/michael/github/bitcoin/depends/SDKs/iPhoneOS12.1.sdk -mlinker-version=253.9 -stdlib=libc++ --version >&5
34Apple LLVM version 10.0.0 (clang-1000.11.45.5)
35Target: aarch64-apple-darwin14
36Thread model: posix
37InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
38configure:3951: $? = 0
39configure:3940: clang++ -target aarch64-apple-darwin14 -arch arm64 -miphoneos-version-min=12.1 --sysroot /Users/michael/github/bitcoin/depends/SDKs/iPhoneOS12.1.sdk -mlinker-version=253.9 -stdlib=libc++ -v >&5
40Apple LLVM version 10.0.0 (clang-1000.11.45.5)
41Target: aarch64-apple-darwin14
42Thread model: posix
43InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
44clang: warning: argument unused during compilation: '-miphoneos-version-min=12.1' [-Wunused-command-line-argument]
45clang: warning: argument unused during compilation: '-stdlib=libc++' [-Wunused-command-line-argument]
46configure:3951: $? = 0
47configure:3940: clang++ -target aarch64-apple-darwin14 -arch arm64 -miphoneos-version-min=12.1 --sysroot /Users/michael/github/bitcoin/depends/SDKs/iPhoneOS12.1.sdk -mlinker-version=253.9 -stdlib=libc++ -V >&5
48clang: error: argument to '-V' is missing (expected 1 value)
49clang: error: no input files
50configure:3951: $? = 1
51configure:3940: clang++ -target aarch64-apple-darwin14 -arch arm64 -miphoneos-version-min=12.1 --sysroot /Users/michael/github/bitcoin/depends/SDKs/iPhoneOS12.1.sdk -mlinker-version=253.9 -stdlib=libc++ -qversion >&5
52clang: error: unknown argument: '-qversion'
53clang: error: no input files
54configure:3951: $? = 1
55configure:3971: checking whether the C++ compiler works
56configure:3993: clang++ -target aarch64-apple-darwin14 -arch arm64 -miphoneos-version-min=12.1 --sysroot /Users/michael/github/bitcoin/depends/SDKs/iPhoneOS12.1.sdk -mlinker-version=253.9 -stdlib=libc++ -pipe -O2 -I/Users/michael/github/bitcoin/depends/aarch64-apple-darwin14/share/../include/ -L/Users/michael/github/bitcoin/depends/aarch64-apple-darwin14/share/../lib conftest.cpp >&5
57ld: library not found for -lc++
58clang: error: linker command failed with exit code 1 (use -v to see invocation)
59configure:3997: $? = 1
60configure:4035: result: no
Rebased and dropped af156c279386fdf6d649c9cff9c61b58d7588eef since we no longer need OpenSSL (#17265). Also bumped the iOs version and aarch64-apple-darwin19.
Unfortunately now I run into a new issue, possibly related to #17118 (though reverting doesn’t help):
0clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]
This config is building and “installing”.
https://travis-ci.com/RandyMcMillan/bitcoin/jobs/258943857
https://github.com/RandyMcMillan/bitcoin/commit/982f4ea45996d88662d3f22b8e4e65a3cbe90d46
@RandyMcMillan I don’t think that new Travis instance is actually building for iOs: https://travis-ci.com/RandyMcMillan/bitcoin/jobs/258943857#L242 . macOS travis machines are pretty scarce at the moment, so we probably can’t add that anyway. Though perhaps we could expand the current macOS machine to cross-compile iOs.
What did the trick for me was using your IOS_SDK=$(xcrun --sdk iphoneos --show-sdk-path)
(which let me drop the symlink) and using shell xcrun -f clang
instead of clang
directly. That does mean you can only build iOs using a macOS host, but that’s fine for now imo; not much you can do in the iOs ecosystem without a Mac anyway.
Co-authored-by: kozyilmaz <kazim@monolytic.com>
Rebased, but it’s not compiling for me anymore:
0Undefined symbols for architecture arm64:
1 "_getauxval", referenced from:
2 crc32c::Extend(unsigned int, unsigned char const*, unsigned long) in libcrc32c.a(libcrc32c_a-crc32c.o)
3ld: symbol(s) not found for architecture arm64
🐙 This pull request conflicts with the target branch and needs rebase.
arm64-apple-darwin
(which is the same target as being added here). I think discussion can continue in #11720 to determine what actually needs to be done.