Add depends 32-bit arm support for bitcoin-qt

TheCharlatan commented at 2:38 pm on July 6, 2018: member

Some hobbyists are used to using the desktop for interfacing with their raspberry pi. This commits adds qt to the arm-linux-gnueabihf target.

TheCharlatan renamed this:
~~Add depends 32-bit arm support for bitcoin-qt~~
[WIP] Add depends 32-bit arm support for bitcoin-qt
on Jul 6, 2018

in depends/packages/packages.mk:8 in 8ef00e8d94 outdated

4@@ -5,6 +5,7 @@ qt_packages = qrencode protobuf zlib
5 
6 qt_x86_64_linux_packages:=qt expat dbus libxcb xcb_proto libXau xproto freetype fontconfig libX11 xextproto libXext xtrans
7 qt_i686_linux_packages:=$(qt_x86_64_linux_packages)
8+qt_arm_linux_packages:=$(qt_x86_64_linux_packages) $(qt_packages)

Sjors commented at 3:20 pm on July 6, 2018:

What’s the role of $(qt_packages), especially given that qt_i686_linux_packages doesn’t have it?

TheCharlatan commented at 2:01 pm on July 8, 2018:

Good findings. I could remove all these changes (I had them in while I was debugging things) and it still seems to run fine. Can you re-run a gitian with b86d49f ?

in depends/packages/qt.mk:67 in 8ef00e8d94 outdated

63@@ -64,7 +64,7 @@ $(package)_config_opts += -prefix $(host_prefix)
64 $(package)_config_opts += -qt-libpng
65 $(package)_config_opts += -qt-libjpeg
66 $(package)_config_opts += -qt-pcre
67-$(package)_config_opts += -system-zlib
68+$(package)_config_opts += -qt-zlib

Sjors commented at 3:20 pm on July 6, 2018:

For future reference: what’s wrong with system-zlib?

in depends/packages/qt.mk:160 in 8ef00e8d94 outdated

156@@ -155,6 +157,9 @@ define $(package)_config_cmds
157   cd ../lupdate/ && ../../../../qtbase/bin/qmake lupdate.pro -o Makefile && cd ../../../..
158 endef
159 
160+#echo "host_build: QT_CONFIG ~= s/system-zlib/zlib" >> mkspecs/qconfig.pri && \

Sjors commented at 3:20 pm on July 6, 2018:

Should this go away?

Sjors commented at 3:20 pm on July 6, 2018: member

Concept ACK. Not just Raspberry Pi, also various clones via Armbian.

Fixes the 32-bit part of #13495.

TheCharlatan commented at 3:32 pm on July 6, 2018: member

The Travis job times out with this change. Not sure how to get it to run through.

Sjors commented at 5:55 pm on July 6, 2018: member

I restarted the Travis job, but that doesn’t seem to help much.

fanquake added the label Build system on Jul 7, 2018

Sjors commented at 12:37 pm on July 7, 2018: member

I’m able to build a binary using Armbian: https://github.com/Sjors/armbian-bitcoin-core/pull/1

However, when I launch QT on an Orange Pi I get:

0bitcoin-qt: /lib/arm-linux-gnueabihf/libm.so.6: version `GLIB_2.27` not found (required by bitcoin-qt)

This might be because I compiled on Bionic and the target machine uses Xenial*, with libc6 version 2.23

I’ll try again with an Armbian on a Ubuntu 16.04 VM.

= because desktop support still doesn’t work for that device with Bionic

TheCharlatan commented at 12:58 pm on July 7, 2018: member

Alternatively you could also apply #13177 and test if it also works for arm.

Sjors commented at 1:29 pm on July 7, 2018: member

Cherry-picking that branch did the trick, nice!

Sjors commented at 3:06 pm on July 7, 2018: member

I’m also trying a gitian build, but that gets stuck at something that seems pretty unrelated (not even the same architecture):

 0  CXX      crypto/crypto_libbitcoin_crypto_sse41_a-sha256_sse41.o
 1  CXX      crypto/crypto_libbitcoin_crypto_avx2_a-sha256_avx2.o
 2crypto/sha256.cpp: In function ‘std::string SHA256AutoDetect()’:
 3crypto/sha256.cpp:537:83: error: inconsistent operand constraints in an ‘asm’
 4   __asm__ ("cpuid" : "=a"(a), "=b"(b), "=c"(c), "=d"(d) : "0"(leaf), "2"(subleaf));
 5                                                                                   ^
 6crypto/sha256.cpp:537:83: error: inconsistent operand constraints in an ‘asm’
 7   __asm__ ("cpuid" : "=a"(a), "=b"(b), "=c"(c), "=d"(d) : "0"(leaf), "2"(subleaf));
 8                                                                                   ^
 9make[2]: *** [crypto/crypto_libbitcoin_crypto_base_a-sha256.o] Error 1
10make[2]: *** Waiting for unfinished jobs....
11make[2]: Leaving directory `/home/ubuntu/build/bitcoin/distsrc-i686-pc-linux-gnu/src'

ken2812221 commented at 3:13 pm on July 7, 2018: contributor

@Sjors See #13538

Sjors commented at 3:41 pm on July 7, 2018: member

@ken2812221 thanks. I just deleted those architectures from the gitian descriptor and tried again.

09c0a172871a27bb727ed27fcdf2f5fd3c6e6b5bc82ea6d7b5745916887a450cb  bitcoin-0.16.99-arm-linux-gnueabihf.tar.gz

Binary works on device as well.

tACK 8ef00e8, assuming Travis gives its blessing

TheCharlatan force-pushed on Jul 8, 2018

TheCharlatan renamed this:
~~[WIP] Add depends 32-bit arm support for bitcoin-qt~~
Add depends 32-bit arm support for bitcoin-qt
on Jul 9, 2018

laanwj commented at 3:49 pm on July 9, 2018: member

my first question when seeing this is: why only 32-bit?—is this already supported for 64-bit, or does 64-bit ARM provide additional challenges?

TheCharlatan commented at 4:37 pm on July 9, 2018: member

@laanwj see my comment in #13495 on the difficulties I am having with 64bit arm.

TheCharlatan force-pushed on Jul 9, 2018

laanwj commented at 1:51 pm on July 10, 2018: member

@laanwj see my comment in #13495 on the difficulties I am having with 64bit arm.

Fair enough. It’s fine, we can do that in a later PR.

Looks like the travis issue is simply a timeout running into the maximum time limit for jobs.

Sjors commented at 2:05 pm on July 10, 2018: member

A particularly stubborn Travis timeout. Maybe the ccache needs to be bigger?

Or perhaps cache /home/travis/build/bitcoin/bitcoin/depends/work/build/arm-linux-gnueabihf as an exception to the general rule of not caching build stuff.

TheCharlatan commented at 4:35 pm on July 10, 2018: member

I’m not sure if your suggestion @Sjors may have unwanted side effects to the Travis jobs. It seems like the linux qt builds are disabled for this reason in the Travis build matrix. Otherwise I’ll remove the Travis change from the pr and hope that enough people run it through gitian to verify?

TheCharlatan force-pushed on Jul 10, 2018

TheCharlatan commented at 10:39 pm on July 10, 2018: member

Removed the Travis patch. To test this pull request, build it locally.

martinpgaunt approved

Sjors commented at 7:44 am on July 11, 2018: member

I restarted the Apple Darwin instance

laanwj requested review from theuni on Jul 11, 2018

in depends/packages/qt.mk:92 in 45c367345e outdated

88@@ -89,7 +89,7 @@ $(package)_config_opts_linux += -system-freetype
89 $(package)_config_opts_linux += -no-feature-sessionmanager
90 $(package)_config_opts_linux += -fontconfig
91 $(package)_config_opts_linux += -no-opengl
92-$(package)_config_opts_arm_linux  = -platform linux-g++ -xplatform $(host)
93+$(package)_config_opts_arm_linux  = -platform linux-g++ -xplatform linux-arm-gnueabihf-g++

theuni commented at 7:54 pm on July 11, 2018:

Let’s make this $(host)-g++. Then below, we can copy to qtbase/mkspecs/$(host)-g++, and use sed to replace arm-linux-gnueabi- with $(host)-.

That way subarches (armv6, for example) just work as intended.

TheCharlatan commented at 8:52 pm on July 11, 2018:

The sed patch is only intended for adding hard float support. I don’t think this should be done for aarch64 as well (which seems to be the only other mkspec that matches the $(host) triplet).

theuni commented at 8:58 pm on July 11, 2018:

Yes, but then we can reuse it for aarch64. All that would be necessary to add it (assuming everything else is working) would be:

0$(package)_config_opts_aarch64_linux = -platform linux-g++ -xplatform $(host)-g++

TheCharlatan commented at 9:07 pm on July 11, 2018:

I get Invalid target platform 'arm-linux-gnueabihf-g++' when I replace it with $(host)-g++.

theuni commented at 9:47 pm on July 11, 2018:

This works for me:

 0diff --git a/depends/packages/qt.mk b/depends/packages/qt.mk
 1index b4d7756..c09c02d 100644
 2--- a/depends/packages/qt.mk
 3+++ b/depends/packages/qt.mk
 4@@ -89,7 +89,7 @@ $(package)_config_opts_linux += -system-freetype
 5 $(package)_config_opts_linux += -no-feature-sessionmanager
 6 $(package)_config_opts_linux += -fontconfig
 7 $(package)_config_opts_linux += -no-opengl
 8-$(package)_config_opts_arm_linux  = -platform linux-g++ -xplatform $(host)
 9+$(package)_config_opts_arm_linux += -platform linux-g++ -xplatform $(host)-g++
10 $(package)_config_opts_i686_linux  = -xplatform linux-g++-32
11 $(package)_config_opts_x86_64_linux = -xplatform linux-g++-64
12 $(package)_config_opts_mingw32  = -no-opengl -xplatform win32-g++ -device-option CROSS_COMPILE="$(host)-"
13@@ -128,6 +128,8 @@ define $(package)_preprocess_cmds
14   cp -f qtbase/mkspecs/macx-clang/Info.plist.app qtbase/mkspecs/macx-clang-linux/ &&\
15   cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\
16   cp -f $($(package)_patch_dir)/mac-qmake.conf qtbase/mkspecs/macx-clang-linux/qmake.conf && \
17+  cp -r qtbase/mkspecs/linux-arm-gnueabi-g++ qtbase/mkspecs/$(host)-g++ && \
18+  sed -i s/arm-linux-gnueabi-/$(host)-/g qtbase/mkspecs/$(host)-g++/qmake.conf && \
19   patch -p1 -i $($(package)_patch_dir)/fix_qt_pkgconfig.patch &&\
20   patch -p1 -i $($(package)_patch_dir)/fix_configure_mac.patch &&\
21   patch -p1 -i $($(package)_patch_dir)/fix_no_printer.patch &&\

TheCharlatan commented at 9:56 pm on July 11, 2018:

Just noticed that the mkspecs is not arm-linux-gnu (like $(host)), but rather linux-arm-gnu. :p

TheCharlatan commented at 9:58 pm on July 11, 2018:

The second chunk could result in undesired behaviour, because you end up copying the arm mkspecs over whatever the $(host)’s mkspecs are supposed to be.

theuni commented at 10:19 pm on July 11, 2018:

Good point. Using bitcoin-$(host)-g++ would guarantee that can’t happen.

TheCharlatan commented at 10:30 pm on July 11, 2018:

I do not understand how that would work. Can you post another patch for it?

theuni commented at 10:43 pm on July 11, 2018:

I guess the spec name doesn’t need to change, all that matters is the sed. Simplified:

 0diff --git a/depends/packages/qt.mk b/depends/packages/qt.mk
 1index b4d7756..22cf4f5 100644
 2--- a/depends/packages/qt.mk
 3+++ b/depends/packages/qt.mk
 4@@ -89,7 +89,7 @@ $(package)_config_opts_linux += -system-freetype
 5 $(package)_config_opts_linux += -no-feature-sessionmanager
 6 $(package)_config_opts_linux += -fontconfig
 7 $(package)_config_opts_linux += -no-opengl
 8-$(package)_config_opts_arm_linux  = -platform linux-g++ -xplatform $(host)
 9+$(package)_config_opts_arm_linux += -platform linux-g++ -xplatform bitcoin-linux-g++
10 $(package)_config_opts_i686_linux  = -xplatform linux-g++-32
11 $(package)_config_opts_x86_64_linux = -xplatform linux-g++-64
12 $(package)_config_opts_mingw32  = -no-opengl -xplatform win32-g++ -device-option CROSS_COMPILE="$(host)-"
13@@ -128,6 +128,8 @@ define $(package)_preprocess_cmds
14   cp -f qtbase/mkspecs/macx-clang/Info.plist.app qtbase/mkspecs/macx-clang-linux/ &&\
15   cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\
16   cp -f $($(package)_patch_dir)/mac-qmake.conf qtbase/mkspecs/macx-clang-linux/qmake.conf && \
17+  cp -r qtbase/mkspecs/linux-arm-gnueabi-g++ qtbase/mkspecs/bitcoin-linux-g++ && \
18+  sed -i s/arm-linux-gnueabi-/$(host)-/g qtbase/mkspecs/bitcoin-linux-g++/qmake.conf && \
19   patch -p1 -i $($(package)_patch_dir)/fix_qt_pkgconfig.patch &&\
20   patch -p1 -i $($(package)_patch_dir)/fix_configure_mac.patch &&\
21   patch -p1 -i $($(package)_patch_dir)/fix_no_printer.patch &&\

The effect is that we essentially have a universal linux mkspec now, it just changes the compiler names to match the HOST passed in.

TheCharlatan commented at 11:07 pm on July 11, 2018:

ACK.

in depends/packages/packages.mk:8 in 45c367345e outdated

4@@ -5,6 +5,7 @@ qt_packages = qrencode protobuf zlib
5 
6 qt_x86_64_linux_packages:=qt expat dbus libxcb xcb_proto libXau xproto freetype fontconfig libX11 xextproto libXext xtrans
7 qt_i686_linux_packages:=$(qt_x86_64_linux_packages)
8+qt_arm_linux_packages:=$(qt_x86_64_linux_packages)

theuni commented at 8:03 pm on July 11, 2018:

Since there’s no point in making the distinction here, let’s just get rid of it.

qt_x86_64_linux_packages can just become qt_linux_packages, then qt_i686_linux_packages and qt_arm_linux_package are no longer needed.

TheCharlatan commented at 8:18 pm on July 11, 2018:

The patch does not work yet for aarch64, so I think we should keep it like this until that it is fixed.

theuni commented at 8:55 pm on July 11, 2018:

Making the suggested change above will make it so that no change is needed here for aarch64.

TheCharlatan commented at 9:16 pm on July 11, 2018:

Substituting qt_x86_64_linux_packages with qt_linux_packages means that qt will be built for all Linux architectures. When I do that and remove the two lines below, a aarch64-linux-gnu depends build will also try to build qt, but will fail as I mentioned in #13495 .

theuni commented at 9:42 pm on July 11, 2018:

Yes, ok, I forgot we purposefully limited that way.

theuni changes_requested

theuni commented at 8:05 pm on July 11, 2018: member

Concept ACK.

Add depends 32-bit arm support for bitcoin-qt

Some hobbyists are used to using the desktop for interfacing with their
raspberry pi. This commits adds qt to the arm-linux-gnueabihf target.

4b69984557

TheCharlatan force-pushed on Jul 11, 2018

TheCharlatan commented at 11:08 pm on July 11, 2018: member

Force-pushed to respond to @theuni’s review.

Fuzzbawls commented at 8:08 am on July 12, 2018: contributor

Concept ACK

aarch64 support can be added later, after the issue with xextproto are dealt with and migration to Bionic for gitian builds is complete.

Sjors commented at 10:59 am on July 12, 2018: member

re-tACK 4b69984 (I retested cross-compile and running on a device, not Gitian)

Sjors commented at 9:28 am on July 13, 2018: member

For some reason make in /depends won’t build Qt if you run it on an ARM device (armv7l-unknown-linux-gnueabihf).

Adding qt_armv7l_linux_packages:=$(qt_x86_64_linux_packages) fixes that and it builds a bunch of things, but then I get:

0Project ERROR: Unknown module(s) in QT: xml
1funcs.mk:242: recipe for target '/home/bitcoin/bitcoin/depends/work/build/armv7l-unknown-linux-gnueabihf/qt/5.9.6-8b61ad4d92c/qtbase/.stamp_configured' failed

If it’s complicated, I’m fine with trying this fix this some other time, I’d rather get this in 0.17

TheCharlatan commented at 9:59 pm on July 13, 2018: member

I get the same xml error when I try to apply @Fuzzbawls patch for aarch64 support.

sipa commented at 8:30 pm on July 14, 2018: member

@theuni Does your “changes requested” still apply?

theuni approved

theuni commented at 10:33 pm on July 16, 2018: member

utACK 4b6998455776ed3e1b7c996c14f15f0c304c9269

Sjors commented at 7:17 am on July 17, 2018: member

Maybe add “for cross-compilation” to the commit message?

ken2812221 commented at 7:36 am on July 17, 2018: contributor

utACK 4b69984

laanwj merged this on Jul 17, 2018

laanwj closed this on Jul 17, 2018

laanwj referenced this in commit 75bea0507f on Jul 17, 2018

TheCharlatan deleted the branch on Jul 17, 2018

TheCharlatan restored the branch on Jul 17, 2018

Sjors commented at 8:03 am on July 18, 2018: member

Gitian builds for master are now broken: #13700

in depends/packages/qt.mk:132 in 4b69984557

127@@ -128,6 +128,8 @@ define $(package)_preprocess_cmds
128   cp -f qtbase/mkspecs/macx-clang/Info.plist.app qtbase/mkspecs/macx-clang-linux/ &&\
129   cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\
130   cp -f $($(package)_patch_dir)/mac-qmake.conf qtbase/mkspecs/macx-clang-linux/qmake.conf && \
131+  cp -r qtbase/mkspecs/linux-arm-gnueabi-g++ qtbase/mkspecs/bitcoin-linux-g++ && \
132+  sed -i s/arm-linux-gnueabi-/$(host)-/g qtbase/mkspecs/bitcoin-linux-g++/qmake.conf && \

Sjors commented at 10:48 am on July 24, 2018:

macOS doesn’t like this, see #13750

sipa added the label Needs release notes on Aug 14, 2018

fanquake removed the label Needs release note on Mar 20, 2019

PastaPastaPasta referenced this in commit a960760931 on Jul 19, 2020

PastaPastaPasta referenced this in commit 020c08f688 on Jul 24, 2020

PastaPastaPasta referenced this in commit d8fa5f1bbc on Jul 27, 2020

UdjinM6 referenced this in commit 4b47a14fea on Jul 27, 2020

UdjinM6 referenced this in commit 2f76c69210 on Jul 27, 2020

hebasto commented at 8:25 am on August 17, 2020: member

Please see #18536.

CryptoCentric referenced this in commit 1ff81bf154 on Jul 2, 2021

DrahtBot locked this on Feb 15, 2022

Add depends 32-bit arm support for bitcoin-qt #13604