travis: Run unit and functional tests on native arm

MarcoFalke commented at 7:08 pm on October 23, 2019: member

This keeps the cross-compilation to make it easy to run the ci on non-arm hardware. To run this locally in qemu-user as it used to be, just export QEMU_USER_CMD="qemu-arm -L /usr/arm-linux-gnueabihf/".

MarcoFalke added the label Tests on Oct 23, 2019

practicalswift commented at 9:14 pm on October 23, 2019: contributor

Concept ACK

Diversity in testing is good

MarcoFalke commented at 9:15 pm on October 23, 2019: member

Even though it shouldn’t fail, this fails with

0/bin/bash: /home/travis/build/bitcoin/bitcoin/depends/arm-linux-gnueabihf/share/../native/bin/uic: cannot execute binary file: Exec format error

I might try to debug this locally some day…

laanwj commented at 11:31 am on October 24, 2019: member

Concept ACK

I think the error has to do with using the output of cross-compilation for normal compilation; the “native” uic will be x86-64, the expected build platform.

MarcoFalke commented at 12:22 pm on October 24, 2019: member

Oh, so we can’t cross compile on arm64 to other arches?

laanwj commented at 5:15 pm on October 24, 2019: member

Oh, so we can’t cross compile on arm64 to other arches?

There is no conceptual reason why not, but maybe not with how depends are set up right now. Not sure.

But I think I misunderstand what is happening. I thought the cross compile was from x86 to armhf. Is it cross-compiling from arm64 to armhf?

(I didn’t think so because I assumed you don’t get the Exec format error between different ARM bitnesses)

What is the “native” platform here, and what is the target platform?

MarcoFalke commented at 5:37 pm on October 24, 2019: member

travis offers arm64 (== aarch64 ) machines, so the cross compilation is on a arm64 host to an arm32 guest

laanwj commented at 10:10 pm on October 24, 2019: member

So we’re sure it’s not using cached arm-linux-gnueabihf depends that have been built on x86-64? I can’t think of any explanation why the native tools like uic (which are by definition not cross-compiled, but built with the native C++ compiler, so that they can be part of the build processs) would be the wrong architecture.

MarcoFalke commented at 10:28 pm on October 24, 2019: member

Good point. The depends cache is not invalidated when the compiler changes

ci: Remove ccache requirement on the host

ccache is only needed to create the cache dir on the host, if it didn't
already exist. The same can be achieved with mkdir, so just use that
instead.

fafa064d2a

MarcoFalke force-pushed on Oct 24, 2019

laanwj commented at 9:52 am on October 25, 2019: member

Looks like it finishes the build now, but the testing stage cannot find the binaries… strange

laanwj commented at 10:29 am on October 27, 2019: member

I think I get the reason for this: you are executing ARM32 executables on ARM64 without dynamic linker and glibc installed. Missing the ELF interpreter will give the same “not found” error as missing the executable itself.

I think this is package libc6:armhf.

0odroid@odroid64:~$ dpkg -S /lib/ld-linux-armhf.so.3
1libc6:armhf: /lib/ld-linux-armhf.so.3

MarcoFalke force-pushed on Oct 28, 2019

MarcoFalke commented at 4:50 pm on October 28, 2019: member

Added libc6-armhf-armel-cross to the install packages

laanwj commented at 7:53 pm on October 28, 2019: member

I think that’s the old package naming. It should work though.

bionic (18.04LTS) (devel): Dummy package to get libc6:armhf installed [universe] 2.27-3ubuntu1cross1: all

MarcoFalke commented at 8:19 pm on October 28, 2019: member

It fails locally and on travis:

 0$ sudo apt install libc6-dev-armhf-armel-cross && make check -j 2
 1Reading package lists... Done
 2Building dependency tree
 3Reading state information... Done
 4libc6-dev-armhf-armel-cross is already the newest version (2.27-3ubuntu1cross1.1).
 50 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
 6Making check in src
 7make[1]: Entering directory '/home/ubuntu/bitcoin-core/src'
 8make[2]: Entering directory '/home/ubuntu/bitcoin-core/src'
 9make[3]: Entering directory '/home/ubuntu/bitcoin-core'
10make[3]: Leaving directory '/home/ubuntu/bitcoin-core'
11make  check-TESTS check-local
12make[3]: Entering directory '/home/ubuntu/bitcoin-core/src'
13Running tests: arith_uint256_tests from test/arith_uint256_tests.cpp
14/bin/bash: test/test_bitcoin: No such file or directory

laanwj commented at 8:32 pm on October 28, 2019: member

What is the output of readelf -a test/test_bitcoin | grep interpreter? does that file exist?

MarcoFalke commented at 8:46 pm on October 28, 2019: member

0$ readelf -a src/test/test_bitcoin | grep interpreter
1      [Requesting program interpreter: /lib/ld-linux-armhf.so.3]
20x59004c <_ZNK5boost7runtime17value_interpreterIbLb0EE9interpretENS_9unit_test13basic_cstringIKcEES6_>: [@0x9c7a78](/bitcoin-bitcoin/contributor/0x9c7a78/)
30x5957bc <_ZNK5boost7runtime17value_interpreterIjLb0EE9interpretENS_9unit_test13basic_cstringIKcEES6_>: [@0x9c807c](/bitcoin-bitcoin/contributor/0x9c807c/)
40x595dc0 <_ZNK5boost7runtime17value_interpreterImLb0EE9interpretENS_9unit_test13basic_cstringIKcEES6_>: [@0x9c8188](/bitcoin-bitcoin/contributor/0x9c8188/)
5 29187: 00000000     0 FILE    LOCAL  DEFAULT  ABS interpreter.cpp
6
7
8$ ll /lib/ld-linux-armhf.so.3
9ls: cannot access '/lib/ld-linux-armhf.so.3': No such file or directory

0$ ll /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 
1lrwxrwxrwx 1 root root 10 Sep 17  2018 /usr/arm-linux-gnueabihf/lib/ld-linux-armhf.so.3 -> ld-2.27.so*

laanwj commented at 11:18 am on October 30, 2019: member

It really needs to be in /lib for multi-arch to work.

Can you try:

0dpkg --add-architecture armhf
1apt-get update
2apt-get install libc6:armhf

MarcoFalke force-pushed on Oct 30, 2019

MarcoFalke commented at 6:27 pm on October 30, 2019: member

I feel dumb for not trying that, but then I remember that it didn’t work locally:

 0# dpkg --add-architecture armhf
 1root@729a87425702:/# apt update
 2Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
 3Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
 4Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
 5Ign:4 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages    
 6Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease              
 7Ign:6 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
 8Ign:7 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages                                                            
 9Ign:8 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages                                                  
10Ign:9 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages                                                                  
11Ign:10 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages                                                   
12Ign:11 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages                                   
13Ign:4 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages                              
14Ign:12 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages    
15Ign:6 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
16Ign:7 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
17Ign:8 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
18Ign:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
19Ign:10 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
20Ign:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
21Ign:4 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
22Ign:15 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
23Ign:6 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
24Ign:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
25Ign:8 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
26Ign:9 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
27Ign:10 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
28Ign:17 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
29Err:4 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
30  404  Not Found [IP: 91.189.88.24 80]
31Ign:18 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
32Ign:6 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
33Ign:11 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages      
34Ign:8 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
35Ign:12 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages    
36Ign:10 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
37Ign:7 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages     
38Ign:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
39Ign:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
40Ign:15 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
41Ign:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
42Ign:9 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
43Ign:17 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
44Ign:18 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
45Ign:11 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
46Ign:12 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
47Err:7 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
48  404  Not Found [IP: 91.189.88.149 80]
49Ign:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
50Ign:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
51Ign:15 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
52Ign:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
53Ign:9 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
54Ign:17 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
55Ign:18 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
56Ign:11 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
57Ign:12 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
58Err:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
59  404  Not Found [IP: 91.189.88.149 80]
60Ign:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
61Ign:15 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
62Ign:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
63Err:17 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
64  404  Not Found [IP: 91.189.88.149 80]
65Ign:18 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
66Reading package lists... Done
67E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/main/binary-armhf/Packages  404  Not Found [IP: 91.189.88.24 80]
68E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/multiverse/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
69E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/universe/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
70E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/main/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
71E: Some index files failed to download. They have been ignored, or old ones used instead.
72root@729a87425702:/# apt-get install libc6:armhf
73Reading package lists... Done
74Building dependency tree       
75Reading state information... Done
76E: Unable to locate package libc6:armhf

laanwj commented at 6:29 pm on October 30, 2019: member

I’m confused;

 0root@odroid64:~# cat /etc/lsb-release 
 1DISTRIB_ID=Ubuntu
 2DISTRIB_RELEASE=18.04
 3DISTRIB_CODENAME=bionic
 4DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
 5
 6root@odroid64:~# apt update
 7Hit:1 http://ports.ubuntu.com/ubuntu-ports bionic InRelease
 8Hit:2 http://deb.odroid.in/c2 bionic InRelease                          
 9Get:3 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease [88.7 kB]
10Get:4 http://ports.ubuntu.com/ubuntu-ports bionic-backports InRelease [74.6 kB]
11Get:5 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease [88.7 kB]
12Get:6 http://ports.ubuntu.com/ubuntu-ports bionic-proposed InRelease [242 kB]
13Get:7 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main Sources [295 kB]
14Get:8 http://ports.ubuntu.com/ubuntu-ports bionic-updates/universe Sources [265 kB]
15Get:9 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 Packages [579 kB]
16Get:10 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main armhf Packages [548 kB]
17Get:11 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main Translation-en [275 kB]
18Get:12 http://ports.ubuntu.com/ubuntu-ports bionic-updates/universe armhf Packages [854 kB]
19Get:13 http://ports.ubuntu.com/ubuntu-ports bionic-updates/universe arm64 Packages [911 kB]
20Get:14 http://ports.ubuntu.com/ubuntu-ports bionic-updates/universe Translation-en [313 kB]
21Get:15 http://ports.ubuntu.com/ubuntu-ports bionic-security/main Sources [127 kB]
22Get:16 http://ports.ubuntu.com/ubuntu-ports bionic-security/main armhf Packages [343 kB]
23Get:17 http://ports.ubuntu.com/ubuntu-ports bionic-security/main arm64 Packages [372 kB]
24Get:18 http://ports.ubuntu.com/ubuntu-ports bionic-security/main Translation-en [182 kB]
25Get:19 http://ports.ubuntu.com/ubuntu-ports bionic-security/universe arm64 Packages [551 kB]
26Get:20 http://ports.ubuntu.com/ubuntu-ports bionic-security/universe armhf Packages [498 kB]
27Get:21 http://ports.ubuntu.com/ubuntu-ports bionic-security/universe Translation-en [206 kB]
28Get:22 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe Sources [15.4 kB]
29Get:23 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/main Sources [27.6 kB]
30Get:24 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/main armhf Packages [35.9 kB]
31Get:25 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/main arm64 Packages [40.6 kB]
32Get:26 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/main Translation-en [27.2 kB]
33Get:27 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe arm64 Packages [24.6 kB]
34Get:28 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe armhf Packages [22.9 kB]
35Get:29 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe Translation-en [16.2 kB]
36Fetched 7,022 kB in 5s (1,534 kB/s)                           
37Reading package lists... Done
38Building dependency tree       
39Reading state information... Done
40
41root@odroid64:~# apt-get install libc6:armhf
42Reading package lists... Done
43Building dependency tree       
44Reading state information... Done
45libc6:armhf is already the newest version (2.27-3ubuntu1).
46libc6:armhf set to manually installed.
470 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.

No idea what could be the difference (though we’re clearly using a different mirror).

MarcoFalke commented at 6:34 pm on October 30, 2019: member

Oh, I was running the dpkg --add-architecture armhf on x86_64 locally. Does that make it not work?

Either way. Looks like on travis and the odroid c2 it worked. Let’s hope it runs the tests now :pray:

laanwj commented at 6:38 pm on October 30, 2019: member

Oh, I was running the dpkg –add-architecture armhf on x86_64 locally. Does that make it not work?

I’m not sure. I think it in principle it should be possible to install armhf as additional architecture on x86 (using qemu-user to execute things), but it’s a much less-traveled path than armhf on aarch64. You might have to mess around with apt configuration.

travis: Run unit and functional tests on native arm facc0da63a

MarcoFalke force-pushed on Oct 30, 2019

MarcoFalke commented at 0:41 am on November 1, 2019: member

@laanwj Thanks for the help. Passes locally and on travis. This is ready for review now!

laanwj commented at 4:04 pm on November 2, 2019: member

Thanks for seeing this through!

LGTM ACK facc0da63a8fa4bd6fc2782cbe92eb9f920f2256

practicalswift commented at 5:33 pm on November 2, 2019: contributor

ACK facc0da63a8fa4bd6fc2782cbe92eb9f920f2256 – diff looks correct and Travis seems happy

Thanks for working on this!

MarcoFalke referenced this in commit def69e5a86 on Nov 4, 2019

MarcoFalke merged this on Nov 4, 2019

MarcoFalke closed this on Nov 4, 2019

MarcoFalke deleted the branch on Nov 4, 2019

sidhujag referenced this in commit 1c6ec8552a on Nov 7, 2019

sidhujag referenced this in commit 15a2c66988 on Nov 10, 2020

MarcoFalke locked this on Dec 16, 2021

travis: Run unit and functional tests on native arm #17233