ci: use depends for s390x #17661

elichai commented at 10:31 PM on December 3, 2019: contributor

Related: #17599

This adds qemu support just like we have in arm and compile the depends.

other than that I also fixed some missing includes to make the depends compile.

fanquake added the label Tests on Dec 3, 2019

DrahtBot commented at 12:07 AM on December 4, 2019: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

in ci/test/00_setup_env_s390x.sh:10 in 2b3eada327 outdated

   5 | @@ -6,9 +6,10 @@
   6 |  
   7 |  export LC_ALL=C.UTF-8
   8 |  
   9 | -export HOST=s390x-unknown-linux-gnu
  10 | -export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev"
  11 | -export NO_DEPENDS=1
  12 | +# The host arch is unknown, so we run the tests through qemu.
  13 | +# If the host is arm and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.

MarcoFalke commented at 3:49 PM on December 4, 2019:

# If the host is s390x and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.

elichai commented at 4:39 PM on December 4, 2019:

Done

in ci/test/00_setup_env_s390x.sh:12 in 2b3eada327 outdated

  10 | -export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev"
  11 | -export NO_DEPENDS=1
  12 | +# The host arch is unknown, so we run the tests through qemu.
  13 | +# If the host is arm and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
  14 | +export QEMU_USER_CMD="${QEMU_USER_CMD:"qemu-s390x"}"
  15 | +export PACKAGES="python3 python3-zmq bsdmainutils qemu-user qtbase5-dev"

MarcoFalke commented at 3:50 PM on December 4, 2019:

export PACKAGES="python3-zmq bsdmainutils qemu-user"

elichai commented at 3:58 PM on December 4, 2019:

I wrote about that in the PR

MarcoFalke commented at 4:00 PM on December 4, 2019:

Hmm. CC @dongcarl and @fanquake for ideas on why this is needed

dongcarl commented at 4:47 PM on December 4, 2019:

Huh... Could you try doing

$(package)_config_opts_s390x_linux = -platform linux-g++ -xplatform bitcoin-linux-g++

in qt.mk and seeing if that works?

elichai commented at 5:10 PM on December 4, 2019:

I'll try

elichai commented at 5:21 PM on December 4, 2019:

Haven't tested that yet, but i'm starting to think that this problem isn't deterministic :O somehow it passes now without it https://travis-ci.org/elichai/bitcoin/jobs/620711630?utm_medium=notification&utm_source=github_status

dongcarl commented at 5:37 PM on December 4, 2019:

Haven't tested that yet, but i'm starting to think that this problem isn't deterministic :O somehow it passes now without it travis-ci.org/elichai/bitcoin/jobs/620711630?utm_medium=notification&utm_source=github_status

The commit hash in the original PR description does not match this one... So perhaps you rebased or made other changes?

MarcoFalke commented at 7:14 PM on December 4, 2019:

I tried cross compilation once from x86_64->s390x and it only worked with the suggestion by @dongcarl

diff --git a/depends/packages/qt.mk b/depends/packages/qt.mk
index a9f82d038..2db7f26b2 100644
--- a/depends/packages/qt.mk
+++ b/depends/packages/qt.mk
@@ -106,6 +106,8 @@ $(package)_config_opts += -no-feature-xml
 
 $(package)_config_opts_darwin = -no-dbus
 
+$(package)_config_opts_s390x_linux = -platform linux-g++ -xplatform bitcoin-linux-g++
+
 ifneq ($(build_os),darwin)
 $(package)_config_opts_darwin += -xplatform macx-clang-linux
 $(package)_config_opts_darwin += -device-option MAC_SDK_PATH=$(OSX_SDK)

Full diff:

diff --git a/ci/test/00_setup_env_s390x.sh b/ci/test/00_setup_env_s390x.sh
index 8116268d3..43a820529 100644
--- a/ci/test/00_setup_env_s390x.sh
+++ b/ci/test/00_setup_env_s390x.sh
@@ -6,10 +6,11 @@
 
 export LC_ALL=C.UTF-8
 
+export HOST=s390x-linux-gnu
 # The host arch is unknown, so we run the tests through qemu.
 # If the host is s390x and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
 export QEMU_USER_CMD="${QEMU_USER_CMD:"qemu-s390x"}"
-export PACKAGES="python3 python3-zmq bsdmainutils qemu-user qtbase5-dev"
+export PACKAGES="python3-zmq g++-s390x-linux-gnu qemu-user"
 export RUN_UNIT_TESTS=true
 export RUN_FUNCTIONAL_TESTS=true
 export GOAL="install"
diff --git a/depends/packages/qt.mk b/depends/packages/qt.mk
index a9f82d038..2db7f26b2 100644
--- a/depends/packages/qt.mk
+++ b/depends/packages/qt.mk
@@ -106,6 +106,8 @@ $(package)_config_opts += -no-feature-xml
 
 $(package)_config_opts_darwin = -no-dbus
 
+$(package)_config_opts_s390x_linux = -platform linux-g++ -xplatform bitcoin-linux-g++
+
 ifneq ($(build_os),darwin)
 $(package)_config_opts_darwin += -xplatform macx-clang-linux
 $(package)_config_opts_darwin += -device-option MAC_SDK_PATH=$(OSX_SDK)

fanquake commented at 7:16 PM on December 4, 2019:

Note: If we are applying this diff, can we make sure we put the $(package)_config_opts_s390x_linux line down with the other Linux options, rather than splitting up the darwin config.

elichai commented at 1:19 PM on December 5, 2019:

It works without that diff. the only problem is the HOST. we could do a trick similiar to QEMU_USER_CMD where we only apply the HOST var if it's in qemu

in ci/test/00_setup_env_s390x.sh:9 in 2b3eada327 outdated

   5 | @@ -6,9 +6,10 @@
   6 |  
   7 |  export LC_ALL=C.UTF-8
   8 |  
   9 | -export HOST=s390x-unknown-linux-gnu

MarcoFalke commented at 3:54 PM on December 4, 2019:

Why?

I presume because the packages are not available?

https://packages.ubuntu.com/eoan/g++-multilib-s390x-linux-gnu https://packages.ubuntu.com/eoan/gcc-s390x-linux-gnu

MarcoFalke commented at 3:55 PM on December 4, 2019:

If so, this should be reported as a bug. Looks like at least debian is working on them: https://packages.debian.org/experimental/gcc-s390x-linux-gnu

elichai commented at 3:57 PM on December 4, 2019:

Yes. that's the reason

MarcoFalke commented at 5:20 PM on December 4, 2019:

Hmm. If the host is not set, it defaults to the arch this script is executed on. This should be the same.

What is the error you were seeing when leaving this line as is?

MarcoFalke commented at 5:37 PM on December 4, 2019:

On travis this is set to s390x-ibm-linux-gnu

elichai commented at 6:01 PM on December 4, 2019:

that it can't find the gcc-s390x-linux-gnu package

MarcoFalke commented at 6:51 PM on December 4, 2019:

Eh? You mean error: provided command 's390x-unknown-linux-gnu-g++' not found?

If so, export HOST=s390x-linux-gnu should work

MarcoFalke commented at 6:55 PM on December 4, 2019:

Sorry, forget my comment. The cross compile packages are missing either way on s390x

MarcoFalke commented at 7:35 PM on December 13, 2019:

On a second try, I think installing gcc on s390x will create the appropriate symlinks in /usr/bin/s390x-linux-gnu-gcc -> gcc-9

This should work:

diff --git a/ci/test/00_setup_env_s390x.sh b/ci/test/00_setup_env_s390x.sh
index c17c890546..637d549553 100644
--- a/ci/test/00_setup_env_s390x.sh
+++ b/ci/test/00_setup_env_s390x.sh
@@ -6,10 +6,11 @@
 
 export LC_ALL=C.UTF-8
 
+export HOST=s390x-linux-gnu
 # The host arch is unknown, so we run the tests through qemu.
 # If the host is s390x and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
 export QEMU_USER_CMD="${QEMU_USER_CMD:"qemu-s390x"}"
-export PACKAGES="python3 python3-zmq bsdmainutils qemu-user"
+export PACKAGES="python3-zmq bsdmainutils qemu-user"
 export RUN_UNIT_TESTS=true
 export RUN_FUNCTIONAL_TESTS=true
 export GOAL="install"

MarcoFalke commented at 3:56 PM on December 4, 2019: member

sweet

elichai commented at 3:57 PM on December 4, 2019: contributor

@MarcoFalke arghh it looks like the rebase screwed some stuff up. fixing

elichai force-pushed on Dec 4, 2019

in ci/test/00_setup_env_s390x.sh:12 in 15eb492458 outdated

   5 | @@ -6,8 +6,10 @@
   6 |  
   7 |  export LC_ALL=C.UTF-8
   8 |  
   9 | -export HOST=s390x-unknown-linux-gnu
  10 | -export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev"
  11 | +# The host arch is unknown, so we run the tests through qemu.
  12 | +# If the host is s390x and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
  13 | +export QEMU_USER_CMD="${QEMU_USER_CMD:"qemu-s390x"}"
  14 | +export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libssl1.0-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev"

fanquake commented at 4:43 PM on December 4, 2019:

Why is libssl1.0-dev being added here? Can 4d238f0cc8f4bf4170f18a27adf7607a6514b79e be squashed into 15eb4924589cdd8127492d67ac94f1ff2cf8651d.

elichai commented at 5:04 PM on December 4, 2019:

arghh, this is because the commits are before libssl was removed and I didn't check things good enough after rebasing, sorry.

elichai commented at 1:17 PM on December 5, 2019:

Done.

MarcoFalke removed the label Tests on Dec 4, 2019

MarcoFalke added the label Build system on Dec 4, 2019

MarcoFalke renamed this:
~~ci: compile and fix depends for s390x~~
build: fix depends for s390x and use it in ci
on Dec 4, 2019

elichai commented at 5:33 PM on December 4, 2019: contributor

I don't get it. the exact same commit passes on my private travis but fails here. https://travis-ci.org/elichai/bitcoin/builds/620711066

MarcoFalke commented at 9:11 PM on December 4, 2019: member

the exact same commit passes on my private travis but fails here.

The depends cache is never invalidated. See bug #17248 . So when it is created once, you can break the build, but travis won't fail.

elichai commented at 10:17 AM on December 5, 2019: contributor

The depends cache is never invalidated. See bug #17248 . So when it is created once, you can break the build, but travis won't fail.

That's bad. I guess I need to remove my cache, and it might also mean that most of the testing i've done were moot :(

elichai commented at 1:12 PM on December 5, 2019: contributor

Lol, I don't really know what happened but i'll clean the commits up now and then it should be ready

elichai force-pushed on Dec 5, 2019

MarcoFalke added the label Needs gitian build on Dec 5, 2019

MarcoFalke commented at 3:12 PM on December 5, 2019: member

ACK a81792804b572f1bfdada2f6b0652f494f49ded5 (didn't review depends build changes)

DrahtBot removed the label Needs gitian build on Dec 7, 2019

MarcoFalke deleted a comment on Dec 9, 2019

DrahtBot added the label Needs rebase on Dec 9, 2019

MarcoFalke commented at 9:12 PM on December 11, 2019: member

@elichai :pray: Trivial rebase. I can pick it up after that.

MarcoFalke added the label Needs gitian build on Dec 11, 2019

elichai force-pushed on Dec 12, 2019

DrahtBot removed the label Needs rebase on Dec 12, 2019

MarcoFalke approved

MarcoFalke commented at 7:37 PM on December 13, 2019: member

ACK after applying my diff to fix the build and adjusting OP

ci-s390x: Add qemu and depends support in the ci script e190000869

elichai force-pushed on Dec 14, 2019

DrahtBot commented at 8:04 AM on December 15, 2019: member

Gitian builds

File	commit c5e318aea6ad098db66e6f174f2790acc197351c<br>(master)	commit 0061b7b5af1463ee0868d08e7ee4f2e0235d6fcb<br>(master and this pull)
bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz	`bea40f9a3e1e3d15...`	`a98f522b7e7ff91b...`
bitcoin-0.19.99-aarch64-linux-gnu.tar.gz	`6062abc6a838a5a6...`	`ebc4dbe686d18f80...`
bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz	`69dcc7a49e222ca9...`	`2a552d16d640fadc...`
bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz	`d0e369041d9fe9b5...`	`c0018bacd165877e...`
bitcoin-0.19.99-i686-pc-linux-gnu-debug.tar.gz	`d3b7c55ff3c2bc2f...`	`84a0133ad9aa9f2c...`
bitcoin-0.19.99-i686-pc-linux-gnu.tar.gz	`ae365922d91820b9...`	`ce52b8ba3748167e...`
bitcoin-0.19.99-osx-unsigned.dmg	`d5388aa2474ac73e...`	`3d5701d1b95e353c...`
bitcoin-0.19.99-osx64.tar.gz	`4baf8f214f059caf...`	`2f8d4aca89d2725d...`
bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz	`6b0e20d11cb5fe6b...`	`55b967d9ccc17e21...`
bitcoin-0.19.99-riscv64-linux-gnu.tar.gz	`0d45828e7aac5e93...`	`e67889265ed13b9a...`
bitcoin-0.19.99-win64-debug.zip	`5669c48e3fad4bc5...`	`507713b1cb0230a8...`
bitcoin-0.19.99-win64-setup-unsigned.exe	`6fc70664b2973325...`	`3f49d90170510f93...`
bitcoin-0.19.99-win64.zip	`576846961cf26ff2...`	`db60fc2d1312a0a7...`
bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz	`591688b38863b8b4...`	`c938e51ff29b599a...`
bitcoin-0.19.99-x86_64-linux-gnu.tar.gz	`e8dcc933264b4d6c...`	`4c3b2c21eec6a881...`
bitcoin-0.19.99.tar.gz	`9d45221fe86cb679...`	`6166d27f5215b5b5...`
bitcoin-core-linux-0.20-res.yml	`2efef3cbffb806a2...`	`b3b604fe76703726...`
bitcoin-core-osx-0.20-res.yml	`d2ea6d1607e9fd97...`	`2f8687cf09b7a1cc...`
bitcoin-core-win-0.20-res.yml	`93ecd38034110860...`	`bc8819618d3b2a1d...`
linux-build.log	`b3d2ce067eb93baa...`	`10693a0201957226...`
osx-build.log	`f3e9977f0c38cddd...`	`8dd6cc0ef0f7e3cf...`
win-build.log	`91466d2d19dec4e6...`	`3bf8f42281562e0c...`
bitcoin-core-linux-0.20-res.yml.diff		`fea7b047d0e61a81...`
bitcoin-core-osx-0.20-res.yml.diff		`fc8e5feb98279eea...`
bitcoin-core-win-0.20-res.yml.diff		`0e7fe61b8dc92eee...`
linux-build.log.diff		`1b744686269d5ca2...`
osx-build.log.diff		`6d66ed27e8c6b710...`
win-build.log.diff		`b9ba672e8d39be29...`

DrahtBot removed the label Needs gitian build on Dec 15, 2019

elichai requested review from dongcarl on Dec 16, 2019

elichai requested review from fanquake on Dec 16, 2019

MarcoFalke commented at 3:57 PM on December 16, 2019: member

ACK e190000 (first commit only, didn't look at second commit)

MarcoFalke commented at 5:36 PM on December 16, 2019: member

Why is the second commit needed? Travis passes without it, no?

elichai commented at 6:04 PM on December 16, 2019: contributor

Why is the second commit needed? Travis passes without it, no?

Hmm is it possible that one of your fixes somehow fixed this too? I'll reset my travis cache (to avoid the bug you talked about) and rerun the tests on that commit only. Because when I opened the PR this was definitely a problem

MarcoFalke commented at 10:17 PM on December 16, 2019: member

I tried it in my repo and it worked: https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/625746803

elichai force-pushed on Dec 17, 2019

elichai commented at 2:54 PM on December 17, 2019: contributor

I tried it in my repo and it worked: https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/625746803

Done. don't think I understand what's going on in the depends anymore.

MarcoFalke commented at 3:02 PM on December 17, 2019: member

Done. don't think I understand what's going on in the depends anymore.

No one does, that I am aware of

MarcoFalke renamed this:
~~build: fix depends for s390x and use it in ci~~
ci: use depends for s390x
on Dec 17, 2019

dongcarl commented at 3:08 PM on December 17, 2019: member

On my TODO list to take a look at this week

MarcoFalke commented at 3:46 PM on December 17, 2019: member

I'll probably just merge this. All it does is set the proper cross compile HOST, remove NO_DEPENDS=1 and remove all the packages from apt to install.

I broke the qemu-wrapper script in the ci system I wrote, so for proper cross-compilation and cross-running the tests in qemu (e.g. on aarch64, riscv64 or x86_64 architecture) I'll need to do a follow up pull anyway.

MarcoFalke referenced this in commit a54c16cdb6 on Dec 17, 2019

MarcoFalke merged this on Dec 17, 2019

MarcoFalke closed this on Dec 17, 2019

sidhujag referenced this in commit ace53d2e4d on Dec 17, 2019

elichai deleted the branch on Dec 18, 2019

sidhujag referenced this in commit 59e1a5ceb4 on Nov 10, 2020

MarcoFalke locked this on Dec 16, 2021