ci: Travis support for PowerPC64

elichai commented at 7:15 pm on November 7, 2019: contributor

Apparently travis has powerpc64 support even though it’s undocumented.

I found a couple of caveats though:

Compact glibc support doesn’t work (should be solved in #14066 )
it fails building OpneSSL from depends (fanquake tried debugging and it looks like it can’t recognize the platform).
Docker doesn’t recognize the platform correctly, so I had to use ppc64le/ubuntu:18.04 instead of library/ubuntu:18.04 (this is also maintained by Docker themselves https://github.com/docker-library/official-images#architectures-other-than-amd64)
functional tests are timing out.

Add ci script to install on powerpc64le 1a05592f13

Add powerpc64 tests to travis 1d684f0534

elichai renamed this:
~~Travis support for PowerPC64~~
ci: Travis support for PowerPC64
on Nov 7, 2019

fanquake added the label Tests on Nov 7, 2019

MarcoFalke commented at 7:23 pm on November 7, 2019: member

Nice find. However, given that it is not documented, it might go away or break any time and we shouldn’t expect travis to provide any support for it.

elichai commented at 7:28 pm on November 7, 2019: contributor

@MarcoFalke I can add allow_failures so the builds won’t fail if it doesn’t work. (altough that means manually checking if PPC passed)

MarcoFalke commented at 7:39 pm on November 7, 2019: member

altough that means manually checking if PPC passed

As if anyone would do that

MarcoFalke commented at 7:44 pm on November 7, 2019: member

Also according to #6466 we were looking for a be build, not le. Travis is already pretty slow these days and adding another build (which is mostly redundant to the other le builds) does not help in that regard.

elichai commented at 7:58 pm on November 7, 2019: contributor

I agree that if it was big endian it would’ve been nicer. But I do think it’s valuable to have CI for every platform we’re aiming to support, as it’s very easy to break support with one the second we use non C++ stuff (i.e. libc, asm etc.)

practicalswift commented at 8:19 pm on November 7, 2019: contributor

Concept ACK

Very nice find! :)

MarcoFalke commented at 8:22 pm on November 7, 2019: member

we use non C++ stuff (i.e. libc, asm etc.)

We use the Ubuntu Bionic libc on all builds, no? So at least that shouldn’t matter.

elichai commented at 9:30 pm on November 7, 2019: contributor

You mean when distributing binaries, right?

MarcoFalke commented at 9:40 pm on November 7, 2019: member

The ci system is running (at least on travis) in ubuntu:bionic docker containers.

laanwj commented at 9:59 pm on November 7, 2019: member

Also according to #6466 we were looking for a be build, not le. Travis is already pretty slow these days and adding another build (which is mostly redundant to the other le builds) does not help in that regard.

I tend to agree. Adding a big-endian build / test (irrespective of platform) would be great. PPC LE is worth testing on intermittently, but don’t think it needs to run for every PR and commit (I made a similar argument for RISC-V in #17089).

MarcoFalke commented at 10:12 pm on November 7, 2019: member

Don’t get me wrong, I appreciate the work you put in here and I am going to steal it for my projects (e.g. intermittent tests as suggested by @laanwj), but it might not be the best fit for the Bitcoin Core repo.

elichai commented at 10:15 am on November 8, 2019: contributor

I guess I agree, about the times. for the same reason RISC-V didn’t get merged, this i guess shouldn’t get merged either.

But this just makes me wonder if travis is hurting us in the long run. Giving up on testcases because “travis is already too slow” is a bad sign (not talking specifically about this PR, for that sense it can be even running valgrind).

Anyway, that’s unrelated to this PR(i’ll see if I have time to look into serious infrastructure (I think this is already a serious enough project to have serious infra)).

MarcoFalke commented at 1:57 pm on November 8, 2019: member

It is possible to send them money to speed it up for us (https://github.com/bitcoin/bitcoin/issues/16148#issuecomment-503677469). However, that will also slow it down for “local” travis testing in forks.

laanwj commented at 12:18 pm on November 10, 2019: member

But this just makes me wonder if travis is hurting us in the long run.

Well a bad craftsman blames his tools. Yes, travis is problematic sometimes, but also our test suite is very much on the heavy side. Both the build and tests somehow take very long compared to other projects.

If it means thorough testing that’s good, of course. But maybe it’d be nice to have a subset of fast tests that we can run as often on as many platforms as possible. Maybe foregoing for ex. depends builds.

elichai commented at 12:31 pm on November 10, 2019: contributor

But this just makes me wonder if travis is hurting us in the long run.

Well a bad craftsman blames his tools. Yes, travis is problematic sometimes, but also our test suite is very much on the heavy side. Both the build and tests somehow take very long compared to other projects.

If it means thorough testing that’s good, of course. But maybe it’d be nice to have a subset of fast tests that we can run as often on as many platforms as possible. Maybe foregoing for ex. depends builds.

I still somewhat blame Travis (i.e. Using a robust Jenkins like server everything is parallel in it’s own spots/machine so new jobs don’t make the tests longer).

But you’re right, maybe a good goal is to start figuring out what takes so much time and how can we fix that. Is it compile time or test time? can we change includes and/or libraries to make compile time faster? can we change test framework(or the way we test) to make tests run faster? Should we introduce ccache into the CI? (These are all open questions in my mind, I never looked into them and the answer might be “no” to all of them)

MarcoFalke commented at 2:02 pm on November 10, 2019: member

Is it compile time or test time?

Without cache it is compile time (due to boost and serialize.h). With cache it is install time (apt-get) and a bit of the functional tests.

can we change includes and/or libraries to make compile time faster?

Yes, see e.g. #17401 and #17307.

can we change test framework(or the way we test) to make tests run faster?

Yes, see e.g. #15858 or #16613.

Should we introduce ccache into the CI?

Yes (we already do for normal compiles), but it is not picked up by depends: #17103

MarcoFalke commented at 5:47 pm on November 13, 2019: member

https://blog.travis-ci.com/2019-11-12-multi-cpu-architecture-ibm-power-ibm-z#what-you-can-expect-from-this-alpha-release

DrahtBot commented at 6:39 am on November 22, 2019: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#17591 (ci: Add big endian platform - s390x by elichai)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

elichai commented at 11:46 am on November 25, 2019: contributor

@MarcoFalke Am I dreaming or are they also supporting s390x which is BE? I’ll test with a small travis to check that it’s actually BE and then PR it :)

EDIT: Definitely Big Endian :)

MarcoFalke referenced this in commit 2c1c43754b on Nov 25, 2019

DrahtBot commented at 8:30 pm on November 25, 2019: member

DrahtBot added the label Needs rebase on Nov 25, 2019

sidhujag referenced this in commit 8a4438f217 on Nov 25, 2019

elichai commented at 10:13 am on December 26, 2019: contributor

I don’t see a good reason to leave this open.

elichai closed this on Dec 26, 2019

sidhujag referenced this in commit a950439921 on Nov 10, 2020

DrahtBot locked this on Feb 15, 2022

ci: Travis support for PowerPC64 #17402

Conflicts