travis-ci integration (pull-tester replacement) #4749

theuni commented at 5:22 PM on August 21, 2014: member

See the doc/travis.txt for a description of what travis is and how it works.

This is an initial descriptor to get things going. It can be changed around at any time, so discussions about what build configs to use, notifications, etc, can all take place in the form of pull-requests in the future.

At the moment, each pull-request will kick off 7 builds:

linux x86_64 full
linux x86_64 minimal
linux i686 full
linux arm no-qt
osx x86_64 full
win x86_64 full
win i686 full

Build results will be similar to this: https://travis-ci.org/coryfields/bitcoin/builds/33137850 .

There are lots of features and bells/whistles to play with down the road, see notifications for an example.

There's also a quick build change in here to make osx build logs actually useful.

build: work around ccache/autotools warning-spamming bug

When using clang and ccache, builds spew lots of:
Clang: warning: argument unused during compilation

Upstream bug: https://bugzilla.samba.org/show_bug.cgi?id=8118

This is harmless, bug annoying. If ccache is being used and the
-Qunused-arguments flag is supported (clang), use it.

386efb7695

BitcoinPullTester commented at 11:22 PM on August 21, 2014: none

I SHALL NOT BE SILENCED!

TheBlueMatt commented at 11:34 PM on August 21, 2014: member

Generally looks good (we can finally kill the damn pull testing bot), though is there any way to access to built binaries after the build finishes?

fanquake commented at 6:41 AM on August 22, 2014: member

@TheBlueMatt The only way to access the binaries would be to upload them to S3 after compilation. http://docs.travis-ci.com/user/deployment/s3/

TheBlueMatt commented at 7:18 AM on August 22, 2014: member

Sounds good to me, can we make that happen?

laanwj commented at 7:33 AM on August 22, 2014: member

Hahahaha @BitcoinPullTester somehow I expected a strong NACK from you.

theuni commented at 3:19 PM on August 22, 2014: member

@TheBlueMatt there are actually several ways to deploy, see here: http://docs.travis-ci.com/user/deployment/ . I figured we'd probably use the custom one to push to some server space we control, but s3 would be fine by me as well.

I left deployment out of the original PR because I figured it'd be worth discussing on its own. Same with notifications. Like everything else with Travis, we don't have to worry about any kind of server-side state, so everything is handled in the .yml file. So all deployment/notification changes will actually happen via git commits.

TheBlueMatt commented at 6:02 PM on August 22, 2014: member

@theuni Yea, we do have a server we can stick crap like this on. As for it being in a separate pull, I dont care, but getting binary output was always one of the nicest features of pull-tester.

laanwj commented at 10:34 AM on August 25, 2014: member

@theuni why is the travis CI build failing here?

theuni commented at 2:30 PM on August 25, 2014: member

@laanwj I forgot to disable the OSX builder for the initial merge. Disabled now and it's all green.

See the note here: https://github.com/bitcoin/bitcoin/pull/4749/files#diff-7dcfe27d5e2ccfc80ea9b88dd14ada7dR34 The OSX SDK url needs to be hidden, since it's not distributable. We can create that encrypted string as a next step, then hook OSX back up.

theuni commented at 2:33 PM on August 25, 2014: member

Note that we'll need to do the same thing for creating secure scp credentials (ssh key and/or password) for uploading build results as well. I figure we can just knock both of those out at once post-merge.

theuni force-pushed on Aug 25, 2014

theuni commented at 3:48 PM on August 25, 2014: member

Squashed and removed osx references. Will re-add that once the sdk fetching is more foolproof.

travis: initial descriptor 9380d019a1

travis: add doc 425c7a8389

BitcoinPullTester commented at 4:01 PM on August 25, 2014: none

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/p4749_425c7a83899f7ccf5343a24319bc19e1e77363da/ for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

in .travis.yml:None in 425c7a8389

   8 | +language: cpp
   9 | +env:
  10 | +  global:
  11 | +    - MAKEJOBS=-j3
  12 | +    - RUN_TESTS=false
  13 | +    - CCACHE_SIZE=100M

sipa commented at 9:20 PM on August 25, 2014:

All .o files in my local installation together is a multiple of 100M already. Does it help at all if it can't even cache one full build's results?

theuni commented at 9:49 PM on August 25, 2014:

@sipa: These are non-debug, so significantly smaller than typical dev objects. They're also gzip compressed (see 2 lines down). A full linux build that way results in a ~17mb ccache dir. For reference, the exact same build with debug is 105mb. That's probably more inline with what you're seeing?

Also, as I see it, there's really no use in keeping more than one version of each object around, since each cache matches a single build config. It either matches the current revision or it can be discarded. So my target with the 100mb figure was the largest reasonable compressed store of one version of all objects.

sipa commented at 9:53 PM on August 25, 2014:

ACK

TheBlueMatt commented at 1:44 AM on August 26, 2014: member

ACK, though I'd like to see build outputs available before we turn @BitcoinPullTester off

theuni commented at 1:57 AM on August 26, 2014: member

@TheBlueMatt agreed, I'm working on that. It proved to be more challenging than anticipated, since Travis assumes that it will publish from commits and/or tags, rather than pull requests (which makes sense in terms of publishing). Will update here when I've got something worked out.

laanwj merged this on Aug 26, 2014

laanwj closed this on Aug 26, 2014

laanwj referenced this in commit 0d763fc9e1 on Aug 26, 2014

laanwj commented at 6:46 AM on August 26, 2014: member

Aren't the executables produced by pull tester mostly useless anyway? The windows ones have been crashing for ages, at least, and I'd expect most Linux people insist on building their own.

theuni commented at 7:13 AM on August 26, 2014: member

@laanwj Historically, probably. But now that they match gitian's functionality, they could be more helpful. I probably shouldn't argue that too strongly though, since it's turning out to be a bit of a pain :p

TheBlueMatt commented at 7:28 AM on August 26, 2014: member

When pull-tester was being actively maintained/developed they were very helpful, I know on several occasions pulls were submitted just to get windows binaries.

MarcoFalke locked this on Sep 8, 2021