ci: Add environment files for all settings

MarcoFalke commented at 8:52 pm on August 15, 2019: member

This moves all environment settings from travis to files in the ci folder. Now, it is possible to easily run each travis configuration with a single command.

MarcoFalke force-pushed on Aug 15, 2019

DrahtBot added the label Docs on Aug 15, 2019

DrahtBot added the label Tests on Aug 15, 2019

promag commented at 11:51 pm on August 15, 2019: member

This approach pollutes the calling bash environment. Should work without export?

Concept ACK.

DrahtBot commented at 0:27 am on August 16, 2019: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#16392 (WIP build: macOS toolchain update by fanquake)
#16161 (util: Fix compilation errors in support/lockedpool.cpp by jkczyz)
#15584 (build: disable BIP70 support by default by fanquake)
#12134 (Build previous releases and run functional tests by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

in .travis.yml:139 in e5ada9ad16 outdated

196-        OSX_SDK=10.11
197-        RUN_UNIT_TESTS=false
198-        RUN_FUNCTIONAL_TESTS=false
199-        GOAL="deploy"
200-        BITCOIN_CONFIG="--enable-gui --enable-reduce-exports --enable-werror"
201+        FILE_ENV="./ci/test/00_setup_env_mac.sh"

fanquake commented at 1:07 am on August 16, 2019:

Should this be 00_setup_env_amd64_mac ? There’s no 00_setup_env_mac.sh file.

MarcoFalke commented at 1:19 pm on August 16, 2019:

Fixed

MarcoFalke force-pushed on Aug 16, 2019

fanquake commented at 4:14 am on August 16, 2019: member

Concept ACK

I’ll run through each of the different ENV builds locally. Have done amd64_asan and amd64_qt5 so far.

For a future change / discussion, it’d be handy if the Docker containers were named. amd64_qt is more useful than peaceful_rubin once there are a bunch floating around.

Also, maybe we could stop (although not destroy) containers automatically after successful completion of the tests?

ci: Add environment files for all settings fa21737ba7

MarcoFalke force-pushed on Aug 16, 2019

MarcoFalke commented at 1:12 pm on August 16, 2019: member

This approach pollutes the calling bash environment.

Can you explain what command exactly does that? I can’t find any.

It will potentially pollute your ccache dir and the git repo with build artifacts, but that has nothing to do with bash.

MarcoFalke commented at 1:20 pm on August 16, 2019: member

Good suggestions @fanquake. I’ll probably leave those as follow-up features that can be added.

Sjors commented at 5:13 pm on August 17, 2019: member

ACK. I would find it easier if the new files were numbered the same as its equivalent Travis machine.

Recipe to verify the move:

delete export from ci/test/00_setup_env_*
git commit -a --amend
git show --color-moved=dimmed-zebra --color-moved-ws=allow-indentation-change

I tried (cloned in /home/travis/bitcoin):

0cd bitcoin
1FILE_ENV="./ci/test/00_setup_env_amd64_qt5.sh" ./ci/test_run_all.sh

This fails at the configure step:

 0travis_fold:start:configure
 1configure: loading site script /home/travis/bitcoin/depends/x86_64-unknown-linux-gnu/share/config.site
 2configure: loading cache config.cache
 3configure: error: `CXXFLAGS' was not set in the previous run
 4configure: error: `CFLAGS' was not set in the previous run
 5configure: error: in `/home/travis/bitcoin/build':
 6configure: error: changes in the environment can compromise the build
 7configure: error: run `make distclean' and/or `rm config.cache' and start over
 8...
 9configure:2987: loading cache config.cache
10configure:3014: error: `CXXFLAGS' was not set in the previous run
11configure:3014: error: `CFLAGS' was not set in the previous run
12configure:3051: error: in `/home/travis/bitcoin/build':
13configure:3053: error: changes in the environment can compromise the build
14configure:3055: error: run `make distclean' and/or `rm config.cache' and start over
15...
16configure: exit 1

It may have gotten poluted from a previous run, so I tried purging docker instances and nuking the git repository:

0docker stop $(docker ps -a -q)
1docker container prune
2git clean -dfx

Now it runs. Because Docker has root privileges, things very quickly turn into permission hell if you try to run more than one instance.

As a followup, it would be nice if you can run all configurations with a single command (stopping if any fails).

MarcoFalke referenced this in commit 8b42db130f on Aug 19, 2019

MarcoFalke commented at 1:41 pm on August 19, 2019: member

Now it runs. Because Docker has root privileges, things very quickly turn into permission hell if you try to run more than one instance.

You can’t run more than one ci instance on the repo at the same time.

As a followup, it would be nice if you can run all configurations with a single command (stopping if any fails).

I’ll leave this as follow-up ;)

MarcoFalke merged this on Aug 19, 2019

MarcoFalke closed this on Aug 19, 2019

MarcoFalke deleted the branch on Aug 19, 2019

Sjors commented at 2:22 pm on August 19, 2019: member

You can’t run more than one ci instance on the repo at the same time.

I was running them in sequence.

MarcoFalke commented at 2:25 pm on August 19, 2019: member

That should work as long as you run sudo git clean -dffx (at your own risk, obviously)

sidhujag referenced this in commit c8c01c9091 on Aug 19, 2019

kittywhiskers referenced this in commit 18646ee093 on Dec 13, 2021

kittywhiskers referenced this in commit 5335524130 on Dec 13, 2021

MarcoFalke locked this on Dec 16, 2021

ci: Add environment files for all settings #16623

Conflicts