ci: Upgrade most ci configs to focal #19267

pull MarcoFalke wants to merge 4 commits into bitcoin:master from MarcoFalke:2006-ciFocal changing 9 files +47 −30
  1. MarcoFalke commented at 1:12 pm on June 13, 2020: member

    Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial. Older OSes (especially with compiler sanitizers) need workarounds that can be dropped by running on a more recent OS.

    This pull changes the asan sanitizer and the experimental multiprocess build to use focal. Also, it runs the no_wallet config on xenial to test against python 3.5, according to doc/dependencies.md.

    Finally, all configs that mimic gitian (win and mac) will stay at bionic.

  2. MarcoFalke added the label Tests on Jun 13, 2020
  3. MarcoFalke force-pushed on Jun 13, 2020
  4. in .travis.yml:130 in fa062788c3 outdated 
    114@@ -129,13 +115,12 @@ jobs:
115         FILE_ENV="./ci/test/00_setup_env_native_fuzz.sh"
116 
117     - stage: test
118-      name: 'x86_64 Linux [GOAL: install]  [bionic]  [multiprocess]'
119-      if: type != pull_request OR commit_message =~ /depends:|multiprocess:/ # Skip on non-depends, non-multiprocess PRs
    hebasto commented at 1:42 pm on June 13, 2020:
    Why this if: removed?
    MarcoFalke commented at 1:45 pm on June 13, 2020:
    I think it is not worth the risk of having the build go broken unnoticed as opposed to the cost of needing one more vm. The travis contract is expiring this month and we will either renew it or buy a contract at a different provider.
  5. MarcoFalke commented at 2:21 pm on June 13, 2020: member

    For some reason this doesn’t link:

    0/usr/bin/ld: qt/libbitcoinqt.a(libbitcoinqt_a-bitcoin.o): in function `GuiMain(int, char**)':
1
2/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:444: undefined reference to `qInitResources_bitcoin()'
3
4/usr/bin/ld: /home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:445: undefined reference to `qInitResources_bitcoin_locale()'
5
6clang: error: linker command failed with exit code 1 (use -v to see invocation)

    locally everything works fine, but on travis not: https://travis-ci.org/github/bitcoin/bitcoin/jobs/697962112#L4091

  6. hebasto commented at 2:42 pm on June 13, 2020: member

    For some reason this doesn’t link:

    0/usr/bin/ld: qt/libbitcoinqt.a(libbitcoinqt_a-bitcoin.o): in function `GuiMain(int, char**)':
1
2/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:444: undefined reference to `qInitResources_bitcoin()'
3
4/usr/bin/ld: /home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/bitcoin.cpp:445: undefined reference to `qInitResources_bitcoin_locale()'
5
6clang: error: linker command failed with exit code 1 (use -v to see invocation)

    locally everything works fine, but on travis not: https://travis-ci.org/github/bitcoin/bitcoin/jobs/697962112#L4091

    Could you try to add #include <QDir> into bitcoin.cpp?

  7. DrahtBot commented at 2:48 pm on June 13, 2020: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #19321 (ci: Run asan ci config on cirrus by MarcoFalke)
    • #19179 ([WIP RFC DONOTMERGE] ci: Run ci configs on cirrus by MarcoFalke)
    • #19077 (wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets by achow101)
    • #18912 (ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors by practicalswift)
    • #18870 (build: Allow BDB between 4.8 and 5.3 without –with-incompatible-bdb by achow101)
    • #18077 (net: Add NAT-PMP port forwarding support by hebasto)
    • #16546 (External signer support - Wallet Box edition by Sjors)
    • #15382 (util: add runCommandParseJSON by Sjors)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  8. kiminuo referenced this in commit 7d70344729 on Jun 13, 2020
  9. kiminuo commented at 3:52 pm on June 13, 2020: contributor
    (@hebasto Different PR but similar error. However, your suggestion did not help me: https://travis-ci.com/github/kiminuo/bitcoin/jobs/348750106 https://travis-ci.com/github/kiminuo/bitcoin/jobs/348750106#L4065)
  10. MarcoFalke force-pushed on Jun 13, 2020
  12. MarcoFalke commented at 4:59 pm on June 13, 2020: member

    Nice catch. Though,

    • Why does this error not abort the build immediately?
    • Why does it not reproduce locally?
  13. hebasto commented at 5:51 pm on June 13, 2020: member
    Travis is happy with 17dc996ec5c7a53627396b0dbd6a2b61804b9ba4 :)
  14. kiminuo referenced this in commit 0ef40ea4f8 on Jun 13, 2020
  15. fanquake commented at 1:16 am on June 14, 2020: member

    Concept ACK.

    Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial.

    If we’re going to do this I think we should make a decision in #18921. Otherwise the way we compile and test in our CI (as well as the way some developers are compiling locally) is starting to diverge from the way we actually build releases.

    For 0.21 this will be Ubuntu Bionic with python3.6.

    Is the minimum supported Python being bumped to 3.6 for 0.21, or are we just not going to test Python 3.5 compat any more? You said here:

    https://packages.ubuntu.com/xenial/python3 is still supported and not EOL until April next year, so we can’t bump to python 3.6 yet.

  16. MarcoFalke force-pushed on Jun 14, 2020
  17. MarcoFalke referenced this in commit fa654ee1ff on Jun 14, 2020
  18. MarcoFalke force-pushed on Jun 14, 2020
  19. MarcoFalke referenced this in commit fa71186ccd on Jun 14, 2020
  20. MarcoFalke force-pushed on Jun 14, 2020
  21. MarcoFalke commented at 12:14 pm on June 14, 2020: member

    starting to diverge from the way we actually build releases.

    I don’t mind bumping gitian to focal, but I think that discussion should happen elsewhere. I kept the three gitian-mimic configs at bionic (win64, mac, and the C++17 linux build). Though, sanitizers are not supported by gitian, so I think requiring them to use the older LTS and patch them with workarounds might not be the best way forward.

    Is the minimum supported Python being bumped to 3.6 for 0.21, or are we just not going to test Python 3.5 compat any more?

    I am testing python 3.5 compat, but most of my testing has moved out of this repo. Though, I’ve switched the existing no_wallet config to xenial in the last commit to make it a bit more explicit that python3.5 compat is not changed by this pull.

  22. MarcoFalke referenced this in commit d8e83f8cc7 on Jun 14, 2020
  23. MarcoFalke force-pushed on Jun 14, 2020
  24. practicalswift commented at 4:57 pm on June 14, 2020: contributor
    Concept ACK
  25. MarcoFalke referenced this in commit fa650d96a8 on Jun 15, 2020
  26. MarcoFalke force-pushed on Jun 15, 2020
  27. MarcoFalke commented at 11:56 am on June 15, 2020: member
    Force pushed to get rid of the already merged commit
  28. laanwj commented at 2:03 pm on June 17, 2020: member

    Concept ACK. I think focal is a better reflection of the compiler and binutils most people are using than bionic, at this point.

    Though I agree with @fanquake’s point that we’re still building releases using bionic, so we need to test that.

    However, there should be at least one config to test compile and run-time compatibility on the oldest supported OS. For 0.21 this will be Ubuntu Bionic with python3.6.

    This is somewhat confusing to me—compile-time and run-time compatibility was never the same. Run-time compatibility (at least for pre-built releases) historically at least tends to back farther than we support building on. But I suppose backward-testing gitian releases isn’t part of the CI so it doesn’t matter here.

  29. travis: Always run multiprocess build fa6ddb2fa1
  30. MarcoFalke referenced this in commit faafabb4a9 on Jun 19, 2020
  31. MarcoFalke force-pushed on Jun 19, 2020
  32. DrahtBot added the label Needs rebase on Jun 19, 2020
  33. MarcoFalke commented at 2:34 pm on June 19, 2020: member
    doc/dependencies.md says it is python 3.5, so I went ahead and updated OP
  34. DrahtBot removed the label Needs rebase on Jun 19, 2020
  35. ci: Have one config run in xenial to test against python3.5 
    Also, bump the travis env to bionic. This shouldn't matter at all
because all ci configs run inside a docker, but it does seem to fix a
bug. See
https://github.com/bitcoin/bitcoin/pull/19267#issuecomment-643630309
    fa880773b4
  36. doc: move doc to ci readme fad6720891
  37. ci: Upgrade most ci configs to focal fa05f44893
  38. MarcoFalke force-pushed on Jun 19, 2020
  39. MarcoFalke commented at 2:45 pm on June 19, 2020: member
    re-arranged commits to minimize diff
  40. Sjors commented at 3:07 pm on June 19, 2020: member
    ACK fa05f44893d228f672f39436d0cb6b3376f81ac2, assuming Travis passes (it hasn’t shown up in the checks yet)
  41. MarcoFalke commented at 4:15 pm on June 19, 2020: member

    (it hasn’t shown up in the checks yet)

    It shows for me:

    Screenshot_2020-06-19 ci Upgrade most ci configs to focal by MarcoFalke · Pull Request #19267 · bitcoin bitcoin

  42. in .travis.yml:120 in fa05f44893 
    118       env: >-
119         FILE_ENV="./ci/test/00_setup_env_native_multiprocess.sh"
120 
121     - stage: test
122-      name: 'x86_64 Linux  [GOAL: install]  [bionic]  [no wallet]'
123+      name: 'x86_64 Linux  [GOAL: install]  [xenial]  [no wallet]'
    hebasto commented at 4:21 pm on June 19, 2020:
    Why xenial?
    MarcoFalke commented at 4:25 pm on June 19, 2020:
    Please refer to the commit message and pull request description for the motivation behind changes
    hebasto commented at 4:32 pm on June 19, 2020:
    I see.
  43. hebasto approved
  44. hebasto commented at 4:33 pm on June 19, 2020: member
    ACK fa05f44893d228f672f39436d0cb6b3376f81ac2
  45. MarcoFalke merged this on Jun 19, 2020
  46. MarcoFalke closed this on Jun 19, 2020
  47. MarcoFalke deleted the branch on Jun 19, 2020
  48. Sjors commented at 4:57 pm on June 19, 2020: member
    I show up for me now as well.
  49. stackman27 referenced this in commit a50fe718f0 on Jun 26, 2020
  50. sidhujag referenced this in commit 558a1b0739 on Jul 7, 2020
  51. janus referenced this in commit 7c67a54475 on Nov 22, 2020
  52. DrahtBot locked this on Feb 15, 2022


MarcoFalke hebasto DrahtBot kiminuo fanquake practicalswift laanwj Sjors

Labels
Tests

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 10:13 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me