New Travis job for CHECK_DOCS steps #11004

1. 
   eklitzke commented at 11:23 pm on August 7, 2017: contributor

   This is a request to change how the Travis job is configured for Bitcoin. I’d submit a PR to do this myself, but the change can only be made by someone who has Travis configuration permissions for the bitcoin repo.

   Background

   Currently there are a number of static-analysis style checks that are run in Travis, guarded by the CHECK_DOC environment variable. This variable is only set for the first test runner (the arm-linux-gnueabihf host), so the checks are only run once. All of these checks run very fast, and do not require building Bitcoin or running the regular C++/Python tests.

   Request

   Configure a new Travis job that runs against PRs, and only runs the CHECK_DOC tests. At the same time, the CHECK_DOC stuff in the existing job would be removed.

   Motivation

   Creating a separate Travis job for these tests would be an improvement because:

   • It would be more immediately obvious that a CHECK_DOC failure is related to one of the check scripts; currently it’s confusing to new contributors (e.g. me) to see an ARM failure, when the failure is not actually related to any new architecture-specific code.
   • The job would either pass quickly or fail quickly, giving faster feedback to developers who submit PRs.
   • The main .travis.yml file in the repo could be cleaned up, by removing the CHECK_DOC guards in the before_script and script sections.
2. jonasschnelli added the label Tests on Aug 8, 2017
3. 
   MarcoFalke commented at 9:45 am on August 8, 2017: member

   There is some overhead in booting and setting up the vm, so I thought it would be fine to misuse the first entry in the build matrix.
4. 
   eklitzke commented at 9:25 pm on August 8, 2017: contributor

   I will defer to you of course, but I’ve found this to be helpful in other projects. As an example, suppose it takes one minute on average to launch a new Travis worker, and the CHECK_DOC tests can run in a few seconds (or at least, less than a minute). Then if someone submits a PR there’s a good chance that they’ll see a red X on their pull request while the tab is still open, rather than an hour later when they go to check how the full test suite did. Feel free to close though if you think the current system makes more sense.
5. 
   MarcoFalke commented at 11:04 am on August 9, 2017: member

   For some reason the red X is not shown as soon as one of the entries in the build matrix failed. It only shows when all entries are finished with evaluation. So we’d need to address that first, if possible. Otherwise the new entry in the build matrix won’t help.
6. 
   jnewbery commented at 11:02 pm on August 9, 2017: member

   Concept ACK. I like the idea of running the static tools separately, both to make it more obvious where problems are and potentially for faster feedback.

   For some reason the red X is not shown as soon as one of the entries in the build matrix failed. It only shows when all entries are finished with evaluation.

   I’ve just tested this and it appears to still be true. Could https://docs.travis-ci.com/user/build-stages/ be a solution? We could run static analysis tools in a pre-build stage, and then build/run tests in a second stage. @eklitzke - Any idea on how much overhead that would add?

7. 
   practicalswift commented at 9:53 pm on August 10, 2017: contributor

   Concept ACK 👍
8. 
   laanwj commented at 9:17 am on August 24, 2017: member

   I like the idea as well, though if it has serious performance repercussions I’d prefer to stay with the current setup. Travis is swamped as it is.

   It’s unfortunate that Travis doesn’t allow reporting error messages to the overview screen from scripts. This would be even better than having a specific sub-build fail.

9. MarcoFalke added the label hacktoberfest on Oct 9, 2017
10. MarcoFalke added the label good first issue on Nov 8, 2017
11. MarcoFalke removed the label hacktoberfest on Nov 8, 2017
12. MarcoFalke removed the label good first issue on Nov 15, 2017
13. MarcoFalke added the label good first issue on Apr 11, 2018
14. MarcoFalke added the label Up for grabs on Apr 11, 2018
15. 
    glaksmono commented at 9:13 am on May 7, 2018: contributor

    Can I take a stab on this? @MarcoFalke @eklitzke

    WIP on https://github.com/bitcoin/bitcoin/pull/13183

16. laanwj closed this on May 9, 2018

    ---

17. laanwj referenced this in commit 9458b05f28 on May 9, 2018
18. MarcoFalke removed the label Up for grabs on May 9, 2018
19. MarcoFalke removed the label good first issue on May 9, 2018
20. TheArbitrator referenced this in commit d0ffee9e4c on Jun 7, 2021
21. MarcoFalke locked this on Sep 8, 2021

Contributors
eklitzke MarcoFalke jnewbery practicalswift laanwj glaksmono

Labels
Tests