Updates to appveyor config for VS2019 and Qt5.9.8 + msvc project fixes #17364

pull sipsorcery wants to merge 1 commits into bitcoin:master from sipsorcery:vs2019_oct31 changing 5 files +43 −21
  1. sipsorcery commented at 2:23 pm on November 4, 2019: member

    Updates to appveyor config:

    • Update build image from Visual Studio 2017 to Visual Studio 2019.
    • Updated Qt static library from Qt5.9.7 to Qt5.9.8.
    • Added commands to update vcpkg port files (this does not update already installed packages).
    • Updated vcpkg package list as per #17309.
    • Removed commands setting common project file options. Now done via common.init.vcxproj include.
    • Changed msbuild verbosity from normal to quiet. Normal writes a LOT of logs and impacts appveyor job duration.

    Updates to msvc project configs:

    • Updated platform toolset from v141 to v142.
    • Updated Qt static library from Qt5.9.7 to Qt5.9.8.
    • Added ignore for linker warning building bitcoin-qt program.
    • Added missing util/str.cpp class file to test_bitcoin project file.

    In order for an existing appveyor job based on the new config to work the cache must be purged. The steps to do this are shown below. The specific appveyor project path will need to be adjusted.

    0export APPVEYOR_TOKEN="<your-api-token>"
1curl -H "Authorization: Bearer $APPVEYOR_TOKEN" -X DELETE https://ci.appveyor.com/api/projects/bitcoin/bitcoin-9ql6k/buildcache
  2. laanwj commented at 2:25 pm on November 4, 2019: member
    Concept ACK
  3. fanquake added the label Windows on Nov 4, 2019
  4. in .appveyor.yml:58 in 4a8ecf1cc4 outdated 
    65 - cmd: src\test_bitcoin.exe -k stdout -e stdout 2> NUL
66 - cmd: src\bench_bitcoin.exe -evals=1 -scaling=0 > NUL
67 - ps:  python test\util\bitcoin-util-test.py
68 - cmd: python test\util\rpcauth-test.py
69-- cmd: python test\functional\test_runner.py --ci --quiet --combinedlogslen=4000 --failfast
70+# Fee estimation test fialing on appveyor with: WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted.
    fanquake commented at 3:11 pm on November 4, 2019:
    nit: s/fialing/failing
  5. fanquake commented at 3:12 pm on November 4, 2019: member
    Concept ACK - There’s some related discussion in #17309 and these changes reflect that. i.e no longer needing OpenSSL as part of the Qt build and adding missing vcpkg packages.
  6. sipsorcery force-pushed on Nov 4, 2019
  7. Sjors commented at 4:17 pm on November 4, 2019: member
    Concept ACK. Works for me if it works for AppVeyor :-)
  8. DrahtBot commented at 5:46 pm on November 4, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #15382 (util: add runCommandParseJSON by Sjors)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  9. in .appveyor.yml:22 in 394d6740d6 outdated 
    26 install:
27-- cmd: pip install --quiet git+https://github.com/frerich/clcache.git@v4.2.0
28 # Disable zmq test for now since python zmq library on Windows would cause Access violation sometimes.
29 # - cmd: pip install zmq
30 - cmd: echo set(VCPKG_BUILD_TYPE release) >> C:\tools\vcpkg\triplets\%PLATFORM%-windows-static.cmake
31+- cmd: cd c:\tools\vcpkg
    fanquake commented at 8:07 pm on November 4, 2019:
    You mentioned here that updating vcpkg for each build would likely be infeasible. Is that no-longer the case, or am I mis-understanding what this does?
    sipsorcery commented at 9:54 pm on November 4, 2019:

    I checked my statement by running the appveyor job with a pull of the vcpkg repo as mentioned in the articles you referenced and I was incorrect about the time it would take.

    The reason being that updating and rebuilding the vcpkg program does not affect already installed packages. In my testing it only added about 2 minutes onto the build time. I wrongly assumed that updating vcpkg would invalidate the existing packages and force them to be re-installed.

    For this PR the vcpkg source does have to be updated to get the latest port files otherwise the build fails (didn’t track down why). After the first successful build thevcpkg update steps can be disabled to save the 2 minutes. It would need to be some kind of conditional disable so that anyone forking the repo still gets a working appveyor config. I can look into it if consensus is the small additional complexity justifies saving 2 minutes.

  10. MarcoFalke commented at 9:19 pm on November 4, 2019: member
    This build took 1h14m. Other builds take typically between 33 min and 49 min. Are you sure caching has no effect on the run time?
  11. sipsorcery commented at 10:05 pm on November 4, 2019: member

    This build took 1h14m. Other builds take typically between 33 min and 49 min. Are you sure caching has no effect on the run time?

    That does warrant some further investigation. I’ll look into it further.

    My build times performed sequentially were as below. They seemed fairly consistent even with appveyor’s unpredictability. But given your results I’ll see if maybe I missed something with clcache. It would certainly be nice to speed things up.

     0Disabled clcache
 151 min 49 sec
 253 min 46 sec
 352 min 23 sec
 453 min 16 sec
 5Failed after 47 min 38 sec
 644 min 4 sec
 744 min 2 sec
 8
 9Enabled clcache
1053 min 13 sec
1152 min 26 sec
1253 min 5 sec
1354 min 45 sec
14Failed after 46 min 26 sec
1553 min 15 sec
1652 min 34 sec
1753 min 9 sec
1853 min 54 sec
19
20Disabled clcache
2157 min 9 sec
22Failed after 1 hr
2359 min 25 sec
2456 min 33 sec
2554 min 55 sec
  12. sipsorcery force-pushed on Nov 5, 2019
  13. sipsorcery force-pushed on Nov 5, 2019
  14. MarcoFalke commented at 11:19 pm on November 5, 2019: member
    Maybe clcache can be removed in a follow-up? Splitting the change into two commits would also help with review.
  15. sipsorcery closed this on Nov 6, 2019
  16. sipsorcery force-pushed on Nov 6, 2019
  17. sipsorcery commented at 10:21 am on November 6, 2019: member
    @MarcoFalke D’accord. Have put back the clcache steps. Now the changes in this PR are focused on getting the msvc build working with VS2019 and Qt5.9.8.
  18. sipsorcery reopened this on Nov 6, 2019
  19. sipsorcery force-pushed on Nov 6, 2019
  20. sipsorcery force-pushed on Nov 6, 2019
  21. sipsorcery force-pushed on Nov 6, 2019
  22. Updated appveyor config: 
     - Update build image from Visual Studio 2017 to Visual Studio 2019.
 - Updated Qt static library from Qt5.9.7 to Qt5.9.8.
 - Added commands to update vcpkg port files (this does not update already installed packages).
 - Updated vcpkg package list as per #17309.
 - Removed commands setting common project file options. Now done via common.init.vcxproj include.
 - Changed msbuild verbosity from normal to quiet. Normal rights a LOT of logs and impacts appveyor job duration.
Updated msvc project configs:
 - Updated platform toolset from v141 to v142.
 - Updated Qt static library from Qt5.9.7 to Qt5.9.8.
 - Added ignore for linker warning building bitcoin-qt program.
 - Added missing util/str.cpp class file to test_bitcoin project file.
    3c84deebaa
  23. sipsorcery force-pushed on Nov 6, 2019
  24. sipsorcery commented at 8:21 am on November 7, 2019: member

    @MarcoFalke the appveyor job will need to have the cache cleared before this PR will build successfully. The Curl command below shows how to do it. You’ll need to replace the project URL.

    All the appveyor builds are failing at the moment due to a new utility class being added to the test_bitcoin program. This PR fixes that.

    0export APPVEYOR_TOKEN="<your-api-token>"
1curl -H "Authorization: Bearer $APPVEYOR_TOKEN" -X DELETE https://ci.appveyor.com/api/projects/sipsorcery/bitcoin-72c17/buildcache
  25. ryanofsky approved
  26. ryanofsky commented at 11:46 am on November 7, 2019: member

    Non-expert code review ACK 3c84deebaa311155ccc7565d09525041eed9747c.

    Options changes look as expected and few lines of shell code added seem to do what’s described in comments. I haven’t looked into details of how appveyor build works, so I probably wouldn’t notice any bugs, but the configuration looks reasonable and is very well commented

    @MarcoFalke the appveyor job will need to have the cache cleared before this PR will build successfully. The Curl command below shows how to do it. You’ll need to replace the project URL.

    Maybe we should have some permanent documentation that says how and when to run this command, if we don’t already

  27. in .appveyor.yml:19 in 3c84deebaa 
    19+  QT_DOWNLOAD_HASH: 'f285cbb02bec3b3f3cc2621e3fa7d5edf0d6a66fa30c57859e583acda954ea80'
20+  QT_LOCAL_PATH: 'C:\Qt5.9.8_x64_static_vs2019'
21+  VCPKG_INSTALL_PATH: 'C:\tools\vcpkg\installed'
22 cache:
23 - C:\tools\vcpkg\installed
24 - C:\Users\appveyor\clcache -> .appveyor.yml, build_msvc\**, **\Makefile.am, **\*.vcxproj.in
    MarcoFalke commented at 1:32 pm on November 7, 2019:

    Why is this cache not cleared?

    https://www.appveyor.com/docs/build-cache/#cleaning-up-cache

  28. MarcoFalke commented at 1:32 pm on November 7, 2019: member 
    0$ curl -H "Authorization: Bearer $APPVEYOR_TOKEN" -X DELETE https://ci.appveyor.com/api/projects/bitcoin/bitcoin-9ql6k/buildcache
1{"message":"Account not found or access denied."}
  29. DrahtBot commented at 1:33 pm on November 7, 2019: member 
    0$ curl -H "Authorization: Bearer $APPVEYOR_TOKEN" -X DELETE https://ci.appveyor.com/api/projects/DrahtBot/bitcoin/buildcache && echo 'Done?'
1Done?
  30. sipsorcery commented at 1:46 pm on November 7, 2019: member
    The cache purge worked. The appveyor job is now installing the vcpkg’s from scratch. The first job will probably fail due to hitting the 1hr time limit. Installing the vcpkg’s (~20mins) + build (~40mins) + test (~15mins) is now too long to do all at once. After the first attempt the vcpkg packages should be cached and subsequent job runs should work.
  31. MarcoFalke commented at 1:54 pm on November 7, 2019: member
    Ok. Could you please update the OP to reflect the current state of this pull request?
  32. sipsorcery renamed this:
    Updated appveyor config for VS2019 and Qt5.9.8
    Updates to appveyor config for VS2019 and Qt5.9.8 + msvc project fixes
    on Nov 7, 2019
  33. sipsorcery commented at 2:00 pm on November 7, 2019: member
    OP and title updated.
  34. MarcoFalke referenced this in commit 46e0e27639 on Nov 7, 2019
  35. MarcoFalke merged this on Nov 7, 2019
  36. MarcoFalke closed this on Nov 7, 2019
  37. MarcoFalke commented at 2:55 pm on November 7, 2019: member
    Just wanted to note that this reverts 0646ca5ea2541d4da324d9aff7de422c6daa0ef9, which now needs to be applied manually on failing builds
  38. fanquake commented at 3:44 pm on November 7, 2019: member

    post-merge ACK 3c84deebaa311155ccc7565d09525041eed9747c

    I’ve just completed a qt build on Windows 10 with these changes:

    windows_17364

  39. Sjors commented at 4:42 pm on November 7, 2019: member

    The commit message was incorrectly formatted:

    0commit 3c84deebaa311155ccc7565d09525041eed9747c
1Author: Aaron Clauson <aaron@sipsorcery.com>
2Date:   Wed Nov 6 22:02:16 2019 +0000
3
4    Updated appveyor config:
5     - Update build image from Visual Studio 2017 to Visual Studio 2019.

    There should be a blank line after the description. Otherwise things like git rebase -i get confused (though not broken), e.g.:

    0pick 037f2405f2 Updated appveyor config:  - Update build image from Visual Studio 2017 to Visual Studio 2019.  - Updated Qt static library from Qt5.9.7 to Qt5.9.8.  - Added commands to update vcpkg port files (this does not update already installed packages).  - Updated vcpkg package list as per [#17309](/bitcoin-bitcoin/17309/).  - Removed commands setting common project file options. Now done via common.init.vcxproj include.  - Changed msbuild verbosity from normal to quiet. Normal rights a LOT of logs and impacts appveyor job duration. Updated msvc project configs:  - Updated platform toolset from v141 to v142.  - Updated Qt static library from Qt5.9.7 to Qt5.9.8.  - Added ignore for linker warning building bitcoin-qt program.  - Added missing util/str.cpp class file to test_bitcoin project file.
1pick 8a5d0954a3 configure: add ax_boost_process
2pick 25405cc429 [build] msvc: add boost::process
  40. MarcoFalke commented at 5:14 pm on November 7, 2019: member
    A linter makes sense to catch these
  41. MarcoFalke commented at 7:54 pm on November 7, 2019: member

    It looks like compile time is back at about 50 minutes (like before): https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/28694137

    Ok, this one is 57 minutes, but close enough.

  42. sipsorcery deleted the branch on Nov 7, 2019
  43. fanquake referenced this in commit 2fb6140d58 on Nov 7, 2019
  44. MarcoFalke referenced this in commit a6f6333ba2 on Nov 11, 2019
  45. sipsorcery referenced this in commit c214c70b01 on Jan 5, 2020
  46. fanquake referenced this in commit cd7b3b254a on Jan 5, 2020
  47. laanwj referenced this in commit bb123c6527 on Jan 8, 2020
  48. MarkLTZ referenced this in commit 2e6abb1507 on Mar 13, 2020
  49. MarkLTZ referenced this in commit ece88bae51 on Mar 13, 2020
  50. jasonbcox referenced this in commit 8809550033 on Nov 5, 2020
  51. hebasto referenced this in commit d1267fdbb0 on Sep 4, 2021
  52. MarcoFalke referenced this in commit 0ebd88fe0b on Sep 5, 2021
  53. arnabsen1729 referenced this in commit 374108550a on Sep 6, 2021
  54. sidhujag referenced this in commit 876f6900bc on Sep 7, 2021
  55. linuxsh2 referenced this in commit 9a52ea9ee4 on Sep 16, 2021
  56. Munkybooty referenced this in commit 861ef02929 on Dec 9, 2021
  57. Munkybooty referenced this in commit fc6f77f098 on Dec 9, 2021
  58. Munkybooty referenced this in commit e32f0d5d86 on Dec 9, 2021
  59. DrahtBot locked this on Dec 16, 2021


sipsorcery laanwj fanquake Sjors DrahtBot MarcoFalke ryanofsky

Labels
Windows

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-29 01:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me