Appveyor improvement - text file for vcpkg package list #17416

1. 
   sipsorcery commented at 1:42 pm on November 8, 2019: member

   #17364 attempted to save a couple of minutes by skipping the vcpkg steps if the vcpkg install directory was already cached.

   The discussion in #15382 highlights the approach used in #17364 does not accommodate adding a new package.

   This PR improves the approach to individually check whether each vcpg package is installed rather than checking for the existence of the vcpkg install directory.

   This PR moves the list of required vcpkg packages into a separate file and uses changes to that file to invalidate the appveyor cache. Whenever the cache is invalidated the vcpkg sources will be updated, the vcpkg binary built and the required packages installed from the latest port files.

2. fanquake added the label Windows on Nov 8, 2019
3. fanquake added the label Tests on Nov 8, 2019
4. 
   MarcoFalke commented at 1:48 pm on November 8, 2019: member

   Would it be possible and easier to put the install script into a separate file and invalidate the cache when that script changes?
5. 
   sipsorcery commented at 2:05 pm on November 8, 2019: member

   @MarcoFalke yep I’m pretty sure that would be possible. My concern would be starting down the path of an elaborate multi-script build config. That could get scary for us Windows devs.

   Is the main goal to have a way of triggering updates without having to touch the appveyor config? What do you think about something simpler like putting the list of packages in a separate text file?

6. 
   MarcoFalke commented at 2:21 pm on November 8, 2019: member

   The installation step is already a separate (one-line) script in build_msvc/README.md, so moving it to a file (or just the list of packages to a file) wouldn’t make it worse. Also, it moves the hard coded required packages names to a single place.
7. sipsorcery force-pushed on Nov 8, 2019
8. sipsorcery force-pushed on Nov 8, 2019
9. in .appveyor.yml:9 in d34028f1f1 outdated

    5@@ -6,39 +6,50 @@ platform: x64
    6 clone_depth: 5
    7 environment:
    8   APPVEYOR_SAVE_CACHE_ON_ERROR: true
    9-  CLCACHE_SERVER: 1
   10-  PACKAGES: berkeleydb boost-filesystem boost-multi-index boost-signals2 boost-test boost-thread libevent openssl rapidcheck zeromq double-conversion
   11+  CLCACHE_SERVER: 1 
   

   ---

   ---

   

   MarcoFalke commented at 6:20 pm on November 8, 2019:

   0  CLCACHE_SERVER: 1
   

10. in .appveyor.yml:35 in d34028f1f1 outdated

    38 - ps: |
    39+      $env:PACKAGES = Get-Content -Path build_msvc\vcpkg-packages.txt
    40       cd c:\tools\vcpkg
    41-      if(!(Test-Path -Path ($env:VCPKG_INSTALL_PATH))) {
    42+      $isPackageMissing = $false
    43+      Foreach($reqdPackage in $env:PACKAGES.split()) {
    

    ---

    ---

    

    MarcoFalke commented at 6:21 pm on November 8, 2019:

    Is this loop still required? vcpkg\installed will be cleared when a package is added or removed to the text file
11. in build_msvc/vcpkg-packages.txt:1 in d34028f1f1 outdated

    0@@ -0,0 +1 @@
    1+berkeleydb boost-filesystem boost-multi-index boost-signals2 boost-test boost-thread libevent openssl rapidcheck zeromq double-conversion
    

    ---

    ---

    

    MarcoFalke commented at 6:22 pm on November 8, 2019:

    nit: Can the install step in build_msvc/README.md be updated to use this?
12. 
    DrahtBot commented at 9:56 pm on November 8, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #15382 (util: add runCommandParseJSON by Sjors)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

13. fanquake added the label Waiting for author on Nov 8, 2019
14. sipsorcery renamed this:
    Appveyor improvement - adds check for all required vcpkg packages
    WIP: Appveyor improvement - adds check for all required vcpkg packages
    on Nov 9, 2019
15. sipsorcery force-pushed on Nov 10, 2019
16. sipsorcery force-pushed on Nov 10, 2019
17. sipsorcery force-pushed on Nov 10, 2019
18. Moves vcpkg list to a text file and updates the appveyor job and readme to use it. 29eb039252
19. sipsorcery force-pushed on Nov 10, 2019
20. sipsorcery renamed this:
    WIP: Appveyor improvement - adds check for all required vcpkg packages
    Appveyor improvement - adds check for all required vcpkg packages
    on Nov 10, 2019
21. sipsorcery renamed this:
    Appveyor improvement - adds check for all required vcpkg packages
    Appveyor improvement - text file for vcpkg package list
    on Nov 10, 2019
22. 
    sipsorcery commented at 3:52 pm on November 10, 2019: member

    This PR is now ready for review. Title and description have been updated.
23. fanquake removed the label Waiting for author on Nov 10, 2019
24. 
    MarcoFalke commented at 2:17 am on November 11, 2019: member

    ACK 29eb039252f35a3bb58b0027f172cfe3d0c79867
25. MarcoFalke referenced this in commit a6f6333ba2 on Nov 11, 2019
26. MarcoFalke merged this on Nov 11, 2019
27. MarcoFalke closed this on Nov 11, 2019

    ---

28. sipsorcery referenced this in commit 57ab95cf81 on Jan 5, 2020
29. fanquake referenced this in commit b0f9b8e648 on Jan 5, 2020
30. laanwj referenced this in commit bb123c6527 on Jan 8, 2020
31. MarkLTZ referenced this in commit 2e6abb1507 on Mar 13, 2020
32. MarkLTZ referenced this in commit ece88bae51 on Mar 13, 2020
33. linuxsh2 referenced this in commit 9a52ea9ee4 on Sep 16, 2021
34. Munkybooty referenced this in commit 861ef02929 on Dec 9, 2021
35. Munkybooty referenced this in commit fc6f77f098 on Dec 9, 2021
36. Munkybooty referenced this in commit e32f0d5d86 on Dec 9, 2021
37. MarcoFalke locked this on Dec 16, 2021

Contributors
sipsorcery MarcoFalke DrahtBot

Labels
Windows Tests