The vcpkg tool has introduced a proper way to use manifests. This PR replaces the custom text file mechanism with the new manifest approach.
It is planned that vckpg manifests will include the ability to version dependencies in the future. Dependency versions would solve a number of issues that currently require workarounds with the appveyor CI.
0@@ -0,0 +1,19 @@
1+{
2+ "name": "bitcoin-core",
3+ "version-string": "0.20.99",
“End users can set these fields to whatever they want, but the name must be all lowercase characters.” From the Link in OP
So I’d suggest to set this to something not entangled with the bitcoin core version. Maybe just "1"?
1 to avoid any perception it relates to any other release or version numbers.
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
Reviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
11@@ -12,10 +12,9 @@ Quick Start
12 The minimal steps required to build Bitcoin Core with the msbuild toolchain are below. More detailed instructions are contained in the following sections.
13
14 ```
15-vcpkg install --triplet x64-windows-static berkeleydb boost-filesystem boost-multi-index boost-signals2 boost-test boost-thread libevent[thread] zeromq double-conversion
16-vcpkg integrate install
17+cd build_msvc
18 py -3 build_msvc\msvc-autogen.py
0py -3 msvc-autogen.py
31- Add-Content "C:\tools\vcpkg\triplets\$env:PLATFORM-windows-static.cmake" "set(VCPKG_BUILD_TYPE release)"
32- .\vcpkg install --triplet $env:PLATFORM-windows-static $env:PACKAGES.split() > $null
33- Write-Host "vcpkg packages installed successfully."
34- .\vcpkg integrate install
35- cd "$env:APPVEYOR_BUILD_FOLDER"
36+ cd $env:APPVEYOR_BUILD_FOLDER
"?
33-- Boost
34-- DoubleConversion
35-- libevent
36-- Qt5
37-- ZeroMQ
38+The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the `build_msvc/vcpkg.json` file. The `msbuild` project files are configured to automatically install the `vcpg` dependencies.
0The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the `build_msvc/vcpkg.json` file. The `msbuild` project files are configured to automatically install the `vcpkg` dependencies.
This looks good now. However can you follow up with:
Please squash and format the commit message with subject line and body
The vcpkg tool has introduced a proper way to use manifests, https://devblogs.microsoft.com/cppblog/vcpkg-accelerate-your-team-development-environment-with-binary-caching-and-manifests/. This PR replaces the custom text file mechanism with the new manifest approach.
It is planned that vckpg manifests will include the ability to version dependencies in the future. Dependency versions would solve a number of issues that currently require workarounds with the appveyor CI.
Set vcpkg manifest version to 1 to avoid any perception it's related to any release or other version numbering.
ACK 712f95d3324d02310dd468e7bfd1e1b0df432e77 - This is a nice simplification. I tested this in a Windows VM; packages were downloaded and installed automatically as required:
I’ve adjusted my commit message to have a better lead (subject) line. Was that the problem?
Thanks. Yes it was.
One other thought. Is there any Appveyor related cache me might need to nuke to make sure this is really working there as expected?
10@@ -11,25 +11,19 @@ environment:
11 QT_DOWNLOAD_HASH: '9a8c6eb20967873785057fdcd329a657c7f922b0af08c5fde105cc597dd37e21'
12 QT_LOCAL_PATH: 'C:\Qt5.9.8_x64_static_vs2019'
13 VCPKG_INSTALL_PATH: 'C:\tools\vcpkg\installed'
14- VCPKG_COMMIT_ID: 'f3f329a048eaff759c1992c458f2e12351486bc7'
15+ VCPKG_COMMIT_ID: '13590753fec479c5b0a3d48dd553dde8d49615fc'
master branch, checked that the dependencies work and used that commit. The vcpkg repo is a bit different to code only repos in that commits are made for the library port configs as well as the source code use to build the vcpkg executable.
23@@ -24,3 +24,4 @@ libtest_util/libtest_util.vcxproj
24 */Win32
25 libbitcoin_qt/QtGeneratedFiles/*
26 test_bitcoin-qt/QtGeneratedFiles/*
27+vcpkg_installed
33-- Boost
34-- DoubleConversion
35-- libevent
36-- Qt5
37-- ZeroMQ
38+The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the `build_msvc/vcpkg.json` file. The `msbuild` project files are configured to automatically install the `vcpkg` dependencies.
0The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the [`build_msvc/vcpkg.json`](https://github.com/bitcoin/bitcoin/tree/master/build_msvc/vcpkg.json) file. The `msbuild` project files are configured to automatically install the `vcpkg` dependencies.
One other thought. Is there any Appveyor related cache me might need to nuke to make sure this is really working there as expected?
Nope. All appveyor caching has already been removed. Predominantly due to nasty issues with the cached vcpkg dependencies getting messed up between different PR branches.
With this new manifest approach it might be feasible to re-introduce a cache of the vcpkg dependencies at some point, and save 20mins per build, but best to leave that until the manifest change is bedded down.
14+ "name": "libevent",
15+ "features": ["thread"]
16+ },
17+ "zeromq"
18+ ]
19+}
Approach ACK 712f95d3324d02310dd468e7bfd1e1b0df432e77, I’ve verified that changes comply MS docs:
msbuild time (at least on appveyor)
@MarcoFalke yes I see what you mean. It seems that build takes approximately 10 minutes longer subsequent to the change in this PR.
One possibility is the vcpkg dependencies may now be being built for both debug and release. Previously the line below restricted them to only release. I’ll do some testing.
0Add-Content "C:\tools\vcpkg\triplets\$env:PLATFORM-windows-static.cmake" "set(VCPKG_BUILD_TYPE release)"
Finally got around to running the build tests to compare the build durations using the original vcpkg install mechanism and the new vcpkg manifest mechanism.
The original way seems to be a few minutes faster but not a smoking gun for a 10 minute increase. Perhaps the manifest + the removal of the set(VCPKG_BUILD_TYPE release) could account for it. The VCPKG_BUILD_TYPE setting should be put back and I’ll create a PR for that.
Build runs with vcpkg installed from command line: Average duration approx. 25 minutes.
0Run 1:
1c:\Tools\vcpkg>vcpkg install --triplet x64-windows-static berkeleydb boost-filesystem boost-multi-index boost-process boost-signals2 boost-test boost-thread libevent[thread] zeromq double-conversion sqlite3 -> 14:12
2c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
3c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 10:37
4
5Run 2:
6c:\Tools\vcpkg>vcpkg install --triplet x64-windows-static berkeleydb boost-filesystem boost-multi-index boost-process boost-signals2 boost-test boost-thread libevent[thread] zeromq double-conversion sqlite3 -> 16:10
7c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
8c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 09:40
9
10Run 3:
11c:\Tools\vcpkg>vcpkg install --triplet x64-windows-static berkeleydb boost-filesystem boost-multi-index boost-process boost-signals2 boost-test boost-thread libevent[thread] zeromq double-conversion sqlite3 -> 15:29
12c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
13c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 09:36
14
15Run 4:
16c:\Tools\vcpkg>vcpkg install --triplet x64-windows-static berkeleydb boost-filesystem boost-multi-index boost-process boost-signals2 boost-test boost-thread libevent[thread] zeromq double-conversion sqlite3 -> 15:12
17c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
18c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 09:39
Build runs with vcpkg installed using manifest mechanism: Average duration approx. 28 minutes.
0Run 1:
1c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
2c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 26:58
3
4Run 2:
5c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
6c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 27:00
7
8Run 3:
9c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
10c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 27:15
11
12Run 4:
13c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
14c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 30:31
15
16Run 5:
17c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
18c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 30:31
19
20Run 6:
21c:\Dev\github\sipsorcery_bitcoin\build_msvc>msvc-autogen.py
22c:\Dev\github\sipsorcery_bitcoin\build_msvc>msbuild /m bitcoin.sln /p:Configuration=Release /p:Platform=x64 /t:clean,build -> 24:51