Just like all the other CI configs, the Windows one should print a single and readable build failure at the end.
Also, includes a bunch of Windows CI refactors.
Just like all the other CI configs, the Windows one should print a single and readable build failure at the end.
Also, includes a bunch of Windows CI refactors.
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
See the guideline for information on the review process.
| Type | Reviewers |
|---|---|
| ACK | hebasto, m3dwards, willcl-ark |
| Concept ACK | l0rinc |
If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.
Reviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
17+ return subprocess.run(cmd, **kwargs)
18+ except Exception as e:
19+ sys.exit(str(e))
20+
21+
22+GENERATE_OPTIONS = {
In fae5e78e1fe
As we are moving this from ci yaml config, which I thoroughly agree with, I woudl prefer to see this go into CMakePresets.json instead, where it can be (re-)used by anybody without needing to use (and likely modify) this python script?
It’s possible to inherit inside of a preset file, so this might look something like (untested etc.):
0diff --git a/.github/ci-windows.py b/.github/ci-windows.py
1index adf5437f870..5855e60c915 100755
2--- a/.github/ci-windows.py
3+++ b/.github/ci-windows.py
4@@ -19,21 +19,9 @@ def run(cmd, **kwargs):
5 sys.exit(str(e))
6
7
8-GENERATE_OPTIONS = {
9- "standard": [
10- "-DBUILD_BENCH=ON",
11- "-DBUILD_KERNEL_LIB=ON",
12- "-DBUILD_UTIL_CHAINSTATE=ON",
13- "-DWERROR=ON",
14- ],
15- "fuzz": [
16- "-DVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON",
17- "-DVCPKG_MANIFEST_FEATURES=wallet",
18- "-DBUILD_GUI=OFF",
19- "-DWITH_ZMQ=OFF",
20- "-DBUILD_FOR_FUZZING=ON",
21- "-DWERROR=ON",
22- ],
23+CI_PRESETS = {
24+ "standard": "vs2022-ci-standard",
25+ "fuzz": "vs2022-ci-fuzz",
26 }
27
28
29@@ -48,11 +36,10 @@ def generate(ci_type):
30 "cmake",
31 "-B",
32 "build",
33- "-Werror=dev",
34 "--preset",
35- "vs2022",
36+ CI_PRESETS[ci_type],
37 f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file}",
38- ] + GENERATE_OPTIONS[ci_type]
39+ ]
40 run(command)
41
42
43@@ -72,7 +59,7 @@ def build():
44
45 def main():
46 parser = argparse.ArgumentParser(description="Utility to run Windows CI steps.")
47- parser.add_argument("ci_type", choices=GENERATE_OPTIONS, help="CI type to run.")
48+ parser.add_argument("ci_type", choices=CI_PRESETS, help="CI type to run.")
49 steps = ["generate", "build"]
50 parser.add_argument("step", choices=steps, help="CI step to perform.")
51 args = parser.parse_args()
52diff --git a/CMakePresets.json b/CMakePresets.json
53index ae9d06dafa4..fa5b8b87b59 100644
54--- a/CMakePresets.json
55+++ b/CMakePresets.json
56@@ -18,6 +18,30 @@
57 "WITH_ZMQ": "ON"
58 }
59 },
60+ {
61+ "name": "vs2022-ci-standard",
62+ "inherits": "vs2022",
63+ "errors": {"dev": true},
64+ "cacheVariables": {
65+ "BUILD_BENCH": "ON",
66+ "BUILD_KERNEL_LIB": "ON",
67+ "BUILD_UTIL_CHAINSTATE": "ON",
68+ "WERROR": "ON"
69+ }
70+ },
71+ {
72+ "name": "vs2022-ci-fuzz",
73+ "inherits": "vs2022",
74+ "errors": {"dev": true},
75+ "cacheVariables": {
76+ "VCPKG_MANIFEST_NO_DEFAULT_FEATURES": "ON",
77+ "VCPKG_MANIFEST_FEATURES": "wallet",
78+ "BUILD_GUI": "OFF",
79+ "WITH_ZMQ": "OFF",
80+ "BUILD_FOR_FUZZING": "ON",
81+ "WERROR": "ON"
82+ }
83+ },
84 {
85 "name": "vs2022-static",
86 "displayName": "Build using 'Visual Studio 17 2022' generator and 'x64-windows-static' triplet",
IMO this takes the same improvement to the codebase, but makes it much more re-usable outside of the CI.
Right. but I am not entirely sure if this is sufficient, given that the toolchain file will also be needed. I guess it can just be passed manually, if someone wants to do that locally. Also, I think there was a discussion about where to put the ci presets (should they live in the main presets file), can they even live in a different file, …?
My preference would be leave this as-is for now and then tackle it in a follow-up. After all, the code change is just a trivial move-only. The conceptual review is the harder part here.
Concept ACK.
Two nice cleanups in one!
I’m heavily in favour of starting to seperate CI configurations out from being embedded in random (GitHub-specific) yaml files, but left a suggestion to move this all the way to the “final logical destination” of a cmake preset.
(FYI I have a draft branch of ~ what such a completed file might look like here)
The verbose build logging stuff all looks OK
37+}
38+
39+
40+def generate(ci_type):
41+ toolchain_file = os.path.join(
42+ os.environ.get("VCPKG_INSTALLATION_ROOT"),
Better to fail on a missing key here. It also make typecheckers more happy, as .get() returns str | None, and silently passsing None here is not correct.
0 os.environ["VCPKG_INSTALLATION_ROOT"],
ACK fa27bc54a358e991e546ae463150fbb7a4dc4572
Left a nit, which you could address if you agree and touch again.
Looks like you may, as a load of unrelated commits seem to have landed in here?
edit: ah I see, we are testing the build failure.
🚧 At least one of the CI tasks failed.
Task ASan + LSan + UBSan + integer: https://github.com/bitcoin/bitcoin/actions/runs/21668109400/job/62468721967
LLM reason (✨ experimental): Compilation failed: unknown type name ‘ElapseTime’ in util_time.cpp during bench_bitcoin build.
Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:
Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.
Leave a comment here, if you need help tracking down a confusing failure.
Example failure, prior command https://github.com/bitcoin/bitcoin/actions/runs/21670047954/job/62475308516?pr=34500#step:11:1008:
0...
1 util_time.cpp
2 bitcoin-qt.cpp
3 uritests.cpp
4 util.cpp
5 addressbooktests.cpp
6D:\a\bitcoin\bitcoin\src\bench\util_time.cpp(18,5): error C2065: 'ElapseTime': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
7D:\a\bitcoin\bitcoin\src\bench\util_time.cpp(18,16): error C2146: syntax error: missing ';' before identifier 'elapse_time' [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
8D:\a\bitcoin\bitcoin\src\bench\util_time.cpp(18,16): error C2065: 'elapse_time': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
9 Creating library D:/a/bitcoin/bitcoin/build/lib/Release/bitcoin-qt.lib and object D:/a/bitcoin/bitcoin/build/lib/Release/bitcoin-qt.exp
10 bitcoin-chainstate.cpp
11 test_kernel.cpp
12 bitcoin-qt.vcxproj -> D:\a\bitcoin\bitcoin\build\bin\Release\bitcoin-qt.exe
13 wallettests.cpp
14 mocs_compilation_Release.cpp
15 wallet_test_fixture.cpp
16 Creating library D:/a/bitcoin/bitcoin/build/lib/Release/bitcoin-chainstate.lib and object D:/a/bitcoin/bitcoin/build/lib/Release/bitcoin-chainstate.exp
17 bitcoin-chainstate.vcxproj -> D:\a\bitcoin\bitcoin\build\bin\Release\bitcoin-chainstate.exe
18 Creating library D:/a/bitcoin/bitcoin/build/lib/Release/test_kernel.lib and object D:/a/bitcoin/bitcoin/build/lib/Release/test_kernel.exp
19 test_kernel.vcxproj -> D:\a\bitcoin\bitcoin\build\bin\Release\test_kernel.exe
20 test_bitcoin-qt.vcxproj -> D:\a\bitcoin\bitcoin\build\bin\Release\test_bitcoin-qt.exe
21Build failure. Verbose build follows.
22+ cmake --build build --config Release -j1 --verbose
23...
And the tail of the failure afterwards:
0Build FAILED.
1
2"D:\a\bitcoin\bitcoin\build\ALL_BUILD.vcxproj" (default target) (1) ->
3"D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj" (default target) (4) ->
4(ClCompile target) ->
5 D:\a\bitcoin\bitcoin\src\bench\util_time.cpp(18,5): error C2065: 'ElapseTime': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
6 D:\a\bitcoin\bitcoin\src\bench\util_time.cpp(18,16): error C2146: syntax error: missing ';' before identifier 'elapse_time' [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
7 D:\a\bitcoin\bitcoin\src\bench\util_time.cpp(18,16): error C2065: 'elapse_time': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
8 D:\a\bitcoin\bitcoin\src\bench\wallet_balance.cpp(36,5): error C2065: 'ElapseTime': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
9 D:\a\bitcoin\bitcoin\src\bench\wallet_balance.cpp(36,16): error C2146: syntax error: missing ';' before identifier 'elapse_time' [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
10 D:\a\bitcoin\bitcoin\src\bench\wallet_balance.cpp(36,16): error C2065: 'elapse_time': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\bench\bench_bitcoin.vcxproj]
11
12
13"D:\a\bitcoin\bitcoin\build\ALL_BUILD.vcxproj" (default target) (1) ->
14"D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj" (default target) (35) ->
15 D:\a\bitcoin\bitcoin\src\test\banman_tests.cpp(20,5): error C2065: 'ElapseTime': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj]
16 D:\a\bitcoin\bitcoin\src\test\banman_tests.cpp(20,16): error C2146: syntax error: missing ';' before identifier 'elapse_time' [D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj]
17 D:\a\bitcoin\bitcoin\src\test\banman_tests.cpp(20,16): error C2065: 'elapse_time': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj]
18 D:\a\bitcoin\bitcoin\src\test\addrman_tests.cpp(998,5): error C2065: 'ElapseTime': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj]
19 D:\a\bitcoin\bitcoin\src\test\addrman_tests.cpp(998,16): error C2146: syntax error: missing ';' before identifier 'elapse_time' [D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj]
20 D:\a\bitcoin\bitcoin\src\test\addrman_tests.cpp(998,16): error C2065: 'elapse_time': undeclared identifier [D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj]
21 D:\a\bitcoin\bitcoin\src\test\addrman_tests.cpp(999,5): error C3861: 'elapse_time': identifier not found [D:\a\bitcoin\bitcoin\build\src\test\test_bitcoin.vcxproj]
22
23 0 Warning(s)
24 13 Error(s)
25
26Time Elapsed 00:00:32.37
27
28Command '['cmake', '--build', 'build', '--config', 'Release', '-j1', '--verbose']' returned non-zero exit status 1.
29Error: Process completed with exit code 1.
I’m heavily in favour of starting to seperate CI configurations out from being embedded in random (GitHub-specific) yaml files
Concept ACK, I really hate those!
ACK fa52680c6a49edd2e8251ee7e298a716ca7c7f6f
range-diff LGTM:
0❯ git range-diff fa27bc54a35...fa52680c6a4
11: fae5e78e1fe ! 1: fabf6da29a6 ci: Refactor Windows CI into script
2 @@ .github/ci-windows.py (new)
3 +
4 +def generate(ci_type):
5 + toolchain_file = os.path.join(
6 -+ os.environ.get("VCPKG_INSTALLATION_ROOT"),
7 ++ os.environ["VCPKG_INSTALLATION_ROOT"],
8 + "scripts",
9 + "buildsystems",
10 + "vcpkg.cmake",
112: fa3ef3940cb = 2: 0000338cce4 ci: [refactor] Add .github/ci-windows.py build step
123: fa27bc54a35 ! 3: fa52680c6a4 ci: Print verbose Windows CI build failure
13 @@ .github/ci-windows.py: def build():
14 "--config",
15 "Release",
16 ]
17 - run(command)
18 +- run(command)
19 + if run(command + ["-j", str(os.process_cpu_count())], check=False).returncode != 0:
20 + print("Build failure. Verbose build follows.")
21 + run(command + ["-j1", "--verbose"])
22 ```
This makes it easier to:
* Run the exact command of any CI type and step locally
* Re-Run older CI tasks on GHA and using the latest merged config.
(.github/ci-windows.py is merged with master on re-runs, but
.github/workflows/ci.yml is NOT)
Also, writing it in Python has benefits:
* Any developer (even non-Windows ones) can read and modify the script.
* Python is already required for tests, so no new dependency is needed.
Note, the use of process_cpu_count() is intentional. It was only added
in Python 3.13, according to
https://docs.python.org/3/library/os.html#os.process_cpu_count .
However, Python 3.13 is also the minimum required version on Windows,
according to
https://github.com/bitcoin/bitcoin/issues/29897#issuecomment-2940318094
to avoid intermittent test failures.
Went ahead and moved all build+checks into the python script. This should make it easier to reproduce the CI locally.
edit: I’ve compared the logs against master and everything looked good to me
ACK fa361d1a895
The latest updates here look OK, although perhaps the PR title/description could be updated to reflect a little more adding all these in?
I did not test running any of this locally on Windows, but the python code LGTM and the CI seems happy.
The latest updates here look OK, although perhaps the PR title/description could be updated to reflect a little more adding all these in?
I’d love to, but GitHub is a broken mess. The pages are corrupt and load slow, and I can’t find the edit button anymore. :shrug:
38+}
39+
40+
41+def generate(ci_type):
42+ toolchain_file = os.path.join(
43+ os.environ["VCPKG_INSTALLATION_ROOT"],
The VCPKG_INSTALLATION_ROOT environment variable is specific to GHA images.
It would be helpful if it would be possible to run the script locally where only VCPKG_ROOT is available, which is used now in presets.
I hit this running locally and simply using VCPKG_ROOT instead didn’t work for me.
Did you pull the latest branch? I don’t think you need to set anything there.
Do the --preset work outside of CI for you?
50+ "-B",
51+ "build",
52+ "-Werror=dev",
53+ "--preset",
54+ "vs2022",
55+ f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file}",
The link doesn’t work for me, I guess GitHub is fully broken :shrug:
Though, I think it is fixed with the last push?
Looks like you have dropped
-DCMAKE_TOOLCHAIN_FILEcompletely now, this now works for me locally but I’m not sure why this was introduced for CI and what the consequences of dropping it are. @hebasto ??
It is not dropped entirely:https://github.com/bitcoin/bitcoin/blob/54bd49c7e3c7de1fdb744d118abd4f12e7dafcd8/CMakePresets.json#L14
58+ "--config",
59+ "Release",
60+ ]
61+ if run(command + ["-j", str(os.process_cpu_count())], check=False).returncode != 0:
62+ print("Build failure. Verbose build follows.")
63+ run(command + ["-j1", "--verbose"])
-j1 is currently ineffective due to the unconditional use of the Multi-ToolTask scheduler here:https://github.com/bitcoin/bitcoin/blob/54bd49c7e3c7de1fdb744d118abd4f12e7dafcd8/CMakeLists.txt#L287-L289
To resolve this, we can make the setting conditional so it can be overridden. Here is the proposed patch (taken from Professional CMake: A Practical Guide 22nd Edition):
0--- a/CMakeLists.txt
1+++ b/CMakeLists.txt
2@@ -286,7 +286,9 @@ if(WIN32)
3 )
4 # Improve parallelism in MSBuild.
5 # See: https://devblogs.microsoft.com/cppblog/improved-parallelism-in-msbuild/.
6- list(APPEND CMAKE_VS_GLOBALS "UseMultiToolTask=true")
7+ if(NOT CMAKE_VS_GLOBALS MATCHES "(^|;)UseMultiToolTask=")
8+ list(APPEND CMAKE_VS_GLOBALS UseMultiToolTask=true)
9+ endif()
10 endif()
11
12 if(MINGW)
This allows passing -DCMAKE_VS_GLOBALS="UseMultiToolTask=false" during configuration to disable it.
-DCMAKE_VS_GLOBALS="UseMultiToolTask=false" during the initial configuration would be sufficient.
Thx, I pushed it, but I wonder:
Thx, I pushed it, but I wonder:
- What it the CI end-to-end runtime difference? How much is it worse?
Let’s see…
- According to https://cmake.org/cmake/help/latest/variable/CMAKE_VS_GLOBALS.html, the variable is meant to be set by a toolchain file, but I don’t have insight into your book, so maybe this is fine.
The toolchain file is provided by vcpkg. We modify CMAKE_VS_GLOBALS in a non-invasive way before adding any new build targets, which should be safe.
Thx, I pushed it, but I wonder:
- What it the CI end-to-end runtime difference? How much is it worse?
Let’s see…
For the “fuzz” job build step: 14m 19s –> 19m 2s.
348- if: matrix.job-type == 'fuzz'
349- working-directory: build
350+ - name: Run tests
351 env:
352- BITCOINFUZZ: '${{ github.workspace }}\build\bin\Release\fuzz.exe'
353+ TEST_RUNNER_EXTRA: "--timeout-factor=${{ env.TEST_RUNNER_TIMEOUT_FACTOR }} ${{ github.event_name != 'pull_request' && '--extended' || '' }}"
This is mostly a refactor, except for using the runner workspace (cwd) for:
* fuzz qa-assets
* functional temp prefix
This makes it easier to run the ci-windows.py locally.
This is mostly a refactor, except for placing the bitcoind.manifest into
a different folder.
This is the standard approach and avoids relying on
VCPKG_INSTALLATION_ROOT and -DCMAKE_TOOLCHAIN_FILE= in the ci-windows.py
script.
This makes it easier to run locally.
They should be sufficient for the task, and are based on containers, so
may be minimally faster to schedule. Ref:
https://docs.github.com/en/actions/reference/runners/github-hosted-runners#single-cpu-runners
ACK fa90277d22e1e6662e827e662eb6bda344cdcb20
Tested verbose output on build failure.
Tested py -3 .github/ci-windows.py standard with generate | check_manifests | prepare_tests | run_tests on Windows 11 24H2. All worked out of the box apart from having to set $env:PYTHONUTF8=1 to let the functional tests pass.
utACK fa90277d22e1e6662e827e662eb6bda344cdcb20
Note that I didn’t test the cmake preset commit locally (no windows), but it looks OK to me.
range-diff since previous ACK:
0$ git range-diff fa361d1a895b63f503731124bc0a88079df1b182...fa90277d22e1e6662e827e662eb6bda344cdcb20
11: fa25e4c572d ! 1: 1111079a16b ci: Add run_tests step to ci-windows.py
2 @@ .github/workflows/ci.yml: jobs:
3 - BITCOINWALLET: '${{ github.workspace }}\build\bin\Release\bitcoin-wallet.exe'
4 - BITCOINCHAINSTATE: '${{ github.workspace }}\build\bin\Release\bitcoin-chainstate.exe'
5 - TEST_RUNNER_EXTRA: ${{ github.event_name != 'pull_request' && '--extended' || '' }}
6 -+ TEST_RUNNER_EXTRA: "--timeout-factor=${{ env.TEST_RUNNER_TIMEOUT_FACTOR }} ${{ github.event_name != 'pull_request' && '--extended' || '' }}"
7 ++ TEST_RUNNER_EXTRA: "--timeout-factor=${{ env.TEST_RUNNER_TIMEOUT_FACTOR }} ${{ case(github.event_name == 'pull_request', '', '--extended') }}"
8 run: |
9 - py -3 test/functional/test_runner.py --jobs $NUMBER_OF_PROCESSORS --quiet --tmpdirprefix="${RUNNER_TEMP}" --combinedlogslen=99999999 --timeout-factor=${TEST_RUNNER_TIMEOUT_FACTOR} ${TEST_RUNNER_EXTRA}
10 -
112: fab467bba6a = 2: fa3f89acaa7 ci: Add check_manifests to ci-windows.py
12-: ----------- > 3: fa9627af9f8 ci: Rely on cmake --preset toolchain file
133: fa361d1a895 = 4: fa90277d22e ci: Use ubuntu-slim for [meta] runners