The use of pwsh was a bit confusing and inconsistent around the exit code. See also #32672
I think it is fine to drop it and purely use Bash/Python.
This also moves a bit of code from the github yaml to the python script.
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
See the guideline for information on the review process.
| Type | Reviewers |
|---|---|
| ACK | hodlinator, m3dwards |
If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.
235- $installationPath = & $vswherePath -latest -property installationPath
236- & "${env:COMSPEC}" /s /c "`"$installationPath\Common7\Tools\vsdevcmd.bat`" -arch=x64 -no_logo && set" | foreach-object {
237- $name, $value = $_ -split '=', 2
238- echo "$name=$value" >> $env:GITHUB_ENV
239- }
240+ run: py -3 .github/ci-windows-cross.py github_env_setup_vs_dev_prompt
.github/ci-windows.py requires an unused positional arg, which is equally confusing. Will leave as-is for now.
Happy to push any diff, if someone writes one, or push any commit, if someone writes one, but I can’t get myself to write it :sweat_smile:
Also, I am happy to review a follow-up doing this.
Uses raw symbols for the function list and execution. Attempts to improve naming.
0diff --git a/.github/ci-windows-cross.py b/.github/ci-windows-cross.py
1index d32233258c..13ca3b4945 100755
2--- a/.github/ci-windows-cross.py
3+++ b/.github/ci-windows-cross.py
4@@ -20,29 +20,6 @@ def run(cmd, **kwargs):
5 sys.exit(str(e))
6
7
8-def setup_vs_dev_prompt():
9- vswhere_path = Path(os.environ["ProgramFiles(x86)"]) / "Microsoft Visual Studio" / "Installer" / "vswhere.exe"
10- installation_path = run(
11- [str(vswhere_path), "-latest", "-property", "installationPath"],
12- capture_output=True,
13- text=True,
14- ).stdout.strip()
15- vsdevcmd = Path(installation_path) / "Common7" / "Tools" / "vsdevcmd.bat"
16- comspec = os.environ["COMSPEC"]
17- output = run(
18- f'"{comspec}" /s /c ""{vsdevcmd}" -arch=x64 -no_logo && set"',
19- capture_output=True,
20- text=True,
21- ).stdout
22- github_env = os.environ["GITHUB_ENV"]
23- with open(github_env, "a") as env_file:
24- for line in output.splitlines():
25- if "=" not in line:
26- continue
27- name, value = line.split("=", 1)
28- env_file.write(f"{name}={value}\n")
29-
30-
31 def print_version():
32 bitcoind = Path.cwd() / "bin" / "bitcoind.exe"
33 run([str(bitcoind), "-version"])
34@@ -158,7 +135,6 @@ def run_unit_tests():
35 def main():
36 parser = argparse.ArgumentParser(description="Utility to run Windows CI steps.")
37 steps = [
38- "github_env_setup_vs_dev_prompt",
39 "print_version",
40 "check_manifests",
41 "prepare_tests",
42@@ -173,9 +149,7 @@ def main():
43 str(Path.cwd() / "previous_releases"),
44 )
45
46- if args.step == "github_env_setup_vs_dev_prompt":
47- setup_vs_dev_prompt()
48- elif args.step == "print_version":
49+ if args.step == "print_version":
50 print_version()
51 elif args.step == "check_manifests":
52 check_manifests()
53diff --git a/.github/ci-windows.py b/.github/ci-windows.py
54index caa2d52c77..212d0de7c5 100755
55--- a/.github/ci-windows.py
56+++ b/.github/ci-windows.py
57@@ -38,6 +38,29 @@ GENERATE_OPTIONS = {
58 }
59
60
61+def github_import_vs_env(ci_type):
62+ vswhere_path = Path(os.environ["ProgramFiles(x86)"]) / "Microsoft Visual Studio" / "Installer" / "vswhere.exe"
63+ installation_path = run(
64+ [str(vswhere_path), "-latest", "-property", "installationPath"],
65+ capture_output=True,
66+ text=True,
67+ ).stdout.strip()
68+ vsdevcmd = Path(installation_path) / "Common7" / "Tools" / "vsdevcmd.bat"
69+ comspec = os.environ["COMSPEC"]
70+ output = run(
71+ f'"{comspec}" /s /c ""{vsdevcmd}" -arch=x64 -no_logo && set"',
72+ capture_output=True,
73+ text=True,
74+ ).stdout
75+ github_env = os.environ["GITHUB_ENV"]
76+ with open(github_env, "a") as env_file:
77+ for line in output.splitlines():
78+ if "=" not in line:
79+ continue
80+ name, value = line.split("=", 1)
81+ env_file.write(f"{name}={value}\n")
82+
83+
84 def generate(ci_type):
85 command = [
86 "cmake",
87@@ -50,7 +73,7 @@ def generate(ci_type):
88 run(command)
89
90
91-def build():
92+def build(ci_type):
93 command = [
94 "cmake",
95 "--build",
96@@ -180,26 +203,18 @@ def run_tests(ci_type):
97 def main():
98 parser = argparse.ArgumentParser(description="Utility to run Windows CI steps.")
99 parser.add_argument("ci_type", choices=GENERATE_OPTIONS, help="CI type to run.")
100- steps = [
101- "generate",
102- "build",
103- "check_manifests",
104- "prepare_tests",
105- "run_tests",
106- ]
107+ steps = list(map(lambda f: f.__name__, [
108+ github_import_vs_env,
109+ generate,
110+ build,
111+ check_manifests,
112+ prepare_tests,
113+ run_tests,
114+ ]))
115 parser.add_argument("step", choices=steps, help="CI step to perform.")
116 args = parser.parse_args()
117
118- if args.step == "generate":
119- generate(args.ci_type)
120- elif args.step == "build":
121- build()
122- elif args.step == "check_manifests":
123- check_manifests(args.ci_type)
124- elif args.step == "prepare_tests":
125- prepare_tests(args.ci_type)
126- elif args.step == "run_tests":
127- run_tests(args.ci_type)
128+ exec(f'{args.step}("{args.ci_type}")')
129
130
131 if __name__ == "__main__":
132diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
133index 4942ac34e8..bc56774e28 100644
134--- a/.github/workflows/ci.yml
135+++ b/.github/workflows/ci.yml
136@@ -227,9 +227,9 @@ jobs:
137
138 - *CHECKOUT
139
140- - &SET_UP_VS
141- name: Set up VS Developer Prompt
142- run: py -3 .github/ci-windows-cross.py github_env_setup_vs_dev_prompt
143+ - &IMPORT_VS_ENV
144+ name: Import Visual Studio env vars
145+ run: py -3 .github/ci-windows.py ${{ matrix.job-type }} github_import_vs_env
146
147 - name: Get tool information
148 run: |
149@@ -411,7 +411,7 @@ jobs:
150 - name: Run bitcoind.exe
151 run: py -3 .github/ci-windows-cross.py print_version
152
153- - *SET_UP_VS
154+ - *IMPORT_VS_ENV
155
156 - name: Check executable manifests
157 run: py -3 .github/ci-windows-cross.py check_manifests
Are you sure the patch applies? It looks corrupt and does not apply locally.
Maybe push a commit, so I can just force push it to this pull?
I don’t think the patch will pass ci as-is, because there is no job-type in the cross test.
So I’ve pushed it with some modifications.
IMPORT_VS_ENV?
Yes, it is needed. Otherwise:
0+ mt.exe -nologo '-inputresource:D:\a\bitcoin\bitcoin\bin\bitcoind.exe' '-out:D:\a\bitcoin\bitcoin\bin\bitcoind.manifest'
1[WinError 2] The system cannot find the file specified
2Error: Process completed with exit code 1.
https://github.com/bitcoin/bitcoin/actions/runs/22061363427/job/63743877223?pr=34583#step:6:11
The yaml anchor made more sense when there was a larger template. As it’s just a call to python now could the anchor just be dropped entirely?
thx. May do, if I have to re-push
Makes sense. Still think “IMPORT_VS_ENV” is clearer than “SET_UP_VS” but it’s tangential to this PR.
thx, done by using your patch
28+ text=True,
29+ ).stdout.strip()
30+ vsdevcmd = Path(installation_path) / "Common7" / "Tools" / "vsdevcmd.bat"
31+ comspec = os.environ["COMSPEC"]
32+ output = run(
33+ f'"{comspec}" /s /c ""{vsdevcmd}" -arch=x64 -no_logo && set"',
"" after /c make me uneasy, but I guess the string parsing might disallow empty strings and so can support keeping track of one level of quoted strings internally.
cmd.exe. I’d be curious to see if there is a way (at all) to do this with an array of strings, but maybe this can be done in a follow-up by a Windows person? (I don’t have Windows, so testing all of this with CI isn’t smooth)
This is just a literal copy of what the pwsh script did (it also had the exact same full string with the exact same double quotes)
It had some back-ticks in there, but I don’t see what difference they would make.
It had some back-ticks in there, but I don’t see what difference they would make.
I think the back-ticks are just some pwsh thing to escape the " char, but I am not 100% sure. (This confusion is one of the reasons to drop pwsh)
In python that isn’t needed, because '-strings can contain " chars as-is.
I am still curious if a Windows person or a Windows LLM can refactor subprocess.run(f'"{comspec}" /s /c ""{vsdevcmd}" -arch=x64 -no_logo && set")' into the run([...]) form.
utACK faba7f616db05ded6cc642821ce540aa40fb8fcb
Wasn’t sure which python file the setup vs prompt belonged but agree it’s perhaps a bit silly to add another file just for this. I wasn’t able to test it on my windows machine as it requires Github environment vars.
Also, change the yaml anchor name and the step name.
Also, small refactors while touching the files.
Seems easier to just use Bash and Python consistently.