ci: Replace wine tests with real tests on Windows? #31071

1. 
   maflcko commented at 8:45 am on October 11, 2024: member

   The Wine tests are annoying, because they frequently fail (for more than a decade now), with no fix in sight.

   Also, they are less useful than running on a real Windows.

   My suggestion would be to replace the win64-cross CI task with one on GitHub Actions, using a real Windows. The cross compilation would still be done normally (to mimic guix), inside WSL. However, the unit tests would be run on the real Windows.

   I won’t be working on this myself, but I am happy to answer any questions on how to implement this, if they arise.

2. maflcko added the label Brainstorming on Oct 11, 2024
3. maflcko added the label Tests on Oct 11, 2024
4. maflcko added the label CI failed on Oct 11, 2024
5. maflcko added the label Windows on Oct 11, 2024
6. 
   maflcko commented at 8:52 am on October 11, 2024: member

   Ugh, this may require WSL2, which isn’t yet available on GHA?
7. 
   willcl-ark commented at 8:58 am on October 11, 2024: member

   I’m presuming that it may need WSL2 for the build, not the test stage?

   If so, we could consider using two GH jobs to split/run this test in two stages; one which does the cross-compile on ubuntu and saves the build artefacts, which, if successful, are then restored by the second (native windows) job to be executed natively under Windows.

   This could prove more tricky to actually execute in practice than it appears to me here though.

8. 
   hebasto commented at 9:00 am on October 11, 2024: member

   Ugh, this may require WSL2, which isn’t yet available on GHA?

   The recent discussion is here.

9. 
   laanwj commented at 4:31 pm on October 22, 2024: member

   Concept ACK. WINE tests are better than nothing, but clearly running the tests on real windows is better. It is not our job to test WINE. They were introduced because there were no usable CI systems for windows at the time. (But yeah we want to test the mingw cross-built Windows binaries, not just MSVC-geneated ones, because those are what we ship)
10. 
    maflcko commented at 1:50 pm on December 3, 2024: member

    Let’s continue discussion in #31176
11. maflcko closed this on Dec 3, 2024

    ---

Contributors
maflcko willcl-ark hebasto laanwj

Labels
Brainstorming Windows Tests CI failed