tests: Some issue about running functional tests on Windows #14446

1. 
   ken2812221 commented at 4:34 pm on October 9, 2018: contributor

   Since #14007, the functional tests would run on Appveyor CI. However the tests would fail occassionally due to several reasons. Here is the list:

   • ConnectionAbortedError: [WinError 10053] might be a race condition Maybe fixed by #14670 For reference:
     https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19366470 https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19344806

   • PermissionError: [WinError 32]: found in feature_notifications. The test script delete the files before the notification command close the file. Maybe fixed by #14465 For reference: https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19363071 https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19342222

   • OSError: [WinError 10022] An invalid argument was supplied: Unknown reason. One-off? For reference: https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19365696

   • AssertionError: Block sync timed out: Unknown reason. Found in wallet_txn_doublespend For reference: Maybe fixed by #15419 https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19360284 https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19334394 https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/20031316

   • PermissionError: [Errno 13] Permission denied: Unknown reason. Found in get_auth_cookie function Maybe fixed by #14788 For reference: https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19324004 https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19393533

   If you find any fail that is not listed here. Please comment below.

2. fanquake added the label Windows on Oct 9, 2018
3. fanquake added the label Tests on Oct 9, 2018
4. 
   meshcollider commented at 10:18 am on October 14, 2018: contributor

   Found a new one? in feature_uacomment.py self.nodes[0].assert_start_raises_init_error(["-uacomment=" + unsafe_char], expected, match=ErrorMatch.FULL_REGEX)

   https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19493806

5. 
   ken2812221 commented at 10:25 am on October 14, 2018: contributor

   Found a new one? in feature_uacomment.py self.nodes[0].assert_start_raises_init_error(["-uacomment=" + unsafe_char], expected, match=ErrorMatch.FULL_REGEX)

   https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19493806

   That is the first situation, you can see ConnectionAbortedError: [WinError 10053] in logs.

6. 
   sipa commented at 9:53 pm on October 17, 2018: member

   This is pretty annoying, as it adds a big red failure sign to your PR. Can we disable appveyor, or prevent it from influencing the CI status of PRs?
7. sipa referenced this in commit d387507aec on Oct 19, 2018
8. 
   sipa commented at 7:02 pm on October 31, 2018: member

   I’m still seeing ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine, for example on https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/19954225.
9. 
   ken2812221 commented at 4:57 am on November 1, 2018: contributor

   @sipa #13501 isn’t merged yet.
10. 
    promag commented at 5:54 pm on December 6, 2018: member

    @ken2812221 #14670 is merged, can you update this?
11. 
    MarcoFalke commented at 6:03 pm on December 7, 2018: member

    I still see the block sync timed out issues.

    Also there is a failure in wallet_multiwallet, which might happen on travis as well. https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/20846360?fullLog=true#L85

    0test_framework.authproxy.JSONRPCException: Wallet file verification failed: Error loading wallet w3. Duplicate -wallet filename specified. (-4)
    

12. MarcoFalke referenced this in commit 2753285be7 on Dec 7, 2018
13. 
    MarcoFalke commented at 6:07 pm on December 7, 2018: member

    Maybe this should be closed and individual issues should be filed for each failure reason?
14. 
    MarcoFalke commented at 5:33 pm on February 5, 2019: member

    If the issues persist, we might want to ignore the exit code for functional tests for now. That way, the output can still be checked manually, but a failing test wouldn’t mark a pull request red in the pull request overview tab.

    Maybe that diff would work:

     0diff --git a/.appveyor.yml b/.appveyor.yml
     1index eb2d32e1bc..492147a222 100644
     2--- a/.appveyor.yml
     3+++ b/.appveyor.yml
     4@@ -54,5 +54,5 @@ test_script:
     5 - cmd: src\bench_bitcoin.exe -evals=1 -scaling=0 > NUL
     6 - ps:  python test\util\bitcoin-util-test.py
     7 - cmd: python test\util\rpcauth-test.py
     8-- cmd: python test\functional\test_runner.py --ci --force --quiet --combinedlogslen=4000 --failfast
     9+- cmd: python test\functional\test_runner.py --ci --force --quiet --combinedlogslen=4000 --failfast || echo "ignore test failure"
    10 deploy: off
    

15. 
    sdaftuar commented at 7:54 pm on February 8, 2019: member

    FYI I tracked down the wallet_txn_doublespend.py test failure – it’s due to an actual bug in the test. More explanation in the OP here: #15360.

    The underlying problem is a race condition between connecting two nodes and generating a block on one of those nodes to trigger block sync on the other. I thought in #15360 that the best way to fix this would be to change the properties of the cached datadir that we create in our test framework, but it seems to me now that probably the test should be modified/rewritten so that the nodes start off outside of IBD, which would eliminate the race condition on block sync issue altogether.

16. MarcoFalke referenced this in commit 8f470ecc53 on Feb 25, 2019
17. MarcoFalke closed this on Feb 25, 2019

    ---

18. Munkybooty referenced this in commit bb975fbc84 on Aug 2, 2021
19. Munkybooty referenced this in commit b9b5e75962 on Aug 3, 2021
20. Munkybooty referenced this in commit bac4b6dc77 on Aug 5, 2021
21. Munkybooty referenced this in commit 6a1747bb5e on Aug 5, 2021
22. Munkybooty referenced this in commit 563ad654c4 on Aug 8, 2021
23. Munkybooty referenced this in commit 7bbd22c68d on Aug 11, 2021
24. Munkybooty referenced this in commit f089273ed0 on Aug 11, 2021
25. Munkybooty referenced this in commit b92a8c2019 on Aug 13, 2021
26. 5tefan referenced this in commit f9cd7ffa58 on Aug 13, 2021
27. 5tefan referenced this in commit d09b13066c on Aug 14, 2021
28. Munkybooty referenced this in commit 29ba5a37de on Aug 15, 2021
29. Munkybooty referenced this in commit e928494a49 on Sep 8, 2021
30. dzutto referenced this in commit 6f296bdb78 on Oct 15, 2021
31. DrahtBot locked this on Dec 16, 2021
32. Munkybooty referenced this in commit f863deb724 on Dec 25, 2021
33. Munkybooty referenced this in commit 675d49a99b on Dec 28, 2021
34. Munkybooty referenced this in commit 19ff5107d9 on Dec 28, 2021
35. Munkybooty referenced this in commit 42048e82a6 on Dec 28, 2021
36. Munkybooty referenced this in commit 08241a7e29 on Dec 28, 2021
37. Munkybooty referenced this in commit 1e0be5a93e on Jan 3, 2022
38. Munkybooty referenced this in commit 1e592a8cf5 on Feb 1, 2022
39. Munkybooty referenced this in commit a0678bdfb3 on Feb 16, 2022
40. UdjinM6 referenced this in commit 5524c42b8c on Sep 13, 2022

Contributors
ken2812221 meshcollider sipa promag MarcoFalke sdaftuar

Labels
Windows Tests