"system_tests/run_command" unit test fails on MSVC

vostrnad commented at 7:29 pm on October 22, 2022: none

I’m building v24.0rc2 on Windows 10 using Visual Studio 2022. The build succeeds without errors and the binary works fine, but there is one failing unit test:

0C:\bitcoin\src>test_bitcoin.exe
1Running 528 test cases...
2C:/bitcoin/src/test/system_tests.cpp(64): error: in "system_tests/run_command": check what.find(expected) != std::string::npos has failed
3
4*** 1 failure is detected in the test module "Bitcoin Core Test Suite"

Also tried with v23.0 and latest master, same error (for v23.0 the line number is different).

vostrnad added the label Bug on Oct 22, 2022

hebasto commented at 5:32 am on October 23, 2022: member

@vostrnad

What is the Boost version you are using to compile your binaries?

Does error happen on your system with release binaries?

Is your Windows localized?

hebasto added the label Windows on Oct 23, 2022

hebasto added the label Tests on Oct 23, 2022

vostrnad commented at 4:10 pm on October 23, 2022: none

@hebasto Thank you for your response. I install dependencies with vcpkg as per the MSVC build instructions, so I assume my Boost version will be correct, but I don’t know how to check what version I’m using.

The release unit test binary has no failing tests, but it says Running 527 test cases... instead of 528. When I compare the outputs of running test_bitcoin.exe -l test_suite, the run_command test case is not present in the release binary output:

0test/system_tests.cpp(24): Entering test suite "system_tests"
1test/system_tests.cpp(28): Entering test case "dummy"
2test/system_tests.cpp(28): Leaving test case "dummy"; testing time: 12072us
3test/system_tests.cpp(24): Leaving test suite "system_tests"; testing time: 15387us

vs

0C:\bitcoin\src\test\system_tests.cpp(24): Entering test suite "system_tests"
1C:\bitcoin\src\test\system_tests.cpp(28): Entering test case "dummy"
2C:\bitcoin\src\test\system_tests.cpp(28): Leaving test case "dummy"; testing time: 9513us
3C:\bitcoin\src\test\system_tests.cpp(35): Entering test case "run_command"
4C:/bitcoin/src/test/system_tests.cpp(64): error: in "system_tests/run_command": check what.find(expected) != std::string::npos has failed
5C:\bitcoin\src\test\system_tests.cpp(35): Leaving test case "run_command"; testing time: 58694us
6C:\bitcoin\src\test\system_tests.cpp(24): Leaving test suite "system_tests"; testing time: 80393us

My Windows is for the most part not localized. I have English (US) and Czech set as preferred languages (in this order, so almost everything is in English) and use the Czech regional format.

hebasto commented at 10:47 am on October 24, 2022: member

@vostrnad

Could you please make another build with a diff as follows:

 0--- a/src/test/system_tests.cpp
 1+++ b/src/test/system_tests.cpp
 2@@ -59,7 +59,7 @@ BOOST_AUTO_TEST_CASE(run_command)
 3         BOOST_CHECK_EXCEPTION(RunCommandParseJSON("invalid_command"), boost::process::process_error, [&](const boost::process::process_error& e) {
 4             const std::string what(e.what());
 5             BOOST_CHECK(what.find("RunCommandParseJSON error:") == std::string::npos);
 6-            BOOST_CHECK(what.find(expected) != std::string::npos);
 7+            BOOST_CHECK_EQUAL(what, std::string());
 8             return true;
 9         });
10     }

then run test_bitcoin.exe --run_test=system_tests/run_command, and post the resulted log with an error?

vostrnad commented at 11:43 am on October 24, 2022: none

@hebasto

0C:\bitcoin\src>test_bitcoin.exe --run_test=system_tests/run_command
1Running 1 test case...
2C:/bitcoin/src/test/system_tests.cpp(64): error: in "system_tests/run_command": check what == std::string() has failed [ CreateProcess failed: Systm neme nalzt uveden soubor. != ]
3
4*** 1 failure is detected in the test module "Bitcoin Core Test Suite"

So it seems some Czech localization does leak out, notably with all characters with diacritics removed. The actual sentence would be “Systém nemůže nalézt uvedený soubor” which means “The system cannot find the file specified”.

hebasto removed the label Bug on Oct 24, 2022

hebasto added the label Questions and Help on Oct 24, 2022

hebasto commented at 11:51 am on October 24, 2022: member

So MSVC-compiled test_bitcoin.exe requires a non-localized environment.

~~Perhaps, it is worth mentioning somewhere in our docs.~~

~~Closing.~~

hebasto closed this on Oct 24, 2022

hebasto reopened this on Oct 24, 2022

hebasto removed the label Questions and Help on Oct 24, 2022

hebasto added the label Bug on Oct 24, 2022

hebasto commented at 12:54 pm on October 24, 2022: member

@vostrnad

Mind testing a fix suggested in #26377?

vostrnad commented at 1:22 pm on October 24, 2022: none

@hebasto Not at all. Can confirm this fixes the problem for me.

maflcko closed this on Oct 28, 2022

sidhujag referenced this in commit f8c56dc55d on Oct 28, 2022

bitcoin locked this on Oct 28, 2023

“system_tests/run_command” unit test fails on MSVC #26368