test: add regression test for #567

vasild commented at 5:34 pm on March 24, 2022: contributor

Add a test that would fail, should #567 resurface.

Also, add a comment and dedup a long expression.

vasild requested review from ryanofsky on Mar 24, 2022

vasild requested review from jonatack on Mar 24, 2022

vasild commented at 5:42 pm on March 24, 2022: contributor

How to test this:

Find your own way. Don’t let me misguide you.

run src/qt/test/test_bitcoin-qt and observe the newly added test passing:

0PASS   : OptionTests::parametersInteraction()

revert the fix from bitcoin-core/gui#568:

 0--- i/src/qt/optionsmodel.cpp
 1+++ w/src/qt/optionsmodel.cpp
 2@@ -168,13 +168,13 @@ void OptionsModel::Init(bool resetSettings)
 3         //
 4         // OptionsModel::Init()
 5         //     this method, can flip -listen from 1 to 0 if fListen=false
 6         //
 7         // AppInitParameterInteraction()
 8         //     error if -listen=0 and -listenonion=1
 9-        gArgs.SoftSetBoolArg("-listenonion", false);
10+        //gArgs.SoftSetBoolArg("-listenonion", false);
11     }
12
13     if (!settings.contains("server")) {
14         settings.setValue("server", false);
15     }
16     if (!gArgs.SoftSetBoolArg("-server", settings.value("server").toBool())) {

and observe the test failing:

0FAIL!  : OptionTests::parametersInteraction() 'gArgs.IsArgSet("-listenonion")' returned FALSE. ()
1   Loc: [qt/test/optiontests.cpp(53)]

in addition to 2., further comment the first check from the test:

 0--- i/src/qt/test/optiontests.cpp
 1+++ w/src/qt/test/optiontests.cpp
 2@@ -47,14 +47,14 @@ void OptionTests::parametersInteraction()
 3
 4     const bool expected{false};
 5
 6     QVERIFY(gArgs.IsArgSet("-listen"));
 7     QCOMPARE(gArgs.GetBoolArg("-listen", !expected), expected);
 8
 9-    QVERIFY(gArgs.IsArgSet("-listenonion"));
10-    QCOMPARE(gArgs.GetBoolArg("-listenonion", !expected), expected);
11+    //QVERIFY(gArgs.IsArgSet("-listenonion"));
12+    //QCOMPARE(gArgs.GetBoolArg("-listenonion", !expected), expected);
13
14     QVERIFY(AppInitParameterInteraction(gArgs));
15
16     // cleanup
17     settings.remove("fListen");
18     QVERIFY(!settings.contains("fListen"));

and observe the second check failing:

0Error: Cannot set -listen=0 together with -listenonion=1
1FAIL!  : OptionTests::parametersInteraction() 'AppInitParameterInteraction(gArgs)' returned FALSE. ()
2   Loc: [qt/test/optiontests.cpp(56)]

vasild cross-referenced this on Mar 24, 2022 from issue options: flip listenonion to false if not listening by vasild

jonatack commented at 7:59 pm on March 24, 2022: contributor

Concept ACK

vasild commented at 9:55 am on March 25, 2022: contributor

I am puzzled why on macOS 12 native [gui, system sqlite only] [no depends] the newly added test fails with:

0Error: Specified blocks directory "" does not exist.
1FAIL!  : OptionTests::parametersInteraction() 'AppInitParameterInteraction(gArgs)' returned FALSE. ()
2   Loc: [qt/test/optiontests.cpp(56)]

While on my computer and on 32-bit + dash [gui] [CentOS 8] (at least) the test passes.

:confused:

ryanofsky commented at 5:28 pm on March 30, 2022: contributor

Thank you for following up with this! :partying_face:

I am puzzled why on macOS 12 native [gui, system sqlite only] [no depends] the newly added test fails

This is mysterious, but actually less mysterious than it appears at first, because of a terrible error message on the following line that checks if one directory exists (GetBlocksDirPath), but then prints the error message about a different directory ("-blocksdir"):

https://github.com/bitcoin-core/gui/blob/f4e5d704f24ad9beba4f9a4838be3351551964df/src/init.cpp#L828-L829

I think this error just happens if tests run in a different order, and I would try adding a gArgs.ClearPathCache() call and forced_settings.erase("data"); to the top of the failing test(s).

in src/qt/optionsmodel.cpp:173 in cb018e7586 outdated

170+        //
171+        // OptionsModel::Init()
172+        //     this method, can flip -listen from 1 to 0 if fListen=false
173+        //
174+        // AppInitParameterInteraction()
175+        //     error if -listen=0 and -listenonion=1

jonatack commented at 8:27 pm on March 30, 2022:

nit suggestion

0         // OptionsModel::Init()
1-        //     this method, can flip -listen from 1 to 0 if fListen=false
2+        //     this method can flip -listen from 1 to 0 if fListen=false
3         //
4         // AppInitParameterInteraction()
5-        //     error if -listen=0 and -listenonion=1
6+        //     raises an error if -listen=0 and -listenonion=1

vasild commented at 1:41 pm on March 31, 2022:

The comma was intentional. Replaced it by brackets. Took the other suggestion.

in src/qt/test/optiontests.cpp:45 in cb018e7586 outdated

34+void OptionTests::parametersInteraction()
35+{
36+    gArgs.LockSettings([&](util::Settings& s) {
37+        s.forced_settings.erase("listen");
38+        s.forced_settings.erase("listenonion");
39+    });

jonatack commented at 8:34 pm on March 30, 2022:

Added asserts here as a sanity check, maybe good to add in case the defaults in test_main.cpp change.

0 void OptionTests::parametersInteraction()
1 {
2+    QVERIFY(gArgs.IsArgSet("-listen"));
3+    QVERIFY(gArgs.IsArgSet("-listenonion"));
4     gArgs.LockSettings([&](util::Settings& s) {
5         s.forced_settings.erase("listen");
6         s.forced_settings.erase("listenonion");
7     }

vasild commented at 1:43 pm on March 31, 2022:

The test unsets those options, so it does not care or rely that they are set when it starts. Thus I think those checks are not necessary. I.e. the test will still work even if those options are not set when it starts.

jonatack commented at 8:39 pm on March 30, 2022: contributor

ACK cb018e75864c973d4d3e9591d7d2347a60aee122 modulo the macOS CI

Note that reverting #568 and running ./src/qt/test/test_bitcoin-qt also prints a couple of Error: Cannot set -listen=0 together with -listenonion=1 messages in the RPCNestedTests and WalletTests output, though the tests don’t fail outright.

I could be missing a command line argument to do this, but added the following to make it easier to see if any test failures happened:

 0--- a/src/qt/test/test_main.cpp
 1+++ b/src/qt/test/test_main.cpp
 2@@ -112,6 +112,10 @@ int main(int argc, char* argv[])
 3         fInvalid = true;
 4     }
 5 #endif
 6
 7+    if (fInvalid) {
 8+        qWarning("\nThere were errors in some of the tests above.\n");
 9+    } else {
10+        qDebug("\nAll tests executed successfully.\n");
11+    }

options: add a comment for -listenonion and dedup a long expression

A followup to https://github.com/bitcoin-core/gui/pull/568

Co-authored-by: Jon Atack <jon@atack.com>

3b82608dd1

vasild force-pushed on Mar 31, 2022

vasild commented at 1:40 pm on March 31, 2022: contributor

cb018e7586...cae12fc803: attempt to fix the CI failure and address suggestions

Invalidates ACK from @jonatack

Thanks for the suggestion, @ryanofsky! I added just gArgs.ClearPathCache() as I think that alone should resolve it. @jonatack I look up the exit status of the test executable (echo $?) and if 1 then search the output for FAIL!, yeah, not very convenient. I too noticed the printout Error: Cannot set -listen=0 together with -listenonion=1 from other tests if the bug resurfaces. I think it is ok.

jonatack commented at 2:42 pm on March 31, 2022: contributor

ACK cae12fc803bc9595dbb4466b4dbbfa29490ce995 provided the CI is happy

in src/qt/test/optiontests.cpp:34 in cae12fc803 outdated

29@@ -29,3 +30,35 @@ void OptionTests::optionTests()
30     });
31     gArgs.WriteSettingsFile();
32 }
33+
34+void OptionTests::parametersInteraction()

jarolrod commented at 1:27 am on April 1, 2022:

This name is fine with me, although it implied general parameter interaction testing.

Absolutely not critical, but while here, as this test is regression specific; maybe we could add a comment similar to the one in the test above?

vasild commented at 8:33 am on April 1, 2022:

Added a comment.

jarolrod commented at 1:28 am on April 1, 2022: member

ACK cae12fc803bc9595dbb4466b4dbbfa29490ce995

This effectively tests the regression described in #567, and fixed in #568.

test: add regression test for bitcoin-core/gui/issues/567 4d4dca43fc

vasild force-pushed on Apr 1, 2022

vasild commented at 8:33 am on April 1, 2022: contributor

cae12fc803...4d4dca43fc: add a comment in the test

Invalidates ACKs from @jonatack and @jarolrod

jarolrod commented at 2:59 pm on April 1, 2022: member

reACK 4d4dca43fc

only change since my last review is the addition of a comment for the added test

jarolrod added the label Tests on Apr 1, 2022

jonatack commented at 10:45 pm on April 2, 2022: contributor

ACK 4d4dca43fc591bf8fae7af74670f6e96650ef34b

hebasto approved

hebasto commented at 7:32 am on April 3, 2022: member

ACK 4d4dca43fc591bf8fae7af74670f6e96650ef34b, tested with reverting changes from bitcoin-core/gui#568, and getting an expected test failure.

shaavan approved

shaavan commented at 1:25 pm on April 3, 2022: contributor

ACK 4d4dca43fc591bf8fae7af74670f6e96650ef34b

The added test seems logically sound and is written in a very easy-to-understand manner.
I was able to verify that:
- The test ran successfully on the PR branch.
- The test failed when the changes done in #568 were reverted.
I also like the comments added along with the changes done in #568. These help the code reader understand the reasoning behind writing the code the way it is written.

hebasto merged this on Apr 4, 2022

hebasto closed this on Apr 4, 2022

vasild deleted the branch on Apr 4, 2022

sidhujag referenced this in commit 3417991add on Apr 4, 2022

ryanofsky cross-referenced this on Apr 5, 2022 from issue interfaces: Expose settings.json methods to GUI by ryanofsky

bitcoin-core locked this on Apr 4, 2023

hebasto commented at 5:03 pm on February 24, 2024: member

Unfortunately, the test_bitcoin-qt.exe fails on Windows (v26.1rc1):

0FAIL!  : OptionTests::parametersInteraction() 'gArgs.IsArgSet("-listen")' returned FALSE.

bitcoin-core unlocked this on Feb 26, 2024

test: add regression test for #567 #569