test: Use Regex Search in Apptests

jarolrod commented at 4:08 am on May 17, 2021: member

This PR refactors our GUI apptests so that it uses regex search to find values in our console/qtextedit output regardless if it is in plaintext, html, or markdown.

This introduces a new function FindInConsole which uses QRegularExpression to search the output of the console. The function must be provided with a perl compatible regex pattern which wants to match a single group. The function then returns the matched group. If no match is found, an empty QString is returned.

We then use this new function in TestRpcCommand to find the current chain value instead of reading with univalue.

This approach can apply to a wider variety of testing scenarios as we can reuse this function to search for values when the console output is exported in a different format than plaintext. As an example, A follow up PR will add tests for console resizing and needs to look for the size in html tags after exporting the console text with toHtml().

hebasto added the label Tests on May 17, 2021

jarolrod force-pushed on May 18, 2021

jarolrod commented at 2:37 pm on May 18, 2021: member

updated from d32b704 -> d342630

Changes:

Address CI failure by explicitly making the “regtest” string a QString
- Note: Using QStringLiteral would be more efficient since it avoids constructing a QString and copying data to it. But we are not in a loop, the performance benefit is minimal, and just QString is cleaner.
While here, include missing QString header

in src/qt/test/apptests.cpp:23 in 2c679ecd57 outdated

21@@ -22,6 +22,7 @@
22 #include <QAction>
23 #include <QEventLoop>
24 #include <QLineEdit>
25+#include <QRegularExpression>

hebasto commented at 6:27 pm on May 20, 2021:

https://github.com/bitcoin/bitcoin/pull/17213#issuecomment-546684680

https://github.com/bitcoin/bitcoin/pull/16432#discussion_r315464914

hebasto commented at 9:37 pm on May 22, 2021:

I verified that Qt itself uses QRegularExpression header, at least for version we use in depends.

So, I’m ok with it.

DrahtBot added the label Needs rebase on May 20, 2021

jarolrod force-pushed on May 21, 2021

jarolrod commented at 2:15 am on May 21, 2021: member

updated from d342630 -> 1a63f44 (pr337.01 -> pr337.02)

Changes:

Rebase due to conflict with #335

DrahtBot removed the label Needs rebase on May 21, 2021

in src/qt/test/apptests.cpp:37 in 1a63f44609 outdated

32 
33 namespace {
34+//! Regex find a string group inside of the console output
35+QString FindInConsole(const QString& output, const QString& pattern)
36+{
37+    QRegularExpression re(pattern);

hebasto commented at 9:58 pm on May 22, 2021:

nit:

0    const QRegularExpression re(pattern);

jarolrod commented at 4:31 am on May 26, 2021:

Addressed in 9bde381

in src/qt/test/apptests.cpp:53 in 1a63f44609 outdated

48@@ -41,10 +49,9 @@ void TestRpcCommand(RPCConsole* console)
49     QTest::keyClick(lineEdit, Qt::Key_Return);
50     QVERIFY(mw_spy.wait(1000));
51     QCOMPARE(mw_spy.count(), 2);
52-    QString output = messagesWidget->toPlainText();
53-    UniValue value;
54-    value.read(output.right(output.size() - output.lastIndexOf(QChar::ObjectReplacementCharacter) - 1).toStdString());
55-    QCOMPARE(value["chain"].get_str(), std::string("regtest"));
56+    const QString output = messagesWidget->toPlainText();
57+    const QString pattern = "\"chain\": \"(\\w+)\"";

hebasto commented at 10:01 pm on May 22, 2021:

nit:

0    const QString pattern = QStringLiteral("\"chain\": \"(\\w+)\"");

jarolrod commented at 4:31 am on May 26, 2021:

Addressed in 9bde381

hebasto commented at 10:03 pm on May 22, 2021: member

Approach ACK 1a63f44609d1c7a4de0f5c4727e0d4cd3d7448ad.

jarolrod force-pushed on May 26, 2021

jarolrod commented at 4:30 am on May 26, 2021: member

updated from 1a63f44 -> 9bde381 (pr337.02, pr337.03, diff)

Changes:

Address @hebasto’s suggestions (1, 2)

DrahtBot added the label Needs rebase on Jun 1, 2021

qt, test: introduce FindInConsole function

Allows for regex searching into the console output.

d09d1cf1a2

qt, test: use regex search in apptests

use the FindInConsole function to regex search for values in apptests instead of Univalue read.

6969b2bb98

jarolrod force-pushed on Jun 1, 2021

jarolrod commented at 6:24 am on June 1, 2021: member

Updated from 9bde381 -> 6969b2b (pr337.03 -> pr337.04)

Changes:

address conflict with #123

DrahtBot removed the label Needs rebase on Jun 1, 2021

hebasto approved

hebasto commented at 6:27 pm on June 5, 2021: member

ACK 6969b2bb98a2f44e1b51c905db92ec2e28345078

shaavan commented at 6:19 pm on August 2, 2021: contributor

ACK 6969b2bb98a2f44e1b51c905db92ec2e28345078 Compiled the PR properly on Ubuntu 20.04

The PR introduces regex search functionality to apptest.cpp file by replacing univalue based search, which requires the qmessagewidget to be output in plaintext to work properly, with a more general Qregularexpression based search which doesn’t explicitly require a plaincheck to function.

Tested if the new apptest is running properly by running ./src/qt/test/test_bitcoin-qt and from there obtained: PASS : AppTests::appTests() For the sake of completion, the final result obtained from running the test_bitcoin-qt file was: Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted, 96ms

This PR is an improvement over the master, as now the search string need not have to be a plaintext, which increases the number of use cases of the search action. An example of an additional use case is if we want to output the qmessagewidget in HTML to examine the values of the current HTML elements.

hebasto merged this on Aug 5, 2021

hebasto closed this on Aug 5, 2021

sidhujag referenced this in commit d001fe4679 on Aug 5, 2021

bitcoin-core locked this on Aug 16, 2022

test: Use Regex Search in Apptests #337