Improve rpc-tests.py

jnewbery commented at 9:00 pm on January 31, 2017: member

rpc-tests.py was converted from a shell script into python in #6616 and it shows. There are several idioms which make sense for a shell script, but are considered bad practice in a python module. This PR cleans up some of the quirks.

This pull request combines several commits which improve the code style of rpc-tests.py and move away from bad practices. Reviews/comments for any or all of the commits are welcomed:

commit 1 replaces the tests_config module and from tests_config import * wildcard import with a .ini file and the Python standard library configparser module. Benefits are:
- config lives in config files and code lives in source files. Duh.
- avoids a wildcard import, which are usually considered bad practice because it’s difficult to know where names are binded.
- replaces some of the ugly workarounds in #8133 - doesn’t leave a compiled tests_config.pyc file lying around, doesn’t add qa/pull_tester to the python path, allows rpc_tests.py to be run from anywhere.
commit 2 replaces the roll-your-own argument parsing code with Python3’s standard library argparse module. Benefits are:
- more compact, cleaner code
- passing unknown arguments down to the individual test scripts and selecting which individual test scripts to run is now cleaner
- it should be easier and less bug-prone to add new arguments in future
commit 3 makes a few miscellaneous improvements to rpc-test.py’s arguments:
- make all arguments start with double dash for consistency. I don’t expect those arguments were being called by automatic tools, so this shouldn’t break any build/test processes.
- improve help text and output
- add nozmq argument to explicitly exclude the ZMQ tests
commit 4 removes global variables and places logic in main(). Benefits:
- allows rpc-tests.py to be imported and runtests() to be called by external modules.

[EDIT: removed commit hashes since I’ve now rebased]

MarcoFalke commented at 10:32 pm on January 31, 2017: member

Great! Concept ACK on all of those. I will take a detailed look later.

in qa/pull-tester/rpc-tests.py: in 553d99c575 outdated

200+        test_list = test_scripts
201     else:
202-        for t in testScripts + testScriptsExt:
203-            if t in opts or re.sub(".py$", "", t) in opts:
204+        test_list = []
205+        for t in test_scripts:

jtimon commented at 0:45 am on February 1, 2017:

One shouldn’t be required to pass –extended to be able to run specific tests from the extended set. You can leave the following here:

0for t in BASE_SCRIPTS + EXTENDED_SCRIPTS:

Mhmm, that would still exclude zmq_test.py because of the other change you made…

jnewbery commented at 2:04 am on February 1, 2017:

Agree. I’ve fixed this by adding a new ALL_SCRIPTS variable which is the concatenation of the three other lists.

in qa/pull-tester/rpc-tests.py: in 553d99c575 outdated

30@@ -30,6 +31,25 @@
31 import tempfile
32 import re
33 
34+# Parse arguments and pass through unrecognised args
35+parser = argparse.ArgumentParser(add_help=False)
36+parser.add_argument('--coverage', action='store_true')
37+parser.add_argument('-extended', action='store_true')
38+parser.add_argument('--help', '-h', '-?', action='store_true')
39+parser.add_argument('--parallel', type=int, default=4)

isle2983 commented at 0:48 am on February 1, 2017:

If we are considering changing this around, my vote would be for renaming --parallel to be --jobs and also giving it a ‘-j’ alias. The advantage is that it matches make where most people know what -j3 means at a glance.

jnewbery commented at 2:04 am on February 1, 2017:

agree. Changed.

in qa/pull-tester/rpc-tests.py: in 553d99c575 outdated

46+
47+for arg in unknown_args:
48+    if arg[:2] == "--":
49+        passon_args.append(arg)
50+    else:
51+        tests.add(arg)

isle2983 commented at 0:48 am on February 1, 2017:

Suggest doing this chunk with list/iterator comprehension like so:

0tests = set(arg for arg in unknown_args if arg[:2] != "--")
1passon_args = [arg for arg in unknown_args if arg[:2] == "--"]

jnewbery commented at 2:04 am on February 1, 2017:

much nicer. Thanks!

in qa/pull-tester/rpc-tests.py: in 553d99c575 outdated

202-        for t in testScripts + testScriptsExt:
203-            if t in opts or re.sub(".py$", "", t) in opts:
204+        test_list = []
205+        for t in test_scripts:
206+            if t in tests or re.sub(".py$", "", t) in tests:
207                 test_list.append(t)

isle2983 commented at 0:48 am on February 1, 2017:

Suggest building test_list with list comprehension:

0test_list = [t for t in test_scripts if
1             (t in tests or re.sub(".py$", "", t) in tests)]

jnewbery commented at 2:04 am on February 1, 2017:

yep. Fixed.

in qa/pull-tester/rpc-tests.py: in fa05213775 outdated

39+parser.add_argument('--coverage', action='store_true', help='generate a basic coverage report for the RPC interface')
40+parser.add_argument('--extended', action='store_true', help='run the extended test suite in addition to the basic tests')
41+parser.add_argument('--help', '-h', '-?', action='store_true', help='print help text and exit')
42+parser.add_argument('--nozmq', action='store_true', help='do not run the zmq tests')
43+parser.add_argument('--parallel', type=int, default=4, help='how many test scripts to run in parallel. Default=4.')
44+parser.add_argument('--win', action='store_true', help='signal that this is running in a Windows environment and that we should run the tests')

jtimon commented at 0:49 am on February 1, 2017:

" and that we should run the tests" sounds weird to me. Are there special tests only for windows? Otherwise we just should run the tests, as always, just taking into account the windows environment. If there’s windows specific tests, what about “signal that this is running in a Windows environment and the windows specific tests sould be run”?

jnewbery commented at 2:06 am on February 1, 2017:

I haven’t changed the text in this argument (the text was previously in the docstring for the module). There are no windows specific tests - this argument simply means “I know I’m running on windows and I definitely want to run the tests.” I haven’t dug into why this argument was added originally, but feel free to open a new PR if you think it can be removed.

in qa/pull-tester/rpc-tests.py: in b90946a124 outdated

90-        print("ERROR: \"import zmq\" failed. Use -nozmq to run without the ZMQ tests."
91-              "To run zmq tests, see dependency info in /qa/README.md.")
92-        raise
93-
94-BASE_SCRIPTS= [
95-    # longest test should go first, to favor running tests in parallel

jtimon commented at 0:53 am on February 1, 2017:

NACK on changing the order and removing the time comments (ie reverting #9276 )

jnewbery commented at 2:06 am on February 1, 2017:

Oops. This was a bad manual merge from a old branch.

Thanks for catching this. I’ve fixed this so #9276 is included.

in qa/pull-tester/rpc-tests.py: in b90946a124 outdated

184+        print("No rpc tests to run. Scripts not found:")
185+        print(tests)
186         sys.exit(0)
187 
188-    coverage = None
189+    runtests(test_list, config["environment"]["SRCDIR"], config["environment"]["BUILDDIR"], config["environment"]["EXEEXT"], args.parallel, args.coverage, passon_args)

jtimon commented at 1:03 am on February 1, 2017:

Maybe stupid question…not sure how main() can call runtests() without having declared it yet.

jnewbery commented at 2:08 am on February 1, 2017:

Not a stupid question :)

main() is called in the final line of the module:

0if __name__ == '__main__':
1    main()

by which point all the functions have been declared. I believe this is a fairly common python module structure.

jtimon commented at 1:04 am on February 1, 2017: contributor

Wouldn’t f1e8e33 be clearer if https://github.com/bitcoin/bitcoin/blob/master/qa/pull-tester/tests_config.py.in was removed in the same commit?

utACK modulo comments.

Use configparser in rpc-tests.py

Remove the use of wildcard imports in rpc-tests.py and replace with
configparser.

1581ecbc33

jnewbery force-pushed on Feb 1, 2017

jnewbery commented at 2:09 am on February 1, 2017: member

@isle2983 @jtimon thanks for the reviews. I’ve addressed all comments in rebased/squashed commits. @jtimon - I’ve removed tests_config.py.in in the first commit (which changes all the commit hashes).

in qa/pull-tester/rpc-tests.py: in 0a49a41909 outdated

190-        test_list = []
191-        for t in test_scripts:
192-            if t in tests or re.sub(".py$", "", t) in tests:
193-                test_list.append(t)
194+        test_list = [t for t in test_scripts if 
195+                     (t in ALL_SCRIPTS or re.sub(".py$", "", t) in ALL_SCRIPTS)]

isle2983 commented at 4:51 pm on February 1, 2017:

This is incorrect and it prevents you from running a single test at a time e.g. $ qa/pull-tester/rpc-tests.py segwit

test_scripts can only be made up of BASE_SCRIPTS, ZMQ_SCRIPTS, and EXTENDED_SCRIPTS. ALL_SCRIPTS is all of those, so test_list will then always be set to all of test_scripts.

I believe it should be:

0test_list = [t for t in test_scripts if (t in tests or re.sub(".py$", "", t) in tests)]

jnewbery commented at 3:17 pm on February 2, 2017:

You’re completely right. I put the ALL_SCRIPTS in the wrong place.

Should be fixed now.

jonasschnelli added the label Tests on Feb 1, 2017

jnewbery force-pushed on Feb 2, 2017

isle2983 commented at 4:20 pm on February 2, 2017: contributor

ACK 117e0f2da241e8491b17a80b0acc34ac245d3842

Tested on Debian 8 environment with normal ./configure LDFLAGS="-L$BDB_BUILD/lib" CPPFLAGS="-I/$BDB_BUILD/include":

with and without --coverage
with and without --extended
with and without --nozmq
individual tests given as args
-j12 and with default jobs

Did not test:

passon_args
--win

jtimon commented at 9:47 pm on February 2, 2017: contributor

Bike-shedding history, I would have preferred that ALL_SCRIPTS is declared and used in https://github.com/bitcoin/bitcoin/pull/9657/commits/0eeb09ecaee969c0f9636b893c0c3a7283006459 otherwise the commit is broken. The comprehensive lists in https://github.com/bitcoin/bitcoin/pull/9657/commits/117e0f2da241e8491b17a80b0acc34ac245d3842 could also be squashed there. It makes sense to keep s/parallel/jobs separated. I’m not sure is worth it for you to fix the order in which things are done at this point, but I wanted to mention it. utACK 117e0f2

jnewbery commented at 10:16 pm on February 2, 2017: member

thanks @jtimon - you’re right about that historic commit being broken. I did try to merge back and rebase, but it caused merge conflicts, so I didn’t bother continuing.

If the maintainers really think this is important, I can go back and fix history, but these commits should be merged as a set, so I don’t think it matters.

jnewbery commented at 10:55 pm on February 2, 2017: member

Apologies for the review churn. I’ve just realised that my last commit introduced a redundant check on the names of the tests to run. I’ve removed that in commit https://github.com/bitcoin/bitcoin/pull/9657/commits/1bd87a63d36c79e5a019bc44e07a5a34d2bd51ea . Once review is complete I’ll squash that into the previous commit.

in qa/pull-tester/rpc-tests.py: in 1bd87a63d3 outdated

150+    enable_zmq = config["components"]["ENABLE_ZMQ"] == "True" and not args.nozmq
151+
152+    if config["environment"]["EXEEXT"] == ".exe" and not args.win:
153+        # https://github.com/bitcoin/bitcoin/commit/d52802551752140cf41f0d9a225a43e84404d3e9
154+        # https://github.com/bitcoin/bitcoin/pull/5677#issuecomment-136646964
155+        print("Win tests currently disabled by default.  Use -win option to enable")

MarcoFalke commented at 2:13 am on February 3, 2017:

Nit: This should use double dash, if you change it above.

MarcoFalke commented at 2:39 am on February 3, 2017: member

so I don’t think it matters.

It will break git bisect. Maybe you squash all commits that are problematic? Though, I won’t block the pull based on this.

Use argparse in rpc_tests.py

This commit replaces the roll-your-own argument parsing in rpc_tests.py
with Python's standard library argparse.

91bfffff5d

Improve rpc-tests.py arguments

A few miscellaneous improvements to rpc-tests.py command line arguments:

 - make all arguments start with double dash for consistency
 - improve help text and output
 - add nozmq argument to explicitly exclude the ZMQ tests
 - change 'parallel' to 'jobs'

afd38e7cc8

Refactor rpc-tests.py

- add main()
- remove global variables

3de3ccdfaa

jnewbery force-pushed on Feb 6, 2017

jnewbery commented at 2:13 pm on February 6, 2017: member

It will break git bisect.

Good point. I’ve now fixed up the history so rpc-tests.py should run at all commits. Branch is now at 3de3ccdfaaa5edcf4e0924860d960e7e0db5a9b9 . The only difference from the previous branch pointer 117e0f2da241e8491b17a80b0acc34ac245d3842 (ACK’ed by @jtimon and @isle2983 ) is that I’ve removed a redundant if test, changed a variable name from run_parallel to jobs and added the double dash to the help text for win.

jtimon commented at 6:40 pm on February 9, 2017: contributor

re-utACK 3de3ccdfaaa5edcf4e0924860d960e7e0db5a9b9 Thanks for cleaning up the history.

in qa/pull-tester/tests_config.ini.in: in 1581ecbc33 outdated

17+BUILDDIR=@abs_top_builddir@
18+EXEEXT=@EXEEXT@
19+
20+[components]
21+# Which components are enabled. These are commented out by `configure` if they were disabled when running config.
22+@ENABLE_WALLET_TRUE@ENABLE_WALLET=True

ryanofsky commented at 8:44 pm on February 16, 2017:

Capitalized “True”/“False” seems out of place in an .ini file. Maybe switch to lowercase “yes”/“no” as shown in the documentation: https://docs.python.org/3/library/configparser.html. “yes”/“no” are also understood by the git config tool (https://git-scm.com/docs/git-config#git-config-true), though the canonical form in git is “true”/“false”.

jnewbery commented at 6:43 pm on February 17, 2017:

Good point. I’ll change these to true and use the getboolean() method

in qa/pull-tester/tests_config.ini.in: in 1581ecbc33 outdated

0@@ -0,0 +1,25 @@
1+# Copyright (c) 2013-2016 The Bitcoin Core developers
2+# Distributed under the MIT software license, see the accompanying
3+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
4+
5+# These environment variables are set by the build process and read by
6+# rpc-tests.py
7+
8+[DEFAULT]

ryanofsky commented at 8:53 pm on February 16, 2017:

This section is redundant and could mislead someone who is reading this file into thinking these options are false when they aren’t. Suggest removing this section and replacing

0ENABLE_WALLET = config["components"]["ENABLE_WALLET"] == "True"

with

0ENABLE_WALLET = config["components"].get("ENABLE_WALLET") == "True"

or similar.

jnewbery commented at 6:34 pm on February 17, 2017:

Great! I wasn’t aware of the get() method. Agree that removing the defaults section is much better.

in qa/pull-tester/rpc-tests.py: in 91bfffff5d outdated

36+parser.add_argument('--coverage', action='store_true')
37+parser.add_argument('-extended', action='store_true')
38+parser.add_argument('--help', '-h', '-?', action='store_true')
39+parser.add_argument('--parallel', type=int, default=4)
40+parser.add_argument('-win', action='store_true')
41+(args, unknown_args) = parser.parse_known_args()

ryanofsky commented at 8:57 pm on February 16, 2017:

Parentheses unnecessary.

jnewbery commented at 6:43 pm on February 17, 2017:

removed

in qa/pull-tester/rpc-tests.py: in 91bfffff5d outdated

82-        passon_args.append(arg)
83-    elif PARALLEL_REGEX.match(arg):
84-        run_parallel = int(arg.split(sep='=', maxsplit=1)[1])
85-    else:
86-        opts.add(arg)
87+print_help = args.help

ryanofsky commented at 8:59 pm on February 16, 2017:

Get rid of these? Why have two names for the same thing?

in qa/pull-tester/rpc-tests.py: in 91bfffff5d outdated

183-    if '-extended' in opts:
184-        test_list = testScripts + testScriptsExt
185-    elif len(opts) == 0 or (len(opts) == 1 and "-win" in opts):
186-        test_list = testScripts
187+    # Build list of tests
188+    if len(tests) != 0:

ryanofsky commented at 9:03 pm on February 16, 2017:

Should avoid if len in python, https://www.python.org/dev/peps/pep-0008/

0Yes: if not seq:
1     if seq:
2
3No: if len(seq):
4    if not len(seq):

More occurrences a few places below, too.

jnewbery commented at 6:55 pm on February 17, 2017:

fixed

in qa/pull-tester/rpc-tests.py: in 91bfffff5d outdated

187+    # Build list of tests
188+    if len(tests) != 0:
189+        # Individual tests have been specified. Run specified tests that exist
190+        # in the ALL_SCRIPTS list. Accept the name with or without .py extension.
191+        test_list = [t for t in ALL_SCRIPTS if
192+                (t in tests or re.sub(".py$", "", t) in tests)]

ryanofsky commented at 9:04 pm on February 16, 2017:

Seems like “t in tests or " is redundant.

jnewbery commented at 7:19 pm on February 17, 2017:

This is required if the user specifies a test with the .py suffix, eg:

rpc-tests.py rpcnamedargs.py

in qa/pull-tester/rpc-tests.py: in 1581ecbc33 outdated

42-RPC_TESTS_DIR = SRCDIR + '/qa/rpc-tests/'
43+# Read config generated by configure.
44+config = configparser.ConfigParser()
45+config.read_file(open(os.path.dirname(__file__) + "/tests_config.ini"))
46+
47+ENABLE_WALLET = config["components"]["ENABLE_WALLET"] == "True"

ryanofsky commented at 9:10 pm on February 16, 2017:

Maybe just use the config object directly below instead of defining these variables, since they are only used once. This will cut down on the number of global variables in this script, and make it clearer from the context where the settings are used where the settings come from.

jnewbery commented at 6:47 pm on February 17, 2017:

yep - I set this as a global variable to match the existing layout. It’s been moved to be a local variable in the subsequent refactor commit.

I could remove the variable entirely and access the config directly in the if test, but I don’t think it makes much difference at this point.

in qa/pull-tester/rpc-tests.py: in afd38e7cc8 outdated

39+parser.add_argument('--coverage', action='store_true', help='generate a basic coverage report for the RPC interface')
40+parser.add_argument('--extended', action='store_true', help='run the extended test suite in addition to the basic tests')
41+parser.add_argument('--help', '-h', '-?', action='store_true', help='print help text and exit')
42+parser.add_argument('--jobs', '-j', type=int, default=4, help='how many test scripts to run in parallel. Default=4.')
43+parser.add_argument('--nozmq', action='store_true', help='do not run the zmq tests')
44+parser.add_argument('--win', action='store_true', help='signal that this is running in a Windows environment and that we should run the tests')

ryanofsky commented at 9:20 pm on February 16, 2017:

Since you’re adding a dash to this argument maybe consider renaming it something comprehensible like –force. And changing help text to “Run tests even on platforms where they are disabled by default (e.g. windows).”

The “signal that this is running in a windows environment” just comes off as strange, since the script obviously knows what environment it is running in.

jnewbery commented at 6:50 pm on February 17, 2017:

Yes. @jtimon made the same point that the help text was nonsense. Good point about changing the name of the parameter since I’m already breaking backward compatibility :)

I’ll change this to –force and update the help text.

in qa/pull-tester/rpc-tests.py: in 3de3ccdfaa outdated

84-        print("ERROR: \"import zmq\" failed. Use -nozmq to run without the ZMQ tests."
85-              "To run zmq tests, see dependency info in /qa/README.md.")
86-        raise
87-
88 BASE_SCRIPTS= [
89+    # Scripts that are run by the travis build process

ryanofsky commented at 9:28 pm on February 16, 2017:

These two lines look like one sentence, would add punctuation.

in qa/pull-tester/rpc-tests.py: in 3de3ccdfaa outdated

202         sys.exit(0)
203 
204-    coverage = None
205+    runtests(test_list, config["environment"]["SRCDIR"], config["environment"]["BUILDDIR"], config["environment"]["EXEEXT"], args.jobs, args.coverage, passon_args)
206+
207+def runtests(test_list, src_dir, build_dir, exeext, jobs=1, enable_coverage=False, args=[]):

ryanofsky commented at 9:36 pm on February 16, 2017:

Maybe call it run_tests for pep8.

ryanofsky commented at 9:38 pm on February 16, 2017: member

Nice changes. Left some comments but they are minor. The one change I would really like to see is removing the DEFAULT section from config.ini, though.

ryanofsky commented at 11:12 am on February 17, 2017: member

utACK 3de3ccdfaaa5edcf4e0924860d960e7e0db5a9b9

Various review markups for rpc-tests.py improvements a6a3e58c48

jnewbery commented at 7:24 pm on February 17, 2017: member

Thanks for the thorough review @ryanofsky . I’ve applied most of those markups in a new commit. I haven’t squashed them into the separate commits since there was quite a lot of code churn within those commits and it’d be tedious to get the history right.

in qa/pull-tester/rpc-tests.py: in 3de3ccdfaa outdated

354@@ -340,10 +355,10 @@ def _get_uncovered_rpc_commands(self):
355 
356         """
357         # This is shared from `qa/rpc-tests/test-framework/coverage.py`
358-        REFERENCE_FILENAME = 'rpc_interface.txt'
359-        COVERAGE_FILE_PREFIX = 'coverage.'
360+        reference_filename = 'rpc_interface.txt'
361+        coverage_file_prefix = 'coverage.'

MarcoFalke commented at 1:30 pm on February 18, 2017:

Even though of local scope, those still denote global constants, so should be ALL_UPPER_CASE?

in qa/pull-tester/rpc-tests.py: in a6a3e58c48

230+        os.environ["BITCOIND"] = build_dir + '/src/bitcoind' + exeext
231+
232+    tests_dir = src_dir + '/qa/rpc-tests/'
233 
234-    if ENABLE_COVERAGE:
235+    flags = ["--srcdir=" + src_dir] + args

MarcoFalke commented at 1:48 pm on February 18, 2017:

I know this has no effect, but this was build_dir previously and should not be changed in this commit.

in qa/pull-tester/rpc-tests.py: in a6a3e58c48

132+                                     formatter_class=argparse.RawTextHelpFormatter)
133+    parser.add_argument('--coverage', action='store_true', help='generate a basic coverage report for the RPC interface')
134+    parser.add_argument('--extended', action='store_true', help='run the extended test suite in addition to the basic tests')
135+    parser.add_argument('--force', '-f', action='store_true', help='run tests even on platforms where they are disabled by default (e.g. windows).')
136+    parser.add_argument('--help', '-h', '-?', action='store_true', help='print help text and exit')
137+    parser.add_argument('--jobs', '-j', type=int, default=4, help='how many test scripts to run in parallel. Default=4.')

MarcoFalke commented at 1:56 pm on February 18, 2017:

If you change the name of the arguments, make sure to update the docs https://github.com/bitcoin/bitcoin/tree/master/qa#running-tests and maybe even put a tiny notice in the release notes that says that some of the arguments to rpc-test.py have changed. Alternatively you could provide aliases for compatibility.

MarcoFalke commented at 2:23 pm on February 18, 2017: member

utACK a6a3e58c483700bd5f2bc7d117a621e0c4af60e3

There might be some minor issues (see feedback), but I think this is ready for merge nonetheless, as it comes with some great features that make future changes to the test framework a lot easier.

The feedback may be addressed in a separate pull, otherwise it would slow down progress in other test related pulls.

MarcoFalke commented at 2:28 pm on February 18, 2017: member

Thanks @ryanofsky for suggesting the getboolean() method, this is indeed cleaner code.

MarcoFalke merged this on Feb 18, 2017

MarcoFalke closed this on Feb 18, 2017

MarcoFalke referenced this in commit 7ff4a538a8 on Feb 18, 2017

jnewbery commented at 4:34 am on February 19, 2017: member

Thanks @MarcoFalke . I’ll make sure to track that feedback and address it early next week.

PastaPastaPasta referenced this in commit 240a3a628a on Dec 28, 2018

PastaPastaPasta referenced this in commit fb3367a916 on Dec 29, 2018

PastaPastaPasta referenced this in commit fa43a84f04 on Dec 29, 2018

UdjinM6 referenced this in commit a49f4123e5 on Jan 3, 2019

PastaPastaPasta referenced this in commit ad25b0049c on Jan 3, 2019

PastaPastaPasta referenced this in commit a679a6a709 on Jan 3, 2019

PastaPastaPasta referenced this in commit e4520e4a44 on Jan 3, 2019

PastaPastaPasta referenced this in commit 62cea47a78 on Jan 5, 2019

PastaPastaPasta referenced this in commit 36f412a5b7 on Jan 7, 2019

PastaPastaPasta referenced this in commit 957622da50 on Jan 25, 2019

PastaPastaPasta referenced this in commit 757e70b116 on Jan 29, 2019

CryptoCentric referenced this in commit c632f583ae on Jul 11, 2019

zkbot referenced this in commit 025bd44543 on Nov 21, 2020

zkbot referenced this in commit 7a0a268054 on Dec 2, 2020

zkbot referenced this in commit c8896f9907 on Dec 2, 2020

MarcoFalke locked this on Sep 8, 2021

Improve rpc-tests.py #9657