[QA] restructure rpc tests directory #6097

• move non-test classes to subdir test-framework
• added all test script into two sections to pull-tester/rpc-tests.sh
• updated readme

rebased

Fixed travis issue. Rebased.

No opinion.

The benefits of this pull is not just another type of structure instead it's a clean distinction between test classes/files and utility classes/files. At the moment you can't distinguish between a runnable test and a utility class within the folder ./qa/rpc-tests. Since #5981 it's even more mixed up. It's hard to get a overview of all possible test we have (thats why i extended rpc-test.sh).

Basically: it's a mess.

IMO messy qa/test setups end up in messy tests and bad test coverage. Therefore it should be clean at the root. This simplifies the test setup and allow upcoming contributors to reduce the learn-curve of how a test should be written.

Concept ACK. I agree that the rpc-tests directory is a bit messy, and I have found it difficult in the past to just find all the tests that I can run. Making rpc-tests.sh be a canonical place to put new scripts so others know to run them seems like a helpful thing which I plan to use.

I don't have much opinion about how to best organize python code, but what you have done with the test_framework directory looks okay to me.

One thing I noticed -- I think script_test.py should probably be commented out of the extended test scripts. Because it's just comparing behavior, it really doesn't make sense to run except with two different bitcoind binaries, which we don't have a way of doing in an automated way right now. If the test were to give an error, it would almost certainly mean that the test is itself broken, and it's extremely slow to run.

concept ACK, tidier is better.

concept ACK, this was in need of some organization for sure.

Concept ACK, I like moving the utility functions to a python package so that the top-level files are runnable scripts (needs rebase, though).

Fixed @sdaftuar's nit (disabled script_test.py). Rebased.

It seems you forgot to update rpcbind_test.py for the new python-bitcoinrpc path.

Speaking of which - let's move the bitcoinrpc directory directly into rpc-tests so that the Add python-bitcoinrpc to module search path hackery (in four places!) is no longer needed and we can just do:

from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException

... directly

Totally agreed. I just remove the python-bitcoinrpc completely and placed the authproxy.py file directly into the new test-framework folder (it's not a submodule so it can be treated as normal utility class). Also fixed the rpcbind_test.py as mentioned by @laanwj

@jonasschnelli Nice solution