Motivation - skeleton.py is a template for RPC tests. instead of creating a copy each time someone creates a new test (difficult to maintain and update fixes), new tests can simply inherit the Skeleton class in skeleton.py.
What i did -
Not all tests may want to have the same setup steps or arguments. Defining a rigid base class limits flexibility.
For example tests may want a different number of nodes, different command line arguments, or nodes with special configuration, etc.
If you take this into account in a base class you may end up with some huge overdesigned system (which people have to learn before they can contribute tests).
For very different test setups, a common base class may not work. The thing is, Gavin just made some changes to the RPC testing(#3863), for now we have only one test that implements the skeleton test, what if we had a 100 that were based on the skeleton .. we would have had to go and fix manually all of them.
Well in the least you then need to have methods on the skeleton object (that can be overridden) that provide all the configurable steps such as command line arguments, node setup, ...
If subclasses end up overriding everything in the skeleton it will be exactly the same and you still need to manually fix everything.
130 | - nodes = start_nodes(2, options.tmpdir) 131 | - connect_nodes(nodes[1], 0) 132 | + def run(self): 133 | + super(ListtransactionsTest, self).main() 134 | + 135 | + def check_array_result(self,object_array, to_match, expected):
This is a generic utility function, no need to make it part of a class (ie, no use of self or global state).
Fixed
12 | +# 1) Change class name to your test 13 | +class MyTest(baseclass): 14 | + 15 | + # 2) Change 'MyTest' to class name 16 | + def run(self): 17 | + super(MyTest, self).main()
Why add this method, and not just call MyTest().main() in step 4?
Fixed it.
I can start creating methods that the parent class can override (init_nodes, stop_nodes,parse_options,etc. ).... but i think the current setup is pretty wide ... you have 200 blocks with 4 nodes ... i think can cover a lot of tests, don't you ?
I do not want to make that judgement for others.
So, I think that the base idea here is good, but it needs finer customizability in the base class. Adding init_nodes, stop_nodes, parse_options etc, as you said, makes sense.
Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/p3956_48b991a930d516a74cf525aaa02159ce0d857aac/ for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.
Closing this because of inactivity.