After long time, I'm trying to pass all the tests. Using current master:
for par in 1 2 4
do
for round in 1 2 3 4 5 6 7 8 9 10
do
echo Par: $par, round: $round
rm -rf qa/cache/; qa/pull-tester/rpc-tests.py -parallel=$par > round-$par-$round.log 2>&1
done
done
And I'm not able to pass the testsuite completely.
On OS X, the testsuite fails randomly with:
Error: Unable to start HTTP server. See debug log for details.
The debug.log of one node then contains something like:
2016-12-10 21:21:34 Binding RPC on address ::1 port 17082 failed.
2016-12-10 21:21:34 Binding RPC on address 127.0.0.1 port 17082 failed.
I have read in other issues (https://github.com/bitcoin/bitcoin/issues/8425#issuecomment-238127542), that the default to run 4 tests in parallel can cause some issues, so I have tried to isolate this issue by removing the parallelisation. But it did not helped completely, but the frequency of such failures is definitely small with less parallelism...
I will post detailed results and analyse of the above script tomorrow.
Similar issue was reported by @fanquake at #8425 (comment)
One of my favourite failure is:
Initializing test directory /var/folders/65/fn0h49r55k7779vg1b_h461r0000gn/T/testye4vq40s/424
Mining blocks...
Assertion failed: 0E-8 != 53.99954800
Stopping nodes
Not cleaning up dir /var/folders/65/fn0h49r55k7779vg1b_h461r0000gn/T/testye4vq40s/424
Failed
stderr:
Error: Unable to start HTTP server. See debug log for details.
File "/private/tmp/bitcoin-master/qa/rpc-tests/test_framework/test_framework.py", line 145, in main
self.run_test()
File "/tmp/bitcoin-master/qa/rpc-tests/wallet.py", line 212, in run_test
assert_equal(self.nodes[2].getbalance(), node_2_bal) #should not be changed because tx was not broadcasted
File "/private/tmp/bitcoin-master/qa/rpc-tests/test_framework/util.py", line 529, in assert_equal
raise AssertionError("%s != %s"%(str(thing1),str(thing2)))
