Fixes another functional test race. disconnect_ban.py bans a subnet with a one second expiry and then immediately tests that the subnet is banned. Due to the #10234, calls to listbanned() now result in a disk flush, which can block on busy systems (for example when running multiple functional tests in parallel).
Fix is to use mocktime to control when the ban expires.
Sorry, but I was not able to reproduce the failure on my systems, so I can't provide tested ACK.
Out of 1000 runs, I have got 8 of this only:
2017-05-10 08:58:54.014000 TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/private/tmp/bitcoin-master/test/functional/test_framework/util.py", line 281, in start_nodes
rpcs.append(start_node(i, dirname, extra_args[i], rpchost, timewait=timewait, binary=binary[i]))
File "/private/tmp/bitcoin-master/test/functional/test_framework/util.py", line 242, in start_node
wait_for_bitcoind_start(bitcoind_processes[i], url, i)
File "/private/tmp/bitcoin-master/test/functional/test_framework/util.py", line 219, in wait_for_bitcoind_start
blocks = rpc.getblockcount()
File "/private/tmp/bitcoin-master/test/functional/test_framework/coverage.py", line 46, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/private/tmp/bitcoin-master/test/functional/test_framework/authproxy.py", line 152, in __call__
response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
File "/private/tmp/bitcoin-master/test/functional/test_framework/authproxy.py", line 125, in _request
self.__conn.request(method, path, postdata, headers)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 712, in create_connection
raise err
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 703, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/private/tmp/bitcoin-master/test/functional/test_framework/test_framework.py", line 147, in main
self.setup_network()
File "/private/tmp/bitcoin-master/test/functional/test_framework/test_framework.py", line 81, in setup_network
self.setup_nodes()
File "/private/tmp/bitcoin-master/test/functional/test_framework/test_framework.py", line 94, in setup_nodes
self.nodes = start_nodes(self.num_nodes, self.options.tmpdir, extra_args)
File "/private/tmp/bitcoin-master/test/functional/test_framework/util.py", line 283, in start_nodes
stop_nodes(rpcs)
File "/private/tmp/bitcoin-master/test/functional/test_framework/util.py", line 303, in stop_nodes
assert not bitcoind_processes.values() # All connections must be gone now
AssertionError
2017-05-10 08:58:54.017000 TestFramework (INFO): Stopping nodes
Traceback (most recent call last):
File "./disconnect_ban.py", line 105, in <module>
DisconnectBanTest().main()
File "/private/tmp/bitcoin-master/test/functional/test_framework/test_framework.py", line 163, in main
self.stop_nodes()
File "/private/tmp/bitcoin-master/test/functional/test_framework/test_framework.py", line 209, in stop_nodes
stop_nodes(self.nodes)
File "/private/tmp/bitcoin-master/test/functional/test_framework/util.py", line 301, in stop_nodes
for i, node in enumerate(nodes):
TypeError: 'NoneType' object is not iterable
Ie. the already known and reported issue #9317 (comment), nothing else.
@paveljanik - can you try running the test when you have a lot of disk access happening in parallel which could block disk i/o for the node under test. One way to do this is to run in parallel with pruning.py which does a lot of stop/starts (and therefore flushes to disk a lot). You can do that by changing the BASE_SCRIPTS list in test_runner to something like:
BASE_SCRIPTS= [
'pruning.py',
'disconnect_ban.py --portseed=1',
'disconnect_ban.py --portseed=2',
'disconnect_ban.py --portseed=3',
'disconnect_ban.py --portseed=4',
...
utACK 3ba2c08980dddb3c109a144a818f23a9188426c3
Concept ACK
I was not able to reproduce, sorry.
But reading the code, utACK https://github.com/bitcoin/bitcoin/commit/3ba2c08980dddb3c109a144a818f23a9188426c3