zapwallettxes.py test failure due to bitcoind return code #10678

In #10677 there was what looks like a new kind of Travis false positive, or at least an intermittent error. It is triggered by the assertion on return code added in #10636:

zapwallettxes.py failed, Duration: 11 s
stdout:
2017-06-27 03:50:43.814000 TestFramework (INFO): Initializing test directory /tmp/bitcoin_test_runner_20170627_034653/zapwallettxes_338
2017-06-27 03:50:46.305000 TestFramework (INFO): Mining blocks...
2017-06-27 03:50:54.465000 TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/test_framework.py", line 153, in main
    self.run_test()
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/zapwallettxes.py", line 67, in run_test
    self.stop_node(0)
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/test_framework.py", line 217, in stop_node
    _stop_node(self.nodes[num_node], num_node)
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/util.py", line 326, in _stop_node
    assert_equal(return_code, 0)
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/util.py", line 440, in assert_equal
    raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(1 == 0)
2017-06-27 03:50:54.466000 TestFramework (INFO): Stopping nodes
stderr:
Traceback (most recent call last):
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/authproxy.py", line 126, in _request
    return self._get_response()
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/authproxy.py", line 169, in _get_response
    http_response = self.__conn.getresponse()
  File "/usr/lib/python3.4/http/client.py", line 1208, in getresponse
    response.begin()
  File "/usr/lib/python3.4/http/client.py", line 380, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.4/http/client.py", line 350, in _read_status
    raise BadStatusLine(line)
http.client.BadStatusLine: ''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/zapwallettxes.py", line 80, in <module>
    ZapWalletTXesTest ().main ()
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/test_framework.py", line 172, in main
    self.stop_nodes()
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/test_framework.py", line 220, in stop_nodes
    _stop_nodes(self.nodes)
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/util.py", line 334, in _stop_nodes
    _stop_node(node, i)
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/util.py", line 321, in _stop_node
    node.stop()
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/coverage.py", line 46, in __call__
    return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/authproxy.py", line 152, in __call__
    response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
  File "/home/travis/build/bitcoin/bitcoin/build/bitcoin-x86_64-unknown-linux-gnu/test/functional/test_framework/authproxy.py", line 130, in _request
    self.__conn.request(method, path, postdata, headers)
  File "/usr/lib/python3.4/http/client.py", line 1125, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1163, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.4/http/client.py", line 1121, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.4/http/client.py", line 951, in _send_output
    self.send(msg)
  File "/usr/lib/python3.4/http/client.py", line 886, in send
    self.connect()
  File "/usr/lib/python3.4/http/client.py", line 863, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python3.4/socket.py", line 512, in create_connection
    raise err
  File "/usr/lib/python3.4/socket.py", line 503, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

I'm not able to reproduce this locally. It's likely that the node crashed in some way, but this doesn't give enough information to extract what happened.

Looks like this was a one-time problem, haven't seen it again.

This is still an issue and I hypothesize it has something to do with setting the return code after a shutdown was requested quickly after startup. Though, I haven't yet looked at a fix...

Perhaps we should just comment out zapwallettxes.py for now? zapwallettxes doesn't actually test anything due to a syntax error (will be fixed in #10330)

Fixed by @TheBlueMatt in #11831