zmq_test.py fails with out-of-tree builds #10268

zmq_test.py, when run from test/functional/test_runner.py, fails when running it with an out-of-tree build:

zmq_test.py failed, Duration: 0 s

stdout:
2017-04-24 13:26:51.336000 TestFramework (INFO): Initializing test directory /tmp/testvfbalauk/558
2017-04-24 13:26:51.460000 TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
  File "/store/orion/projects/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 150, in main
    self.setup_network()
  File "/store/orion/projects/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 62, in setup_n
etwork
    self.nodes = self.setup_nodes()
  File "/home/orion/projects/bitcoin/bitcoin/test/functional/zmq_test.py", line 32, in setup_nodes
    config.read_file(open(os.path.dirname(__file__) + "/config.ini"))
FileNotFoundError: [Errno 2] No such file or directory: '/home/orion/projects/bitcoin/bitcoin/test/functional/config.ini'
2017-04-24 13:26:51.477000 TestFramework (INFO): Stopping nodes


stderr:
Traceback (most recent call last):
  File "/home/orion/projects/bitcoin/bitcoin/test/functional/zmq_test.py", line 110, in <module>
    ZMQTest ().main ()
  File "/store/orion/projects/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 166, in main
    stop_nodes(self.nodes)
  File "/store/orion/projects/bitcoin/bitcoin/test/functional/test_framework/util.py", line 380, in stop_nodes
    for i, node in enumerate(nodes):
TypeError: 'NoneType' object is not iterable

I think the problem was introduced in 6803e09e6eeb2dbc820bb1025475a50a4352e30d. This added config.read_file(open(os.path.dirname(__file__) + "/config.ini")) to test/functional/zmq_test.py.

Note that with out-of-tree builds, only test/functional/test_runner.py is symlinked, not all of the tests. The rest is simply executed in-place in the source directory. So individual tests won't find a config.ini. A potential solution would be to allow overriding the config.ini location through an environment variable or command line argument.

Same happens when launching the test manually:

BITCOIND=$BUILDDIR/src/bitcoind $SRCDIR/bitcoin/test/functional/zmq_test.py 

In a pinch it can be worked around by adding a link in the source directory (ugly!)

ln -s $BUILDDIR/test/functional/config.ini $SRCDIR/test/functional/config.ini

Yes, this was almost certainly broken by my change in #10097 :( I don't generally build out-of-tree and expect Travis to pick up the cases where building out of tree could cause problems. In this case, Travis doesn't fail because it's building bitcoin in a distdir, which does include all the test and config files in test/functional.

#10307 should fix this by passing a new --configfile parameter from the test_runner to the individual tests. I've tested locally on an out-of-tree build and it seems to work.

I don't generally build out-of-tree and expect Travis to pick up the cases where building out of tree could cause problems

Ah yes I also wondered why Travis wouldn't pick it up. Anyhow, if you break out-of-tree builds you can be sure that I notice :)