RPC tests don't use correct bitcoind in ordinary build layout #7002

bitcoin $ qa/rpc-tests/bip65-cltv-p2p.py
Initializing test directory /tmp/testY3PdTa
MiniNode: Connecting to Bitcoin Node IP # 127.0.0.1:11198
Test 1: PASS [100]
Test 2: PASS [849]
Test 3: PASS [850]
Assertion failed: Test failed at test 4
  File "/home/luke-jr/Projects/Education/Tonal/BitCoin/bitcoin/qa/rpc-tests/test_framework/test_framework.py", line 135, in main
    self.run_test()
  File "qa/rpc-tests/bip65-cltv-p2p.py", line 54, in run_test
    test.run()
  File "/home/luke-jr/Projects/Education/Tonal/BitCoin/bitcoin/qa/rpc-tests/test_framework/comptool.py", line 323, in run
    raise AssertionError("Test failed at test %d" % test_number)
Stopping nodes
Cleaning up
Failed

This fails even on commit 3e187f2acdb69ceb97a875f8f3bf361c11ef543c (#6894), so I'm not sure how that got merged?

Strange - travis is passing.

Doesn't it include this test?

FWIW it succeeds on my machine (both 3e187f2 and master).

Can you run that test with --nocleanup and share your debug file somewhere?

(Also this is an extended test, so travis doesn't run it. Perhaps it should.)

Not sure what debug file that is. http://luke.dashjr.org/tmp/code/testQ8BQP7.txz

Looks like you're running a version of bitcoind that doesn't have BIP65 support? From tmp/testQ8BQP7/node0/regtest/debug.log:

2015-11-13 11:44:34 Bitcoin version v0.11.1.0-gcf33f19 (Mon, 12 Oct 2015 20:58:31 +0200)

Is it possible you have some environment variables set that point at an 0.11.1 binary? I think the python testing framework will search for other binaries in the $BITCOIND environment variable before defaulting to the bitcoind built in your checked out directory.

Ugh, the default really ought to be <builddir>/src/ so it finds the right binary. Setting BITCOIND explicitly fixed this.

Is there a reason why the default isn't set a reasonable way?

How does it detect what bitcoind to use, and how should it detect it?

Looks like it does exactly what you say it should be doing. It writes the BUILDDDIR into tests_config.py then uses src/bitcoind of that: https://github.com/bitcoin/bitcoin/blob/master/qa/pull-tester/rpc-tests.py#L65

@luke-jr Mind to share steps to reproduce?

This is an old issue and since then, test_runner.py has been introduced. I ran this specific test twice, first with a bitcoind process running from /usr/bin/ and second time after killing it. Notice on first run, there is now a warning message if pidof returns an id for bitcoind.

$ ./test/functional/test_runner.py bip65-cltv-p2p.py 
Temporary test directory at /tmp/bitcoin_test_runner_20180114_205646
WARNING! There is already a bitcoind process running on this system. Tests may fail unexpectedly due to resource contention!
........................
bip65-cltv-p2p.py passed, Duration: 12 s

TEST              | STATUS    | DURATION

bip65-cltv-p2p.py | ✓ Passed  | 12 s

ALL               | ✓ Passed  | 12 s (accumulated) 
Runtime: 12 s

$ ./test/functional/test_runner.py bip65-cltv-p2p.py 
Temporary test directory at /tmp/bitcoin_test_runner_20180114_205729
.........................
bip65-cltv-p2p.py passed, Duration: 13 s

TEST              | STATUS    | DURATION

bip65-cltv-p2p.py | ✓ Passed  | 13 s

ALL               | ✓ Passed  | 13 s (accumulated) 
Runtime: 13 s

As of commit 3e187f2 the folder with the built bitcoind was not detected correctly. This was fixed in #8018