Add "it works" test for bitcoin-tx #4624

This is our first execution test (executing a util, checking the result). The intention is to introduce a very simple test for bitcoin-tx, to better gauge the needs to execution tests.

Good to have tests for bitcoin-tx.

Nit: I'd prefer to use Python for the test executor. It allows doing some more advanced checks without getting a very messy shell script when processing JSON and such (the tests in qa/rpc-tests were moved to Python for this reason too).

@laanwj Well, two comments on that,

• The RPC tests appeared to be ultimately driven by shell scripts, which was the genesis of this direction. I suppose I could add another layer here, but I don't how you could get much more simple than this.
• For a command line util, a shell script + statically prepared data output for cmp(1) is typically all you need.

That said, I am OK with adding some prepared python test driver if you have a favorite one pre-written.

For now, a simple shell 'for' loop that (1) executes a command line and (2) compares output with statically prepared output should cover a wide range of tests with one small shell script.

Well for the RPC tests we got all kinds of very subtle edge cases. Making a robust shell script that doesn't shoot you in the foot in unexpected conditions (which can be as mild as spaces in directory names) is harder then it seems. And don't forget the subtle differences between shells on different platforms. That's all a lot easier to get right with Python. Although granted, the case here is easier as there is no need to set up test nodes and kill them afterwards.

I don't insist on converting it to Python, but it would be unfortunate to have the same realizations again with yet another built-in test framework.

Anyhow - ACK after this passed pulltester

@laanwj Yes. I think "AM_TESTS_ENVIRONMENT" is a new feature, and it doesn't exist in our older pulltester/gitian environment. Gotta spin up a VM and check it out.

@jgarzik If we're relying on new automake behavior, then we need to set the minimum supported version in AM_INIT_AUTOMAKE. A quick google isn't turning up which version introduced it, though...

@theuni Just need some way to communicate $(top_srcdir) to the test script. Thus perhaps there is a way to accomplish that without AM_TESTS_ENVIRONMENT.

@jgarzik If it's just this one, you can just move it to bitcoin-tx-test.sh.in and use @top_srcdir@. If there's reason to think we'll need to do it again, we should add some kind of stub test-runner.

After a good bit of research on this one, turns out there's a stupidly easy answer. srcdir is exported already.

diff --git a/src/test/bitcoin-tx-test.sh b/src/test/bitcoin-tx-test.sh
index 5887b1c..b255e7d 100755
--- a/src/test/bitcoin-tx-test.sh
+++ b/src/test/bitcoin-tx-test.sh
@@ -1,7 +1,7 @@
 #!/bin/sh -x

 TMP_FN=testcmp.$$
-BASE_FN=$TOP_SRCDIR/src/test/data/blanktx.hex
+BASE_FN=$srcdir/test/data/blanktx.hex

 ./bitcoin-tx -create > $TMP_FN
 if [ $? -ne 0 ]; then

Then the env stuff can go away and we don't have to worry about automake versions.

@theuni doesn't seem to work for me;

=============================================
   Bitcoin Core 0.9.99: src/test-suite.log
=============================================

# TOTAL: 3
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: test/bitcoin-tx-test.sh
=============================

+ TMP_FN=testcmp.27109
+ BASE_FN=./src/test/data/blanktx.hex
+ ./bitcoin-tx -create
+ [ 0 -ne 0 ]
+ cmp --verbose testcmp.27109 ./src/test/data/blanktx.hex
cmp: ./src/test/data/blanktx.hex: No such file or directory
+ RET=2
+ rm -f testcmp.27109
+ [ 2 -ne 0 ]
+ exit 1

Looks like $srcdir is relative to the root of the project, and the test is executed in src/.

Updated to switch to python framework.

NOTE: "make check" works, but "make distcheck" does not. Need to dig into that, perhaps with @theuni help...

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/p4624_fb14452c6cadb8d977c405dddb0a94115250d7c4/ for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

ACK

How do I use this? Currently Travis is complaining about a recent pull request I raised (https://travis-ci.org/bitcoin/bitcoin/jobs/38996464) and it appears to be this where the error is appearing. Is there any documentation on how to debug when it fails?

@rebroad Standard autoconf test suite execution, standard autoconf test documentation.

bitcoin-util-test.py executes tests from test/data/bitcoin-util-test.json.

Run "make check" and individual results will be found in *.log and *.trs files.