Unit test logs should be written to build path, not source path #17224

When doing out-of-tree builds it's not expected that the build (or tests) clutter the source path. All build output should be written to the build path (the current directory when configure is run) and subdirectories of that. (theoretically it should even be possible to have the source on a read-only mount, but that's not really important imo, the point is to not clutter the git repository with build output)

However, after running make check there will be a while slew of *.log files in the source directory:

bitcoin$ ./autogen.sh
bitcoin$ make build
bitcoin$ cd build
bitcoin/build$ ../configure ...
bitcoin/build$ make check
bitcoin/build$ cd ..
bitcoin$ ls src/test/*.log
src/test/addrman_tests.cpp.log            src/test/compress_tests.cpp.log         src/test/netbase_tests.cpp.log          src/test/sigopcount_tests.cpp.log
src/test/allocator_tests.cpp.log          src/test/crypto_tests.cpp.log           src/test/net_tests.cpp.log              src/test/skiplist_tests.cpp.log
src/test/amount_tests.cpp.log             src/test/cuckoocache_tests.cpp.log      src/test/pmt_tests.cpp.log              src/test/streams_tests.cpp.log
src/test/arith_uint256_tests.cpp.log      src/test/dbwrapper_tests.cpp.log        src/test/policyestimator_tests.cpp.log  src/test/sync_tests.cpp.log
...

Either these files should be written to a temporary path (like the other test output), or to the build directory.

@laanwj Hi. I searched the code for this issue and i'm wondering if i should take this actions:

1. In Make.test.include changing all test/... to ../build/test/... which seems not to be an efficient solution :(.

OR

2. In Makefile:line 2241 if test -f "./$$f"; then dir=./; \ elif test -f "$$f"; then dir=; \ else dir="$(srcdir)/"; fi; \ tst=$$dir$$f; log='$@'; I think we should change $(srcdir) here to build directory so that test logs will save in build sub directory which seems much better.

In Make.test.include changing all test/... to ../build/test/... which seems not to be an efficient solution :(.

I think that would be incorrect. The source should stay in the source path.

I think we should change $(srcdir) here to build directory so that test logs will save in build sub directory which seems much better.

Yes, I think that would be better. The idea is to get the .log files to be written somewhere else, but keep everything else the same.

Where can i change this? It seems this file (Makefile in src/) is created after make and does not exist before making process.

Hi,

After some investigation and reading and trying out various Makefile settings, I think I've found the reason why the logs always land in tests directory.

It has to do with the fact that autogen.sh is automatically generating test-driver in build/aux.

test-driver itself has an option, --log-file, but because there is currently no way to access it before the compilation is kicked off, the settings fall back to their default: generate log files in current dir.

Here's the snippet of test-driver's params:

while test $# -gt 0; do
  case $1 in
  --help) print_usage; exit $?;;
  --version) echo "test-driver $scriptversion"; exit $?;;
  --test-name) test_name=$2; shift;;
  --log-file) log_file=$2; shift;;
  --trs-file) trs_file=$2; shift;;
  --color-tests) color_tests=$2; shift;;
  --expect-failure) expect_failure=$2; shift;;
  --enable-hard-errors) enable_hard_errors=$2; shift;;
  --) shift; break;;
  -*) usage_error "invalid option: '$1'";;
   *) break;;
  esac
  shift
done

Further down in its code there is also a line where the logs get written.

# Test script is run here.
"$@" >$log_file 2>&1

I've seen some projects keeping the test-driver under revision control, because they prefer to build & configure it without autogen.sh. Maybe this strategy could be applied here as well? Just in case it's fitting to the overall architecture, of course.

Regards,

@brakmic thanks for the thorough investigation!

Anyone is working on this? Otherwise, I would like to take a stab on it

Anyone is working on this? Otherwise, I would like to take a stab on it @glaksmono Feel free to work on it.

hey @fanquake, @laanwj,

Should we make a temp folder under ./tests/temp or ./src/tests/temp for all of the log files?

I presume that the temp folder should be created the first time we're running make check?

Thoughts?

Just putting them in $build_patch/src/test/ should be enough, no?

Not sure how automake works, but there should be a symbol abs_top_builddir somewhere, which is the build path

This issue has been around for a while so I thought I'd take a stab at it. @MarcoFalke thanks for the suggestion about using abs_top_builddir.

@MarcoFalke Sorry for the confusion with the comments. I deleted my comment and in the 10 seconds it took me to write a new comment you replied xD

@MarcoFalke could we move the discussion to the PR?

Is this issue fixed by @yancyribbens PR? I would like to work on this if something else needs to be done.

I think this can be closed now?