Intermittent FAIL: test/test

laanwj commented at 3:25 pm on February 22, 2017: member

This seems to happen at least on 0.14:

 0FAIL: test/test_bitcoin
 1=======================
 2test_bitcoin: /home/travis/build/bitcoin/bitcoin/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/recursive_mutex.hpp:113: void boost::recursive_mutex::lock(): Assertion `!pthread_mutex_lock(&m)' failed.
 3Running 228 test cases...
 4unknown location(0): fatal error: in "rpc_tests/rpc_rawparams": signal: SIGABRT (application abort requested)
 5test/rpc_tests.cpp(29): last checkpoint
 6test_bitcoin: key.cpp:300: void ECC_Start(): Assertion `secp256k1_context_sign == __null' failed.
 7unknown location(0): fatal error: in "rpc_tests/rpc_format_monetary_values": signal: SIGABRT (application abort requested)
 8...
 9*** 37 failures are detected in the test module "Bitcoin Test Suite"
10test_bitcoin: /home/travis/build/bitcoin/bitcoin/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/condition_variable_fwd.hpp:102: boost::condition_variable::~condition_variable(): Assertion `!ret' failed.

Somehow it fails to lock a mutex even before the tests start, and all tests after that fail. I cannot reproduce it locally.

laanwj added the label Tests on Feb 22, 2017

laanwj commented at 7:34 am on February 23, 2017: member

Another one: https://travis-ci.org/bitcoin/bitcoin/jobs/204158785

0test_bitcoin: tpp.c:62: __pthread_tpp_change_priority: Assertion `new_prio == -1 || (new_prio >= __sched_fifo_min_prio && new_prio <= __sched_fifo_max_prio)' failed.
1Running 228 test cases...
2unknown location(0): fatal error: in "rpc_tests/rpc_rawparams": signal: SIGABRT (application abort requested)
3test/rpc_tests.cpp(29): last checkpoint
4...

laanwj commented at 7:54 am on February 23, 2017: member

Another one on 0.14: https://travis-ci.org/bitcoin/bitcoin/jobs/204167533

0FAIL: test/test_bitcoin
1=======================
2Running 228 test cases...
3unknown location(0): fatal error: in "rpc_tests/rpc_rawsign": signal: illegal operand; address of failing instruction: 0x2b73cbf4cf3b
4test/rpc_tests.cpp(29): last checkpoint

laanwj commented at 9:15 am on February 23, 2017: member

@theuni Do you perhaps have any guess what this could be?

I think the random errors are pretty scary. I’ve tried running a few test_bitcoins in parallel on a trusty VM locally but never get this error. It could be that some of their testing machines have broken CPUs but that doesn’t explain why it always happens in test_bitcoin and not during the RPC tests or compilation.

laanwj commented at 8:51 am on February 24, 2017: member

Apparantly this was reproduced by someone outside of travis in #9850

laanwj commented at 7:26 am on February 27, 2017: member

There’s a new wave of probably related travis failures. Here, Travis will just time out during run of make check without any output. Most likely this is test_bitcoin hanging at start.

laanwj referenced this in commit 4d60a7451b on Feb 27, 2017

laanwj referenced this in commit 91890d3478 on Feb 27, 2017

laanwj referenced this in commit d4e00de4dd on Feb 27, 2017

ryanofsky commented at 3:27 pm on February 27, 2017: member

I’m not sure I understand how #9875 would fix #9825. Isn’t the pthread_mutex_lock/new_prio assertion failing before any test even runs?

I do think it’s likely that this bug started with #9773. As I mentioned during the IRC meeting, I was seeing these crashes first over a week ago, and looking back at travis pull request logs (https://travis-ci.org/bitcoin/bitcoin/pull_requests), the first instance of this bug that I could find was https://travis-ci.org/bitcoin/bitcoin/builds/202600549 from #9773.

laanwj commented at 2:59 pm on February 28, 2017: member

I’m not sure I understand how #9875 would fix #9825. Isn’t the pthread_mutex_lock/new_prio assertion failing before any test even runs?

I haven’t seen the issue since. Until I see it again, I’m not going to spend any time on it.

Also remember: messages can be out of order if one is written to stdout and the other to stderr. I think that may very well be the case here.

MarcoFalke commented at 3:09 pm on February 28, 2017: member

Closing for now, please let me know when the issue appears again.

MarcoFalke closed this on Feb 28, 2017

ryanofsky commented at 3:10 pm on February 28, 2017: member

I see, if the messages are out of order that makes perfect sense.

DrahtBot locked this on Sep 8, 2021

Intermittent FAIL: test/test_bitcoin in Travis #9825