deadlock on shutdown #11763

issue dooglus opened this issue on November 25, 2017

dooglus commented at 6:40 AM on November 25, 2017: contributor

I just tried shutting down bitcoin-qt using the 'stop' RPC. It hung.

I built from a recent master branch commit 99bc0b428b03b571afbc311b7f18fd3a707ac5af with a couple of @laanwj's debugging commits (see here) merged in.

Here's are the stack traces:

(gdb) thread apply all where

Thread 10 (Thread 0x7f5dd77ae700 (LWP 20622)):
[#0](/bitcoin-bitcoin/0/)  0x00007f5e427476ad in poll () at ../sysdeps/unix/syscall-template.S:84
[#1](/bitcoin-bitcoin/1/)  0x00007f5e4007e9f6 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#2](/bitcoin-bitcoin/2/)  0x00007f5e4007eb0c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#3](/bitcoin-bitcoin/3/)  0x00007f5e44d0f04f in QEventDispatcherGlib::processEvents (this=0x7f5dc80008c0, flags=...) at kernel/qeventdispatcher_glib.cpp:423
[#4](/bitcoin-bitcoin/4/)  0x00007f5e44cb89ca in QEventLoop::exec (this=this@entry=0x7f5dd77adcc0, flags=..., flags@entry=...) at kernel/qeventloop.cpp:212
[#5](/bitcoin-bitcoin/5/)  0x00007f5e44ae60f3 in QThread::exec (this=<optimized out>) at thread/qthread.cpp:507
[#6](/bitcoin-bitcoin/6/)  0x00007f5e44aeada8 in QThreadPrivate::start (arg=0x55692e0ccb10) at thread/qthread_unix.cpp:368
[#7](/bitcoin-bitcoin/7/)  0x00007f5e441a8494 in start_thread (arg=0x7f5dd77ae700) at pthread_create.c:333
[#8](/bitcoin-bitcoin/8/)  0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 9 (Thread 0x7f5dfbfff700 (LWP 20605)):
[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
[#1](/bitcoin-bitcoin/1/)  0x000055692afb895b in leveldb::(anonymous namespace)::PosixEnv::BGThread (this=<optimized out>) at leveldb/util/env_posix.cc:649
[#2](/bitcoin-bitcoin/2/)  leveldb::(anonymous namespace)::PosixEnv::BGThreadWrapper (arg=<optimized out>) at leveldb/util/env_posix.cc:564
[#3](/bitcoin-bitcoin/3/)  0x00007f5e441a8494 in start_thread (arg=0x7f5dfbfff700) at pthread_create.c:333
[#4](/bitcoin-bitcoin/4/)  0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 8 (Thread 0x7f5e0bfff700 (LWP 20601)):
[#0](/bitcoin-bitcoin/0/)  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
[#1](/bitcoin-bitcoin/1/)  0x00007f5e42fda140 in std::__atomic_futex_unsigned_base::_M_futex_wait_until(unsigned int*, unsigned int, bool, std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
[#2](/bitcoin-bitcoin/2/)  0x000055692ad39110 in std::__atomic_futex_unsigned<2147483648u>::_M_load_and_test_until (this=<optimized out>, __assumed=<optimized out>, __operand=1, 
    __equal=255, __mo=std::memory_order_acquire, __has_timeout=false, __s=..., __ns=...)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/atomic_futex.h:102
[#3](/bitcoin-bitcoin/3/)  std::__atomic_futex_unsigned<2147483648u>::_M_load_and_test (this=<optimized out>, __assumed=<optimized out>, __operand=1, __equal=255, 
    __mo=std::memory_order_acquire) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/atomic_futex.h:121
[#4](/bitcoin-bitcoin/4/)  std::__atomic_futex_unsigned<2147483648u>::_M_load_when_equal (this=<optimized out>, __val=1, __mo=std::memory_order_acquire)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/atomic_futex.h:162
[#5](/bitcoin-bitcoin/5/)  std::__future_base::_State_baseV2::wait (this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/future:328
[#6](/bitcoin-bitcoin/6/)  std::__basic_future<void>::wait (this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/future:662
[#7](/bitcoin-bitcoin/7/)  sendrawtransaction (request=...) at rpc/rawtransaction.cpp:977
[#8](/bitcoin-bitcoin/8/)  0x000055692ad4467c in CRPCTable::execute (this=<optimized out>, request=...) at rpc/server.cpp:496
[#9](/bitcoin-bitcoin/9/)  0x000055692ae365c6 in HTTPReq_JSONRPC (req=<optimized out>) at httprpc.cpp:188
[#10](/bitcoin-bitcoin/10/) 0x000055692acf3bf9 in std::_Function_handler<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), bool (*)(HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::_M_invoke(std::_Any_data const&, HTTPRequest*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (__functor=..., 
    __args=<error reading variable: Cannot access memory at address 0x8>, __args=<error reading variable: Cannot access memory at address 0x8>)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:1716
[#11](/bitcoin-bitcoin/11/) 0x000055692ae46aa5 in std::function<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::operator()(HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const (
    __args=<error reading variable: Cannot access memory at address 0x80000008>, __args=<error reading variable: Cannot access memory at address 0x80000008>, 
    this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:2127
[#12](/bitcoin-bitcoin/12/) HTTPWorkItem::operator() (this=0xfffffffffffffe00) at httpserver.cpp:53
[#13](/bitcoin-bitcoin/13/) 0x000055692ae4721f in WorkQueue<HTTPClosure>::Run (this=<optimized out>) at httpserver.cpp:134
[#14](/bitcoin-bitcoin/14/) 0x000055692ae40f2a in HTTPWorkQueueRun (queue=0x7f5db8001860) at httpserver.cpp:362
[#15](/bitcoin-bitcoin/15/) 0x000055692ae48458 in std::_Bind_simple<void (*(WorkQueue<HTTPClosure>*))(WorkQueue<HTTPClosure>*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) (
    this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:1390
[#16](/bitcoin-bitcoin/16/) std::_Bind_simple<void (*(WorkQueue<HTTPClosure>*))(WorkQueue<HTTPClosure>*)>::operator()() (this=<optimized out>)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:1380
[#17](/bitcoin-bitcoin/17/) std::thread::_State_impl<std::_Bind_simple<void (*(WorkQueue<HTTPClosure>*))(WorkQueue<HTTPClosure>*)> >::_M_run() (this=0xfffffffffffffe00)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/thread:197
[#18](/bitcoin-bitcoin/18/) 0x00007f5e42fdbe6f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
[#19](/bitcoin-bitcoin/19/) 0x00007f5e441a8494 in start_thread (arg=0x7f5e0bfff700) at pthread_create.c:333
[#20](/bitcoin-bitcoin/20/) 0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 7 (Thread 0x7f5e19f5c700 (LWP 20598)):
[#0](/bitcoin-bitcoin/0/)  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
[#1](/bitcoin-bitcoin/1/)  0x00007f5e42fda140 in std::__atomic_futex_unsigned_base::_M_futex_wait_until(unsigned int*, unsigned int, bool, std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
[#2](/bitcoin-bitcoin/2/)  0x000055692ad39110 in std::__atomic_futex_unsigned<2147483648u>::_M_load_and_test_until (this=<optimized out>, __assumed=<optimized out>, __operand=1, 
    __equal=255, __mo=std::memory_order_acquire, __has_timeout=false, __s=..., __ns=...)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/atomic_futex.h:102
[#3](/bitcoin-bitcoin/3/)  std::__atomic_futex_unsigned<2147483648u>::_M_load_and_test (this=<optimized out>, __assumed=<optimized out>, __operand=1, __equal=255, 
    __mo=std::memory_order_acquire) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/atomic_futex.h:121
[#4](/bitcoin-bitcoin/4/)  std::__atomic_futex_unsigned<2147483648u>::_M_load_when_equal (this=<optimized out>, __val=1, __mo=std::memory_order_acquire)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/bits/atomic_futex.h:162
[#5](/bitcoin-bitcoin/5/)  std::__future_base::_State_baseV2::wait (this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/future:328
[#6](/bitcoin-bitcoin/6/)  std::__basic_future<void>::wait (this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/future:662
[#7](/bitcoin-bitcoin/7/)  sendrawtransaction (request=...) at rpc/rawtransaction.cpp:977
[#8](/bitcoin-bitcoin/8/)  0x000055692ad4467c in CRPCTable::execute (this=<optimized out>, request=...) at rpc/server.cpp:496
[#9](/bitcoin-bitcoin/9/)  0x000055692ae365c6 in HTTPReq_JSONRPC (req=<optimized out>) at httprpc.cpp:188
[#10](/bitcoin-bitcoin/10/) 0x000055692acf3bf9 in std::_Function_handler<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), bool (*)(HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::_M_invoke(std::_Any_data const&, HTTPRequest*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (__functor=..., 
    __args=<error reading variable: Cannot access memory at address 0x8>, __args=<error reading variable: Cannot access memory at address 0x8>)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:1716
[#11](/bitcoin-bitcoin/11/) 0x000055692ae46aa5 in std::function<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>::operator()(HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const (
    __args=<error reading variable: Cannot access memory at address 0x80000008>, __args=<error reading variable: Cannot access memory at address 0x80000008>, 
    this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:2127
[#12](/bitcoin-bitcoin/12/) HTTPWorkItem::operator() (this=0xfffffffffffffe00) at httpserver.cpp:53
[#13](/bitcoin-bitcoin/13/) 0x000055692ae4721f in WorkQueue<HTTPClosure>::Run (this=<optimized out>) at httpserver.cpp:134
[#14](/bitcoin-bitcoin/14/) 0x000055692ae40f2a in HTTPWorkQueueRun (queue=0x7f5dbc000af0) at httpserver.cpp:362
[#15](/bitcoin-bitcoin/15/) 0x000055692ae48458 in std::_Bind_simple<void (*(WorkQueue<HTTPClosure>*))(WorkQueue<HTTPClosure>*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) (
    this=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:1390
[#16](/bitcoin-bitcoin/16/) std::_Bind_simple<void (*(WorkQueue<HTTPClosure>*))(WorkQueue<HTTPClosure>*)>::operator()() (this=<optimized out>)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/functional:1380
[#17](/bitcoin-bitcoin/17/) std::thread::_State_impl<std::_Bind_simple<void (*(WorkQueue<HTTPClosure>*))(WorkQueue<HTTPClosure>*)> >::_M_run() (this=0xfffffffffffffe00)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/thread:197
[#18](/bitcoin-bitcoin/18/) 0x00007f5e42fdbe6f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
[#19](/bitcoin-bitcoin/19/) 0x00007f5e441a8494 in start_thread (arg=0x7f5e19f5c700) at pthread_create.c:333
[#20](/bitcoin-bitcoin/20/) 0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 6 (Thread 0x7f5e26e62700 (LWP 20594)):
[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
[#1](/bitcoin-bitcoin/1/)  0x00007f5e42fd650c in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
[#2](/bitcoin-bitcoin/2/)  0x000055692ae462db in WorkQueue<HTTPClosure>::WaitExit (this=<optimized out>) at httpserver.cpp:149
[#3](/bitcoin-bitcoin/3/)  0x000055692ae41dab in StopHTTPServer () at httpserver.cpp:489
[#4](/bitcoin-bitcoin/4/)  0x000055692abfe659 in Shutdown () at init.cpp:188
[#5](/bitcoin-bitcoin/5/)  0x000055692aab5e92 in BitcoinCore::shutdown (this=0x55692da40e10) at qt/bitcoin.cpp:319
[#6](/bitcoin-bitcoin/6/)  0x000055692aab5b6f in BitcoinCore::qt_static_metacall (_o=0x7f5e14044fcc, _c=128, _id=<optimized out>, _a=0x7f5e441ae15f <pthread_cond_wait@@GLIBC_2.3.2+191>)
    at ./qt/bitcoin.moc:91
[#7](/bitcoin-bitcoin/7/)  0x00007f5e44ce7499 in QObject::event (this=0x55692da40e10, e=<optimized out>) at kernel/qobject.cpp:1263
[#8](/bitcoin-bitcoin/8/)  0x00007f5e45629b8c in QApplicationPrivate::notify_helper (this=<optimized out>, receiver=0x55692da40e10, e=0x55692dd89a90) at kernel/qapplication.cpp:3799
[#9](/bitcoin-bitcoin/9/)  0x00007f5e45631341 in QApplication::notify (this=0x7ffc3c340d10, receiver=0x55692da40e10, e=0x55692dd89a90) at kernel/qapplication.cpp:3556
[#10](/bitcoin-bitcoin/10/) 0x00007f5e44cba9e0 in QCoreApplication::notifyInternal2 (receiver=0x55692da40e10, event=event@entry=0x55692dd89a90) at kernel/qcoreapplication.cpp:988
[#11](/bitcoin-bitcoin/11/) 0x00007f5e44cbd16d in QCoreApplication::sendEvent (event=0x55692dd89a90, receiver=<optimized out>)
    at ../../include/QtCore/../../src/corelib/kernel/qcoreapplication.h:231
[#12](/bitcoin-bitcoin/12/) QCoreApplicationPrivate::sendPostedEvents (receiver=receiver@entry=0x0, event_type=event_type@entry=0, data=0x55692da40c70) at kernel/qcoreapplication.cpp:1649
[#13](/bitcoin-bitcoin/13/) 0x00007f5e44cbd5d8 in QCoreApplication::sendPostedEvents (receiver=receiver@entry=0x0, event_type=event_type@entry=0) at kernel/qcoreapplication.cpp:1503
[#14](/bitcoin-bitcoin/14/) 0x00007f5e44d0ec43 in postEventSourceDispatch (s=0x7f5e14001700) at kernel/qeventdispatcher_glib.cpp:276
[#15](/bitcoin-bitcoin/15/) 0x00007f5e4007e7f7 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#16](/bitcoin-bitcoin/16/) 0x00007f5e4007ea60 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#17](/bitcoin-bitcoin/17/) 0x00007f5e4007eb0c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#18](/bitcoin-bitcoin/18/) 0x00007f5e44d0f04f in QEventDispatcherGlib::processEvents (this=0x7f5e140008c0, flags=...) at kernel/qeventdispatcher_glib.cpp:423
[#19](/bitcoin-bitcoin/19/) 0x00007f5e44cb89ca in QEventLoop::exec (this=this@entry=0x7f5e26e61cc0, flags=..., flags@entry=...) at kernel/qeventloop.cpp:212
[#20](/bitcoin-bitcoin/20/) 0x00007f5e44ae60f3 in QThread::exec (this=<optimized out>) at thread/qthread.cpp:507
[#21](/bitcoin-bitcoin/21/) 0x00007f5e44aeada8 in QThreadPrivate::start (arg=0x55692d47b2e0) at thread/qthread_unix.cpp:368
[#22](/bitcoin-bitcoin/22/) 0x00007f5e441a8494 in start_thread (arg=0x7f5e26e62700) at pthread_create.c:333
[#23](/bitcoin-bitcoin/23/) 0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 5 (Thread 0x7f5e27707700 (LWP 20589)):
[#0](/bitcoin-bitcoin/0/)  0x00007f5e427476ad in poll () at ../sysdeps/unix/syscall-template.S:84
[#1](/bitcoin-bitcoin/1/)  0x00007f5e4007e9f6 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#2](/bitcoin-bitcoin/2/)  0x00007f5e4007eb0c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#3](/bitcoin-bitcoin/3/)  0x00007f5e44d0f04f in QEventDispatcherGlib::processEvents (this=0x7f5e1c0008c0, flags=...) at kernel/qeventdispatcher_glib.cpp:423
[#4](/bitcoin-bitcoin/4/)  0x00007f5e44cb89ca in QEventLoop::exec (this=this@entry=0x7f5e27706c90, flags=..., flags@entry=...) at kernel/qeventloop.cpp:212
[#5](/bitcoin-bitcoin/5/)  0x00007f5e44ae60f3 in QThread::exec (this=this@entry=0x7f5e44f95d60 <(anonymous namespace)::Q_QGS__q_manager::innerFunction()::holder>)
    at thread/qthread.cpp:507
[#6](/bitcoin-bitcoin/6/)  0x00007f5e44f206d5 in QDBusConnectionManager::run (this=0x7f5e44f95d60 <(anonymous namespace)::Q_QGS__q_manager::innerFunction()::holder>)
    at qdbusconnection.cpp:178
[#7](/bitcoin-bitcoin/7/)  0x00007f5e44aeada8 in QThreadPrivate::start (arg=0x7f5e44f95d60 <(anonymous namespace)::Q_QGS__q_manager::innerFunction()::holder>)
    at thread/qthread_unix.cpp:368
[#8](/bitcoin-bitcoin/8/)  0x00007f5e441a8494 in start_thread (arg=0x7f5e27707700) at pthread_create.c:333
[#9](/bitcoin-bitcoin/9/)  0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 4 (Thread 0x7f5e2d98e700 (LWP 20588)):
[#0](/bitcoin-bitcoin/0/)  0x00007f5e427476ad in poll () at ../sysdeps/unix/syscall-template.S:84
[#1](/bitcoin-bitcoin/1/)  0x00007f5e4007e9f6 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#2](/bitcoin-bitcoin/2/)  0x00007f5e4007ed82 in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#3](/bitcoin-bitcoin/3/)  0x00007f5e3412b656 in ?? () from /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
[#4](/bitcoin-bitcoin/4/)  0x00007f5e400a63d5 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#5](/bitcoin-bitcoin/5/)  0x00007f5e441a8494 in start_thread (arg=0x7f5e2d98e700) at pthread_create.c:333
[#6](/bitcoin-bitcoin/6/)  0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 3 (Thread 0x7f5e2e1df700 (LWP 20587)):
[#0](/bitcoin-bitcoin/0/)  0x00007f5e427476ad in poll () at ../sysdeps/unix/syscall-template.S:84
[#1](/bitcoin-bitcoin/1/)  0x00007f5e4007e9f6 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#2](/bitcoin-bitcoin/2/)  0x00007f5e4007eb0c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#3](/bitcoin-bitcoin/3/)  0x00007f5e4007eb51 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#4](/bitcoin-bitcoin/4/)  0x00007f5e400a63d5 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#5](/bitcoin-bitcoin/5/)  0x00007f5e441a8494 in start_thread (arg=0x7f5e2e1df700) at pthread_create.c:333
[#6](/bitcoin-bitcoin/6/)  0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 2 (Thread 0x7f5e37e27700 (LWP 20586)):
[#0](/bitcoin-bitcoin/0/)  0x00007f5e427476ad in poll () at ../sysdeps/unix/syscall-template.S:84
[#1](/bitcoin-bitcoin/1/)  0x00007f5e3df4b150 in ?? () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
[#2](/bitcoin-bitcoin/2/)  0x00007f5e3df4cee9 in xcb_wait_for_event () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
[#3](/bitcoin-bitcoin/3/)  0x00007f5e3a588b69 in QXcbEventReader::run (this=0x55692d292ed0) at qxcbconnection.cpp:1343
[#4](/bitcoin-bitcoin/4/)  0x00007f5e44aeada8 in QThreadPrivate::start (arg=0x55692d292ed0) at thread/qthread_unix.cpp:368
[#5](/bitcoin-bitcoin/5/)  0x00007f5e441a8494 in start_thread (arg=0x7f5e37e27700) at pthread_create.c:333
[#6](/bitcoin-bitcoin/6/)  0x00007f5e42750aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 1 (Thread 0x7f5e468f0900 (LWP 20585)):
[#0](/bitcoin-bitcoin/0/)  0x00007f5e427476ad in poll () at ../sysdeps/unix/syscall-template.S:84
[#1](/bitcoin-bitcoin/1/)  0x00007f5e4007e9f6 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#2](/bitcoin-bitcoin/2/)  0x00007f5e4007eb0c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
[#3](/bitcoin-bitcoin/3/)  0x00007f5e44d0f04f in QEventDispatcherGlib::processEvents (this=0x55692d31cd70, flags=...) at kernel/qeventdispatcher_glib.cpp:423
[#4](/bitcoin-bitcoin/4/)  0x00007f5e44cb89ca in QEventLoop::exec (this=this@entry=0x7ffc3c340b00, flags=..., flags@entry=...) at kernel/qeventloop.cpp:212
[#5](/bitcoin-bitcoin/5/)  0x00007f5e44cc113c in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1261
[#6](/bitcoin-bitcoin/6/)  0x000055692aab8eeb in main (argc=<optimized out>, argv=<optimized out>) at qt/bitcoin.cpp:716

dooglus commented at 6:48 AM on November 25, 2017: contributor

The end of the debug log shows this:

2017-11-25 06:24:41 connect() to [2001:0:5ef5:79fb:c11:dce:3f9c:ef0]:8333 failed: Network is unreachable (101)
2017-11-25 06:25:04 tor: Thread interrupt
2017-11-25 06:25:04 addcon thread exit
2017-11-25 06:25:04 opencon thread exit
2017-11-25 06:25:04 net thread exit
2017-11-25 06:25:04 scheduler thread interrupt
2017-11-25 06:25:04 torcontrol thread exit
2017-11-25 06:25:04 Shutdown: In progress...
2017-11-25 06:25:04 msghand thread exit

jonasschnelli commented at 6:56 AM on November 25, 2017: contributor

Thanks for reporting (@dooglus reports are really helpful)!

Two threads hang here: https://github.com/bitcoin/bitcoin/blob/99bc0b428b03b571afbc311b7f18fd3a707ac5af/src/rpc/rawtransaction.cpp#L977

Should be fixed in master via #11738. Can you confirm @dooglus?
dooglus commented at 7:03 AM on November 25, 2017: contributor

I can confirm that I had sendrawtransaction hang a couple of times earlier today. I had no idea why, but forgot about it until now.

So this is very likely the same issue. I can't really confirm that it's fixed since I don't know how to reproduce it, but I don't think there's any value in keeping this issue open.
dooglus closed this on Nov 25, 2017
dooglus commented at 7:21 AM on November 25, 2017: contributor

Actually, maybe of interest is that the transaction which hung when I did the 'sendrawtransaction' wasn't already in the mempool. I had just created it for the first time so it couldn't have been. I did then try sending it again twice more after that which would account for the two threads you saw, but the first hang is unexplained. Maybe it was just being slow for some reason, I don't know.
TheBlueMatt commented at 3:01 PM on November 25, 2017: member

What is "the first hang" in this context? There are only two threads blocking shutdown.

On November 24, 2017 11:21:08 PM PST, Chris Moore notifications@github.com wrote:

Actually, maybe of interest is that the transaction which hung when I did the 'sendrawtransaction' wasn't already in the mempool. I had just created it for the first time so it couldn't have been. I did then try sending it again twice more after that which would account for the two threads you saw, but the first hang is unexplained. Maybe it was just being slow for some reason, I don't know.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/bitcoin/bitcoin/issues/11763#issuecomment-346924025
dooglus commented at 4:39 PM on November 25, 2017: contributor

I had a shell function which creates, signs, and sends raw transactions. I ran it yesterday, it hung when trying to send the signed transaction. That's the "first hang" I'm referring to.

The function leaves the signed transaction in a shell variable $p after it has run.

Seeing it hang, I hit control-C, then ran "bitcoin-cli sendrawtransaction $p" to try sending it again. That hung again. I tried it a second time, and it hung yet again. Those are the 2nd and 3rd hangs.

I waited 5 minutes and tried again. That time it didn't hang. Presumably by then the transaction was in a block, and so no longer in the mempool.

My point is that the very first time I tried sending this raw transaction, sendrawtransaction hung and had to be interrupted to get back to a shell prompt, and so the hanging isn't only caused by the transaction already being in the mempool - since this was a new transaction that I had only just created.
dooglus commented at 4:41 PM on November 25, 2017: contributor

I guess I only waited about 5 to 10 seconds before giving up on the first send before interrupting it. It may not have been permanently locked up. Maybe it would have completed if I gave it more time. Since only 2 threads were locked in the stacktrace I'm guessing it must have completed eventually.
TheBlueMatt commented at 6:36 PM on November 25, 2017: member

If you have a large wallet that was busy syncing when you sent the transaction, it is, indeed, very likely that it was simply slow to respond. I suppose for now nothing to be done but see if it happens again.

On November 25, 2017 11:39:45 AM EST, Chris Moore notifications@github.com wrote:

I had a shell function which creates, signs, and sends raw transactions. I ran it yesterday, it hung when trying to send the signed transaction. That's the "first hang" I'm referring to.

The function leaves the signed transaction in a shell variable $p after it has run.

Seeing it hang, I hit control-C, then ran "bitcoin-cli sendrawtransaction $p" to try sending it again. That hung again. I tried it a second time, and it hung yet again. Those are the 2nd and 3rd hangs.

I waited 5 minutes and tried again. That time it didn't hang. Presumably by then the transaction was in a block, and so no longer in the mempool.

My point is that the very first time I tried sending this raw transaction, sendrawtransaction hung and had to be interrupted to get back to a shell prompt, and so the hanging isn't only caused by the transaction already being in the mempool - since this was a new transaction that I had only just created.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/bitcoin/bitcoin/issues/11763#issuecomment-346951562
MarcoFalke locked this on Sep 8, 2021

Contributors

dooglus

jonasschnelli

TheBlueMatt

Linked (view graph)

#1 JSON-RPC support for mobile devices ("ultra-lightweight" clients)#2 Long-term, safe, store-of-value #3 Encrypt wallet #4 Export/Import wallet in a human readable, future-proof format #5 Make the version number the protocol version and not the client version #6 Treat wallet as a generic keystore #7 Block-header-only, faster startup client #8 RPC command to sign text with wallet private key #9 Fix for GUI on Macs and latest wxWidgets #10 Add address to listtransactions output #11 Nolisten patch #12 Monitor transactions and/or blocks #13 Messages with or about transactions #14 bitcoin: URI and/or bitcoin-request MIME type for click-to-pay #15 Option to specify external IP address #16 Mac UI issues #17 listaccounts method #18 Error when trying to send very precise amount #19 bitcoin(d) --help should output version number #20 JSON-RPC callback #21 Add time to category:move transactions #22 Update the list of hard-coded node IP addresses #23 CORS support #11355 LevelDB read failure: Corruption: block checksum mismatch