UI Hangs #10140

issue achow101 openend this issue on April 2, 2017
  1. achow101 commented at 3:56 pm on April 2, 2017: member

    Describe the issue

    The UI frequently hangs for me, either requiring me to wait a very long time for it to become responsive again, or killing bitcoin-qt and restarting it.

    Can you reliably reproduce the issue?

    If so, please list the steps to reproduce below:

    1. Start Bitcoin Core
    2. Wait a few minutes
    3. Minimize to the taskbar
    4. Open the main window by clicking the icon in the taskbar and clicking “Show / Hide”
    5. Attempt to do anything with the UI. I have found that opening the debug window and attempting to change tabs will usually cause it to hang.

    Expected behaviour

    It shouldn’t hang, and I shouldn’t need to kill it.

    Actual behaviour

    It hangs, and usually requires force killing in order make the node come back online (it seems to go offline when it hangs). I am unable to do anything with the GUI.

    What version of bitcoin-core are you using?

    v0.14.99.0-f34cdcb

    Machine specs:

    • OS: Ubuntu 16.04
    • CPU: i7-6800k
    • RAM: 32 GB
  2. fanquake added the label GUI on Apr 2, 2017
  3. laanwj commented at 4:13 pm on April 3, 2017: member

    This unfortunately doesn’t give enough information to reproduce or diagnose the issue.

    • Is there anything being logged in debug.log when this happened? This might have the same cause as #10005 and #5851
    • Is the process (any of its threads) still using CPU?
    • If possible: attach a debugger when this happens and do thread apply all bt (in case of gdb) to get an idea what hangs
  4. achow101 commented at 7:03 pm on April 3, 2017: member

    Is there anything being logged in debug.log when this happened? This might have the same cause as #10005 and #5851

    No, nothing unusual.

    Is the process (any of its threads) still using CPU?

    It doesn’t seem like it. %CPU in the system monitor is 0 and top didn’t even list the process.

    If possible: attach a debugger when this happens and do thread apply all bt (in case of gdb) to get an idea what hangs

    Is it normal that after I attach gdb that gdb will pause immediately and let me enter commands? Here is all of the output of that command after I attached it when it hung:

      0Thread 24 (Thread 0x7f12e37fe700 (LWP 25080)):
      1[#0](/bitcoin-bitcoin/0/)  0x00007f134cc9db5d in poll () at ../sysdeps/unix/syscall-template.S:84
      2[#1](/bitcoin-bitcoin/1/)  0x00007f134bde838c in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      3[#2](/bitcoin-bitcoin/2/)  0x00007f134bde849c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      4[#3](/bitcoin-bitcoin/3/)  0x00007f134f06f7cf in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
      5   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
      6[#4](/bitcoin-bitcoin/4/)  0x00007f134f016b4a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
      7   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
      8[#5](/bitcoin-bitcoin/5/)  0x00007f134ee33834 in QThread::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
      9[#6](/bitcoin-bitcoin/6/)  0x00007f134ee387be in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
     10[#7](/bitcoin-bitcoin/7/)  0x00007f134e2ae6ba in start_thread (arg=0x7f12e37fe700) at pthread_create.c:333
     11[#8](/bitcoin-bitcoin/8/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
     12
     13Thread 23 (Thread 0x7f12e3fff700 (LWP 25078)):
     14[#0](/bitcoin-bitcoin/0/)  0x00007f134cc9db5d in poll () at ../sysdeps/unix/syscall-template.S:84
     15[#1](/bitcoin-bitcoin/1/)  0x00007f134bde838c in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
     16[#2](/bitcoin-bitcoin/2/)  0x00007f134bde849c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
     17[#3](/bitcoin-bitcoin/3/)  0x00007f134f06f7eb in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
     18   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
     19[#4](/bitcoin-bitcoin/4/)  0x00007f134f016b4a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
     20   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
     21[#5](/bitcoin-bitcoin/5/)  0x00007f134ee33834 in QThread::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
     22[#6](/bitcoin-bitcoin/6/)  0x00007f134ee387be in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
     23[#7](/bitcoin-bitcoin/7/)  0x00007f134e2ae6ba in start_thread (arg=0x7f12e3fff700) at pthread_create.c:333
     24[#8](/bitcoin-bitcoin/8/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
     25
     26Thread 22 (Thread 0x7f12f0e39700 (LWP 25071)):
     27[#0](/bitcoin-bitcoin/0/)  0x00007f134e2b79ff in __libc_send (fd=55, buf=0x7f129234fcc0, n=999997, flags=flags@entry=16384)
     28    at ../sysdeps/unix/sysv/linux/x86_64/send.c:26
     29[#1](/bitcoin-bitcoin/1/)  0x0000562a8983f9f7 in CConnman::SocketSendData (this=this@entry=0x7f1334d8a5b0, 
     30    pnode=pnode@entry=0x7f12e83d6130) at net.cpp:844
     31[#2](/bitcoin-bitcoin/2/)  0x0000562a898411ec in CConnman::PushMessage(CNode*, CSerializedNetMsg&&) (this=this@entry=0x7f1334d8a5b0, 
     32    pnode=pnode@entry=0x7f12e83d6130, 
     33    msg=msg@entry=<unknown type in /home/andy/bitcoin/bitcoin/src/qt/bitcoin-qt, CU 0x1d2dfc3, DIE 0x1edc23f>)
     34    at net.cpp:2770
     35[#3](/bitcoin-bitcoin/3/)  0x0000562a89879670 in ProcessGetData (pfrom=pfrom@entry=0x7f12e83d6130, consensusParams=..., connman=..., 
     36    interruptMsgProc=...) at net_processing.cpp:1042
     37[#4](/bitcoin-bitcoin/4/)  0x0000562a8987d822 in ProcessMessage (pfrom=pfrom@entry=0x7f12e83d6130, strCommand="", vRecv=..., 
     38    nTimeReceived=<optimized out>, chainparams=..., connman=..., interruptMsgProc=...) at net_processing.cpp:1598
     39[#5](/bitcoin-bitcoin/5/)  0x0000562a8988e1a2 in ProcessMessages (pfrom=0x7f12e83d6130, connman=..., interruptMsgProc=...)
     40    at net_processing.cpp:2714
     41[#6](/bitcoin-bitcoin/6/)  0x0000562a8984753b in boost::function3<bool, CNode*, CConnman&, std::atomic<bool>&>::operator() (a2=..., 
     42    a1=..., a0=<optimized out>, this=<optimized out>) at /usr/include/boost/function/function_template.hpp:773
     43[#7](/bitcoin-bitcoin/7/)  boost::signals2::detail::call_with_tuple_args<bool>::m_invoke<bool, boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)>, 0u, 1u, 2u, CNode*&, CConnman&, std::atomic<bool>&>(bool*, boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)>&, boost::signals2::detail::unsigned_meta_array<0u, 1u, 2u>, std::tuple<CNode*&, CConnman&, std::atomic<bool>&>) const (args=..., func=..., this=<optimized out>)
     44    at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:87
     45[#8](/bitcoin-bitcoin/8/)  boost::signals2::detail::call_with_tuple_args<bool>::operator()<boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)>, CNode*&, CConnman&, std::atomic<bool>&, 3ul>(boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)>&, std::tuple<CNode*&, CConnman&, std::atomic<bool>&>, mpl_::size_t<3ul>) const (args=..., func=..., 
     46    this=<optimized out>) at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:81
     47[#9](/bitcoin-bitcoin/9/)  boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>::m_invoke<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >(boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > const&, ...) const (connectionBody=..., this=<optimized out>)
     48    at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:132
     49[#10](/bitcoin-bitcoin/10/) boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>::operator()<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >(boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::m---Type <return> to continue, or q <return> to quit---
     50utex> > const&) const (connectionBody=..., this=<optimized out>)
     51    at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:120
     52[#11](/bitcoin-bitcoin/11/) boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> >::dereference() const (this=0x7f12f0e386f0)
     53    at /usr/include/boost/signals2/detail/slot_call_iterator.hpp:82
     54[#12](/bitcoin-bitcoin/12/) boost::iterators::iterator_core_access::dereference<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >(boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > const&) (f=...) at /usr/include/boost/iterator/iterator_facade.hpp:540
     55[#13](/bitcoin-bitcoin/13/) boost::iterators::detail::iterator_facade_base<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> >, bool, boost::iterators::single_pass_traversal_tag, bool const&, long, false, false>::operator*() const (this=0x7f12f0e386f0)
     56    at /usr/include/boost/iterator/iterator_facade.hpp:646
     57[#14](/bitcoin-bitcoin/14/) CombinerAll::operator()<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >(boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> >, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boos
     58t::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> >) const (last=..., first=..., this=<optimized out>) at net.h:419
     59[#15](/bitcoin-bitcoin/15/) boost::signals2::detail::combiner_invoker<bool>::operator()<CombinerAll, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >(CombinerAll&, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> >, boost::signals2::detail::slot_call_iterator_t<boost:---Type <return> to continue, or q <return> to quit---
     60:signals2::detail::variadic_slot_invoker<bool, CNode*, CConnman&, std::atomic<bool>&>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<bool (CNode*, CConnman&, std::atomic<bool>&), boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)> >, boost::signals2::mutex> >) const (
     61    last=..., first=..., combiner=..., this=<optimized out>)
     62    at /usr/include/boost/signals2/detail/result_type_wrapper.hpp:53
     63[#16](/bitcoin-bitcoin/16/) boost::signals2::detail::signal_impl<bool (CNode*, CConnman&, std::atomic<bool>&), CombinerAll, int, std::less<int>, boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)>, boost::function<bool (boost::signals2::connection const&, CNode*, CConnman&, std::atomic<bool>&)>, boost::signals2::mutex>::operator()(CNode*, CConnman&, std::atomic<bool>&) (args#2=..., args#1=..., args#0=0x7f12e83d6130, this=0x562a8a9f4b50)
     64    at /usr/include/boost/signals2/detail/signal_template.hpp:246
     65[#17](/bitcoin-bitcoin/17/) boost::signals2::signal<bool (CNode*, CConnman&, std::atomic<bool>&), CombinerAll, int, std::less<int>, boost::function<bool (CNode*, CConnman&, std::atomic<bool>&)>, boost::function<bool (boost::signals2::connection const&, CNode*, CConnman&, std::atomic<bool>&)>, boost::signals2::mutex>::operator()(CNode*, CConnman&, std::atomic<bool>&)
     66    (args#2=..., args#1=..., args#0=0x7f12e83d6130, this=0x562a8a3121c0 <g_signals>)
     67    at /usr/include/boost/signals2/detail/signal_template.hpp:718
     68[#18](/bitcoin-bitcoin/18/) CConnman::ThreadMessageHandler (this=0x7f1334d8a5b0) at net.cpp:1969
     69[#19](/bitcoin-bitcoin/19/) 0x0000562a8981fe5f in std::function<void ()>::operator()() const (this=0x7f12f0e38d30)
     70    at /usr/include/c++/5/functional:2267
     71[#20](/bitcoin-bitcoin/20/) TraceThread<std::function<void ()> >(char const*, std::function<void ()>) (name=0x562a89f41c6d "msghand", 
     72    func=...) at util.h:228
     73[#21](/bitcoin-bitcoin/21/) 0x0000562a89850c3b in std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=<optimized out>)
     74    at /usr/include/c++/5/functional:1531
     75[#22](/bitcoin-bitcoin/22/) std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::operator()() (this=<optimized out>) at /usr/include/c++/5/functional:1520
     76[#23](/bitcoin-bitcoin/23/) std::thread::_Impl<std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)> >::_M_run() (this=<optimized out>) at /usr/include/c++/5/thread:115
     77[#24](/bitcoin-bitcoin/24/) 0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
     78[#25](/bitcoin-bitcoin/25/) 0x00007f134e2ae6ba in start_thread (arg=0x7f12f0e39700) at pthread_create.c:333
     79[#26](/bitcoin-bitcoin/26/) 0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
     80
     81Thread 21 (Thread 0x7f12f163a700 (LWP 25070)):
     82[#0](/bitcoin-bitcoin/0/)  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
     83[#1](/bitcoin-bitcoin/1/)  0x00007f134e2b0e42 in __GI___pthread_mutex_lock (mutex=0x7f1334dda8c8) at ../nptl/pthread_mutex_lock.c:115
     84[#2](/bitcoin-bitcoin/2/)  0x0000562a89708ea5 in boost::recursive_mutex::lock (this=<optimized out>)
     85    at /usr/include/boost/thread/pthread/recursive_mutex.hpp:113
     86[#3](/bitcoin-bitcoin/3/)  AnnotatedMixin<boost::recursive_mutex>::lock (this=<optimized out>) at ./sync.h:60
     87[#4](/bitcoin-bitcoin/4/)  boost::unique_lock<CCriticalSection>::lock (this=0x7f12f1639770)
     88    at /usr/include/boost/thread/lock_types.hpp:346
     89[#5](/bitcoin-bitcoin/5/)  CMutexLock<CCriticalSection>::Enter (pszName=<optimized out>, pszFile=<optimized out>, nLine=<optimized out>, 
     90    this=0x7f12f1639770) at ./sync.h:124
     91[#6](/bitcoin-bitcoin/6/)  CMutexLock<CCriticalSection>::CMutexLock (this=0x7f12f1639770, mutexIn=..., pszName=<optimized out>, 
     92    pszFile=<optimized out>, nLine=<optimized out>, fTry=<optimized out>) at ./sync.h:145
     93[#7](/bitcoin-bitcoin/7/)  0x0000562a8984c968 in CConnman::ThreadOpenConnections (this=<optimized out>) at net.cpp:1726
     94[#8](/bitcoin-bitcoin/8/)  0x0000562a8981fe5f in std::function<void ()>::operator()() const (this=0x7f12f1639d30)
     95    at /usr/include/c++/5/functional:2267
     96[#9](/bitcoin-bitcoin/9/)  TraceThread<std::function<void ()> >(char const*, std::function<void ()>) (name=0x562a89f41c65 "opencon", 
     97    func=...) at util.h:228
     98[#10](/bitcoin-bitcoin/10/) 0x0000562a89850c3b in std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=<optimized out>)
     99    at /usr/include/c++/5/functional:1531
    100[#11](/bitcoin-bitcoin/11/) std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::operator()() (this=<optimized out>) at /usr/include/c++/5/functional:1520
    101[#12](/bitcoin-bitcoin/12/) std::thread::_Impl<std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)> >::_M_run() (this=<optimized out>) at /usr/include/c++/5/thread:115
    102[#13](/bitcoin-bitcoin/13/) 0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    103[#14](/bitcoin-bitcoin/14/) 0x00007f134e2ae6ba in start_thread (arg=0x7f12f163a700) at pthread_create.c:333
    104[#15](/bitcoin-bitcoin/15/) 0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    105
    106Thread 20 (Thread 0x7f12f1e3b700 (LWP 25069)):
    107[#0](/bitcoin-bitcoin/0/)  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
    108[#1](/bitcoin-bitcoin/1/)  0x00007f134e2b0e42 in __GI___pthread_mutex_lock (mutex=0x7f1334dda8c8) at ../nptl/pthread_mutex_lock.c:115
    109[#2](/bitcoin-bitcoin/2/)  0x0000562a89708ea5 in boost::recursive_mutex::lock (this=<optimized out>)
    110    at /usr/include/boost/thread/pthread/recursive_mutex.hpp:113
    111---Type <return> to continue, or q <return> to quit---
    112[#3](/bitcoin-bitcoin/3/)  AnnotatedMixin<boost::recursive_mutex>::lock (this=<optimized out>) at ./sync.h:60
    113[#4](/bitcoin-bitcoin/4/)  boost::unique_lock<CCriticalSection>::lock (this=0x7f12f1e3a630)
    114    at /usr/include/boost/thread/lock_types.hpp:346
    115[#5](/bitcoin-bitcoin/5/)  CMutexLock<CCriticalSection>::Enter (pszName=<optimized out>, pszFile=<optimized out>, nLine=<optimized out>, 
    116    this=0x7f12f1e3a630) at ./sync.h:124
    117[#6](/bitcoin-bitcoin/6/)  CMutexLock<CCriticalSection>::CMutexLock (this=0x7f12f1e3a630, mutexIn=..., pszName=<optimized out>, 
    118    pszFile=<optimized out>, nLine=<optimized out>, fTry=<optimized out>) at ./sync.h:145
    119[#7](/bitcoin-bitcoin/7/)  0x0000562a8984012c in CConnman::GetAddedNodeInfo (this=this@entry=0x7f1334d8a5b0) at net.cpp:1835
    120[#8](/bitcoin-bitcoin/8/)  0x0000562a8984dfab in CConnman::ThreadOpenAddedConnections (this=0x7f1334d8a5b0) at net.cpp:1882
    121[#9](/bitcoin-bitcoin/9/)  0x0000562a8981fe5f in std::function<void ()>::operator()() const (this=0x7f12f1e3ad30)
    122    at /usr/include/c++/5/functional:2267
    123[#10](/bitcoin-bitcoin/10/) TraceThread<std::function<void ()> >(char const*, std::function<void ()>) (name=0x562a89f41c5e "addcon", 
    124    func=...) at util.h:228
    125[#11](/bitcoin-bitcoin/11/) 0x0000562a89850c3b in std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=<optimized out>)
    126    at /usr/include/c++/5/functional:1531
    127[#12](/bitcoin-bitcoin/12/) std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::operator()() (this=<optimized out>) at /usr/include/c++/5/functional:1520
    128[#13](/bitcoin-bitcoin/13/) std::thread::_Impl<std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)> >::_M_run() (this=<optimized out>) at /usr/include/c++/5/thread:115
    129[#14](/bitcoin-bitcoin/14/) 0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    130[#15](/bitcoin-bitcoin/15/) 0x00007f134e2ae6ba in start_thread (arg=0x7f12f1e3b700) at pthread_create.c:333
    131[#16](/bitcoin-bitcoin/16/) 0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    132
    133Thread 19 (Thread 0x7f12f2e3d700 (LWP 25067)):
    134[#0](/bitcoin-bitcoin/0/)  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
    135[#1](/bitcoin-bitcoin/1/)  0x00007f134e2b0e42 in __GI___pthread_mutex_lock (mutex=0x7f12e83d61d8) at ../nptl/pthread_mutex_lock.c:115
    136[#2](/bitcoin-bitcoin/2/)  0x0000562a89708ea5 in boost::recursive_mutex::lock (this=<optimized out>)
    137    at /usr/include/boost/thread/pthread/recursive_mutex.hpp:113
    138[#3](/bitcoin-bitcoin/3/)  AnnotatedMixin<boost::recursive_mutex>::lock (this=<optimized out>) at ./sync.h:60
    139[#4](/bitcoin-bitcoin/4/)  boost::unique_lock<CCriticalSection>::lock (this=0x7f12f2e2bef0)
    140    at /usr/include/boost/thread/lock_types.hpp:346
    141[#5](/bitcoin-bitcoin/5/)  CMutexLock<CCriticalSection>::Enter (pszName=<optimized out>, pszFile=<optimized out>, nLine=<optimized out>, 
    142    this=0x7f12f2e2bef0) at ./sync.h:124
    143[#6](/bitcoin-bitcoin/6/)  CMutexLock<CCriticalSection>::CMutexLock (this=0x7f12f2e2bef0, mutexIn=..., pszName=<optimized out>, 
    144    pszFile=<optimized out>, nLine=<optimized out>, fTry=<optimized out>) at ./sync.h:145
    145[#7](/bitcoin-bitcoin/7/)  0x0000562a8984a2fd in CConnman::ThreadSocketHandler (this=0x7f1334d8a5b0) at net.cpp:1293
    146[#8](/bitcoin-bitcoin/8/)  0x0000562a8981fe5f in std::function<void ()>::operator()() const (this=0x7f12f2e3cd30)
    147    at /usr/include/c++/5/functional:2267
    148[#9](/bitcoin-bitcoin/9/)  TraceThread<std::function<void ()> >(char const*, std::function<void ()>) (name=0x562a89f70209 "net", 
    149    func=...) at util.h:228
    150[#10](/bitcoin-bitcoin/10/) 0x0000562a89850c3b in std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=<optimized out>)
    151    at /usr/include/c++/5/functional:1531
    152[#11](/bitcoin-bitcoin/11/) std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)>::operator()() (this=<optimized out>) at /usr/include/c++/5/functional:1520
    153[#12](/bitcoin-bitcoin/12/) std::thread::_Impl<std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)> >::_M_run() (this=<optimized out>) at /usr/include/c++/5/thread:115
    154[#13](/bitcoin-bitcoin/13/) 0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    155[#14](/bitcoin-bitcoin/14/) 0x00007f134e2ae6ba in start_thread (arg=0x7f12f2e3d700) at pthread_create.c:333
    156[#15](/bitcoin-bitcoin/15/) 0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    157
    158Thread 18 (Thread 0x7f12f363e700 (LWP 25066)):
    159[#0](/bitcoin-bitcoin/0/)  0x00007f134cca9e23 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
    160[#1](/bitcoin-bitcoin/1/)  0x00007f134d82fb18 in ?? () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
    161[#2](/bitcoin-bitcoin/2/)  0x00007f134d819e2d in event_base_loop () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
    162[#3](/bitcoin-bitcoin/3/)  0x0000562a89924b5f in TorControlThread () at torcontrol.cpp:672
    163[#4](/bitcoin-bitcoin/4/)  0x0000562a89852a40 in TraceThread<void (*)()> (name=0x562a89f82a3c "torcontrol", 
    164    func=0x562a89924ac0 <TorControlThread()>) at util.h:228
    165[#5](/bitcoin-bitcoin/5/)  0x00007f13500d55d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
    166[#6](/bitcoin-bitcoin/6/)  0x00007f134e2ae6ba in start_thread (arg=0x7f12f363e700) at pthread_create.c:333
    167[#7](/bitcoin-bitcoin/7/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    168
    169Thread 17 (Thread 0x7f130aebd700 (LWP 24652)):
    170[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    171[#1](/bitcoin-bitcoin/1/)  0x0000562a89b4f83b in leveldb::(anonymous namespace)::PosixEnv::BGThread (this=0x7f1334cc7590)
    172    at leveldb/util/env_posix.cc:574
    173---Type <return> to continue, or q <return> to quit---
    174[#2](/bitcoin-bitcoin/2/)  leveldb::(anonymous namespace)::PosixEnv::BGThreadWrapper (arg=0x7f1334cc7590)
    175    at leveldb/util/env_posix.cc:521
    176[#3](/bitcoin-bitcoin/3/)  0x00007f134e2ae6ba in start_thread (arg=0x7f130aebd700) at pthread_create.c:333
    177[#4](/bitcoin-bitcoin/4/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    178
    179Thread 16 (Thread 0x7f1323fff700 (LWP 24651)):
    180[#0](/bitcoin-bitcoin/0/)  0x00007f134cca9e23 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
    181[#1](/bitcoin-bitcoin/1/)  0x00007f13509a10ec in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5
    182[#2](/bitcoin-bitcoin/2/)  0x00007f13509cc7fa in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5
    183[#3](/bitcoin-bitcoin/3/)  0x00007f134e2ae6ba in start_thread (arg=0x7f1323fff700) at pthread_create.c:333
    184[#4](/bitcoin-bitcoin/4/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    185
    186Thread 15 (Thread 0x7f1328ffc700 (LWP 24650)):
    187[#0](/bitcoin-bitcoin/0/)  0x00007f134cca9e23 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
    188[#1](/bitcoin-bitcoin/1/)  0x00007f13509a10ec in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5
    189[#2](/bitcoin-bitcoin/2/)  0x00007f13509cc7fa in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5
    190[#3](/bitcoin-bitcoin/3/)  0x00007f134e2ae6ba in start_thread (arg=0x7f1328ffc700) at pthread_create.c:333
    191[#4](/bitcoin-bitcoin/4/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    192
    193Thread 14 (Thread 0x7f132a7fc700 (LWP 24649)):
    194[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    195[#1](/bitcoin-bitcoin/1/)  0x00007f134d53e91c in std::condition_variable::wait(std::unique_lock<std::mutex>&) ()
    196   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    197[#2](/bitcoin-bitcoin/2/)  0x0000562a899ccceb in WorkQueue<HTTPClosure>::Run (this=0x7f1334148340) at httpserver.cpp:126
    198[#3](/bitcoin-bitcoin/3/)  HTTPWorkQueueRun (queue=0x7f1334148340) at httpserver.cpp:354
    199[#4](/bitcoin-bitcoin/4/)  0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    200[#5](/bitcoin-bitcoin/5/)  0x00007f134e2ae6ba in start_thread (arg=0x7f132a7fc700) at pthread_create.c:333
    201[#6](/bitcoin-bitcoin/6/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    202
    203Thread 13 (Thread 0x7f132affd700 (LWP 24648)):
    204[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    205[#1](/bitcoin-bitcoin/1/)  0x00007f134d53e91c in std::condition_variable::wait(std::unique_lock<std::mutex>&) ()
    206   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    207[#2](/bitcoin-bitcoin/2/)  0x0000562a899ccceb in WorkQueue<HTTPClosure>::Run (this=0x7f1334148340) at httpserver.cpp:126
    208[#3](/bitcoin-bitcoin/3/)  HTTPWorkQueueRun (queue=0x7f1334148340) at httpserver.cpp:354
    209[#4](/bitcoin-bitcoin/4/)  0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    210[#5](/bitcoin-bitcoin/5/)  0x00007f134e2ae6ba in start_thread (arg=0x7f132affd700) at pthread_create.c:333
    211[#6](/bitcoin-bitcoin/6/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    212
    213Thread 12 (Thread 0x7f132b7fe700 (LWP 24647)):
    214[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    215[#1](/bitcoin-bitcoin/1/)  0x00007f134d53e91c in std::condition_variable::wait(std::unique_lock<std::mutex>&) ()
    216   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    217[#2](/bitcoin-bitcoin/2/)  0x0000562a899ccceb in WorkQueue<HTTPClosure>::Run (this=0x7f1334148340) at httpserver.cpp:126
    218[#3](/bitcoin-bitcoin/3/)  HTTPWorkQueueRun (queue=0x7f1334148340) at httpserver.cpp:354
    219[#4](/bitcoin-bitcoin/4/)  0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    220[#5](/bitcoin-bitcoin/5/)  0x00007f134e2ae6ba in start_thread (arg=0x7f132b7fe700) at pthread_create.c:333
    221[#6](/bitcoin-bitcoin/6/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    222
    223Thread 11 (Thread 0x7f132bfff700 (LWP 24646)):
    224[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    225[#1](/bitcoin-bitcoin/1/)  0x00007f134d53e91c in std::condition_variable::wait(std::unique_lock<std::mutex>&) ()
    226   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    227[#2](/bitcoin-bitcoin/2/)  0x0000562a899ccceb in WorkQueue<HTTPClosure>::Run (this=0x7f1334148340) at httpserver.cpp:126
    228[#3](/bitcoin-bitcoin/3/)  HTTPWorkQueueRun (queue=0x7f1334148340) at httpserver.cpp:354
    229[#4](/bitcoin-bitcoin/4/)  0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    230[#5](/bitcoin-bitcoin/5/)  0x00007f134e2ae6ba in start_thread (arg=0x7f132bfff700) at pthread_create.c:333
    231[#6](/bitcoin-bitcoin/6/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    232
    233Thread 10 (Thread 0x7f1330ffc700 (LWP 24645)):
    234[#0](/bitcoin-bitcoin/0/)  0x00007f134cca9e23 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
    235[#1](/bitcoin-bitcoin/1/)  0x00007f134d82fb18 in ?? () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
    236[#2](/bitcoin-bitcoin/2/)  0x00007f134d819e2d in event_base_loop () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
    237[#3](/bitcoin-bitcoin/3/)  0x0000562a899cb034 in ThreadHTTP (base=0x7f1334147cb0, http=<optimized out>) at httpserver.cpp:305
    238[#4](/bitcoin-bitcoin/4/)  0x0000562a899d20b1 in std::__invoke<bool (*)(event_base*, evhttp*), event_base*, evhttp*>(bool (*)(event_base*, evhttp*), event_base*&&, evhttp*&&) (__f=<optimized out>) at /usr/include/c++/5/functional:227
    239[#5](/bitcoin-bitcoin/5/)  std::reference_wrapper<bool (*)(event_base*, evhttp*)>::operator()<event_base*, evhttp*>(event_base*&&, evhttp*---Type <return> to continue, or q <return> to quit---
    240&&) const (this=<optimized out>) at /usr/include/c++/5/functional:428
    241[#6](/bitcoin-bitcoin/6/)  std::_Bind_simple<std::reference_wrapper<bool (*)(event_base*, evhttp*)> (event_base*, evhttp*)>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=<optimized out>) at /usr/include/c++/5/functional:1531
    242[#7](/bitcoin-bitcoin/7/)  std::_Bind_simple<std::reference_wrapper<bool (*)(event_base*, evhttp*)> (event_base*, evhttp*)>::operator()()
    243    (this=<optimized out>) at /usr/include/c++/5/functional:1520
    244[#8](/bitcoin-bitcoin/8/)  std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<bool>, std::__future_base::_Result_base::_Deleter>, std::_Bind_simple<std::reference_wrapper<bool (*)(event_base*, evhttp*)> (event_base*, evhttp*)>, bool>::operator()() const (this=0x7f1330ffbd30) at /usr/include/c++/5/future:1319
    245[#9](/bitcoin-bitcoin/9/)  std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<bool>, std::__future_base::_Result_base::_Deleter>, std::_Bind_simple<std::reference_wrapper<bool (*)(event_base*, evhttp*)> (event_base*, evhttp*)>, bool> >::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/5/functional:1857
    246[#10](/bitcoin-bitcoin/10/) 0x0000562a899d1e89 in std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>::operator()() const (this=<optimized out>) at /usr/include/c++/5/functional:2267
    247[#11](/bitcoin-bitcoin/11/) std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*) (this=0x7f1334148960, __f=<optimized out>, 
    248    __did_set=0x7f1330ffbcd7) at /usr/include/c++/5/future:527
    249[#12](/bitcoin-bitcoin/12/) 0x00007f134e2b5a99 in __pthread_once_slow (once_control=0x7f1334148978, 
    250    init_routine=0x7f134d542ac0 <__once_proxy>) at pthread_once.c:116
    251[#13](/bitcoin-bitcoin/13/) 0x0000562a899d2a4f in __gthread_once (__func=<optimized out>, __once=0x7f1334148978)
    252    at /usr/include/x86_64-linux-gnu/c++/5/bits/gthr-default.h:699
    253[#14](/bitcoin-bitcoin/14/) std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)
    254    (__once=..., 
    255    __f=__f@entry=<unknown type in /home/andy/bitcoin/bitcoin/src/qt/bitcoin-qt, CU 0x334a62a, DIE 0x33c70ad>)
    256    at /usr/include/c++/5/mutex:738
    257[#15](/bitcoin-bitcoin/15/) 0x0000562a899d2cb0 in std::__future_base::_State_baseV2::_M_set_result(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) (__ignore_failure=false, 
    258    __res=..., this=0x7f1334148960) at /usr/include/c++/5/future:387
    259[#16](/bitcoin-bitcoin/16/) std::__future_base::_Task_state<bool (*)(event_base*, evhttp*), std::allocator<int>, bool (event_base*, evhttp*)>::_M_run(event_base*&&, evhttp*&&) (
    260    __args#1=<unknown type in /home/andy/bitcoin/bitcoin/src/qt/bitcoin-qt, CU 0x334a62a, DIE 0x33c7107>, 
    261    __args#0=<unknown type in /home/andy/bitcoin/bitcoin/src/qt/bitcoin-qt, CU 0x334a62a, DIE 0x33c7102>, 
    262    this=0x7f1334148960) at /usr/include/c++/5/future:1403
    263[#17](/bitcoin-bitcoin/17/) std::packaged_task<bool (event_base*, evhttp*)>::operator()(event_base*, evhttp*) (__args#1=0x7f1334148270, 
    264    __args#0=0x7f1334147cb0, this=<optimized out>) at /usr/include/c++/5/future:1547
    265[#18](/bitcoin-bitcoin/18/) std::_Bind_simple<std::packaged_task<bool (event_base*, evhttp*)> (event_base*, evhttp*)>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=<optimized out>) at /usr/include/c++/5/functional:1531
    266[#19](/bitcoin-bitcoin/19/) std::_Bind_simple<std::packaged_task<bool (event_base*, evhttp*)> (event_base*, evhttp*)>::operator()() (
    267    this=<optimized out>) at /usr/include/c++/5/functional:1520
    268[#20](/bitcoin-bitcoin/20/) std::thread::_Impl<std::_Bind_simple<std::packaged_task<bool (event_base*, evhttp*)> (event_base*, evhttp*)> >::_M_run() (this=<optimized out>) at /usr/include/c++/5/thread:115
    269[#21](/bitcoin-bitcoin/21/) 0x00007f134d543c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    270[#22](/bitcoin-bitcoin/22/) 0x00007f134e2ae6ba in start_thread (arg=0x7f1330ffc700) at pthread_create.c:333
    271[#23](/bitcoin-bitcoin/23/) 0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    272
    273Thread 9 (Thread 0x7f13317fd700 (LWP 24644)):
    274[#0](/bitcoin-bitcoin/0/)  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
    275[#1](/bitcoin-bitcoin/1/)  0x0000562a89aca0ba in boost::condition_variable::do_wait_until (timeout=..., m=..., this=0x562a8af6ff30)
    276    at /usr/include/boost/thread/pthread/condition_variable.hpp:108
    277[#2](/bitcoin-bitcoin/2/)  boost::condition_variable::wait_until (tp=..., lk=..., this=0x562a8af6ff30)
    278    at /usr/include/boost/thread/pthread/condition_variable_fwd.hpp:253
    279[#3](/bitcoin-bitcoin/3/)  boost::condition_variable::wait_until<boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > > (
    280    t=<synthetic pointer>, lock=..., this=0x562a8af6ff30)
    281    at /usr/include/boost/thread/pthread/condition_variable_fwd.hpp:166
    282[#4](/bitcoin-bitcoin/4/)  CScheduler::serviceQueue (this=0x562a8af6ff00) at scheduler.cpp:59
    283[#5](/bitcoin-bitcoin/5/)  0x0000562a8981fe5f in std::function<void ()>::operator()() const (this=0x7f13317fcd20)
    284    at /usr/include/c++/5/functional:2267
    285[#6](/bitcoin-bitcoin/6/)  TraceThread<std::function<void ()> >(char const*, std::function<void ()>) (name=0x562a89f310f2 "scheduler", 
    286    func=...) at util.h:228
    287[#7](/bitcoin-bitcoin/7/)  0x0000562a898214e1 in boost::_bi::list2<boost::_bi::value<char const*>, boost::_bi::value<std::function<void ()> > >::operator()<void (*)(char const*, std::function<void ()>), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(char const*, std::function<void ()>), boost::_bi::list0&, int) (f=<optimized out>, a=<synthetic pointer>, 
    288---Type <return> to continue, or q <return> to quit---
    289    this=<optimized out>) at /usr/include/boost/bind/bind.hpp:313
    290[#8](/bitcoin-bitcoin/8/)  boost::_bi::bind_t<void, void (*)(char const*, std::function<void ()>), boost::_bi::list2<boost::_bi::value<char const*>, boost::_bi::value<std::function<void ()> > > >::operator()() (this=<optimized out>)
    291    at /usr/include/boost/bind/bind.hpp:893
    292[#9](/bitcoin-bitcoin/9/)  boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(char const*, std::function<void ()>), boost::_bi::list2<boost::_bi::value<char const*>, boost::_bi::value<std::function<void ()> > > > >::run() (
    293    this=0x7f1334147470) at /usr/include/boost/thread/detail/thread.hpp:116
    294[#10](/bitcoin-bitcoin/10/) 0x00007f13500d55d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
    295[#11](/bitcoin-bitcoin/11/) 0x00007f134e2ae6ba in start_thread (arg=0x7f13317fd700) at pthread_create.c:333
    296[#12](/bitcoin-bitcoin/12/) 0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    297
    298Thread 8 (Thread 0x7f13297fd700 (LWP 24643)):
    299[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    300[#1](/bitcoin-bitcoin/1/)  0x0000562a89988aab in boost::condition_variable::wait (m=..., this=0x562a8a315248 <scriptcheckqueue+40>)
    301    at /usr/include/boost/thread/pthread/condition_variable.hpp:73
    302[#2](/bitcoin-bitcoin/2/)  CCheckQueue<CScriptCheck>::Loop (this=0x562a8a315220 <scriptcheckqueue>, fMaster=false) at checkqueue.h:102
    303[#3](/bitcoin-bitcoin/3/)  0x00007f13500d55d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
    304[#4](/bitcoin-bitcoin/4/)  0x00007f134e2ae6ba in start_thread (arg=0x7f13297fd700) at pthread_create.c:333
    305[#5](/bitcoin-bitcoin/5/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    306
    307Thread 7 (Thread 0x7f1331ffe700 (LWP 24642)):
    308[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    309[#1](/bitcoin-bitcoin/1/)  0x0000562a89988aab in boost::condition_variable::wait (m=..., this=0x562a8a315248 <scriptcheckqueue+40>)
    310    at /usr/include/boost/thread/pthread/condition_variable.hpp:73
    311[#2](/bitcoin-bitcoin/2/)  CCheckQueue<CScriptCheck>::Loop (this=0x562a8a315220 <scriptcheckqueue>, fMaster=false) at checkqueue.h:102
    312[#3](/bitcoin-bitcoin/3/)  0x00007f13500d55d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
    313[#4](/bitcoin-bitcoin/4/)  0x00007f134e2ae6ba in start_thread (arg=0x7f1331ffe700) at pthread_create.c:333
    314[#5](/bitcoin-bitcoin/5/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    315
    316Thread 6 (Thread 0x7f1338817700 (LWP 24641)):
    317[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    318[#1](/bitcoin-bitcoin/1/)  0x0000562a89988aab in boost::condition_variable::wait (m=..., this=0x562a8a315248 <scriptcheckqueue+40>)
    319    at /usr/include/boost/thread/pthread/condition_variable.hpp:73
    320[#2](/bitcoin-bitcoin/2/)  CCheckQueue<CScriptCheck>::Loop (this=0x562a8a315220 <scriptcheckqueue>, fMaster=false) at checkqueue.h:102
    321[#3](/bitcoin-bitcoin/3/)  0x00007f13500d55d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
    322[#4](/bitcoin-bitcoin/4/)  0x00007f134e2ae6ba in start_thread (arg=0x7f1338817700) at pthread_create.c:333
    323[#5](/bitcoin-bitcoin/5/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    324
    325Thread 5 (Thread 0x7f1339018700 (LWP 24640)):
    326[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    327[#1](/bitcoin-bitcoin/1/)  0x0000562a89988aab in boost::condition_variable::wait (m=..., this=0x562a8a315248 <scriptcheckqueue+40>)
    328    at /usr/include/boost/thread/pthread/condition_variable.hpp:73
    329[#2](/bitcoin-bitcoin/2/)  CCheckQueue<CScriptCheck>::Loop (this=0x562a8a315220 <scriptcheckqueue>, fMaster=false) at checkqueue.h:102
    330[#3](/bitcoin-bitcoin/3/)  0x00007f13500d55d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
    331[#4](/bitcoin-bitcoin/4/)  0x00007f134e2ae6ba in start_thread (arg=0x7f1339018700) at pthread_create.c:333
    332[#5](/bitcoin-bitcoin/5/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    333
    334Thread 4 (Thread 0x7f1339819700 (LWP 24639)):
    335[#0](/bitcoin-bitcoin/0/)  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
    336[#1](/bitcoin-bitcoin/1/)  0x0000562a89988aab in boost::condition_variable::wait (m=..., this=0x562a8a315248 <scriptcheckqueue+40>)
    337    at /usr/include/boost/thread/pthread/condition_variable.hpp:73
    338[#2](/bitcoin-bitcoin/2/)  CCheckQueue<CScriptCheck>::Loop (this=0x562a8a315220 <scriptcheckqueue>, fMaster=false) at checkqueue.h:102
    339[#3](/bitcoin-bitcoin/3/)  0x00007f13500d55d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
    340[#4](/bitcoin-bitcoin/4/)  0x00007f134e2ae6ba in start_thread (arg=0x7f1339819700) at pthread_create.c:333
    341[#5](/bitcoin-bitcoin/5/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    342
    343Thread 3 (Thread 0x7f133a22c700 (LWP 24638)):
    344[#0](/bitcoin-bitcoin/0/)  0x00007f134cc9db5d in poll () at ../sysdeps/unix/syscall-template.S:84
    345[#1](/bitcoin-bitcoin/1/)  0x00007f134bde838c in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
    346[#2](/bitcoin-bitcoin/2/)  0x00007f134bde849c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
    347[#3](/bitcoin-bitcoin/3/)  0x00007f134f06f7eb in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
    348   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    349[#4](/bitcoin-bitcoin/4/)  0x00007f134f016b4a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
    350   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    351[#5](/bitcoin-bitcoin/5/)  0x00007f134ee33834 in QThread::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    352[#6](/bitcoin-bitcoin/6/)  0x00007f134ee387be in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    353[#7](/bitcoin-bitcoin/7/)  0x00007f134e2ae6ba in start_thread (arg=0x7f133a22c700) at pthread_create.c:333
    354---Type <return> to continue, or q <return> to quit---
    355[#8](/bitcoin-bitcoin/8/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    356
    357Thread 2 (Thread 0x7f1343859700 (LWP 24636)):
    358[#0](/bitcoin-bitcoin/0/)  0x00007f134cc9db5d in poll () at ../sysdeps/unix/syscall-template.S:84
    359[#1](/bitcoin-bitcoin/1/)  0x00007f1349beac62 in ?? () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
    360[#2](/bitcoin-bitcoin/2/)  0x00007f1349bec8d7 in xcb_wait_for_event () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
    361[#3](/bitcoin-bitcoin/3/)  0x00007f13463e0629 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5XcbQpa.so.5
    362[#4](/bitcoin-bitcoin/4/)  0x00007f134ee387be in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    363[#5](/bitcoin-bitcoin/5/)  0x00007f134e2ae6ba in start_thread (arg=0x7f1343859700) at pthread_create.c:333
    364[#6](/bitcoin-bitcoin/6/)  0x00007f134cca982d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
    365
    366Thread 1 (Thread 0x7f1350c6b8c0 (LWP 24633)):
    367[#0](/bitcoin-bitcoin/0/)  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
    368[#1](/bitcoin-bitcoin/1/)  0x00007f134e2b0e42 in __GI___pthread_mutex_lock (mutex=0x7f12e83d61b0) at ../nptl/pthread_mutex_lock.c:115
    369[#2](/bitcoin-bitcoin/2/)  0x0000562a89708ea5 in boost::recursive_mutex::lock (this=<optimized out>)
    370    at /usr/include/boost/thread/pthread/recursive_mutex.hpp:113
    371[#3](/bitcoin-bitcoin/3/)  AnnotatedMixin<boost::recursive_mutex>::lock (this=<optimized out>) at ./sync.h:60
    372[#4](/bitcoin-bitcoin/4/)  boost::unique_lock<CCriticalSection>::lock (this=0x7ffef9fa0690)
    373    at /usr/include/boost/thread/lock_types.hpp:346
    374[#5](/bitcoin-bitcoin/5/)  CMutexLock<CCriticalSection>::Enter (pszName=<optimized out>, pszFile=<optimized out>, nLine=<optimized out>, 
    375    this=0x7ffef9fa0690) at ./sync.h:124
    376[#6](/bitcoin-bitcoin/6/)  CMutexLock<CCriticalSection>::CMutexLock (this=0x7ffef9fa0690, mutexIn=..., pszName=<optimized out>, 
    377    pszFile=<optimized out>, nLine=<optimized out>, fTry=<optimized out>) at ./sync.h:145
    378[#7](/bitcoin-bitcoin/7/)  0x0000562a8983c273 in CNode::copyStats (this=this@entry=0x7f12e83d6130, stats=...) at net.cpp:654
    379[#8](/bitcoin-bitcoin/8/)  0x0000562a89840c3b in CConnman::GetNodeStats (this=0x7f1334d8a5b0, 
    380    vstats=std::vector of length 16, capacity 21 = {...}) at net.cpp:2470
    381[#9](/bitcoin-bitcoin/9/)  0x0000562a8973d245 in PeerTablePriv::refreshPeers (this=0x562a8af93470) at qt/peertablemodel.cpp:61
    382[#10](/bitcoin-bitcoin/10/) 0x0000562a89738984 in PeerTableModel::refresh (this=0x562a8af929a0) at qt/peertablemodel.cpp:224
    383[#11](/bitcoin-bitcoin/11/) 0x00007f134f047d2a in QMetaObject::activate(QObject*, int, int, void**) ()
    384   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    385[#12](/bitcoin-bitcoin/12/) 0x00007f134f0545c8 in QTimer::timerEvent(QTimerEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    386[#13](/bitcoin-bitcoin/13/) 0x00007f134f048bb3 in QObject::event(QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    387[#14](/bitcoin-bitcoin/14/) 0x00007f134f98a05c in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
    388   from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
    389[#15](/bitcoin-bitcoin/15/) 0x00007f134f98f516 in QApplication::notify(QObject*, QEvent*) ()
    390   from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
    391[#16](/bitcoin-bitcoin/16/) 0x00007f134f01938b in QCoreApplication::notifyInternal(QObject*, QEvent*) ()
    392   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    393[#17](/bitcoin-bitcoin/17/) 0x00007f134f06e5ed in QTimerInfoList::activateTimers() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    394[#18](/bitcoin-bitcoin/18/) 0x00007f134f06eb29 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    395[#19](/bitcoin-bitcoin/19/) 0x00007f134bde8197 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
    396[#20](/bitcoin-bitcoin/20/) 0x00007f134bde83f0 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
    397[#21](/bitcoin-bitcoin/21/) 0x00007f134bde849c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
    398[#22](/bitcoin-bitcoin/22/) 0x00007f134f06f7cf in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
    399   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    400[#23](/bitcoin-bitcoin/23/) 0x00007f134f016b4a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
    401   from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    402[#24](/bitcoin-bitcoin/24/) 0x00007f134f01ebec in QCoreApplication::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
    403[#25](/bitcoin-bitcoin/25/) 0x0000562a896d7eaf in main (argc=3, argv=<optimized out>) at qt/bitcoin.cpp:691
    
  5. theuni commented at 9:16 pm on April 3, 2017: member
    0Thread 22 (Thread 0x7f12f0e39700 (LWP 25071)):
    1[#0](/bitcoin-bitcoin/0/)  0x00007f134e2b79ff in __libc_send (fd=55, buf=0x7f129234fcc0, n=999997, flags=flags@entry=16384)
    2   at ../sysdeps/unix/sysv/linux/x86_64/send.c:26
    3[#1](/bitcoin-bitcoin/1/)  0x0000562a8983f9f7 in CConnman::SocketSendData (this=this@entry=0x7f1334d8a5b0, 
    4   pnode=pnode@entry=0x7f12e83d6130) at net.cpp:844
    

    This one’s suspicious, it should be really rare to catch in a backtrace. The only way you would be likely to catch that is if MSG_DONTWAIT isn’t used, and the call is blocking.

    When you build, do you see a warning about MSG_DONTWAIT being redefined? I suspect you might’ve not re-run autogen after #9921.

    Either way, please try rebuilding with a fresh autogen/configure/make.

  6. achow101 commented at 9:26 pm on April 3, 2017: member
    @theuni Ahh. I do see a warning about MSG_DONTWAIT being redefined. I have rebuilt with a new autogen.sh and configure. I’ll see if the same issue still happens.
  7. achow101 commented at 9:35 pm on April 3, 2017: member
    Seems like it is working. I’ll close this for now.
  8. achow101 closed this on Apr 3, 2017

  9. MarcoFalke locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-24 18:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me