Early shutdown results in freeze (torControlThread not stopping) #16376

1. 
   mzumsande commented at 0:54 am on July 12, 2019: member

   Using Ubuntu 18.04.2 LTS, with up-to-date, self-compiled master. When I run ./bitcoind -regtest -whitebind=bla I expect a clean shutdown with Error: Cannot resolve -whitebind address: 'bla'. However sometimes my node doesn’t stop and freezes (shutdown never finishes). This happens ~30% of the time for me, both on testnet and regtest.

   I believe that this is because torControlThread.join(); never returns if it the thread is stopped right after its creation - it is started right before the whitebind part in AppInitMain(). If I insert a sleep before whitebind parameters are processed in Step 12 of AppInitMain() (or if I just disable the TorControl thread) the error does not occur.

   Could someone try to reproduce? I encountered this testing #16248, but it is not restricted to this branch.

2. fanquake added the label Linux/Unix on Jul 12, 2019
3. 
   ajtowns commented at 1:51 am on July 12, 2019: member

   I’ve seen the “node gets stuck in shutdown” thing myself, caused by the “get dns peers” thread taking forever. If you check in debug.log, you should see “dnsseed thread start” without a corresponding “dnsseed thread exit”. I think if you wait about 5 minutes you should see it finish up.
4. 
   mzumsande commented at 10:45 am on July 12, 2019: member

   In my case it’s not dnsseed, which has not been launched at the point of startup where the shutdown happens, plus I’ve been waiting for 30 minutes without anything happening. But of course the root cause could be related.
5. 
   ajtowns commented at 2:00 pm on July 16, 2019: member

   Yeah, I think there’s a race if shutdown happens early:

   init                       torcontrol
   StartTor()
     event_base_new()
     std::thread()
   InterruptTor()
     event_base_loopbreak()
                              TorControlThread()
                                   TorController()
                                   event_base_dispatch()
   StopTorControl()
     torControlThread.join()    
   

   If _loopbreak is called before _dispatch has started, it won’t have any effect. Calling event_base_loopexit(gBase,nullptr) instead (or as well) seems like it fixes it.

6. 
   promag commented at 11:48 pm on July 16, 2019: member

   I also hit the “hang” regarding dnsseed thread: #15657 (comment)

   I’ll try to reproduce the race.

7. 
   ajtowns commented at 10:39 am on July 17, 2019: member

   I found it reproduced pretty easily on master by invoking bitcoind with -proxy=127.0.0.1:9050 -listen=1 -bind=127.0.0.1 -whitebind=127.0.0.1:0 fwiw.
8. MarcoFalke added this to the milestone 0.19.0 on Jul 17, 2019
9. laanwj closed this on Jul 18, 2019

   ---

10. laanwj referenced this in commit 65d12110d4 on Jul 18, 2019
11. 
    sriharikapu commented at 5:50 am on September 3, 2019: none

    thanks @ajtowns this works perfectly fine for me bitcoind -proxy=127.0.0.1:9050 -listen=1 -bind=127.0.0.1 -whitebind=127.0.0.1:0
12. PastaPastaPasta referenced this in commit 9cc8ebed8a on Jun 27, 2021
13. PastaPastaPasta referenced this in commit 59d599fb5b on Jun 28, 2021
14. PastaPastaPasta referenced this in commit 643bae5bd3 on Jun 29, 2021
15. PastaPastaPasta referenced this in commit 008b7889e8 on Jul 1, 2021
16. PastaPastaPasta referenced this in commit eba7908278 on Jul 1, 2021
17. PastaPastaPasta referenced this in commit d47f4f9b2a on Jul 12, 2021
18. PastaPastaPasta referenced this in commit f2bb2844a0 on Jul 13, 2021
19. MarcoFalke locked this on Dec 16, 2021
20. gades referenced this in commit 8bca0ed535 on Apr 20, 2022

Contributors
mzumsande ajtowns promag sriharikapu

Labels
Linux/Unix

Milestone
0.19.0