I performed a bit of hand-rolled static analysis on the Bitcoin source to check for potential deadlocks, by examining partial lock orders (like DEBUG_LOCKORDER but more thorough). I found a few, which I checked manually.
This list is not comprehensive, as indirect calls (functions pointers, virtual functions, such as in the RPC dispatcher) are currently ignored by my static checker.
These cases are the exceptions. The cases under "counterexample" are the "normal" order, or at least the most common one.
Case 4 involves TRY_LOCKs, so it is probably OK.
1.
SendMessages LOCKs cs_inventory lock on a node
→ CWallet::GetTransaction LOCKs the cs_wallet lock.
Counterexample:
CWallet::ResendWalletTransactions LOCKs cs_wallet lock
→ CWallet::RelayWalletTransaction
→ RelayMessage
→ RelayInventory
→ CNode::PushInventory which LOCKs the cs_inventory lock
2.
CTxMemPool::accept(CTxDB&, CTransaction&, bool, bool*) LOCKs local static lock for rate-limit
→ CWallet::GetDebit(CTransaction const&) const
→CWallet::GetDebit(CTxIn const&) const LOCKs CWallet lock
Counterexample:
CWallet::CommitTransaction(CWalletTx&, CReserveKey&) LOCKs CWallet lock
→ CMerkleTx::AcceptToMemoryPool()
→ CTxMemPool::accept(CTxDB&, CTransaction&, bool, bool*) LOCKs local static lock for rate-limit
3.
CWalletTx::AcceptWalletTransaction(CTxDB&, bool) LOCKs CTxMemPool:cs
→ CTxMemPool::accept(CTxDB&, CTransaction&, bool, bool*)
→CWallet::GetDebit(CTransaction const&)
→CWallet::GetDebit(CTxIn const&) LOCKs CWallet:cs_wallet
CreateNewBlock(CReserveKey&) LOCKs CTxMemPool:cs
→ CBlock::ConnectBlock(CTxDB&, CBlockIndex*, bool)
→ CWallet::AddToWalletIfInvolvingMe(CTransaction const&, CBlock const*, bool, bool) LOCKs CWallet:cs_wallet
Counterexample:
CWallet::CommitTransaction(CWalletTx&, CReserveKey&) LOCKs CWallet:cs_wallet
→ CMerkleTx::AcceptToMemoryPool()
→ CTransaction::ClientConnectInputs() LOCKs CTxMemPool:cs
4.
ThreadSocketHandler2(void*) LOCKs cs_vNodes, then TRYLOCKs CNode:cs_vSend (when disconnecting nodes)
Counterexample:
ThreadMessageHandler2(void*) TRYLOCKs CNode:cs_vSend
→ SendMessages(CNode*, bool) LOCKs cs_vNodes (for address refresh broadcast)
