Occasional ipc_test failures (macOS?)

Sjors commented at 8:09 am on February 6, 2025: member

I’m seeing this test fail occasionally, both locally (on macOS) and on CI: https://github.com/bitcoin/bitcoin/actions/runs/13161509993/job/36731182334?pr=31802#step:7:2568

Not sure if it’s a macOS specific thing, that’s just based on the above CI run that also happens to be macOS. I’ll keep an eye on it.

fanquake commented at 10:47 am on February 6, 2025: member

Copied the output so we aren’t relying on GH actions to retain it.

 0test/ipc_tests.cpp:11: Entering test suite "ipc_tests"
 1test/ipc_tests.cpp:12: Entering test case "ipc_tests"
 2libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
 32025-02-05T16:12:45.463402Z [unknown] [test/util/random.cpp:48] [SeedRandomStateForTest] Setting random seed for current tests to RANDOM_CTX_SEED=40e3ae01ebf0dcb2dc43849f963c573142c9f0ec38096c9feb4f14563c55cb78
 42025-02-05T16:12:45.472556Z [test] [init/common.cpp:153] [LogPackageVersion] Bitcoin Core version v28.99.0-cd63f7b213c7-dirty (release build)
 52025-02-05T16:12:45.472716Z [test] [kernel/context.cpp:20] [operator()] Using the 'arm_shani(1way,2way)' SHA256 implementation
 62025-02-05T16:12:45.483530Z [test] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client first request from current thread, constructing waiter
 72025-02-05T16:12:45.483734Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.add$Params (a = 1, b = 2)
 82025-02-05T16:12:45.483815Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#1](/bitcoin-core-multiprocess/1/) FooInterface.add$Params (a = 1, b = 2)
 92025-02-05T16:12:45.483854Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#1](/bitcoin-core-multiprocess/1/) FooInterface.add$Results (result = 3)
102025-02-05T16:12:45.483883Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.add$Results (result = 3)
112025-02-05T16:12:45.484420Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.passOutPoint$Params (arg = "d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000")
122025-02-05T16:12:45.484443Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#2](/bitcoin-core-multiprocess/2/) FooInterface.passOutPoint$Params (arg = "d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000")
132025-02-05T16:12:45.484458Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#2](/bitcoin-core-multiprocess/2/) FooInterface.passOutPoint$Results (result = "d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000")
142025-02-05T16:12:45.484477Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.passOutPoint$Results (result = "d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000")
152025-02-05T16:12:45.484688Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.passUniValue$Params (arg = "{\\"i\\":1,\\"s\\":\\"two\\"}")
162025-02-05T16:12:45.484710Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#3](/bitcoin-core-multiprocess/3/) FooInterface.passUniValue$Params (arg = "{\\"i\\":1,\\"s\\":\\"two\\"}")
172025-02-05T16:12:45.484724Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#3](/bitcoin-core-multiprocess/3/) FooInterface.passUniValue$Results (result = "{\\"i\\":1,\\"s\\":\\"two\\"}")
182025-02-05T16:12:45.484741Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.passUniValue$Results (result = "{\\"i\\":1,\\"s\\":\\"two\\"}")
192025-02-05T16:12:45.484770Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.passTransaction$Params (arg = "\\002\\000\\000\\000\\001d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000\\000\\377\\377\\377\\377\\001\\000\\341\\365\\005\\000\\000\\000\\000\\000\\003\\000\\000\\000")
202025-02-05T16:12:45.484790Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#4](/bitcoin-core-multiprocess/4/) FooInterface.passTransaction$Params (arg = "\\002\\000\\000\\000\\001d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000\\000\\377\\377\\377\\377\\001\\000\\341\\365\\005\\000\\000\\000\\000\\000\\003\\000\\000\\000")
212025-02-05T16:12:45.484966Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#4](/bitcoin-core-multiprocess/4/) FooInterface.passTransaction$Results (result = "\\002\\000\\000\\000\\001d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000\\000\\377\\377\\377\\377\\001\\000\\341\\365\\005\\000\\000\\000\\000\\000\\003\\000\\000\\000")
222025-02-05T16:12:45.485063Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.passTransaction$Results (result = "\\002\\000\\000\\000\\001d\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\000\\310\\000\\000\\000\\000\\377\\377\\377\\377\\001\\000\\341\\365\\005\\000\\000\\000\\000\\000\\003\\000\\000\\000")
232025-02-05T16:12:45.485124Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.passVectorChar$Params (arg = "Hello")
242025-02-05T16:12:45.485171Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#5](/bitcoin-core-multiprocess/5/) FooInterface.passVectorChar$Params (arg = "Hello")
252025-02-05T16:12:45.485198Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#5](/bitcoin-core-multiprocess/5/) FooInterface.passVectorChar$Results (result = "Hello")
262025-02-05T16:12:45.485239Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.passVectorChar$Results (result = "Hello")
272025-02-05T16:12:45.485430Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.passBlockState$Params (arg = (mode = 1, result = 8, rejectReason = "reject reason", debugMessage = "debug message"))
282025-02-05T16:12:45.485488Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#6](/bitcoin-core-multiprocess/6/) FooInterface.passBlockState$Params (arg = (mode = 1, result = 8, rejectReason = "reject reason", debugMessage = "debug message"))
292025-02-05T16:12:45.485525Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#6](/bitcoin-core-multiprocess/6/) FooInterface.passBlockState$Results (result = (mode = 1, result = 8, rejectReason = "reject reason", debugMessage = "debug message"))
302025-02-05T16:12:45.485571Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.passBlockState$Results (result = (mode = 1, result = 8, rejectReason = "reject reason", debugMessage = "debug message"))
312025-02-05T16:12:45.485619Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.passBlockState$Params (arg = (mode = 0, result = 0, rejectReason = "", debugMessage = ""))
322025-02-05T16:12:45.485663Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#7](/bitcoin-core-multiprocess/7/) FooInterface.passBlockState$Params (arg = (mode = 0, result = 0, rejectReason = "", debugMessage = ""))
332025-02-05T16:12:45.485693Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#7](/bitcoin-core-multiprocess/7/) FooInterface.passBlockState$Results (result = (mode = 0, result = 0, rejectReason = "", debugMessage = ""))
342025-02-05T16:12:45.485737Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.passBlockState$Results (result = (mode = 0, result = 0, rejectReason = "", debugMessage = ""))
352025-02-05T16:12:45.485793Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client send FooInterface.passScript$Params (arg = "[")
362025-02-05T16:12:45.485833Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server recv request  [#8](/bitcoin-core-multiprocess/8/) FooInterface.passScript$Params (arg = "[")
372025-02-05T16:12:45.485858Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server send response [#8](/bitcoin-core-multiprocess/8/) FooInterface.passScript$Results (result = "[")
382025-02-05T16:12:45.485898Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/b-test-54829} IPC client recv FooInterface.passScript$Results (result = "[")
392025-02-05T16:12:45.486162Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} IPC server destroy N2mp11ProxyServerIN3gen12FooInterfaceEEE
402025-02-05T16:12:45.486271Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} EventLoop::loop done, cancelling event listeners.
412025-02-05T16:12:45.486283Z [unknown] [test/ipc_test.cpp:60] [operator()] LOG0: {IpcPipeTest-22316/54830} EventLoop::loop bye.
42
43132/140 Test [#135](/bitcoin-core-multiprocess/135/): wallet_crypto_tests ..................   Passed    3.05 sec
44133/140 Test [#134](/bitcoin-core-multiprocess/134/): spend_tests ..........................   Passed    3.68 sec
45134/140 Test   [#8](/bitcoin-core-multiprocess/8/): secp256k1_exhaustive_tests ...........   Passed   27.97 sec
46135/140 Test [#139](/bitcoin-core-multiprocess/139/): walletload_tests .....................   Passed    4.45 sec
47136/140 Test  [#10](/bitcoin-core-multiprocess/10/): bench_sanity_check_high_priority .....   Passed   28.82 sec
48137/140 Test [#136](/bitcoin-core-multiprocess/136/): wallet_tests .........................   Passed    9.35 sec
49138/140 Test [#127](/bitcoin-core-multiprocess/127/): coinselector_tests ...................   Passed   20.56 sec
50139/140 Test   [#6](/bitcoin-core-multiprocess/6/): secp256k1_noverify_tests .............   Passed   42.48 sec
51Errors while running CTest
52140/140 Test   [#7](/bitcoin-core-multiprocess/7/): secp256k1_tests ......................   Passed   59.80 sec
53
5499% tests passed, 1 tests failed out of 140
55
56Total Test time (real) =  59.82 sec
57
58The following tests FAILED:
59	140 - ipc_tests (Subprocess aborted)

ryanofsky commented at 2:26 pm on February 6, 2025: collaborator

This is probably caused by the same bug reported in #128 based on the “mutex lock failed: Invalid argument” error. But the version reported here might be easier to debug because the crash is happening in a unit test instead of in a python test.

If you can reproduce this problem locally sometimes, maybe you could run it in a loop until it crashes and try to get a stack trace? This script might work:

0#!/bin/bash
1while true; do
2    lldb --batch -o "run" -o "bt" -- build/src/test/test_bitcoin -t ipc_tests
3    status=$?
4    if [[ $status -ne 0 ]]; then
5        echo "Program crashed with exit code $status. Stopping."
6        break
7    fi
8done

(script from chatgpt of course…)

ryanofsky commented at 5:40 pm on February 6, 2025: collaborator

I couldn’t reproduce this but I think I can see how this problem might be happening.

The “mutex lock failed” exception is thrown from libc++ mutex::lock method when there’s any failure to lock the mutex.

In this case I think the exception might being thrown in the the ~CapnpProtocol destructor when it is calling EventLoop::removeClient to decrement the last reference count on the event loop. Decrementing the count will cause the event loop to exit, and for the event loop thread to destroy it here.

The problem is that while the event loop thread is destroying the event loop object, the ~CapnpProtocol destructor thread could be trying to lock the EventLoop::m_mutex member here after it calls write(post_fd).

I think a simple fix for this would look like:

 0--- a/src/ipc/libmultiprocess/src/mp/proxy.cpp
 1+++ b/src/ipc/libmultiprocess/src/mp/proxy.cpp
 2@@ -240,10 +240,12 @@ void EventLoop::removeClient(std::unique_lock<std::mutex>& lock)
 3     if (done(lock)) {
 4         m_cv.notify_all();
 5         int post_fd{m_post_fd};
 6-        Unlock(lock, [&] {
 7-            char buffer = 0;
 8-            KJ_SYSCALL(write(post_fd, &buffer, 1)); // NOLINT(bugprone-suspicious-semicolon)
 9-        });
10+        // Intentionally do not unlock/relock the mutex around this final write
11+        // call, so clients are free to destroy the EventLoop object immediately
12+        // after the loop() method exits, and code here will not be trying to
13+        // relock the event loop mutex after it was destroyed.
14+        char buffer = 0;
15+        KJ_SYSCALL(write(post_fd, &buffer, 1)); // NOLINT(bugprone-suspicious-semicolon)
16     }
17 }
18

ryanofsky referenced this in commit baea320be0 on Feb 7, 2025

ryanofsky commented at 12:17 pm on February 7, 2025: collaborator

The change in https://github.com/bitcoin/bitcoin/pull/31815 should be a better fix for this issue than the change suggested https://github.com/chaincodelabs/libmultiprocess/issues/154#issuecomment-2640564671, which is a less direct fix and could also theoretically cause deadlocks if the write(post_fd) call blocks.

Assuming the issue is the CapnpProtocol::startLoop calling m_loop.reset() too early, adding a mutex around that call is a more direct fix than holding onto the lock in EventLoop::removeClient so EventLoop::loop call can’t exit and m_loop.reset() is indirectly delayed.

Sjors referenced this in commit 1c1b0e7f94 on Feb 7, 2025

Sjors referenced this in commit ada66eb4e4 on Feb 7, 2025

Sjors referenced this in commit 05a08fed63 on Feb 7, 2025

Sjors referenced this in commit 106fd58563 on Feb 7, 2025

ryanofsky referenced this in commit be6ad9f955 on Feb 10, 2025

ryanofsky referenced this in commit f8cbd0d34a on Feb 10, 2025

ryanofsky referenced this in commit 61bf9456f9 on Feb 10, 2025

ryanofsky referenced this in commit 48d01bcca7 on Feb 10, 2025

ryanofsky referenced this in commit 26b9f3dda4 on Feb 10, 2025

ryanofsky commented at 5:02 am on February 10, 2025: collaborator

Since the change in https://github.com/bitcoin/bitcoin/pull/31815 didn’t work (see comments there), I set up a macos vm using quickemu and was able to reproduce this bug locally, get a stack trace, and fix it.

Will keep this issue open to see if the fix works in CI, but https://github.com/chaincodelabs/libmultiprocess/pull/159 is the change, and the stack trace looked like:

 0(lldb) bt
 1* thread [#1](/bitcoin-core-multiprocess/1/), name = 'b-test', queue = 'com.apple.main-thread', stop reason = signal SIGABRT
 2  * frame [#0](/bitcoin-core-multiprocess/0/): 0x00007ff80922a196 libsystem_kernel.dylib`__pthread_kill + 10
 3    frame [#1](/bitcoin-core-multiprocess/1/): 0x00007ff809261ee6 libsystem_pthread.dylib`pthread_kill + 263
 4    frame [#2](/bitcoin-core-multiprocess/2/): 0x00007ff809188b45 libsystem_c.dylib`abort + 123
 5    frame [#3](/bitcoin-core-multiprocess/3/): 0x000000010290aa23 libc++abi.1.dylib`abort_message + 195
 6    frame [#4](/bitcoin-core-multiprocess/4/): 0x00000001028e8179 libc++abi.1.dylib`demangling_terminate_handler() + 233
 7    frame [#5](/bitcoin-core-multiprocess/5/): 0x0000000102908e76 libc++abi.1.dylib`std::__terminate(void (*)()) + 6
 8    frame [#6](/bitcoin-core-multiprocess/6/): 0x0000000102908e18 libc++abi.1.dylib`std::terminate() + 56
 9    frame [#7](/bitcoin-core-multiprocess/7/): 0x000000010000e6c2 test_bitcoin`__clang_call_terminate + 18
10    frame [#8](/bitcoin-core-multiprocess/8/): 0x00000001011f3ab2 test_bitcoin`mp::EventLoop::post(std::__1::function<void ()> const&) [inlined] mp::UnlockGuard<std::__1::unique_lock<std::__1::mutex> >::~UnlockGuard(this=0x00007ff7bfefe1c8) at util.h:138:29 [opt]
11    frame [#9](/bitcoin-core-multiprocess/9/): 0x00000001011f3aa6 test_bitcoin`mp::EventLoop::post(std::__1::function<void ()> const&) [inlined] mp::UnlockGuard<std::__1::unique_lock<std::__1::mutex> >::~UnlockGuard(this=0x00007ff7bfefe1c8) at util.h:138:20 [opt]
12    frame [#10](/bitcoin-core-multiprocess/10/): 0x00000001011f3aa6 test_bitcoin`mp::EventLoop::post(std::__1::function<void ()> const&) [inlined] void mp::Unlock<std::__1::unique_lock<std::__1::mutex>, mp::EventLoop::post(std::__1::function<void ()> const&)::$_1>(lock=0x00007ff7bfefe1f0, callback=<unavailable>)> const&)::$_1&&) at util.h:147:1 [opt]
13    frame [#11](/bitcoin-core-multiprocess/11/): 0x00000001011f3aa6 test_bitcoin`mp::EventLoop::post(this=0x0000000102e12240, fn=0x00007ff7bfefe250)> const&) at proxy.cpp:231:5 [opt]
14    frame [#12](/bitcoin-core-multiprocess/12/): 0x0000000100fc5143 test_bitcoin`std::__1::__function::__func<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'(), std::__1::allocator<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()>, void ()>::operator()() [inlined] void mp::EventLoop::sync<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()::operator()() const::'lambda'()>(this=<unavailable>, callable=0x00007ff7bfefe240)::'lambda0'()::operator()() const::'lambda'()&&) at proxy-io.h:155:9 [opt]
15    frame [#13](/bitcoin-core-multiprocess/13/): 0x0000000100fc511c test_bitcoin`std::__1::__function::__func<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'(), std::__1::allocator<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()>, void ()>::operator()() [inlined] mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(this=<unavailable>)::'lambda0'()::operator()() const at proxy-io.h:426:38 [opt]
16    frame [#14](/bitcoin-core-multiprocess/14/): 0x0000000100fc50f1 test_bitcoin`std::__1::__function::__func<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'(), std::__1::allocator<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()>, void ()>::operator()() [inlined] decltype(__f=<unavailable>)::'lambda0'()&>()()) std::__1::__invoke[abi:ne190107]<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()&>(mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()&) at invoke.h:149:25 [opt]
17    frame [#15](/bitcoin-core-multiprocess/15/): 0x0000000100fc50f1 test_bitcoin`std::__1::__function::__func<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'(), std::__1::allocator<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()>, void ()>::operator()() [inlined] void std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ne190107]<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(__args=<unavailable>)::'lambda0'()&>(mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()&) at invoke.h:224:5 [opt]
18    frame [#16](/bitcoin-core-multiprocess/16/): 0x0000000100fc50f1 test_bitcoin`std::__1::__function::__func<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'(), std::__1::allocator<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()>, void ()>::operator()() [inlined] std::__1::__function::__alloc_func<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'(), std::__1::allocator<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()>, void ()>::operator(this=<unavailable>)[abi:ne190107]() at function.h:171:12 [opt]
19    frame [#17](/bitcoin-core-multiprocess/17/): 0x0000000100fc50f1 test_bitcoin`std::__1::__function::__func<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'(), std::__1::allocator<mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::ProxyClientBase(ipc::capnp::messages::Init::Client, mp::Connection*, bool)::'lambda0'()>, void ()>::operator(this=<unavailable>)() at function.h:313:10 [opt]
20    frame [#18](/bitcoin-core-multiprocess/18/): 0x00000001011f6a07 test_bitcoin`mp::CleanupRun(std::__1::list<std::__1::function<void ()>, std::__1::allocator<std::__1::function<void ()> > >&) [inlined] std::__1::__function::__value_func<void ()>::operator(this=0x00007ff7bfefe2b0)[abi:ne190107]() const at function.h:430:12 [opt]
21    frame [#19](/bitcoin-core-multiprocess/19/): 0x00000001011f69f8 test_bitcoin`mp::CleanupRun(std::__1::list<std::__1::function<void ()>, std::__1::allocator<std::__1::function<void ()> > >&) [inlined] std::__1::function<void ()>::operator(this=0x00007ff7bfefe2b0)() const at function.h:989:10 [opt]
22    frame [#20](/bitcoin-core-multiprocess/20/): 0x00000001011f69f8 test_bitcoin`mp::CleanupRun(fns=size=0)>, std::__1::allocator<std::__1::function<void ()> > >&) at proxy.h:46:9 [opt]
23    frame [#21](/bitcoin-core-multiprocess/21/): 0x0000000100fc558e test_bitcoin`mp::ProxyClientBase<ipc::capnp::messages::Init, interfaces::Init>::~ProxyClientBase(this=0x000060000171e880) at proxy-io.h:449:5 [opt]
24    frame [#22](/bitcoin-core-multiprocess/22/): 0x0000000100ff1a38 test_bitcoin`mp::ProxyClient<ipc::capnp::messages::Init>::~ProxyClient() [inlined] mp::ProxyClientCustom<ipc::capnp::messages::Init, interfaces::Init>::~ProxyClientCustom(this=0x000060000171e880) at proxy.h:112:7 [opt]
25    frame [#23](/bitcoin-core-multiprocess/23/): 0x0000000100ff1a30 test_bitcoin`mp::ProxyClient<ipc::capnp::messages::Init>::~ProxyClient() [inlined] mp::ProxyClient<ipc::capnp::messages::Init>::~ProxyClient(this=0x000060000171e880) at init.capnp.proxy-types.c++:7:82 [opt]
26    frame [#24](/bitcoin-core-multiprocess/24/): 0x0000000100ff1a1d test_bitcoin`mp::ProxyClient<ipc::capnp::messages::Init>::~ProxyClient() [inlined] mp::ProxyClient<ipc::capnp::messages::Init>::~ProxyClient(this=0x000060000171e880) at init.capnp.proxy-types.c++:7:57 [opt]
27    frame [#25](/bitcoin-core-multiprocess/25/): 0x0000000100ff1a1d test_bitcoin`mp::ProxyClient<ipc::capnp::messages::Init>::~ProxyClient(this=0x000060000171e880) at init.capnp.proxy-types.c++:7:57 [opt]
28    frame [#26](/bitcoin-core-multiprocess/26/): 0x0000000100f9a69b test_bitcoin`IpcSocketPairTest() [inlined] std::__1::default_delete<interfaces::Init>::operator(this=0x00007ff7bfefe3e8, __ptr=<unavailable>)[abi:ne190107](interfaces::Init*) const at unique_ptr.h:80:5 [opt]
29    frame [#27](/bitcoin-core-multiprocess/27/): 0x0000000100f9a695 test_bitcoin`IpcSocketPairTest() [inlined] std::__1::unique_ptr<interfaces::Init, std::__1::default_delete<interfaces::Init> >::reset[abi:ne190107](this=0x00007ff7bfefe3e8, __p=0x0000000000000000) at unique_ptr.h:292:7 [opt]
30    frame [#28](/bitcoin-core-multiprocess/28/): 0x0000000100f9a67e test_bitcoin`IpcSocketPairTest() at ipc_test.cpp:149:17 [opt]
31    frame [#29](/bitcoin-core-multiprocess/29/): 0x0000000100a0587a test_bitcoin`ipc_tests::ipc_tests::test_method(this=0x00007ff7bfefe710) at ipc_tests.cpp:16:5 [opt]
32    frame [#30](/bitcoin-core-multiprocess/30/): 0x0000000100a05742 test_bitcoin`ipc_tests::ipc_tests_invoker() at ipc_tests.cpp:12:1 [opt]
33    frame [#31](/bitcoin-core-multiprocess/31/): 0x0000000100054581 test_bitcoin`boost::detail::function::function_obj_invoker<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) [inlined] boost::function_n<void>::operator(this=<unavailable>)() const at function_template.hpp:789:14 [opt]
34    frame [#32](/bitcoin-core-multiprocess/32/): 0x000000010005456e test_bitcoin`boost::detail::function::function_obj_invoker<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) [inlined] boost::detail::forward::operator(this=<unavailable>)() at execution_monitor.ipp:1405:32 [opt]
35    frame [#33](/bitcoin-core-multiprocess/33/): 0x000000010005456b test_bitcoin`boost::detail::function::function_obj_invoker<boost::detail::forward, int>::invoke(function_obj_ptr=<unavailable>) at function_template.hpp:79:18 [opt]
36    frame [#34](/bitcoin-core-multiprocess/34/): 0x0000000100018786 test_bitcoin`boost::execution_monitor::catch_signals(boost::function<int ()> const&) [inlined] boost::function_n<int>::operator(this=<unavailable>)() const at function_template.hpp:789:14 [opt]
37    frame [#35](/bitcoin-core-multiprocess/35/): 0x0000000100018770 test_bitcoin`boost::execution_monitor::catch_signals(boost::function<int ()> const&) [inlined] int boost::detail::do_invoke<boost::shared_ptr<boost::detail::translator_holder_base>, boost::function<int ()> >(tr=0x0000000101660ca0, F=<unavailable>)> const&) at execution_monitor.ipp:318:30 [opt]
38    frame [#36](/bitcoin-core-multiprocess/36/): 0x000000010001875c test_bitcoin`boost::execution_monitor::catch_signals(this=0x0000000101660c88, F=0x00007ff7bfeff030)> const&) at execution_monitor.ipp:920:16 [opt]
39    frame [#37](/bitcoin-core-multiprocess/37/): 0x0000000100018a2b test_bitcoin`boost::execution_monitor::execute(this=0x0000000101660c88, F=0x00007ff7bfeff030)> const&) at execution_monitor.ipp:1318:16 [opt]
40    frame [#38](/bitcoin-core-multiprocess/38/): 0x0000000100013d4e test_bitcoin`boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned long) [inlined] boost::execution_monitor::vexecute(this=0x0000000101660c88, F=0x0000000102c21018)> const&) at execution_monitor.ipp:1414:5 [opt]
41    frame [#39](/bitcoin-core-multiprocess/39/): 0x0000000100013d2f test_bitcoin`boost::unit_test::unit_test_monitor_t::execute_and_translate(this=0x0000000101660c88, func=0x0000000102c21018, timeout_microseconds=0)> const&, unsigned long) at unit_test_monitor.ipp:49:9 [opt]
42    frame [#40](/bitcoin-core-multiprocess/40/): 0x0000000100015543 test_bitcoin`boost::unit_test::framework::state::execute_test_tree(this=0x0000000101660d60, tu_id=66179, timeout_microseconds=0, p_random_generator=0x00007ff7bfeff300) at framework.ipp:815:44 [opt]
43    frame [#41](/bitcoin-core-multiprocess/41/): 0x0000000100015b68 test_bitcoin`boost::unit_test::framework::state::execute_test_tree(this=0x0000000101660d60, tu_id=131, timeout_microseconds=0, p_random_generator=0x00007ff7bfeff4a0) at framework.ipp:740:54 [opt]
44    frame [#42](/bitcoin-core-multiprocess/42/): 0x0000000100015b68 test_bitcoin`boost::unit_test::framework::state::execute_test_tree(this=0x0000000101660d60, tu_id=1, timeout_microseconds=0, p_random_generator=0x0000000000000000) at framework.ipp:740:54 [opt]
45    frame [#43](/bitcoin-core-multiprocess/43/): 0x0000000100012ca8 test_bitcoin`boost::unit_test::framework::run(id=1, continue_test=<unavailable>) at framework.ipp:1722:29 [opt]
46    frame [#44](/bitcoin-core-multiprocess/44/): 0x00000001000248c3 test_bitcoin`boost::unit_test::unit_test_main(init_func=<unavailable>, argc=<unavailable>, argv=<unavailable>)(int, char**), int, char**) at unit_test_main.ipp:250:9 [opt]
47    frame [#45](/bitcoin-core-multiprocess/45/): 0x00007ff808f07418 dyld`start + 1896

Sjors commented at 8:29 am on February 10, 2025: member

Included in the latest push: https://github.com/bitcoin/bitcoin/pull/30975#event-16243241286

ryanofsky commented at 4:03 pm on February 11, 2025: collaborator

Will close this issue since I am pretty confident about the fix in #159 and we haven’t seen this problem recur since it’s been incorporated.

ryanofsky closed this on Feb 11, 2025

Occasional ipc_test failures (macOS?) #154