29.x: using a local libmultiprocess install will no-longer work #33576

Given that the 29.x branch doesn't have the changes to accomodate upstream changes in libmultiprocess, anyone who builds and installs libmultiprocess, and then tries to build Core with -DWITH_MULTIPROCESS=ON, will get a compile failure:

[ 34%] Building CXX object src/ipc/CMakeFiles/bitcoin_ipc.dir/capnp/protocol.cpp.o
/bitcoin/src/ipc/capnp/protocol.cpp: In destructor 'virtual ipc::capnp::{anonymous}::CapnpProtocol::~CapnpProtocol()':
/bitcoin/src/ipc/capnp/protocol.cpp:45:62: error: no matching function for call to 'std::unique_lock<std::mutex>::unique_lock(mp::Mutex&)'
   45 |             std::unique_lock<std::mutex> lock(m_loop->m_mutex);
      |                                                              ^
In file included from /usr/include/c++/13/mutex:46,
                 from /bitcoin/src/threadsafety.h:9,
                 from /bitcoin/src/logging.h:10,
                 from /bitcoin/src/interfaces/node.h:10,
                 from /bitcoin/src/interfaces/init.h:11,
                 from /bitcoin/src/ipc/capnp/protocol.cpp:5:
/usr/include/c++/13/bits/unique_lock.h:68:16: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   68 |       explicit unique_lock(mutex_type& __m)
      |                ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:68:40: note:   no known conversion for argument 1 from 'mp::Mutex' to 'std::unique_lock<std::mutex>::mutex_type&' {aka 'std::mutex&'}
   68 |       explicit unique_lock(mutex_type& __m)
      |                            ~~~~~~~~~~~~^~~
/usr/include/c++/13/bits/unique_lock.h:97:9: note: candidate: 'template<class _Rep, class _Period> std::unique_lock<_Mutex>::unique_lock(mutex_type&, const std::chrono::duration<_Rep, _Period>&) [with _Period = _Rep; _Mutex = std::mutex]'
   97 |         unique_lock(mutex_type& __m,
      |         ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:97:9: note:   template argument deduction/substitution failed:
/bitcoin/src/ipc/capnp/protocol.cpp:45:62: note:   candidate expects 2 arguments, 1 provided
   45 |             std::unique_lock<std::mutex> lock(m_loop->m_mutex);
      |                                                              ^
/usr/include/c++/13/bits/unique_lock.h:90:9: note: candidate: 'template<class _Clock, class _Duration> std::unique_lock<_Mutex>::unique_lock(mutex_type&, const std::chrono::time_point<_Clock, _Duration>&) [with _Duration = _Clock; _Mutex = std::mutex]'
   90 |         unique_lock(mutex_type& __m,
      |         ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:90:9: note:   template argument deduction/substitution failed:
/bitcoin/src/ipc/capnp/protocol.cpp:45:62: note:   candidate expects 2 arguments, 1 provided
   45 |             std::unique_lock<std::mutex> lock(m_loop->m_mutex);
      |                                                              ^
/usr/include/c++/13/bits/unique_lock.h:112:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(std::unique_lock<_Mutex>&&) [with _Mutex = std::mutex]'
  112 |       unique_lock(unique_lock&& __u) noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:112:33: note:   no known conversion for argument 1 from 'mp::Mutex' to 'std::unique_lock<std::mutex>&&'
  112 |       unique_lock(unique_lock&& __u) noexcept
      |                   ~~~~~~~~~~~~~~^~~
/usr/include/c++/13/bits/unique_lock.h:83:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&, std::adopt_lock_t) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   83 |       unique_lock(mutex_type& __m, adopt_lock_t) noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:83:7: note:   candidate expects 2 arguments, 1 provided
/usr/include/c++/13/bits/unique_lock.h:79:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&, std::try_to_lock_t) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   79 |       unique_lock(mutex_type& __m, try_to_lock_t)
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:79:7: note:   candidate expects 2 arguments, 1 provided
/usr/include/c++/13/bits/unique_lock.h:75:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&, std::defer_lock_t) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   75 |       unique_lock(mutex_type& __m, defer_lock_t) noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:75:7: note:   candidate expects 2 arguments, 1 provided
/usr/include/c++/13/bits/unique_lock.h:64:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock() [with _Mutex = std::mutex]'
   64 |       unique_lock() noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:64:7: note:   candidate expects 0 arguments, 1 provided
/bitcoin/src/ipc/capnp/protocol.cpp:46:21: error: 'class mp::EventLoop' has no member named 'removeClient'
   46 |             m_loop->removeClient(lock);
      |                     ^~~~~~~~~~~~
/bitcoin/src/ipc/capnp/protocol.cpp: In lambda function:
/bitcoin/src/ipc/capnp/protocol.cpp:87:66: error: no matching function for call to 'std::unique_lock<std::mutex>::unique_lock(mp::Mutex&)'
   87 |                 std::unique_lock<std::mutex> lock(m_loop->m_mutex);
      |                                                                  ^
/usr/include/c++/13/bits/unique_lock.h:68:16: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   68 |       explicit unique_lock(mutex_type& __m)
      |                ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:68:40: note:   no known conversion for argument 1 from 'mp::Mutex' to 'std::unique_lock<std::mutex>::mutex_type&' {aka 'std::mutex&'}
   68 |       explicit unique_lock(mutex_type& __m)
      |                            ~~~~~~~~~~~~^~~
/usr/include/c++/13/bits/unique_lock.h:97:9: note: candidate: 'template<class _Rep, class _Period> std::unique_lock<_Mutex>::unique_lock(mutex_type&, const std::chrono::duration<_Rep, _Period>&) [with _Period = _Rep; _Mutex = std::mutex]'
   97 |         unique_lock(mutex_type& __m,
      |         ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:97:9: note:   template argument deduction/substitution failed:
/bitcoin/src/ipc/capnp/protocol.cpp:87:66: note:   candidate expects 2 arguments, 1 provided
   87 |                 std::unique_lock<std::mutex> lock(m_loop->m_mutex);
      |                                                                  ^
/usr/include/c++/13/bits/unique_lock.h:90:9: note: candidate: 'template<class _Clock, class _Duration> std::unique_lock<_Mutex>::unique_lock(mutex_type&, const std::chrono::time_point<_Clock, _Duration>&) [with _Duration = _Clock; _Mutex = std::mutex]'
   90 |         unique_lock(mutex_type& __m,
      |         ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:90:9: note:   template argument deduction/substitution failed:
/bitcoin/src/ipc/capnp/protocol.cpp:87:66: note:   candidate expects 2 arguments, 1 provided
   87 |                 std::unique_lock<std::mutex> lock(m_loop->m_mutex);
      |                                                                  ^
/usr/include/c++/13/bits/unique_lock.h:112:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(std::unique_lock<_Mutex>&&) [with _Mutex = std::mutex]'
  112 |       unique_lock(unique_lock&& __u) noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:112:33: note:   no known conversion for argument 1 from 'mp::Mutex' to 'std::unique_lock<std::mutex>&&'
  112 |       unique_lock(unique_lock&& __u) noexcept
      |                   ~~~~~~~~~~~~~~^~~
/usr/include/c++/13/bits/unique_lock.h:83:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&, std::adopt_lock_t) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   83 |       unique_lock(mutex_type& __m, adopt_lock_t) noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:83:7: note:   candidate expects 2 arguments, 1 provided
/usr/include/c++/13/bits/unique_lock.h:79:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&, std::try_to_lock_t) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   79 |       unique_lock(mutex_type& __m, try_to_lock_t)
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:79:7: note:   candidate expects 2 arguments, 1 provided
/usr/include/c++/13/bits/unique_lock.h:75:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock(mutex_type&, std::defer_lock_t) [with _Mutex = std::mutex; mutex_type = std::mutex]'
   75 |       unique_lock(mutex_type& __m, defer_lock_t) noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:75:7: note:   candidate expects 2 arguments, 1 provided
/usr/include/c++/13/bits/unique_lock.h:64:7: note: candidate: 'std::unique_lock<_Mutex>::unique_lock() [with _Mutex = std::mutex]'
   64 |       unique_lock() noexcept
      |       ^~~~~~~~~~~
/usr/include/c++/13/bits/unique_lock.h:64:7: note:   candidate expects 0 arguments, 1 provided
/bitcoin/src/ipc/capnp/protocol.cpp:88:25: error: 'class mp::EventLoop' has no member named 'addClient'
   88 |                 m_loop->addClient(lock);
      |                         ^~~~~~~~~
gmake[2]: *** [src/ipc/CMakeFiles/bitcoin_ipc.dir/build.make:178: src/ipc/CMakeFiles/bitcoin_ipc.dir/capnp/protocol.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1208: src/ipc/CMakeFiles/bitcoin_ipc.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

Unless we backport a number of ipc related changes to this branch (unlikely), this will be broken for the rest of this branches life. This seems like something we might want to document.

cc @Sjors @ryanofsky

Maybe we should tag the last libmultiprocess commit that is expected to work with 29.x and recommend that in either the libmultiprocess README or backported documentation here?

Unless we backport a number of ipc related changes to this branch (unlikely), this will be broken for the rest of this branches lief. This seems like something we might want to document.

This does seem like something good to document. Maybe the 29.x dependencies.md file could document the version of multiprocess known to work with v29, consistent with the version used in depends https://github.com/bitcoin/bitcoin/blob/8bcb90d7e3cd80db3d24e29ed8c72a55fcba90bc/depends/packages/native_libmultiprocess.mk#L2.

Could also create branches and tags in the libmultiprocess repo to provide clearer versioning. Maybe v0 for the branch that works with bitcoin 29, v1 for the branch that works with bitcoin 30.0, etc. Libmultiprocess versions and bitcoin major versions would not necessarily increase in lockstep since libmultiprocess API should be be pretty stable and not need to increase with every new bitcoin release, and conversely there could also be libmultiprocess API changes that are not backwards compatible made between bitcoin minor versions.

This does seem like something good to document. Maybe the 29.x dependencies.md file could document the version of multiprocess known to work with v29, consistent with the version used in depends

Yeah, makes sense to document this. (Looks like it isn't documented at all right now). Just documenting the exact commit should be good enough and I think tagging isn't needed, but no strong opinion.

I think this is resolved with #33623. The dependencies document now mentions which versions are compatible https://github.com/bitcoin/bitcoin/blob/29.x/doc/dependencies.md#optional