Add windows support #231

pull ryanofsky wants to merge 22 commits into bitcoin-core:master from ryanofsky:pr/win changing 19 files +505 −151
  1. ryanofsky commented at 8:31 AM on October 22, 2025: collaborator

    This PR is based on #274

    Add support for running on windows. These changes make the libmultiprocess API more generic, using stream types instead of file descriptors. All features are supported, including spawning processes with socket connections to the parent process. These changes were originally made in https://github.com/bitcoin/bitcoin/pull/32387

  2. DrahtBot commented at 8:31 AM on October 22, 2025: none

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #269 (proxy: add local connection limit to ListenConnections by enirox001)
    • #218 (Better error and log messages by ryanofsky)
    • #209 (cmake: Increase cmake policy version by ryanofsky)
    • #175 (Set cmake_minimum_required(VERSION 3.22) by maflcko)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • nonunix -> non-Unix [“nonunix” is not standard English and may confuse readers.]

    <sup>2026-04-22 21:07:39</sup>

  3. DrahtBot added the label Needs rebase on Jan 13, 2026
  4. pavlenex referenced this in commit 9800329587 on Mar 24, 2026
  5. ryanofsky force-pushed on Apr 3, 2026
  6. ryanofsky commented at 9:43 PM on April 3, 2026: collaborator

    <!-- begin push-2 -->

    Rebased 7d2f3c4371f8689260061b4b98f0f15dff9d95f7 -> 2975facdd1ddb3b52d052aaaac611566f3ff27ae (pr/win.1 -> pr/win.2, compare)<!-- end --> due to conflicts with #237

  7. DrahtBot removed the label Needs rebase on Apr 3, 2026
  8. pavlenex commented at 8:58 AM on April 8, 2026: none

    We have miners using Stratum V2 that are on Windows and would like to see Windows IPC support landing, is there any estimate when they can expect this to materialize? We have a dozen of users who can help with testing if that's a blocker.

  9. Sjors commented at 10:20 AM on April 8, 2026: member

    @pavlenex testing would definitely be useful. I guess for that to work we'd need a stack of pull requests: this one here in libmultiprocess, one in Bitcoin Core that enables IPC support in the Windows Guix build, and then an SRI pull request that uses it. Testers would then have download (or build themselves) the custom bitcoin core and SRI binaries.

    I've added Windows support to my v32 wish list: https://github.com/bitcoin/bitcoin/issues/33777

    No guarantees obviously.

  10. ryanofsky commented at 1:18 PM on April 8, 2026: collaborator

    I'll try to get this PR ready for review this week, split up into smaller commits and with ci passing. From there as Sjors mentioned there is a lot more work to do: more code changes in https://github.com/bitcoin/bitcoin/pull/32387 that need to be made in bitcoin core, enabling IPC into windows builds in bitcoin core, enabling it in windows CI jobs with pycapnp, adding client support, probably adding a windows CI job to this repository.

    v32 sounds like a good target though and it is very useful to know there is demand for this feature, because it hasn't been a priority so far

  11. doc: Bump version 11 > 12 b15d63e9d8
  12. util, refactor: Add ProcessId type alias and use it
    Add ProcessId = int type alias and apply it to WaitProcess, SpawnProcess
    (pid output argument), and callers.
    36c91a0c73
  13. util, refactor: Add SocketId type alias and use it
    Add SocketId = int and SocketError = -1 type aliases and apply SocketId
    to SpawnProcess (return type and callback parameter) and callers.
    94af41bb55
  14. util, refactor: Add ConnectInfo type alias and use it
    Add ConnectInfo type alias to pass socket handle from parent process to
    child process in more platform independent way.
    beaa50a046
  15. ryanofsky closed this on Apr 15, 2026

  16. ryanofsky force-pushed on Apr 15, 2026
  17. ryanofsky reopened this on Apr 15, 2026

  18. ryanofsky commented at 4:42 PM on April 15, 2026: collaborator

    PR is split up into commits now and should be reviewable. CI is not passing but failures look like IWYU errors. I also opened https://github.com/bitcoin/bitcoin/pull/35084 with corresponding bitcoin core changes. Windows support for bitcoin core can be tested with https://github.com/bitcoin/bitcoin/pull/32387 which combines both PRs and enables IPC by default in windows builds.

    <!-- begin push-3 -->

    Rebased 2975facdd1ddb3b52d052aaaac611566f3ff27ae -> cb16d2e37ece4ce9326f03624f7f195ae0e55462 (pr/win.2 -> pr/win.3, compare)<!-- end --> splitting changes up into more reviewable commits

    <!-- begin push-4 -->

    Updated cb16d2e37ece4ce9326f03624f7f195ae0e55462 -> e563c968b4e13ac54aaec474e2a6bd98cd0c1c36 (pr/win.3 -> pr/win.4, compare)<!-- end --> to fix various ci failures #231 (comment): macos/freebsd shutdownwrite fail, bitcoin core ci jobs api incompatibility, iwyu and olddeps fixes

    <!-- begin push-5 -->

    Updated e563c968b4e13ac54aaec474e2a6bd98cd0c1c36 -> d9fcac6e30d0b9ea2cf24814bb014be70e6a2ade (pr/win.4 -> pr/win.5, compare)<!-- end --> to fix IWYU errors https://github.com/bitcoin-core/libmultiprocess/actions/runs/24539430990/job/71741856575?pr=231

    <!-- begin push-6 -->

    Added 1 commits d9fcac6e30d0b9ea2cf24814bb014be70e6a2ade -> a1748e2fae3abf7ba41ffa78db82eae4761f6723 (pr/win.5 -> pr/win.6, compare)<!-- end --> to fix bitcoin core macos exception type error https://github.com/bitcoin-core/libmultiprocess/actions/runs/24541950788/job/71749481354?pr=231

    <!-- begin push-7 -->

    Updated a1748e2fae3abf7ba41ffa78db82eae4761f6723 -> 18fc18846e38dfec4b92a36b7a20d4f9be74a874 (pr/win.6 -> pr/win.7, compare)<!-- end --> with updates from https://github.com/bitcoin/bitcoin/pull/32387 pr/ipc-win.23

    <!-- begin push-8 -->

    Updated 18fc18846e38dfec4b92a36b7a20d4f9be74a874 -> 7fd5ec40bc8c2a1fa0e2645d2b587ce2c1c3d17d (pr/win.7 -> pr/win.8, compare)<!-- end --> adding workaround for ubuntu packaging bug exposed by cmake change https://github.com/bitcoin-core/libmultiprocess/actions/runs/24746487506/job/72399345861, also fixing olddeps include and iwyu errors https://github.com/bitcoin-core/libmultiprocess/actions/runs/24746487510, also fixing more MSVC errors, and rearranging commits

  19. ryanofsky force-pushed on Apr 16, 2026
  20. ryanofsky force-pushed on Apr 17, 2026
  21. util, refactor: Handle forking inside ExecProcess
    gen.cpp used fork() directly via <unistd.h> to invoke the capnp compiler as a
    subprocess, but fork() is not available on Windows, so shouldn't be used in
    application code.
    
    Add an ExecProcess(const std::vector<std::string>& args) function to
    util.h/util.cpp that spawns a process and returns its ProcessId, leaving
    the caller responsible for WaitProcess. On POSIX it uses fork() (via
    KJ_SYSCALL) + execvp; on Windows it can use CreateProcess.
    
    Update gen.cpp to replace the inline fork/exec/wait with
    mp::WaitProcess(mp::ExecProcess(args)).
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    b16f8c4b47
  22. util, refactor: Add SocketPair() and use it in SpawnProcess
    Extract socket pair creation from SpawnProcess into a standalone
    SocketPair() function, and use it to replace the inline socketpair()
    call. No behavior change.
    022b29b776
  23. util: Clear FD_CLOEXEC on child socket before exec
    Explicitly clear FD_CLOEXEC on the child's socket before calling exec,
    so the fd survives into the spawned process regardless of how the socket
    was created. Previously this relied on socketpair() not setting
    FD_CLOEXEC by default, which is not guaranteed if the caller creates
    sockets with SOCK_CLOEXEC or if the flag gets set by other means.
    24c5e57fdd
  24. proxy, refactor: Replace EventLoop wakeup fd integers with KJ stream objects
    Replace the m_wait_fd/m_post_fd raw int members with
    m_wait_stream/m_post_stream kj::Own<kj::AsyncIoStream> and
    m_post_writer kj::Own<kj::OutputStream>.
    
    The constructor uses provider->newTwoWayPipe() instead of calling
    socketpair() directly. The loop() and post() methods write through
    m_post_writer instead of calling write() with a raw fd, and
    EventLoopRef::reset does the same.
    3c81cf27ea
  25. cmake: Bump minimum required Cap'n Proto version to 0.9
    kj::AsyncIoStream::getFd() was added in capnproto 0.9 (commit
    d27bfb8a4175b32b783de68d93dd1dbafadddea5, first released in 0.9.0). The
    code now uses getFd() in proxy.cpp, so 0.7 is no longer a sufficient
    minimum.
    
    Set olddeps version to 0.9.2, which is the patched 0.9.x release for
    CVE-2022-46149.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    17a1952eb5
  26. proxy, refactor: Change ConnectStream and ServeStream to accept stream objects
    Instead of accepting raw file descriptor integers and wrapping them
    internally, ConnectStream and ServeStream now accept
    kj::Own<kj::AsyncIoStream> directly. This removes the assumption that
    the transport is always a local unix fd, making the API easier to adapt
    to other I/O types (e.g. Windows handles).
    
    The Stream type alias (kj::Own<kj::AsyncIoStream>) is added as a
    convenience, along with StreamSocketId() to extract the underlying fd
    from a Stream when needed.
    
    Callers are updated to wrap their fd with wrapSocketFd() before calling.
    091f5e16dc
  27. proxy: Call shutdownWrite() in Connection destructor
    Flush pending Cap'n Proto release messages before closing the stream.
    When one side of a socket pair closes, the other side does not receive
    an onDisconnect event, so it relies on receiving release messages from
    the closing side to free its ProxyServer objects and shut down cleanly.
    Without this, Server objects are not freed by Cap'n Proto on
    disconnection.
    bfc2db7b51
  28. util, refactor: Fix PtrOrValue constructor for move-only types on MSVC
    MSVC error when building multiprocess.vcxproj:
    
      mp/util.h(146,46): error C2280:
        'std::variant<T *,T>::variant(const std::variant<T *,T> &)':
        attempting to reference a deleted function [with T=mp::Lock]
    
    The PtrOrValue constructor used a ternary expression to initialize data:
    
      data(ptr ? ptr : std::variant<T*, T>{std::in_place_type<T>, args...})
    
    Both arms are prvalues of type std::variant<T*,T>, so under C++17's
    mandatory copy elision no copy/move constructor should be invoked. GCC
    and Clang apply this correctly. MSVC does not apply guaranteed copy
    elision to ternary expressions in this context: it materializes the
    temporary and then attempts to copy-construct data from it. Since
    std::variant<Lock*,Lock> has a deleted copy constructor (Lock holds a
    std::unique_lock which is move-only), MSVC fails.
    
    Fix by initializing data to hold T*=ptr in the member initializer list,
    then emplacing T in-place in the constructor body if ptr is null. This
    avoids the ternary entirely and requires only the in-place constructor
    of T, not any variant copy or move.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    1060a95de2
  29. Sjors commented at 1:57 PM on April 20, 2026: member

    How realistic is it to add a Windows CI job here? (can be cross-compiled)

  30. Sjors referenced this in commit d42af54c56 on Apr 20, 2026
  31. Sjors referenced this in commit 7de24f776c on Apr 20, 2026
  32. Sjors referenced this in commit 2f549eb17d on Apr 20, 2026
  33. ryanofsky commented at 5:21 PM on April 21, 2026: collaborator

    How realistic is it to add a Windows CI job here? (can be cross-compiled)

    I think this does need a windows CI job to exist in order to be merged, otherwise windows support is very likely to break with future changes, so thanks for opening #272.

    I think it may also make sense to split this PR up to separate the commits which are needed to support windows but don't actually add any windows code, from the one commit which actually does add windows code.

  34. Sjors commented at 7:00 PM on April 21, 2026: member

    it may also make sense to split this PR up to separate the commits which are needed to support windows but don't actually add any windows code

    Yes it would be good to land those changes to keep this PR focussed.

  35. ryanofsky force-pushed on Apr 21, 2026
  36. proxy, refactor: Fix C4305 truncation warning in Accessor on MSVC
    MSVC warns (C4305, treated as error) about truncation from 'int' to
    'const bool' when initializing static const bool members from integer
    bitwise-and expressions. Use constexpr bool with explicit != 0 to
    make the boolean conversion unambiguous.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    362d416844
  37. type-interface, refactor: Fix typename decltype() SFINAE in CustomBuildField on MSVC
    MSVC cannot parse 'typename decltype(expr)::Member' syntax and fails
    with a hard error (C2039, C2146) instead of a SFINAE substitution
    failure. Use Decay<> wrapper to provide the extra template indirection
    that MSVC needs, consistent with the unique_ptr and shared_ptr overloads
    of CustomBuildField which already use this pattern.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    3fd227ce24
  38. ci: Check out bitcoin/bitcoin PR #35084 instead of master
    This repo has introduced API changes to add Windows support to
    libmultiprocess (HANDLE-based IPC alongside the existing fd-based IPC).
    These changes require corresponding updates to Bitcoin Core, which are
    pending in bitcoin/bitcoin#35084. Until that PR merges, the Bitcoin Core
    CI jobs fail against master because Bitcoin Core has not yet been updated
    to use the new API.
    
    Switch the Bitcoin Core checkout in both jobs to use
    refs/pull/35084/merge so CI tests against the compatible version. A
    BITCOIN_CORE_REF env var is introduced at the top of the file; once
    (and keep the var in place for any future API compatibility cycles).
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    926ae3562e
  39. proxy: Fix shutdownWrite() exception handling on macOS with dynamic libraries
    On macOS, when libcapnp is built as a dynamic library and Bitcoin Core
    REDUCE_EXPORT option is used the RTTI typeinfo for kj::Exception has a
    different address in libcapnp.dylib versus the calling binary. This
    means catch (const kj::Exception& e) in the calling binary silently
    fails to match exceptions thrown by capnp, so the DISCONNECTED exception
    from shutdownWrite() propagates as a fatal uncaught exception instead of
    being suppressed as intended.
    
    This causes the Bitcoin Core macOS native CI job to fail with:
      Fatal uncaught kj::Exception: kj/async-io-unix.c++:491: disconnected:
        shutdown(fd, SHUT_WR): Socket is not connected
    
    The fix is to use kj::runCatchingExceptions/kj::throwRecoverableException,
    which use KJ's own thread-level exception interception mechanism rather
    than C++ RTTI-based matching, and therefore work correctly across dynamic
    library boundaries. This is the same approach used elsewhere in the
    codebase (proxy.cpp EventLoop::post, type-context.h server request handler)
    for the same reason.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    28e4c7fd2e
  40. ipc: Wrap mpgen main() in try-catch to print errors
    On MSVC, std::terminate() does not print the exception message before
    calling abort()/fastfail, so exceptions thrown during mpgen execution
    appear as a bare 0xC0000409 exit code with no diagnostic output. Wrap
    main() in a try-catch to explicitly print the error to stderr and
    return 1 instead of crashing.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    f6aa627aa4
  41. doc: Remove trailing whitespace
    Bitcoin Core linter rejects it:
    https://github.com/bitcoin/bitcoin/actions/runs/24568789956/job/71835997334?pr=32387
    7f513a47dc
  42. cmake: Replace capnp_PREFIX path construction with cmake-provided symbols
    Use target_compile_definitions on mpgen to expose CAPNP_EXECUTABLE,
    CAPNPC_CXX_EXECUTABLE (via $<TARGET_FILE:...> generator expressions on
    the CapnProto::capnp_tool and CapnProto::capnpc_cpp imported targets),
    and CAPNP_INCLUDE_DIRS (from the CAPNP_INCLUDE_DIRS variable set by
    find_package). gen.cpp uses these directly instead of constructing paths
    from capnp_PREFIX. Remove capnp_PREFIX from config.h.in as it is no
    longer needed there. Add compat fallbacks in compat_config.cmake to
    synthesize the tool imported targets and CAPNP_INCLUDE_DIRS from older
    variables when using an older CapnProto package.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    c9aa8060ec
  43. cmake: Fix CapnProto tool paths broken by Ubuntu Noble packaging bug
    Ubuntu Noble's libcapnp-dev 1.0.1 cmake config file is installed under
    /usr/lib/x86_64-linux-gnu/cmake/CapnProto/ but its _IMPORT_PREFIX
    calculation goes up only 3 directory levels to /usr/lib instead of 4
    levels to /usr, so IMPORTED_LOCATION for CapnProto::capnp_tool is set
    to /usr/lib/bin/capnp (non-existent) rather than /usr/bin/capnp.
    
    The previous compat_config.cmake fallback only fired when the target
    didn't exist at all (NOT TARGET), so it didn't catch this case where
    the target exists but has a wrong path.
    
    Add a validation pass that iterates over both tool targets after they
    are created (either by the package or by our own fallback). For each
    target, check whether any IMPORTED_LOCATION (config-specific or
    generic) resolves to an existing file. If none do, use find_program
    (with capnp_PREFIX/bin as a hint) to locate the actual binary and
    override all stored locations on that target.
    
    Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
    7cb83a5d53
  44. util: Add Windows support
    Add Windows-specific code to support building and running on Windows:
    
    - util.h: Guard ProcessId/SocketId/SocketError type aliases with WIN32
      ifdefs so they use SOCKET/uintptr_t on Windows and int on Unix.
      Add winsock2.h include on Windows.
    - util.cpp: Guard Unix-specific system headers with WIN32 ifdefs. Add
      Windows-specific includes (windows.h, winsock2.h). Guard MaxFd() with
      #ifndef WIN32. Add GetCurrentThreadId() branch in ThreadName(). Add
      win32Socketpair() forward-declare. Add Windows branch in SocketPair()
      using win32Socketpair(). Add CommandLineFromArgv() helper needed to
      construct CreateProcess command lines. Add Windows branch in
      SpawnProcess() using named pipes and WSADuplicateSocket to pass socket
      to child. Add Windows branch in StartSpawned() reading socket from
      named pipe. Add Windows branch in WaitProcess() using
      WaitForSingleObject/GetExitCodeProcess.
    - proxy-io.h: Add Windows branch in StreamSocketId() using
      getWin32Handle().
    - proxy.cpp: Add SocketOutputStream class on Windows (analogous to
      FdOutputStream but using SOCKET/send()). Add Windows branch in
      EventLoop constructor to create m_post_writer using SocketOutputStream.
    4f58c8c981
  45. util: drop POSIX/pthread dependencies to enable MSVC builds
    Remove POSIX and pthread calls from util.cpp to avoid relying on MinGW's POSIX
    compatibility layer. This lets code be compiled with MSVC.
    7fd5ec40bc
  46. ryanofsky force-pushed on Apr 22, 2026
  47. Sjors referenced this in commit 8db8bf06ee on Apr 23, 2026
  48. Sjors referenced this in commit baf5d5e571 on Apr 23, 2026
  49. Sjors referenced this in commit 36b415c486 on Apr 23, 2026
  50. Sjors referenced this in commit a20bfd86d2 on Apr 23, 2026

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin-core/libmultiprocess. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-11 12:30 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me