ipc: add windows support #32387

commit e1dcc6eb1825c78772bf0ccf7eaee27c476c3dac

Remove recently introduced SpawnConnectInfo and SpawnConnectInfoToArgsFn type
aliases since they are the same on all platforms and might obscure the fact
that connect info should be treated as an opaque string.

Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>

Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>

Add Windows-specific code to support building and running on Windows:

- util.h: Guard ProcessId/SocketId/SocketError type aliases with WIN32
  ifdefs so they use SOCKET/uintptr_t on Windows and int on Unix.
  Add winsock2.h include on Windows.
- util.cpp: Guard Unix-specific system headers with WIN32 ifdefs. Add
  Windows-specific includes (windows.h, winsock2.h). Guard MaxFd() with
  #ifndef WIN32. Add GetCurrentThreadId() branch in ThreadName(). Add
  win32Socketpair() forward-declare. Add Windows branch in SocketPair()
  using win32Socketpair(). Add CommandLineFromArgv() helper needed to
  construct CreateProcess command lines. Add Windows branch in
  SpawnProcess() using named pipes and WSADuplicateSocket to pass socket
  to child. Add Windows branch in StartSpawned() reading socket from
  named pipe. Add Windows branch in WaitProcess() using
  WaitForSingleObject/GetExitCodeProcess.
- proxy.cpp: Add SocketOutputStream class on Windows (analogous to
  FdOutputStream but using SOCKET/send()). Add Windows branch in
  EventLoop constructor to create m_post_writer using SocketOutputStream.

Allow code to compile without pthreads available, as required for MSVC
compatibility. Avoid unconditional POSIX calls (fork, posix_spawn,
pthread_getname_np) by moving them into #ifndef WIN32 or HAVE_PTHREAD_*
guards. When pthreads is available on Windows (detected via cmake
HAVE_PTHREAD_* checks), still use it for thread name reporting since it
provides useful information at low cost.

Also add Threads::Threads as an explicit dependency of the multiprocess
library. proxy.cpp directly uses thread_local, std::this_thread, and
std::thread, and the dependency was previously satisfied only through
transitive linkage from CapnProto::kj-async.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Replace POSIX-only headers (sys/socket.h, sys/un.h, unistd.h) with
Windows equivalents (afunix.h via util.h), guard them with #ifdef WIN32,
use TCP sockets instead of Unix sockets for Wine compatibility, replace
mkdtemp/unlink/rmdir with std::filesystem equivalents, and use
SocketId/SocketError types instead of int/-1 for socket handles so the
file compiles and works with MinGW.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

listen_tests.cpp tests fail on Windows with:
  "expected m_fd != SocketError [18446744073709551615 != 18446744073709551615]"

socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) returns INVALID_SOCKET because
WSAStartup() has not been called. The mp library calls WSAStartup() only inside
ConnectSocketToProcess(), which listen_tests.cpp never reaches — it creates
sockets directly using the BSD API.

Fix: add a static initializer that calls WSAStartup(MAKEWORD(2,2), ...) at
program startup before any test runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- shell.nix: add `windows` parameter that selects pkgs.pkgsCross.mingwW64
  as the cross target; also change crossPkgs default from import<nixpkgs>{}
  to null (cleaner API). When windows=true, add native pkgs.capnproto to
  nativeBuildInputs so capnp/capnpc-c++ are in PATH for cmake code generation,
  and add wine64Packages.staging so ctest can run mptest.exe via wine.
  Change llvmBase to always use pkgs (native) instead of crossPkgs.

- ci/configs/windows.bash: new config that cross-compiles with mingw,
  sets CMAKE_SYSTEM_NAME=Windows, CMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER
  (so cmake finds native capnp from PATH), CMAKE_CROSSCOMPILING_EMULATOR=wine
  (so ctest runs mptest.exe via wine), and sets MPGEN_PRE_BUILD=1.

- ci/scripts/ci.sh: add MPGEN_PRE_BUILD support: when set, build native mpgen
  in $CI_DIR-native before the main cross build, then inject
  -DMPGEN_EXECUTABLE into CMAKE_ARGS. This is needed because cmake's
  add_custom_command does not use CMAKE_CROSSCOMPILING_EMULATOR, so the
  cross-compiled mpgen.exe cannot be used as a code generator directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Error seen (MSVC native build):
  mpgen error: mp\util.cpp:332: failed: CreateProcessA(...): #2 The system
  cannot find the file specified.

Problem: mpgen constructs capnp tool paths as capnp_PREFIX + "/bin/capnp"
and capnp_PREFIX + "/bin/capnpc-c++". capnp_PREFIX is derived from the
capnproto include directory parent, so on a vcpkg install it resolves to
something like vcpkg_installed/x64-windows-release. But vcpkg installs
executables under tools/capnproto/, not bin/, so the constructed path
does not exist and CreateProcessA fails with ERROR_FILE_NOT_FOUND.

Fix: pass the CAPNP_EXECUTABLE and CAPNPC_CXX_EXECUTABLE cmake variables
from CapnProtoConfig.cmake as compile definitions to mpgen, and use them
in gen.cpp instead of constructing paths from capnp_PREFIX.

Using these cmake variables rather than reading the CapnProto::capnp_tool
imported target IMPORTED_LOCATION directly is important: on Debian/Ubuntu,
IMPORTED_LOCATION is set to a non-existent path due to a multiarch cmake
packaging bug in libcapnp-dev (see
https://github.com/bitcoin-core/libmultiprocess/issues/328). The cmake
variables avoid this because CapnProtoConfig.cmake hardcodes the correct
paths on Debian/Ubuntu, and on other platforms (including vcpkg) the
variables hold generator expressions that target_compile_definitions
evaluates correctly at generation time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

MinGW has a bug where thread_local destructors can run on already-freed memory
at thread exit, causing heap corruption. MSVC builds and non-Windows platforms
are unaffected.

This caused intermittent STATUS_HEAP_CORRUPTION in MinGW CI builds when
~ThreadContext walked freed memory at thread exit. See CurrentThread() in
proxy-io.h for the full explanation and upstream bug references.

Workaround: change CurrentThread() accessor in MinGW builds to return a
heap-allocated object held by a trivially-destructible pointer, skipping
destructor registration at thread exit. The object is deliberately
leaked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Various small fixes for making ENABLE_IPC build work on windows.

MSVC does not provide mkstemp (a POSIX function available in MinGW but
not the MSVC runtime), causing:

  error C3861: 'mkstemp': identifier not found
  error C4996: 'close': The POSIX name for this item is deprecated

Fix by using _mktemp_s on WIN32, which fills in the XXXXXX suffix
in-place without creating a file (so no fd to close or file to remove).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

On Windows, connect() to a Unix domain socket path that doesn't exist
may return a WSA error code (e.g. WSAECONNREFUSED) that doesn't map to
std::errc::no_such_file_or_directory via std::system_category(), unlike
POSIX where ENOENT is returned. After a failed connect, explicitly check
if the socket path exists and throw the expected error if not.

Also add an fs::exists(path, ec) overload to fs.h: the file's existing
exists() wrappers shadow std::filesystem::exists(path, error_code&),
causing the new call in process.cpp to fail to compile under MinGW.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

When the IPC socket file exists but no node is listening, ProcessImpl::connect
throws std::system_error(WSAECONNREFUSED, std::system_category()). MinGW
libstdc++'s system_category() does not map Winsock error values to std::errc
conditions, so the std::errc::connection_refused check in connectAddress fails
to catch it, and bitcoin-cli -ipcconnect=auto throws CConnectionFailed instead
of falling back to HTTP.

This is a workaround: the proper fix is for ProcessImpl::connect to throw
std::errc::connection_refused (using generic_category) rather than relying on
system_category() to map Winsock codes. As a workaround, check for
WSAECONNREFUSED explicitly in system_category on WIN32.

MSVC's STL does map WSAECONNREFUSED to errc::connection_refused, so this
check is redundant but harmless there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Bug: MSVC C2131 "expression did not evaluate to a constant" on line:
  static_assert(ipc::capnp::messages::MAX_DOUBLE == std::numeric_limits<double>::max())

Root cause: Capnp floating-point constants can't be constexpr on MSVC.
In capnp's generated-header-support.h, CAPNP_NON_INT_CONSTEXPR_DECL_INIT
is defined as empty on MSVC (with the comment "MSVC doesn't understand
floating-point constexpr yet"), so MAX_DOUBLE is declared as
`static const double MAX_DOUBLE;` without an initializer in the header.
A `const double` (unlike `const int`) is not a constant expression in C++,
so it can't be used in static_assert on MSVC.

Fix: Guard with `#if !defined(_MSC_VER) || defined(__clang__)` to match
the exact condition capnp itself uses for the same limitation. The other
three static_asserts in the same block use integer constants (MAX_MONEY,
DEFAULT_BLOCK_RESERVED_WEIGHT, DEFAULT_COINBASE_OUTPUT_MAX_ADDITIONAL_SIGOPS)
which don't have this MSVC limitation and continue to run.

Alternatives considered:
- Convert to a runtime BOOST_CHECK: would require moving into a test
  function, more invasive change.
- Use std::numeric_limits<double>::max() literal: wouldn't help since
  MAX_DOUBLE is the non-constexpr side.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

On Windows, _wexecvp (like all _exec* variants in the MSVC CRT) works by
calling CreateProcess to start the child, then immediately calling
ExitProcess(0) on the parent. Unlike POSIX execvp, which replaces the
current process image so the parent's PID becomes the child, this means
the child is orphaned from the original caller's perspective: the shell
or test framework waiting on bitcoin.exe sees it exit with code 0 while
bitcoind.exe continues running in the background independently.

This breaks functional tests that invoke bitcoin.exe as a wrapper (e.g.
`bitcoin node -regtest`) because the test process has no way to wait for
the node to stop or retrieve its exit code.

Fix by switching to _wspawnvp(_P_WAIT, ...), which spawns the child and
blocks until it exits, then calling _exit() with the child's exit code.
From the caller's perspective ExecVp still never returns on success (the
parent process exits inside it), so call sites don't need to change. On
error the existing errno / -1 return convention is preserved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

_wspawnvp (the Windows implementation of ExecVp) requires the full filename
including ".exe" when given an explicit path. Unlike POSIX execvp, it does
not add the extension automatically.

Also, msvcrt's _wspawnvp returns EINVAL (not ENOENT) when a file is not
found — both when ".exe" is absent and when the file simply doesn't exist.
ucrt returns ENOENT correctly. The allow_notfound check in try_exec only
caught ENOENT, so on msvcrt it never fell through to the next search path.

Without both fixes, all IPC functional tests failed immediately with:
  "execvp failed to execute '...\libexec\bitcoin-node': Invalid argument"

Fix: append ".exe" when the path has no extension, and catch both ENOENT
and EINVAL as "file not found" on WIN32.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Now that util::ExecVp on Windows uses _wspawnvp(_P_WAIT) instead of
_wexecvp, the bitcoin wrapper process blocks until the child exits and
Python can capture its stdout/stderr and exit code normally. Remove the
Windows skip added in #33229.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Python's asyncio doesn't support create_unix_connection() on Windows. Implement
it by patching asyncio.ProactorEventLoop, using ctypes ws2_32.connect() and
a non-blocking select() loop. See ipc_util.py for details on why each of those
choices was necessary.

Also: run blocking do_fn() callables in run_in_executor() in wait_and_do();
unlink stale AF_UNIX socket files before node restart on Windows (test_node.py);
set UNIX_PATH_MAX = 108 for Windows (test_node.py); catch OSError in the
connection retry loop in interface_ipc_mining.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

On POSIX, connect() to a stopped node returns ECONNREFUSED and bitcoin-cli
prints "error: Connection refused" followed by help text. On Windows the
equivalent WSAECONNREFUSED produces a different OS error string without help
text. Fix the expected string and switch to a prefix match since the Windows
message has no appended help text.

Also, on Windows, bitcoind removes the IPC socket file on clean shutdown
(unlike Linux where the file persists and connect() returns ECONNREFUSED).
A subsequent connect attempt sees ENOENT ("No such file or directory").
Update test_cli() to accept a tuple of alternative prefixes and use a
two-entry tuple for the Windows case covering both error strings.

Also add diagnostic logging retained for future debugging: ipc_socket_path
in both backslash and forward-slash forms with encoded byte length (to
diagnose MinGW path format issues), socket file existence before each
explicit-path test, and per-call debug logging of returncode and stdout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add a helper function that generates a target-specific resource file from a
template and links it along with an application manifest, replacing the static
per-binary *-res.rc files previously used by non-GUI executables and the
bitcoin-qt-res.rc previously used by GUI executables.

The new add_windows_exe_resources(target description [ICON_DIR dir]) function
in AddWindowsResources.cmake handles both GUI and non-GUI executables through a
single cmake/windows-res.rc.in template. When ICON_DIR is provided, the
function prepends icon declarations to the generated RC file and sets
ProductName to the project name, matching the previous GUI-specific behavior.

Also adds the missing add_windows_exe_resources() call for bitcoin-node, which
previously had no resource section and caused the Windows CI manifest
validation check (mt.exe -validate_manifest) to fail.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Remove -DENABLE_IPC=OFF from both win64 cross-build environment files
and install pycapnp in the Windows cross-build test runner. The
depends/ system already builds capnp for the MinGW target (NO_IPC is
not set), so cmake will find it via the toolchain file. This lets
interface_ipc.py and related tests run in the "Windows, test
cross-built" CI jobs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add an "ipc" feature to vcpkg.json with capnproto as a dependency and
include it in default-features, so the Windows native VS CI job installs
capnproto when ENABLE_IPC is ON (the default). The fuzz build already
uses VCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON, so it is unaffected.

Also install pycapnp in ci-windows.py alongside pyzmq so the IPC
functional tests (interface_ipc.py) have their required Python library,
matching what every other CI platform already does.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The fuzz build in ci-windows.py set -DVCPKG_MANIFEST_FEATURES=wallet but
omitted ipc and tests, so vcpkg installed wallet dependencies but not
capnproto. The build then failed with:
  "Cap'n Proto is required but was not found"

Fix: extend VCPKG_MANIFEST_FEATURES to ipc;tests;wallet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

WARNING: DO NOT MERGE — this draft PR modifies the libmultiprocess
subtree directly (intentionally, for testing purposes) which causes
the subtree link check to fail. Disabling the check here to avoid
masking unrelated lint failures while the PR is open for testing.

Restore the omitted entry before any merge attempt.

Note: removing src/ipc/libmultiprocess from get_subtrees() also removes
it from get_pathspecs_default_excludes(), so all other lint checks
(trailing newline, whitespace, etc.) now run on files in that directory
as well. This is why src/ipc/libmultiprocess/include/mp/type-unordered-set.h
needs a trailing newline added here even though it is not otherwise
modified by this PR: it has always been missing the newline, but was
previously hidden from the lint check by the subtree exclusion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

This reverts commit 789ba90418822714c8f9c09e7daab188c652f3e3.