Error runing ./configure and make with --enable-multiprocess or MULTIPROCESS=1 #25207

Hi,

I am getting an error when building bitcoin-master

during make, it fails with:

CXX ipc/capnp/libbitcoin_ipc_a-protocol.o ipc/capnp/protocol.cpp:7:10: fatal error: ipc/capnp/init.capnp.h: No such file or directory 7 | #include <ipc/capnp/init.capnp.h> | ^~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.

I pulled git clone from the source repository here on github. My make procedure for capnproto was as per the instrucions in capnproto/c++

Architecture x86_64 Debian 11 + Kali Linux 2022.1 GCC-11 and GCC-12 system compilers Everything is largely up to date and most dependencies should be in place.

I also had issues trying to follow bitcoin recommendations of running MULTIPROCESS=1 (with capnproto and libmultitool still on my system, not yet uninstalled) as per docs here:

https://github.com/bitcoin/bitcoin/blob/ac219dcbcc18a48210d58cdfaf7cf27ca5e7435b/doc/multiprocess.md

Please advise as this is outside my field of knowledge as per why this file would be missing when doing make.

I had a lot of difficulty with libmultiprocess not uilding with the debian package on capnproto, and debian needs to update capnproto as well, on a side note, to fix atomic and libatomic being requested in CapnProtoConfig.cmake by default. Thankfully the master tree here had not issue, but being on an x86_64 system, it was troublesome having the debian package request atomic, I tried installing libatomic as a dependency. Master tree here on github installed both capnproto and libmultitool with no issue.

But something doesn't want to add up for make on the bitcoin master.

I will note, bitcoin was configured with:

CONFIG_SITE=

{BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include" --enable-wallet --enable-usdt --with-qrencode --with-zmq --with-mpgen=yes --with-sqlite3 --with-gui --with-miniupnpc --with-natpmp --enable-hardening

after

cd <BITCOIN_SOURCE_DIRECTORY> make -C depends NO_QT=1 MULTIPROCESS=1

I also reproduced the same error configuring with:

./configure BDB_LIBS="-L

{BDB_PREFIX}/include" --enable-wallet --enable-usdt --with-qrencode --with-zmq --with-mpgen=yes --with-sqlite3 --with-gui --with-miniupnpc --with-natpmp --enable-hardening --enable-multiprocess

and previously I had the same issue

I have had no issue building with all the other flags without multiprocess, I will mention, this was a reinstall to test multiprocess feature.

Also, I have libcapnp-dev debian package installed via apt repository

If you're using CONFIG_SITE, it should be pointing to the config.site file in the depends prefix that is generated after make -C depends. i.e CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site In that case, passing any BDB_* vars are also irrelevant.

Yes I am doing just that and getting the error:

make[2]: *** [/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/include/mpgen.mk:4: ipc/capnp/init.capnp.c++] Error 127 ipc/capnp/protocol.cpp:7:10: fatal error: ipc/capnp/init.capnp.h: No such file or directory 7 | #include <ipc/capnp/init.capnp.h> | ^~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.

And you are correct on the directory location for config.site as well my command is:

CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure --enable-wallet --enable-usdt --with-qrencode --with-zmq --with-mpgen=yes --with-sqlite3 --with-gui --with-miniupnpc --with-natpmp --enable-hardening

sudo make -j4

I believe that there is an issue with capnp (i.e. libcapnp-dev or capnproto). It might be located in libmultitool. Both were built directly from the current github source repositories with git clone (less two days ago)

Again, on x86_64 Debian 11 + Kali 2022.1

I will print what configure reads as well in case there is something to spot there in terms of dependency if it helps to locate:

CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure --enable-wallet --enable-usdt --with-qrencode --with-zmq --with-mpgen=yes --with-sqlite3 --with-gui --with-miniupnpc --with-natpmp --enable-hardening
configure: loading site script /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/share/config.site checking for x86_64-pc-linux-gnu-pkg-config... /usr/bin/pkg-config --static checking pkg-config is at least version 0.9.0... yes checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for x86_64-pc-linux-gnu-strip... strip checking for a race-free mkdir -p... /usr/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether to enable maintainer-specific portions of Makefiles... yes checking whether make supports nested variables... (cached) yes checking whether the C++ compiler works... yes checking for C++ compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether the compiler supports GNU C++... yes checking whether g++ -m64 accepts -g... yes checking for g++ -m64 option to enable C++11 features... none needed checking whether make supports the include directive... yes (GNU style) checking dependency style of g++ -m64... gcc3 checking whether g++ -m64 supports C++17 features with -std=c++17... yes checking whether std::atomic can be used without link library... yes checking whether std::filesystem can be used without link library... yes checking for x86_64-pc-linux-gnu-g++... g++ -m64 -std=c++17 checking whether the compiler supports GNU Objective C++... no checking whether g++ -m64 -std=c++17 accepts -g... no checking dependency style of g++ -m64 -std=c++17... gcc3 checking how to print strings... printf checking for x86_64-pc-linux-gnu-gcc... gcc -m64 checking whether the compiler supports GNU C... yes checking whether gcc -m64 accepts -g... yes checking for gcc -m64 option to enable C11 features... none needed checking whether gcc -m64 understands -c and -o together... yes checking dependency style of gcc -m64... gcc3 checking for a sed that does not truncate output... /usr/bin/sed checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for fgrep... /usr/bin/grep -F checking for ld used by gcc -m64... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... nm checking the name lister (nm) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for x86_64-pc-linux-gnu-file... no checking for file... file checking for x86_64-pc-linux-gnu-objdump... no checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for x86_64-pc-linux-gnu-dlltool... no checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for x86_64-pc-linux-gnu-ar... ar checking for archiver @FILE support... @ checking for x86_64-pc-linux-gnu-strip... (cached) strip checking for x86_64-pc-linux-gnu-ranlib... ranlib checking command to parse nm output from gcc -m64 object... ok checking for sysroot... no checking for a working dd... /usr/bin/dd checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 checking for x86_64-pc-linux-gnu-mt... no checking for mt... mt checking if mt is a manifest tool... no checking for stdio.h... yes checking for stdlib.h... yes checking for string.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for strings.h... yes checking for sys/stat.h... yes checking for sys/types.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc -m64 supports -fno-rtti -fno-exceptions... no checking for gcc -m64 option to produce PIC... -fPIC -DPIC checking if gcc -m64 PIC flag -fPIC -DPIC works... yes checking if gcc -m64 static flag -static works... yes checking if gcc -m64 supports -c -o file.o... yes checking if gcc -m64 supports -c -o file.o... (cached) yes checking whether the gcc -m64 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking how to run the C++ preprocessor... g++ -m64 -std=c++17 -E checking for ld used by g++ -m64 -std=c++17... /usr/bin/ld -m elf_x86_64 checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes checking whether the g++ -m64 -std=c++17 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking for g++ -m64 -std=c++17 option to produce PIC... -fPIC -DPIC checking if g++ -m64 -std=c++17 PIC flag -fPIC -DPIC works... yes checking if g++ -m64 -std=c++17 static flag -static works... yes checking if g++ -m64 -std=c++17 supports -c -o file.o... yes checking if g++ -m64 -std=c++17 supports -c -o file.o... (cached) yes checking whether the g++ -m64 -std=c++17 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... (cached) GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking for x86_64-pc-linux-gnu-ar... no checking for ar... (cached) ar checking for x86_64-pc-linux-gnu-gcov... no checking for gcov... /usr/bin/gcov checking for x86_64-pc-linux-gnu-llvm-cov... no checking for llvm-cov... no checking for lcov... no checking for python3.6... no checking for python3.7... no checking for python3.8... no checking for python3.9... /usr/bin/python3.9 checking for genhtml... no checking for git... /usr/bin/git checking for ccache... no checking for xgettext... /usr/bin/xgettext checking for hexdump... /usr/bin/hexdump checking for x86_64-pc-linux-gnu-objcopy... no checking for objcopy... /usr/bin/objcopy checking for doxygen... no checking whether C++ compiler accepts -Werror... yes checking whether the linker accepts -Wl,--fatal-warnings... yes checking whether C++ compiler accepts -fno-extended-identifiers... yes checking whether C++ compiler accepts -msse4.2... yes checking whether C++ compiler accepts -msse4.1... yes checking whether C++ compiler accepts -mavx -mavx2... yes checking whether C++ compiler accepts -msse4 -msha... yes checking whether C++ compiler accepts -mpclmul... yes checking for SSE4.2 intrinsics... yes checking for SSE4.1 intrinsics... yes checking for AVX2 intrinsics... yes checking for x86 SHA-NI intrinsics... yes checking whether C++ compiler accepts -march=armv8-a+crc+crypto... no checking whether C++ compiler accepts -march=armv8-a+crc+crypto... (cached) no checking for ARMv8 CRC32 intrinsics... no checking for ARMv8 SHA-NI intrinsics... no checking whether byte ordering is bigendian... no checking how to run the C preprocessor... gcc -m64 -E checking whether gcc -m64 is Clang... no checking whether pthreads work with "-pthread" and "-lpthread"... yes checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE checking whether more special flags are required for pthreads... no checking for PTHREAD_PRIO_INHERIT... yes checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... no checking for g++ -m64 -std=c++17 options needed to detect all undeclared functions... none needed checking whether strerror_r is declared... yes checking whether strerror_r returns char *... yes checking for library containing clock_gettime... none required checking whether C++ compiler accepts -fPIC... yes checking whether C++ compiler accepts -fstack-reuse=none... yes checking whether C++ compiler accepts -Wstack-protector... yes checking whether C++ compiler accepts -fstack-protector-all... yes checking whether the linker accepts -fcf-protection=full... yes checking whether C++ compiler accepts -fstack-clash-protection... yes checking whether C++ preprocessor accepts -D_FORTIFY_SOURCE=2... yes checking whether C++ preprocessor accepts -U_FORTIFY_SOURCE... yes checking whether the linker accepts -Wl,--enable-reloc-section... no checking whether the linker accepts -Wl,--dynamicbase... no checking whether the linker accepts -Wl,--nxcompat... no checking whether the linker accepts -Wl,--high-entropy-va... no checking whether the linker accepts -Wl,-z,relro... yes checking whether the linker accepts -Wl,-z,now... yes checking whether the linker accepts -Wl,-z,separate-code... yes checking whether the linker accepts -fPIE -pie... yes checking for endian.h... yes checking for sys/endian.h... no checking for byteswap.h... yes checking for stdio.h... (cached) yes checking for stdlib.h... (cached) yes checking for unistd.h... (cached) yes checking for strings.h... (cached) yes checking for sys/types.h... (cached) yes checking for sys/stat.h... (cached) yes checking for sys/select.h... yes checking for sys/prctl.h... yes checking for sys/sysctl.h... no checking for vm/vm_param.h... no checking for sys/vmmeter.h... no checking for sys/resources.h... no checking whether getifaddrs is declared... yes checking whether ifaddrs funcs can be used without link library... yes checking whether freeifaddrs is declared... yes checking whether ifaddrs funcs can be used without link library... yes checking whether fork is declared... yes checking whether setsid is declared... yes checking whether pipe2 is declared... yes checking whether le16toh is declared... yes checking whether le32toh is declared... yes checking whether le64toh is declared... yes checking whether htole16 is declared... yes checking whether htole32 is declared... yes checking whether htole64 is declared... yes checking whether be16toh is declared... yes checking whether be32toh is declared... yes checking whether be64toh is declared... yes checking whether htobe16 is declared... yes checking whether htobe32 is declared... yes checking whether htobe64 is declared... yes checking whether bswap_16 is declared... yes checking whether bswap_32 is declared... yes checking whether bswap_64 is declared... yes checking for __builtin_clzl... yes checking for __builtin_clzll... yes checking for getmemoryinfo... yes checking for mallopt M_ARENA_MAX... yes checking for posix_fallocate... yes checking for default visibility attribute... yes checking for dllexport attribute... no checking for thread_local support... yes checking for gmtime_r... yes checking for Linux getrandom syscall... yes checking for getentropy via random.h... yes checking for sysctl... no checking for sysctl KERN_ARND... no checking for if type char equals int8_t... no checking for fdatasync... yes checking for F_FULLFSYNC... no checking for O_CLOEXEC... yes checking for __builtin_prefetch... yes checking for _mm_prefetch... yes checking for strong getauxval support in the system headers... yes checking for std::system... yes checking for ::_wsystem... no checking for Qt5Core >= 5.11.3... yes checking for Qt5Gui >= 5.11.3... yes checking for Qt5Widgets >= 5.11.3... yes checking for Qt5Network >= 5.11.3... yes checking for Qt5Test >= 5.11.3... yes checking for Qt5DBus >= 5.11.3... yes checking for static Qt... yes checking for Qt5AccessibilitySupport... yes checking for Qt5DeviceDiscoverySupport... yes checking for Qt5EdidSupport... yes checking for Qt5EventDispatcherSupport... yes checking for Qt5FbSupport... yes checking for Qt5FontDatabaseSupport... yes checking for Qt5ThemeSupport... yes checking for Qt5InputSupport... yes checking for Qt5ServiceSupport... yes checking for Qt5XcbQpa... yes checking for Qt5XkbCommonSupport... yes checking for QMinimalIntegrationPlugin (-lqminimal)... yes checking for QXcbIntegrationPlugin (-lqxcb)... yes checking whether -fPIE can be used with this Qt config... yes checking for moc-qt5... no checking for moc5... no checking for moc... /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/native/bin/moc checking for uic-qt5... no checking for uic5... no checking for uic... /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/native/bin/uic checking for rcc-qt5... no checking for rcc5... no checking for rcc... /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/native/bin/rcc checking for lrelease-qt5... no checking for lrelease5... no checking for lrelease... /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/native/bin/lrelease checking for lupdate-qt5... no checking for lupdate5... no checking for lupdate... /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/native/bin/lupdate checking for lconvert-qt5... no checking for lconvert5... no checking for lconvert... /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/native/bin/lconvert checking whether to build Bitcoin Core GUI... yes (Qt5) checking whether main function is needed for fuzz binary... checking whether the linker accepts ... no yes checking for __builtin_mul_overflow... yes checking for Berkeley DB C++ headers... default checking for main in -ldb_cxx-4.8... yes checking for sqlite3 >= 3.7.17... yes checking whether to build wallet with support for sqlite... yes checking whether Userspace, Statically Defined Tracing tracepoints are supported... yes checking for miniupnpc/miniupnpc.h... yes checking for upnpDiscover in -lminiupnpc... yes checking for miniupnpc/upnpcommands.h... yes checking for upnpDiscover in -lminiupnpc... (cached) yes checking for miniupnpc/upnperrors.h... yes checking for upnpDiscover in -lminiupnpc... (cached) yes checking whether miniUPnPc API version is supported... yes checking for natpmp.h... yes checking for initnatpmp in -lnatpmp... yes checking for boostlib >= 1.64.0 (106400) includes in "/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/include"... yes checking for boostlib >= 1.64.0 (106400) lib path in "/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/lib/x86_64-linux-gnu"... no checking for boostlib >= 1.64.0 (106400) lib path in "/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/lib64"... no checking for boostlib >= 1.64.0 (106400) lib path in "/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/libx32"... no checking for boostlib >= 1.64.0 (106400) lib path in "/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/lib"... yes checking for boostlib >= 1.64.0 (106400)... yes checking whether Boost.Process can be used... yes checking for seccomp-bpf (Linux x86-64)... yes checking for libevent >= 2.1.8... yes checking for libevent_pthreads >= 2.1.8... yes checking if evhttp_connection_get_peer expects const char**... no checking for libqrencode... yes checking for libzmq >= 4... yes checking for libmultiprocess... yes checking whether to build bitcoind... yes checking whether to build bitcoin-cli... yes checking whether to build bitcoin-tx... yes checking whether to build bitcoin-wallet... yes checking whether to build bitcoin-util... yes checking whether to build experimental bitcoin-chainstate... no checking whether to build libraries... yes checking if ccache should be used... no checking if wallet should be enabled... yes checking whether to build with support for UPnP... yes checking whether to build with UPnP enabled by default... no checking whether to build with support for NAT-PMP... yes checking whether to build with NAT-PMP enabled by default... no checking whether to build GUI with support for D-Bus... yes checking whether to build GUI with support for QR codes... yes checking whether to build test_bitcoin-qt... yes checking whether to build test_bitcoin... yes checking whether to reduce exports... no checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating libbitcoinconsensus.pc config.status: creating Makefile config.status: creating src/Makefile config.status: creating doc/man/Makefile config.status: creating share/setup.nsi config.status: creating share/qt/Info.plist config.status: creating test/config.ini config.status: creating contrib/devtools/split-debug.sh config.status: creating src/config/bitcoin-config.h config.status: src/config/bitcoin-config.h is unchanged config.status: executing depfiles commands config.status: executing libtool commands === configuring in src/secp256k1 (/home/username/bitcoin-master/src/secp256k1) configure: running /bin/bash ./configure --disable-option-checking '--prefix=/usr/local' '--enable-wallet' '--enable-usdt' '--with-qrencode' '--with-zmq' '--with-mpgen=yes' '--with-sqlite3' '--with-gui' '--with-miniupnpc' '--with-natpmp' '--enable-hardening' '--disable-shared' '--with-pic' '--enable-benchmark=no' '--enable-module-recovery' '--enable-module-schnorrsig' --cache-file=/dev/null --srcdir=. configure: loading site script /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/share/config.site checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for x86_64-pc-linux-gnu-strip... strip checking for a race-free mkdir -p... /usr/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether make supports nested variables... (cached) yes checking for x86_64-pc-linux-gnu-gcc... gcc -m64 checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether the compiler supports GNU C... yes checking whether gcc -m64 accepts -g... yes checking for gcc -m64 option to enable C11 features... none needed checking whether gcc -m64 understands -c and -o together... yes checking whether make supports the include directive... yes (GNU style) checking dependency style of gcc -m64... gcc3 checking dependency style of gcc -m64... gcc3 checking for x86_64-pc-linux-gnu-ar... ar checking the archiver (ar) interface... ar checking how to print strings... printf checking for a sed that does not truncate output... /usr/bin/sed checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for fgrep... /usr/bin/grep -F checking for ld used by gcc -m64... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... nm checking the name lister (nm) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for x86_64-pc-linux-gnu-file... no checking for file... file checking for x86_64-pc-linux-gnu-objdump... no checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for x86_64-pc-linux-gnu-dlltool... no checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for x86_64-pc-linux-gnu-ar... (cached) ar checking for archiver @FILE support... @ checking for x86_64-pc-linux-gnu-strip... (cached) strip checking for x86_64-pc-linux-gnu-ranlib... ranlib checking command to parse nm output from gcc -m64 object... ok checking for sysroot... no checking for a working dd... /usr/bin/dd checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 checking for x86_64-pc-linux-gnu-mt... no checking for mt... mt checking if mt is a manifest tool... no checking for stdio.h... yes checking for stdlib.h... yes checking for string.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for strings.h... yes checking for sys/stat.h... yes checking for sys/types.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc -m64 supports -fno-rtti -fno-exceptions... no checking for gcc -m64 option to produce PIC... -fPIC -DPIC checking if gcc -m64 PIC flag -fPIC -DPIC works... yes checking if gcc -m64 static flag -static works... yes checking if gcc -m64 supports -c -o file.o... yes checking if gcc -m64 supports -c -o file.o... (cached) yes checking whether the gcc -m64 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... no checking whether to build static libraries... yes checking if gcc -m64 supports -Werror=unknown-warning-option... no checking if gcc -m64 supports -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef... yes checking if gcc -m64 supports -Wno-overlength-strings... yes checking if gcc -m64 supports -Wall... yes checking if gcc -m64 supports -Wno-unused-function... yes checking if gcc -m64 supports -Wextra... yes checking if gcc -m64 supports -Wcast-align... yes checking if gcc -m64 supports -Wcast-align=strict... yes checking if gcc -m64 supports -Wconditional-uninitialized... no checking if gcc -m64 supports -fvisibility=hidden... yes checking for x86_64 assembly availability... yes checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating Makefile config.status: creating libsecp256k1.pc config.status: creating src/libsecp256k1-config.h config.status: src/libsecp256k1-config.h is unchanged config.status: executing depfiles commands config.status: executing libtool commands

Build Options: with external callbacks = no with benchmarks = no with tests = yes with coverage = no with examples = no module ecdh = no module recovery = yes module extrakeys = yes module schnorrsig = yes

asm = x86_64 ecmult window size = 15 ecmult gen prec. bits = 4

valgrind = yes CC = gcc -m64 CPPFLAGS = -I/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/include/ SECP_CFLAGS = -O2 -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wcast-align=strict -fvisibility=hidden CFLAGS = -pipe -O2 LDFLAGS = -L/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/lib

Options used to compile and link: external signer = yes multiprocess = yes with experimental syscall sandbox support = yes with libs = yes with wallet = yes with sqlite = yes with bdb = yes with gui / qt = yes with qr = yes with zmq = yes with test = yes with fuzz binary = yes with bench = yes with upnp = yes with natpmp = yes use asm = yes USDT tracing = yes sanitizers = debug enabled = no gprof enabled = no werror = no LTO = no

target os = linux-gnu build os = linux-gnu

CC = gcc -m64 CFLAGS = -pthread -pipe -O2 CPPFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -DHAVE_BUILD_INFO -DPROVIDE_FUZZ_MAIN_FUNCTION -I/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/include/ CXX = g++ -m64 -std=c++17 CXXFLAGS = -fstack-reuse=none -Wstack-protector -fstack-protector-all -fcf-protection=full -fstack-clash-protection -fno-extended-identifiers -pipe -O2 LDFLAGS = -lpthread -Wl,-z,relro -Wl,-z,now -Wl,-z,separate-code -pie -L/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/lib ARFLAGS = cr

My make procedure for capnproto was as per the instrucions in capnproto/c++

Building our own depends should be enough. @SatoriHoshiAiko

What are the last few lines of make -C depends NO_QT=1 MULTIPROCESS=1 output?

Also could you post the content of depends/x86_64-pc-linux-gnu/share/config.site?

Ran make clean beforehand for safekeeping, then -

make -C depends NO_QT=1 MULTIPROCESS=1

make: Entering directory '/home/username/bitcoin-master/depends' copying packages: native_libmultiprocess native_capnp boost libevent bdb sqlite miniupnpc libnatpmp systemtap zeromq libmultiprocess capnp to: /home/username/bitcoin-master/depends/x86_64-pc-linux-gnu make: Leaving directory '/home/username/bitcoin-master/depends'

From there at depends/x86_64-pc-linux-gnu/share/config.site

prints -

# shellcheck shell=sh disable=SC2034 # Many variables set will be used in
                                     # ./configure but shellcheck doesn't know
                                     # that, hence: disable=SC2034

true  # Dummy command because shellcheck treats all directives before first
      # command as file-wide, and we only want to disable for one line.
      #
      # See: https://github.com/koalaman/shellcheck/wiki/Directive

# shellcheck disable=SC2154
depends_prefix="$(cd "$(dirname "$ac_site_file")/.." && pwd)"

cross_compiling=maybe
host_alias="x86_64-pc-linux-gnu"
ac_tool_prefix="${host_alias}-"

if test -z "$with_boost"; then
  with_boost="$depends_prefix"
fi
if test -z "$with_qt_plugindir"; then
  with_qt_plugindir="${depends_prefix}/plugins"
fi
if test -z "$with_qt_translationdir"; then
  with_qt_translationdir="${depends_prefix}/translations"
fi
if test -z "$with_qt_bindir" && test -z "1"; then
  with_qt_bindir="${depends_prefix}/native/bin"
fi
if test -z "$with_mpgen" && test -n "1"; then
  with_mpgen="${depends_prefix}/native"
fi

if test -z "$with_qrencode" && test -n ""; then
  with_qrencode=no
fi

if test -z "$enable_wallet" && test -n ""; then
  enable_wallet=no
fi

if test -z "$with_bdb" && test -n ""; then
  with_bdb=no
fi

if test -z "$with_sqlite" && test -n ""; then
  with_sqlite=no
fi

if test -z "$enable_multiprocess" && test -n "1"; then
  enable_multiprocess=yes
fi

if test -z "$with_miniupnpc" && test -n ""; then
  with_miniupnpc=no
fi

if test -z "$with_natpmp" && test -n ""; then
  with_natpmp=no
fi

if test -z "$with_gui" && test -n "1"; then
  with_gui=no
fi

if test -n "" && test -z "1" && test "$with_gui" != "no"; then
  with_gui=qt5_debug
fi

if test -z "$enable_zmq" && test -n ""; then
  enable_zmq=no
fi

if test -z "$enable_usdt" && test -n ""; then
  enable_usdt=no
fi

if test "linux" = darwin; then
  BREW=no
fi

PKG_CONFIG="$(which pkg-config) --static"

# These two need to remain exported because pkg-config does not see them
# otherwise. That means they must be unexported at the end of configure.ac to
# avoid ruining the cache. Sigh.
export PKG_CONFIG_PATH="${depends_prefix}/share/pkgconfig:${depends_prefix}/lib/pkgconfig"
if test -z ""; then
  export PKG_CONFIG_LIBDIR="${depends_prefix}/lib/pkgconfig"
fi

CPPFLAGS="-I${depends_prefix}/include/ ${CPPFLAGS}"
LDFLAGS="-L${depends_prefix}/lib ${LDFLAGS}"

if test -n "gcc -m64" -a -z "${CC}"; then
  CC="gcc -m64"
fi
if test -n "g++ -m64" -a -z "${CXX}"; then
  CXX="g++ -m64"
fi
PYTHONPATH="${depends_prefix}/native/lib/python3/dist-packages${PYTHONPATH:+${PATH_SEPARATOR}}${PYTHONPATH}"

if test -n "ar"; then
  AR="ar"
  ac_cv_path_ac_pt_AR="${AR}"
fi

if test -n "ranlib"; then
  RANLIB="ranlib"
  ac_cv_path_ac_pt_RANLIB="${RANLIB}"
fi

if test -n "nm"; then
  NM="nm"
  ac_cv_path_ac_pt_NM="${NM}"
fi

if test -n "strip"; then
  STRIP="strip"
  ac_cv_path_ac_pt_STRIP="${STRIP}"
fi

if test "linux" = darwin; then
  if test -n ""; then
    OTOOL=""
    ac_cv_path_ac_pt_OTOOL="${OTOOL}"
  fi

  if test -n ""; then
    INSTALL_NAME_TOOL=""
    ac_cv_path_ac_pt_INSTALL_NAME_TOOL="${INSTALL_NAME_TOOL}"
  fi

  if test -n ""; then
    DSYMUTIL=""
    ac_cv_path_ac_pt_DSYMUTIL="${DSYMUTIL}"
  fi
fi

if test -n ""; then
  enable_reduce_exports=no
fi

if test -n "-pipe -O2"; then
  CFLAGS="-pipe -O2 ${CFLAGS}"
fi
if test -n "-pipe -O2"; then
  CXXFLAGS="-pipe -O2 ${CXXFLAGS}"
fi
if test -n ""; then
  CPPFLAGS=" ${CPPFLAGS}"
fi
if test -n ""; then
  LDFLAGS=" ${LDFLAGS}"
fi

Thanks again

Applied code formatting in #25207 (comment).

So, packages libmultiprocess and capnp have been built successfully in depends.

Could you try

./autogen.sh
./configure CONFIG_SITE=/home/username/bitcoin-master/depends/x86_64-pc-linux-gnu/share/config.site
make clean
make

one more time please?

It seems like if there is a bug, it's located in the bitcoin build, not depends build, since error about ipc/capnp/init.capnp.h (which is a generated file) not being found is an error about the build failing to find a file that it is supposed to be generating. The most useful thing to see would probably be make output or make V=1 output, and maybe run find -name init.capnp.h to see if the file is being generated.

I had a lot of difficulty with libmultiprocess not uilding with the debian package on capnproto, and debian needs to update capnproto as well, on a side note, to fix atomic and libatomic being requested in CapnProtoConfig.cmake by default. Thankfully the master tree here had not issue, but being on an x86_64 system, it was troublesome having the debian package request atomic, I tried installing libatomic as a dependency. Master tree here on github installed both capnproto and libmultitool with no issue.

Note that this is basically a separate issue, which is reported in https://github.com/chaincodelabs/libmultiprocess/issues/68. Debian libcapnp-dev package includes a patch which causes it to install a non-working /usr/lib/x86_64-linux-gnu/cmake/CapnProto/CapnProtoConfig.cmake file, and there is a small workaround I can add to libmultiprocess to make it skip the broken part of this file. This issue doesn't happen in the depends build because the depends build is just building from source without debian's patches or packages.

Okay successfully built. Thanks fellas,

Looks like running make -C depends with the flag NO_QT=1

and following the steps mentioned by @hebasto to run ./autogen.sh were what was needed

I additionally ran ./configure with BDB flags for Berkeley 4.8 DB alongside the other flags for a full installation (this second part is unnecessary as the config.site keeps this information), but the BDB flag is then turned on.

make install has in usr/bin

bitcoind bitcoin-qt bitcoin-cli bitcoin-tx bitcoin-util bitcoin-wallet

Yes this appear to be a bug in the code and I think hebasto was right to locate that depends was struggling to successfully set the dependencies and correctly install them. It was also the first time that this group of dependencies cleanly installed with depends, and previous the NO-QT=1 flag was off, and before that I ran make -C -i depends with warning flags as errors

I can assume, capnp wasn't fully or correctly installed, and the config.site had incomplete information to create init.capnp.h

Much appreciated, great work!

I am hoping the additions of multiprocess will grow the bitcoin-core project as I know this is more recent in the code. Personally I am highly invested into the security in bitcoin nodes, and this is a wonderful addition to make use of.

In cybersecurity my rule of thumb is, separate entities into 3, and protect the bridges and keep each entity secure (hash/password). Rule of 3s are no different in 2FA, in Government Power spread and accountability. Consider it a principal to be effective at security. Very well thought to the team that developed that.

I know the multiprocess runs identical to multiprocess disabled under normal use, but this is a direction that holds a lot of value on the security front early on. My interest is in how json-rpc commands can be isolated, and when applied on front end, provide added privacy when introduced into a json-rpc plugins or in communication with another wallets using rpc communication.

Either way, I will be putting this to further testing and don't see myself needing anything here until bitcoin-core receives its next major updates.

Best of luck,

SatoriHoshiAiko

Thanks for the feedback and testing SatoriHoshiAiko!

I was able to reproduce your original bug by running make -C src ipc/capnp/libbitcoin_ipc_a-protocol.o in a clean directory after configure:

make: Entering directory '/home/russ/work/bitcoin/src'
  CXX      ipc/capnp/libbitcoin_ipc_a-protocol.o
ipc/capnp/protocol.cpp:7:10: fatal error: ipc/capnp/init.capnp.h: No such file or directory
    7 | #include <ipc/capnp/init.capnp.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:9114: ipc/capnp/libbitcoin_ipc_a-protocol.o] Error 1
make: Leaving directory '/home/russ/work/bitcoin/src'

The fix for the bug is:

diff --git a/src/Makefile.am b/src/Makefile.am
index 7f82d188f0c..0f72182f23c 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -330,7 +330,7 @@ obj/build.h: FORCE
          "$(abs_top_srcdir)"
 libbitcoin_util_a-clientversion.$(OBJEXT): obj/build.h
 
-ipc/capnp/libbitcoin_ipc_a-ipc.$(OBJEXT): $(libbitcoin_ipc_mpgen_input:=.h)
+ipc/capnp/libbitcoin_ipc_a-protocol.$(OBJEXT): $(libbitcoin_ipc_mpgen_input:=.h)
 
 # server: shared between bitcoind and bitcoin-qt
 # Contains code accessing mempool and chain state that is meant to be separated

The bug is hard to reproduce and doesn't happen reliably because make doesn't build targets in a predictable order, so if a manual dependency declaration is not correct, it might not be noticeable.

I'll followup with a PR to fix this. Thanks again!