depends: fix multiprocess build on OpenBSD (apply capnp patch, correct SHA256SUM command) #32690

This PR fixes the multiprocess depends build for OpenBSD by applying upstream patch https://github.com/capnproto/capnproto/pull/2308 and switching the SHA256SUM command to output hash sums in the expected format (the default is BSD format [1], but we need GNU format [2], see commit message for details). Note that the hashing issue is only prevailing for packages defining the $(package)_local_dir variable (introduced in 5d105fb8c3ffa39c3e716c3147ee74795b129113, part of #31741), where the following line of the fetch_local_dir_sha256 function leads to the wrong output: https://github.com/bitcoin/bitcoin/blob/ae024137bda9fe189f4e7ccf26dbaffd44cbbeb6/depends/funcs.mk#L57

The first commit can be replaced with a simple capnp version bump once this is available in a release.

Tested on OpenBSD 7.7 (x86_64) via

$ gmake -C depends MULTIPROCESS=1 NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_QR=1 NO_WALLET=1 NO_ZMQ=1 NO_USDT=1

[1] example output: SHA256 (/home/thestack/.vimrc) = 6ba69d100e8c5ca0488ded6293d4e5f740a6a5d5ace96cbcf0599c18d27389e4 [2] example output: 6ba69d100e8c5ca0488ded6293d4e5f740a6a5d5ace96cbcf0599c18d27389e4 /home/thestack/.vimrc

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32690.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process.

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

LLM Linter (✨ experimental)

Possible typos and grammar issues:

• may also solves -> may also solve [correct verb form]

^{drahtbot_id_4_m}

Concept ACK 8713e8060d504f561fed705b4aa5af7b96c36e75.

https://github.com/hebasto/bitcoin-core-nightly/actions/runs/15485455235/job/43599018182 looks good.

Why 8713e8060d504f561fed705b4aa5af7b96c36e75 is necessary for libmultiprocess, but not for other packages?

I tested this on top of #31802 on a VM running OpenBSD 7.7 (dropping NO_IPC for OpenBSB).

Without the patch I get a bunch of errors like:

[  2%] Building CXX object src/kj/CMakeFiles/kj.dir/cidr.c++.o
/root/src/bitcoin/depends/work/build/aarch64-unknown-openbsd7.7/native_capnp/1.1.0-f5dbdc0b2f0/src/kj/cidr.c++:116:71: error: member access into incomplete type 'const struct sockaddr_in6'
        otherBits = reinterpret_cast<const struct sockaddr_in6*>(addr)->sin6_addr.s6_addr;

With the patch the build still fails:

gmake -C depends NO_QT=1 NO_QR=1 NO_ZMQ=1 NO_USDT=1
...
[ 95%] Linking CXX executable capnp
ld: warning: parser.c++(parser.c++.o:(kj::Maybe<decltype(apply(instance<capnp::compiler::CapnpParser::CapnpParser(capnp::Orphanage, capnp::compiler::ErrorReporter&)::'lambda38'(capnp::compiler::(anonymous namespace)::Located<capnp::Text::Reader>&&, kj::Maybe<capnp::Orphan<capnp::compiler::LocatedInteger>>&&, capnp::compiler::(anonymous namespace)::Located<kj::Array<kj::Maybe<capnp::compiler::(anonymous namespace)::Located<capnp::Text::Reader>>>>&&, capnp::Orphan<capnp::compiler::Expression>&&, kj::Array<capnp::Orphan<capnp::compiler::Declaration::AnnotationApplication>>&&)&>(), instance<kj::parse::OutputType_<decltype(instance<kj::parse::Sequence_<kj::parse::TransformOrReject_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)0, &capnp::compiler::Token::Reader::getIdentifier() const>> const&, capnp::compiler::(anonymous namespace)::ExactString>, kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)0, &capnp::compiler::Token::Reader::getIdentifier() const>> const&, kj::parse::Optional_<kj::parse::ParserRef<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>, capnp::Orphan<capnp::compiler::LocatedInteger>>&>, kj::parse::Transform_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::List<capnp::List<capnp::compiler::Token, (capnp::Kind)3>, (capnp::Kind)6>::Reader, (capnp::compiler::Token::Which)5, &capnp::compiler::Token::Reader::getParenthesizedList() const>> const&, capnp::compiler::(anonymous namespace)::ParseListItems<kj::parse::OneOf_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)0, &capnp::compiler::Token::Reader::getIdentifier() const>> const&, kj::parse::TransformWithLocation_<kj::parse::TransformOrReject_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)4, &capnp::compiler::Token::Reader::getOperator() const>> const&, capnp::compiler::(anonymous namespace)::ExactString>, capnp::compiler::CapnpParser::CapnpParser(capnp::Orphanage, capnp::compiler::ErrorReporter&)::'lambda37'(kj::parse::Span<capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>)>>&>>, kj::parse::TransformOrReject_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)4, &capnp::compiler::Token::Reader::getOperator() const>> const&, capnp::compiler::(anonymous namespace)::ExactString>, kj::parse::ParserRef<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>, capnp::Orphan<capnp::compiler::Expression>>&, kj::parse::Many_<kj::parse::ParserRef<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>, capnp::Orphan<capnp::compiler::Declaration::AnnotationApplication>>&, false>>&>()(instance<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>&>()))>::Type&&>()))> kj::parse::Transform_<kj::parse::Sequence_<kj::parse::TransformOrReject_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)0, &capnp::compiler::Token::Reader::getIdentifier() const>> const&, capnp::compiler::(anonymous namespace)::ExactString>, kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)0, &capnp::compiler::Token::Reader::getIdentifier() const>> const&, kj::parse::Optional_<kj::parse::ParserRef<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>, capnp::Orphan<capnp::compiler::LocatedInteger>>&>, kj::parse::Transform_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::List<capnp::List<capnp::compiler::Token, (capnp::Kind)3>, (capnp::Kind)6>::Reader, (capnp::compiler::Token::Which)5, &capnp::compiler::Token::Reader::getParenthesizedList() const>> const&, capnp::compiler::(anonymous namespace)::ParseListItems<kj::parse::OneOf_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)0, &capnp::compiler::Token::Reader::getIdentifier() const>> const&, kj::parse::TransformWithLocation_<kj::parse::TransformOrReject_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)4, &capnp::compiler::Token::Reader::getOperator() const>> const&, capnp::compiler::(anonymous namespace)::ExactString>, capnp::compiler::CapnpParser::CapnpParser(capnp::Orphanage, capnp::compiler::ErrorReporter&)::'lambda37'(kj::parse::Span<capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>)>>&>>, kj::parse::TransformOrReject_<kj::parse::TransformOrReject_<kj::parse::Any_ const&, capnp::compiler::(anonymous namespace)::MatchTokenType<capnp::Text::Reader, (capnp::compiler::Token::Which)4, &capnp::compiler::Token::Reader::getOperator() const>> const&, capnp::compiler::(anonymous namespace)::ExactString>, kj::parse::ParserRef<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>, capnp::Orphan<capnp::compiler::Expression>>&, kj::parse::Many_<kj::parse::ParserRef<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>, capnp::Orphan<capnp::compiler::Declaration::AnnotationApplication>>&, false>>, capnp::compiler::CapnpParser::CapnpParser(capnp::Orphanage, capnp::compiler::ErrorReporter&)::'lambda38'(capnp::compiler::(anonymous namespace)::Located<capnp::Text::Reader>&&, kj::Maybe<capnp::Orphan<capnp::compiler::LocatedInteger>>&&, capnp::compiler::(anonymous namespace)::Located<kj::Array<kj::Maybe<capnp::compiler::(anonymous namespace)::Located<capnp::Text::Reader>>>>&&, capnp::Orphan<capnp::compiler::Expression>&&, kj::Array<capnp::Orphan<capnp::compiler::Declaration::AnnotationApplication>>&&)>::operator()<kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>>(kj::parse::IteratorInput<capnp::compiler::Token::Reader, capnp::_::IndexingIterator<capnp::List<capnp::compiler::Token, (capnp::Kind)3>::Reader const, capnp::compiler::Token::Reader>>&) const) in archive libcapnpc.a): warning: strcpy() is almost always misused, please use strlcpy()
ld: error: undefined symbol: std::__throw_bad_array_new_length()
>>> referenced by module-loader.c++
>>>               CMakeFiles/capnp_tool.dir/compiler/module-loader.c++.o:(capnp::compiler::ModuleLoader::Impl::loadModule(kj::ReadableDirectory const&, kj::PathPtr))
>>> referenced by compiler.c++
>>>               compiler.c++.o:(std::_Hashtable<capnp::compiler::Compiler::Node*, std::pair<capnp::compiler::Compiler::Node* const, unsigned int>, std::allocator<std::pair<capnp::compiler::Compiler::Node* const, unsigned int>>, std::__detail::_Select1st, std::equal_to<capnp::compiler::Compiler::Node*>, std::hash<capnp::compiler::Compiler::Node*>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true>>::_M_rehash(unsigned long, unsigned long const&)) in archive libcapnpc.a
>>> referenced by compiler.c++
>>>               compiler.c++.o:(std::_Hashtable<capnp::compiler::Module*, std::pair<capnp::compiler::Module* const, kj::Own<capnp::compiler::Compiler::CompiledModule, std::nullptr_t>>, std::allocator<std::pair<capnp::compiler::Module* const, kj::Own<capnp::compiler::Compiler::CompiledModule, std::nullptr_t>>>, std::__detail::_Select1st, std::equal_to<capnp::compiler::Module*>, std::hash<capnp::compiler::Module*>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true>>::_M_rehash(unsigned long, unsigned long const&)) in archive libcapnpc.a
>>> referenced 2 more times
collect2: error: ld returned 1 exit status
gmake[3]: *** [src/capnp/CMakeFiles/capnp_tool.dir/build.make:122: src/capnp/capnp] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:480: src/capnp/CMakeFiles/capnp_tool.dir/all] Error 2
gmake[1]: *** [Makefile:136: all] Error 2
gmake[1]: Leaving directory '/home/sjors/src/bitcoin/depends/work/build/aarch64-unknown-openbsd7.7/capnp/1.1.0-adf16ebd094'
gmake: *** [funcs.mk:342: /home/sjors/src/bitcoin/depends/work/build/aarch64-unknown-openbsd7.7/capnp/1.1.0-adf16ebd094/./.stamp_built] Error 2
gmake: Leaving directory '/home/sjors/src/bitcoin/depends'

(this also happens with just this PR at 8713e8060d504f561fed705b4aa5af7b96c36e75)

Though maybe there's something wrong with my VM? #32691 In particular, I symlinked egcc and eg++ to /usr/local/bin/g{cc,++}.

ACK 8713e8060d504f561fed705b4aa5af7b96c36e75

Switched to an x86_64 based VM and it builds fine. I also ran the Bitcoin Core test suite.

Can you add a note to the OpenBSD instruction in depends/README.md to also install cmake?

Why 8713e80 is necessary for libmultiprocess, but not for other packages?

The libmultiprocess package is the only one which sets the $(package)_local_dir variable, causing execution of the fetch_local_dir_sha256 shell script where the hash is extracted from the file manually, expecting the hash in the first row (=GNU format): https://github.com/bitcoin/bitcoin/blob/ae024137bda9fe189f4e7ccf26dbaffd44cbbeb6/depends/funcs.mk#L57

Another instance where we do this manual extraction is in the calculation of $(1)_all_file_checksums: https://github.com/bitcoin/bitcoin/blob/ae024137bda9fe189f4e7ccf26dbaffd44cbbeb6/depends/funcs.mk#L62

But this variable is only a temporary for calculating $(1)_recipe_hash, which is done by feeding the hashing tool from stdin, where the output is as expected [1]: https://github.com/bitcoin/bitcoin/blob/ae024137bda9fe189f4e7ccf26dbaffd44cbbeb6/depends/funcs.mk#L69

So this was unnoticed, but the recipe hash calculation was not entirely correct, so I'd suggest to backport this commit.

Note that instances where a hash list is verified via -c are not affected, as this options accepts both the BSD and the GNU formats. Also, if the input to the hashing command is piped in (i.e. coming from stdin), the output is also in the expected format even without specifying -r.

[1] e.g.

$ echo "foobar" | sha256
aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f

@Sjors:

Can you add a note to the OpenBSD instruction in depends/README.md to also install cmake?

Seems like material for a different PR, considering this affects other BSDs as well, and at first glance it seems there are even other packages missing, like e.g. curl (which AFAIR, is also not part of OpenBSD's base system). Will open a follow-up later.

ACK 8713e8060d504f561fed705b4aa5af7b96c36e75.

ACK 8713e8060d504f561fed705b4aa5af7b96c36e75

Backported to 29.x in #32690.

Backported to 29.x in #32690.

Note that this became a partial backport, just 8713e8060d504f561fed705b4aa5af7b96c36e75.