proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values

Sjors commented at 8:57 am on February 20, 2026: member

Taken from: https://github.com/bitcoin/bitcoin/pull/34020#issuecomment-3624001654

Let applications override CustomHasField so they can decide to treat certain capnproto values as being unset. For example, when converting List(Data) to vector<shared_ptr<CTransaction>>, mapping empty Data fields to null pointers.

This safe to do in special cases, like in this example because serialized CTransaction representations are never empty. It is also useful to do in this case because Cap’n Proto doesn’t currently provide any API for distinguishing between unset and empty data values in a list (although they can be distinguished on the wire).

DrahtBot commented at 8:57 am on February 20, 2026: none

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	ryanofsky

If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#243 (mpgen: support primitive std::optional struct fields by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Sjors commented at 8:58 am on February 20, 2026: member

So far I just took https://github.com/bitcoin/bitcoin/commit/b14778374b0694f7255f774cb657c7049cb4256e verbatim, minus the IPC tests which go into Bitcoin Core. Let me know if you need more adjustments. Looks like the LLM found some typos.

Sjors force-pushed on Feb 20, 2026

Sjors commented at 9:48 am on February 20, 2026: member

Whipped up a test inspired by the one in https://github.com/bitcoin/bitcoin/commit/b14778374b0694f7255f774cb657c7049cb4256e on the Bitcoin Core side.

I fixed the typos and also added a commit to fix two missing includes, that would otherwise need to be added in the test.

Sjors force-pushed on Feb 20, 2026

Sjors commented at 10:06 am on February 20, 2026: member

Added missing #include <vector> to mp/gen.cpp.

DrahtBot added the label Needs rebase on Feb 21, 2026

Sjors force-pushed on Feb 21, 2026

DrahtBot removed the label Needs rebase on Feb 21, 2026

in include/mp/proxy-types.h:197 in 413f915979

194+//! unset when Cap'n Proto fields have certain values. But internally the
195+//! functions get called in different ways. This is because in C++, unlike in
196+//! Cap'n Proto not every C++ type is default constructible, and it may be
197+//! impossible to leave certain C++ values unset. For example if a C++ method
198+//! requires function parameters, there's no way to call the function
199+//! constructing values for each of the parameters. Similarly there's no way to

ryanofsky commented at 3:44 pm on February 25, 2026:

In commit “ipc: Serialize null CTransactionRef as empty Data” (413f91597932921176904492ca09673549750fd1)

I think there is a missing word and this supposed to say “call the function without constructing values”

ryanofsky approved

ryanofsky commented at 3:59 pm on February 25, 2026: collaborator

Code review ACK 413f91597932921176904492ca09673549750fd1. Code here looks good. I think PR title & description need to be updated (also title & description of main commit) because they are referencing CTransactionRef which is not in this repository and also not affected by this change (it requires the CustomHasField(TypeList<CTransaction>...) overload in the original commit).

A good summary of changes here might be “Add CustomHasValue overload to allow mapping non-null Cap’n Proto values to null C++ values” and probably the LLM can turn that into a good title & description not referring to CTransactionRef

Also wondering if you’d want this added to https://github.com/bitcoin/bitcoin/pull/34422 if merged (seems reasonable)

Sjors commented at 5:43 pm on February 27, 2026: member

Also wondering if you’d want this added to https://github.com/bitcoin/bitcoin/pull/34422 if merged (seems reasonable)

That’s not necessary. I need this for https://github.com/bitcoin/bitcoin/pull/34020 which can wait for v32.

Sjors renamed this:
~~ipc: Serialize null CTransactionRef as empty Data~~
proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values
on Feb 27, 2026

Sjors force-pushed on Feb 27, 2026

Sjors commented at 6:06 pm on February 27, 2026: member

Tweaked the commit message and add the missing “without”.

ryanofsky commented at 7:30 pm on February 27, 2026: collaborator

Code review ACK 2d3cc775e655965b6e70c6c22b1e14c5fd527eb7, with commit message rewrite and comment fix since last review.

FWIW I think new commit message is clear, but new PR description is pretty random and doesn’t say what the change does. Not important, but something like this might be better:

Let applications override CustomHasField so they can decide to treat certain capnproto values as being unset. For example, when converting List(Data) to vector<shared_ptr<CTransaction>>, mapping empty Data fields to null pointers. This safe to do in special cases, like in this example because serialized CTransaction representations are never empty. It is also useful to do in this case because Cap’n Proto doesn’t currently provide any API for distinguishing between unset and empty data values in a list (although they can be distinguished on the wire).

refactor: add missing includes to mp/type-data.h 8c2f10252c

proxy-types: add CustomHasField hook for nullable decode paths

Add a CustomHasField(TypeList<...>, InvokeContext&, const Input&) extension
point used by ReadField/CustomReadField to decide whether an input should
materialize a C++ value. The default behavior remains input.has().

This enables targeted mappings from specific non-null Cap'n Proto values to
null C++ values (for example empty Data in List(Data), where Cap'n Proto C++
cannot currently distinguish null vs empty), without CTransactionRef-specific
logic in libmultiprocess.

Apply the hook across nullable read paths and add a test covering
round-tripping data pointers including null values.

Originally proposed here:
https://github.com/bitcoin/bitcoin/pull/34020#issuecomment-3624001654

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>

97d877053b

Sjors force-pushed on Mar 4, 2026

Sjors commented at 9:41 pm on March 4, 2026: member

Rebased for easier subtree updating in https://github.com/bitcoin/bitcoin/pull/34020. No other changes.

ryanofsky commented at 4:47 pm on March 5, 2026: collaborator

Code review ACK 97d877053b655dd041bc3487310e863ec2532f3a. Confirmed no changes since last rebase. @Sjors could you maybe review #246? I want to bump the version after https://github.com/bitcoin/bitcoin/issues/28722 was merged. Then merge newer PR’s like this.

proxy-types: add CustomHasField hook to map Cap’n Proto values to null C++ values #242

Reviews

Conflicts