kernel: support for serializing witness-stripped transactions/blocks? #34960

Please describe the feature you'd like to see added.

In the course of writing a BIP54 compliance checking tool using libbitcoinkernel, I've locally added a new C API function for serializing transactions without witness data (btck_transaction_to_bytes_witness_stripped) to check for rule 3. This alone is probably not a convincing use case (witness-stripping the serialized tx by hand seems feasible, particularly for this case where only 1-in-1-out txs are candidates), but I'm wondering whether it's generally in scope of the project to add support for serializing witness-stripped transactions/blocks. On first sight this seems unnecessary due to widespread segwit support, but my thinking is that potential node implementations built on bitcoinkernel would e.g. still need it to serve peers that don't signal NODE_WITNESS in their version message.

Thoughts? Happy to hear API suggestions (new parameter to existing to_bytes functions vs. separate functions?) and submit a PR if it's considered in scope.

Is your feature related to a problem, if so please describe it.

No response

Describe the solution you'd like

No response

Describe any alternatives you've considered

No response

Please leave any additional context

No response

but I'm wondering whether it's generally in scope of the project to add support for serializing witness-stripped transactions/blocks.

I've been going back and forth on this, and I also realize this is probably not a big deal/something we can change later either way. Currently I'm leaning towards it not being in scope, with nuance.

The kernel interface needs a serialization format: at the very minimum, it needs to accept blocks. As a utility, it's helpful to also let users export data. Internally, kernel also needs to persist to and read from disk, accept new blocks etc. For pragmatic purposes, it makes a lot of sense to use the same format for all of that, and stick to the format in which we expect to receive blocks from the network. And of course, validation internally use this format too, e.g. for (w)txid calculation.

However: architecturally, kernel should not depend on p2p, and vice versa. That means either should own and be able to change its serialization format independently. For that reason, I think we should consider withdrawing the "This is consensus serialization that is also used for the P2P network." statements in the btck_transaction_to_bytes and btck_block_to_bytes function documentation.

If that is true, a user who wants to network with legacy nodes should implement (or use a p2p library for) the wire format instead of relying on serializations that currently match but aren't guaranteed to stay so. If kernel users need witness-stripped serialize data for other purposes (e.g. to get a txid, weight, discount, ...) we should expose that functionality directly, e.g. btck_transaction_get_weight().

I'm currently exploring an approach to expose a btck_transaction_visit(cb) pattern, where cb is a struct of callbacks - one for each type of field in the transaction. visit then iterates over the entire transaction, allowing any serialization logic in a single API call. This way, a p2p library could implement a TX_WITH_WITNESS and TX_NO_WITNESS callback for transactions, and neither p2p nor kernel would depend on each other. Of course, this would be a lot more complex to implement, so I'm very much not sure yet if it's a good idea though.

If we'd rather keep things simple and just add a seralization flag to btck_{block,transaction}_to_bytes(), I'm open to that too. The coupling already exists and this is a small incremental addition.