removed proprietary and por fields #1038

This removes the proprietary and por fields from the PSBT specifications.

First off, at face value they have nothing to do with the operations intrinsically required to finalize a valid transaction from PSBT manipulation.

Moreover, whatever information content they can provide for non-standard PSBT manipulation, that content could stay in the unknown field without any loss of generality. How to structure and deal with unknown data would be the responsibility of proprietary software or users wanting to provide proof-of-reserve. As long as BIP174 clearly prescribes that unknown data must be kept during PSBT manipulation, that should be enough.

Let me stress the above point: I have a project where we include proprietary information in the PSBT. Any PSBT software supporting unknown data gently keeps our proprietary information and our proprietary software retrieves that data from serialized PSBT with no problem. There is no need for a PSBT implementation to provide explicit support for proprietary and proof-of-reserves types.

My last conclusion is reinforced by the evidence of all PSBT implementations I know of, including bitcoin core and HWI, not implementing proprietary and proof-of-reserve types. There is a high probability that part of BIP174 would be just ignored.

My last conclusion is reinforced by the evidence of all PSBT implementations I know of, including bitcoin core and HWI, not implementing proprietary and proof-of-reserve types.

Bitcoin Knots includes support for proprietary, at least.

Proprietary types are being implemented in Core in https://github.com/bitcoin/bitcoin/pull/17034. I think it is important for the BIP to define how proprietary fields should work to avoid people just picking a free type and then accidentally causing conflicts. While it is not necessary for every implementation to implement it, it should still be part of the spec for those who do want to use proprietary fields.

The proof of reserves commitment is included to be clear that that field is no longer available if new types are to be added.

NACK

Post-close comment, just as another data point.

https://github.com/Simplexum/python-bitcointx/ supports proprietary fields, and we use them.

To expand on what @achow101 said, in addition to being a protection from vendors just choosing an arbitrary free type that might be used by the standard later, proprietary fields also provide a conflict resolution mechanism for vendors, via proprietary prefix. That prefix is supposed to be vendor-specific (and likely to be globally unique, like their domain name, for example, or maybe even a GUID).

I agree that it is important for the BIP to define how proprietary fields should work. Let me reiterate that I am also relying on "proprietary" data for a project of mine: anyway, an explicit proprietary field is not needed at all.

To avoid vendors picking up a free type that might be later used in the standard, it would be enough to reserve the 0xFC field for data that will **always be treated as unknown**. This would solve any potential conflict down the line. I could amend/re-open my PR for documenting that.

As for the rest, the burden of specifying proprietary standard details should be left out of the PSBT standard, provided that BIP174 has a clear way to deal with them. Reserving 0xFC would be enough.

IMHO https://github.com/bitcoin/bitcoin/pull/17034 just proves the code bloat introduced in core for supporting features that are exogenous to core itself. And this burden would be passed to any standard compliant PSBT implementation.

You're missing the vendor interoperability issue. If there's no way to clearly distinguish proprietary data from different vendors, and if 0xFC is the sole marker, implementations from one vendor can erroneously mangle or fail on data added by another vendor. There would be a need for vendors to come to consensus for interoperability. The whole concept of PSBT is about interoperability between implementations, and thus defining interoperability-related structures for deconflicting is clearly in scope of the BIP.

If the vendor-specific extension is clearly identifiable, other vendors can simply ignore unsupported extensions or they can decide to support the extension. This allows for sub-industry-specific sub-specifications to emerge without the need to amend the BIP.

The whole concept of PSBT is about interoperability between implementations, and thus defining interoperability-related structures for deconflicting is clearly in scope of the BIP.

I agree it is in the BIP scope, I disagree with the proposed solution. To look for proprietary data in the unknown field instead of the proprietary one (provided that 0xFC is preserved for unknown) would change nothing but simplifying the implementation of this BIP.

You're missing the vendor interoperability issue.

I don't think I am.

If there's no way to clearly distinguish proprietary data from different vendors, and if 0xFC is the sole marker, implementations from one vendor can erroneously mangle or fail on data added by another vendor. There would be a need for vendors to come to consensus for interoperability.

Vendor interoperability is all about agreeing with prefix, subtype, and key data in the {0xFC}|<prefix>|{subtype}|{key data} data structure. This is obviously not covered in BIP174, so vendors will have to reach an agreement elsewhere.

If the vendor-specific extension is clearly identifiable, other vendors can simply ignore unsupported extensions or they can decide to support the extension. This allows for sub-industry-specific sub-specifications to emerge without the need to amend the BIP.

Again: an implementation disregarding the proprietary field but honouring the unknown one does work with both truly unknown data and "recognized" proprietary data. If your software does not recognize the data, then it fittingly keeps it in the unknown field. If your software knows how to manage proprietary data, then it can do it even if it is stored in the unknown field, without bothering that it has been filed as unknown instead of proprietary.

In summary, I am basically advocating a simpler specification with no loss of content or functionality. Hard to disagree with... but considering how quickly this PR has been closed and how little attention its arguments have received, I am probably too late in the game or I am touching sensitivities here...

Vendor interoperability is all about agreeing with prefix

It is all about allowing vendors to do anything they want under their own prefix without stepping on the toes of another vendors. Without prefix, there would be a need for central registry for vendors to reserve the types to avoid conflicts. With prefixes a vendor can just use a prefix that would be definitely unique. Agreement between vendors is not required, only requirement is to pick the prefix that is not likely to be used by others (like your domain name, or GUID)

If your software knows how to manage proprietary data, then it can do it even if it is stored in the unknown field, without bothering that it has been filed as unknown instead of proprietary.

The problem is software from different vendors picking the same way to pack proprietary data (which is likely to happen without deconfliction), but using different formats, making their implementations in best case incompatible, and in worst case corrupting each other's data in PSBT.

If an implementation does not want to deal with proprietary fields at all, it can just ignore 0xFC type and treat it as unknown. So proprietary field format adds some complexity to the spec, but this is a necessary complexity with clear purpose - deconfliction. An implementation that treats 0xFC as unknown does not have to deal with that complexity at all.