[POLICY] Make multiple OP_RETURNS in a single TX standard #17272

Bushstar commented at 10:16 AM on October 27, 2019: contributor

This proposed policy change would allow multiple OP_RETURN outputs per transaction, the Bitcoin protocol allows for multiple of these opcodes per transaction but the current policy does not. We are seeing an inefficient use of OP_RETURN for systems that rely on this feature. For example, an asset layer solution making use of OP_RETURN currently has to create multiple transactions when sending to multiple recipients.

There are several regular sources of these types of transactions on the network making up a significant proportion of daily transaction, OmniLayer, one successful and popular asset layer, on the 21st Oct 2019 was responsible for 8% of transactions that day (Omni 28,313 TXs [1] / Bitcoin 351,632 TXs [2]). Allowing multiple OP_RETURN opcodes would enable asset layer solutions to move multiple assets to multiple recipients in a single transaction, this would result in an overall reduced data size for the movement of those assets reducing transaction load on the network.

The original Pull Request to add OP_RETURN linked below bemusingly set the fail reason for multiple instances of it to “mucho-data”, the opposite is now true, single OP_RETURN per transaction only result in more data, not less.

https://github.com/bitcoin/bitcoin/pull/2738/files

A service bit could be added to show whether a node would accept and relay a transaction with multiple OP_RETURN opcodes, such transactions would then only be relayed to nodes indicating support. If this change warrants such a thing I’d be happy to update this PR to include it.

Bushstar force-pushed on Oct 27, 2019

DrahtBot added the label Tests on Oct 27, 2019

DrahtBot added the label TX fees and policy on Oct 27, 2019

luke-jr commented at 1:21 PM on October 27, 2019: member

There should probably be a BIP first.

Why is it using OP_RETURN at all? I would expect assets to just use coloured coins...

luke-jr commented at 1:22 PM on October 27, 2019: member

(Also, the BIP and policy change should be explicitly for the asset stuff, not a general OP_RETURN policy)

CraigSellars commented at 3:48 PM on October 27, 2019: none

Why is it using OP_RETURN at all? I would expect assets to just use coloured coins...

Two reasons: first, colored coins can only perform explicit transactions, such as sends of assets from one address to another. They cannot support implicit actions, like sending an asset from a recipient in response to a single transaction from a sender (this would include crowdsale/ICO-like functionality as well as decentralized exchange capability). Second, colored coins using a particular number of satoshis in an output as an identifier are always at risk of being inadvertently destroyed by non-compliant clients/wallets that receive them and don't recognize the importance of that output being preserved.

laanwj removed the label Tests on Oct 28, 2019

Relaxo143 commented at 6:27 PM on October 28, 2019: none

Each OP_RETURN output is being paid for (fees), so there is no reason to limit the amount of such outputs. Great PR in my opinion!

luke-jr commented at 12:16 AM on October 29, 2019: member

Two reasons: first, colored coins can only perform explicit transactions, such as sends of assets from one address to another. They cannot support implicit actions, like sending an asset from a recipient in response to a single transaction from a sender (this would include crowdsale/ICO-like functionality as well as decentralized exchange capability). @CraigSellars Bitcoin transactions don't have an identified sender.

Second, colored coins using a particular number of satoshis in an output as an identifier are always at risk of being inadvertently destroyed by non-compliant clients/wallets that receive them and don't recognize the importance of that output being preserved.

That's no more or less true than using OP_RETURN.

Again, where's the BIP? That should hopefully make a lot of questions go away...

Each OP_RETURN output is being paid for (fees), so there is no reason to limit the amount of such outputs.Each OP_RETURN output is being paid for (fees), so there is no reason to limit the amount of such outputs. @Relaxo143 This is nonsense. Fees do not pay for outputs or transactions, they merely bribe miners (who do not carry the burden) to include them in blocks. These policy spam filters exist to help prevent spam regardless of the fee included.

dexX7 commented at 7:01 AM on October 29, 2019: contributor

Bitcoin transactions don't have an identified sender.

But certain overlay protocols do and this is what this is about.

Also related: #12033

in src/policy/policy.cpp:119 in d8c035db96 outdated

 118 |          }
 119 |  
 120 | -        if (whichType == TX_NULL_DATA)
 121 | -            nDataOut++;
 122 | -        else if ((whichType == TX_MULTISIG) && (!permit_bare_multisig)) {
 123 | +        if ((whichType == TX_MULTISIG) && (!permit_bare_multisig)) {

theStack commented at 2:17 PM on October 29, 2019:

Note that your policy change leads to the execution of the dust check (IsDust()) for OP_RETURN outputs.

theStack changes_requested

theStack commented at 2:21 PM on October 29, 2019: member

The policy changes should also be updated in the corresponding unit test case test_IsStandard. It only passed because of an error in https://github.com/bitcoin/bitcoin/pull/17272/commits/5fe6f052bd37a16b2849e05f5cf18d7e194bc705 (see my review comment in policy.cpp), leading to a dust rejection reason, instead of the expected multi-op-return. See also PR #17299 which adds reason checks for non-standard transactions.

in test/functional/mempool_accept.py:288 in d8c035db96 outdated

 279 | @@ -281,13 +280,6 @@ def run_test(self):
 280 |              result_expected=[{'txid': tx.rehash(), 'allowed': False, 'reject-reason': 'dust'}],
 281 |              rawtxs=[tx.serialize().hex()],
 282 |          )
 283 | -        tx.deserialize(BytesIO(hex_str_to_bytes(raw_tx_reference)))
 284 | -        tx.vout[0].scriptPubKey = CScript([OP_RETURN, b'\xff'])
 285 | -        tx.vout = [tx.vout[0]] * 2
 286 | -        self.check_mempool_result(
 287 | -            result_expected=[{'txid': tx.rehash(), 'allowed': False, 'reject-reason': 'multi-op-return'}],

theStack commented at 3:42 PM on October 29, 2019:

I'd suggest not removing the test case, but rather simply adapting the expected result (to 'allowed': True instead of False and removing the 'reject-reason').

Relaxo143 commented at 4:04 PM on October 29, 2019: none

Two reasons: first, colored coins can only perform explicit transactions, such as sends of assets from one address to another. They cannot support implicit actions, like sending an asset from a recipient in response to a single transaction from a sender (this would include crowdsale/ICO-like functionality as well as decentralized exchange capability).

@CraigSellars Bitcoin transactions don't have an identified sender.

Second, colored coins using a particular number of satoshis in an output as an identifier are always at risk of being inadvertently destroyed by non-compliant clients/wallets that receive them and don't recognize the importance of that output being preserved.

That's no more or less true than using OP_RETURN.

Again, where's the BIP? That should hopefully make a lot of questions go away...

Each OP_RETURN output is being paid for (fees), so there is no reason to limit the amount of such outputs.Each OP_RETURN output is being paid for (fees), so there is no reason to limit the amount of such outputs.

@Relaxo143 This is nonsense. Fees do not pay for outputs or transactions, they merely bribe miners (who do not carry the burden) to include them in blocks. These policy spam filters exist to help prevent spam regardless of the fee included. @luke-jr What is spam is highly subjective. The author of this PR doesn't think OP_RETURN outputs are spam and thus proposes to make them standard (more than 1 in a tx). This will actually save blockchain space. If a person has to or wants to use OP_RETURN and needs 5 outputs for their data they'll have to compose 5 different TXs with at least one input. Inputs take up more space than outputs so by leaving >1 OP_RETURN outputs non-standard we enforce higher blockchain usage. there is no reason why we should limit them in my opinion. Just like batch sending is possible with multiple outputs (addresses), batch data embedding should be possible too.

Bushstar force-pushed on Oct 29, 2019

luke-jr commented at 8:19 PM on October 29, 2019: member

@Relaxo143 I'm not saying this is spam. I'm saying the argument you made is nonsensical.

DrahtBot commented at 8:28 PM on October 29, 2019: member

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#17480 (test: add unit test for non-standard txs with too large scriptSig by theStack)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

theStack commented at 8:35 PM on October 29, 2019: member

Each OP_RETURN output is being paid for (fees), so there is no reason to limit the amount of such outputs.Each OP_RETURN output is being paid for (fees), so there is no reason to limit the amount of such outputs.

@Relaxo143 This is nonsense. Fees do not pay for outputs or transactions, they merely bribe miners (who do not carry the burden) to include them in blocks. These policy spam filters exist to help prevent spam regardless of the fee included.

What exactly do you mean by "burden" here? If arbitrary data is stored in the blockchain without using OP_RETURN (e.g. by just misusing the 20-byte hash in a P2PKH output, which can't even be prevented), the burden is actually much higher since it has to be kept in the UTXO set forever. Why is exactly one output of OP_RETURN considered as non-spam, but more than one is?

MarcoFalke referenced this in commit ecad0a8019 on Oct 29, 2019

DrahtBot added the label Needs rebase on Oct 30, 2019

[POLICY] Make multiple OP_RETURNS in a single TX standard 87c312bad9

[test] Remove multi-op-return failure from mempool tests 4013ba495b

Bushstar force-pushed on Oct 30, 2019

Bushstar commented at 7:06 AM on October 30, 2019: contributor

The process of using multiple TXs to send multiple OP_RETURN outputs is inefficient when those outputs could be sent in one TX, this PR exists to reduce inefficiencies.

Rebased to fix conflicts. @theStack Thanks for spotting those issues, I've updated to not call IsDust if the output is OP_RETURN and have removed the additional tests.

DrahtBot removed the label Needs rebase on Oct 30, 2019

luke-jr commented at 4:45 PM on November 4, 2019: member

Generic OP_RETURN is accepted to commit to a hash of external data. That external data can be of any size without increasing the size of the hash or requiring multiple hashes. It can even be a merkle tree, if multiple independent commitments are desired.

Considering no jurification is given for multiple OP_RETURNs, and the desired goal is more properly accomplished by implementing a policy that understands the asset layer (to some extent) directly, Concept NACK.

jgarzik commented at 5:08 PM on November 4, 2019: contributor

Generic OP_RETURN is accepted to commit to a hash of external data. That external data can be of any size without increasing the size of the hash or requiring multiple hashes. It can even be a merkle tree, if multiple independent commitments are desired.

True.

Considering no jurification is given for multiple OP_RETURNs, and the desired goal is more properly accomplished by implementing a policy that understands the asset layer (to some extent) directly, Concept NACK.

The assumption that a single-output+merkle-tree==sufficient misses the bitcoin-supported case where multiple parties can collaborate on assembling a multi-party transaction, SIGHASH_NONE, SIGHASH_SINGLE and the other less common forms of transaction. That would seem to provide some justification for multiple OP_RETURN outputs.

luke-jr commented at 5:33 PM on November 4, 2019: member

In such cases, couldn't they concatenate their hashes and put a hash of all of them together in a single OP_RETURN?

jgarzik commented at 6:29 PM on November 4, 2019: contributor

In such cases, couldn't they concatenate their hashes and put a hash of all of them together in a single OP_RETURN?

Part of the point of SIGHASH_SINGLE and friends is that aggregation of that sort may require burdensome cooperation across competing or independent parties, leading to the need to aggregate at the tx input/output level instead. It seems like the same train of thought and reasoning that lead to SIGHASH_NONE and SIGHASH_SINGLE can be applied here to multiple/single OP_RETURN values.

Sjors commented at 11:15 AM on November 14, 2019: member

As Luke-Jr already said: this needs a BIP. The bitcoin-dev mailinglist is a more appropriate place for discussion. The BIP can point to this PR as its example implementation.

DrahtBot added the label Needs rebase on Nov 15, 2019

DrahtBot commented at 8:26 PM on November 15, 2019: member

Needs rebase

fanquake commented at 11:04 AM on April 12, 2020: member

As mentioned, this needs a BIP / mailing list discussion. Going to close this, you can still point back to the changes if needed.

fanquake closed this on Apr 12, 2020

DrahtBot locked this on Feb 15, 2022