The maximum size for OP_RETURN outputs used to be 80 bytes, then got changed to 40 bytes to be on the safe side. We have now been running with 40 bytes for about 9 months, and nothing catastrophic happened to the Blockchain, so I am proposing to increase it back to 80 bytes.
Also, that value is now configurable through the datacarriersize option, so miners who want to stay on 40 bytes (or any other value) can easily do so.
The value can be changed through the '-datacarriersize' option, this
is modifying the default value for that option.
The maximum length for the payload of an OP_RETURN output is now
80 bytes, and unit tests must be modified to account for the change.
Let me play devil’s advocat and ask: why should one use OP_RETURN at all? As long as alternatives are cheaper (and faster at this point), there is not much inceive to do so.
I took a look at cost as transaction size in relation to payload sizes and it turns out 40 byte OP_RETURN isn’t really a great choice.
80 byte however, assuming one pay-to-pubkey change output, beats efficient bare multisig encoding (that is: spending multisig outputs!).
I’m in the middle here. If this is because the alternative is people storing data in other non-prunable ways anyway, then it’s obviously a win. For some use cases, there is no way around that.
But for many there is, and this is just providing them with an easier path that may result in an ecosystem with higher costs for everyone. I don’t like encouraging that, and I don’t think it’s in the best interest of Bitcoin nodes to help people do so.
I think OP_RETURN has shown itself to be seriously problematic; and we continue to have problems with people beleving that storing non-bitcoin related data in the chain (as opposed, e.g. to simple commitments or things like ECDH nonces) is an approved, correct, non-antisocial use of the system. We have people selling insane data storage services, etc. It’s a bad place to be.
Meanwhile, many of the externalized cost creating services which could use op_return (e.g. data they’re encoding is small) still don’t bother. Rather than a success, I think it’s hard to say that it hasn’t been much of a success. Though: at least a few things that were going to encode data did switch to hashes, so I think the limitations were at least somewhat informative.
That said, there are some cases where I think more data can make sense and be useful. For example, I think of the ECDH address negoiation stuff isn’t a terrible use… a bit inefficient compared to using an external system, but not unreasonable… and that application wants the ability to push in additional payment identifiers and the like (see petertodd’s latest update to the stealth address additions).
So, I’d prefer to make any increase take the form of allowing more pushes, e.g. you can have up to {limit} bytes, in the form of pushes up to 40 bytes. Beyond furthering the message that this isn’t an endorcement for externalizing your storage and transmission costs to the increasingly shrinking public network that our decenteralization depends on; this option also is permissive enough that there isn’t a {well you can use checkmultsig instead} that is more permissive.
If people who would support this would also support that, I’ll submit a PR.
—–BEGIN PGP SIGNED MESSAGE—– Hash: SHA256
@jgarzik No, #5075 allows multiple outputs. What @gmaxwell is suggesting is still a single output, but with multiple pushdata opcodes.
I have a pull-req open to allow that.
As for this pull-req, utACK —–BEGIN PGP SIGNATURE—– Version: APG v1.1.1
iQFQBAEBCAA6BQJUgM3EMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhexHB/4xpXq3P3+AC3wUsJID YgI/oLJIOS74mVYobmfi3qjF8TWtsi6IHpHKAPRS1JCakSSFRa8wviLfwvrrglYv wW3ZusJXxjHnMsJblbhMMJl+2mF1W2RIMoYi2jiIhz4nMMFsrU6rl8vvVERXKRec WslksFr23Tx441FCVk4K9LpmzjkeCqr5NxUGKaudVKTM9RqSId+frsGF+Teo2uNx QVtprvTzvxqHShSEKGL+y+cMlaWZ2p8HH3nXtN1tkoEqxdqb0LEauV8Hk0MSkRz2 6kXZnFTtxa+alwVzfsEdR6mU/5cl9Sg/RmpbQraUYUAw6SEtytVeUSAwY2NfO87V xJsb =ioIv —–END PGP SIGNATURE—–
untested ACK.
So we’ve got three different suggestions for how to do this, and two “maybe it is a bad idea to do it at all.”
Of the three suggestions: I like this one best, it is simplest conceptually.
And I still think the benefits of a prunable 80-byte OP_RETURN (and 40-byte OP_RETURN) outweigh the costs.
Removing the single PUSHDATA restriction is orthogonal to how many bytes are allowed, and mostly orthogonal to how many OP_RETURN outputs are allowed.
On 4 December 2014 21:35:19 GMT+00:00, Gavin Andresen notifications@github.com wrote:
untested ACK.
So we’ve got three different suggestions for how to do this, and two “maybe it is a bad idea to do it at all.”
Of the three suggestions: I like this one best, it is simplest conceptually.
And I still think the benefits of a prunable 80-byte OP_RETURN (and 40-byte OP_RETURN) outweigh the costs.
Reply to this email directly or view it on GitHub: #5286 (comment)
this option also is permissive enough that there isn’t a that is more permissive option.
Creating unspendable outputs is another option that is more permissive. I’m of course supporting that in the coloured coin standard I’m working on, an approach being taken by many projects. (that particular application has no show-stopper issues even with P2SH²)
I think OP_RETURN has shown itself to be seriously problematic; and we continue to have problems with people beleving that storing non-bitcoin related data in the chain (as opposed, e.g. to simple commitments or things like ECDH nonces) is an approved, correct, non-antisocial use of the system
I fully agree. However, practically, until the time that we have a solution for spam in unspendable outputs (cluttering the UTXO set which is so much worse), at least encouraging the least antisocial method for people that insist on doing it is advantageous, I’d suppose…
cluttering the UTXO set which is so much worse
I believe an increase in OP_RETURN’s default size would mainly serve as inceive to move away from bare multisig encoding which is currently “cheaper”. Getting the attention of (ab-) users who create unspendable outputs, such as cryptograffiti.info, appears to be another topic and an increase to 80 byte probably has no effect in that context.
Hey @petertodd: the chart was created based on the following assumptions:
Please let me know, if there are flaws or more efficient ways to transport data, but I came up with the following scripts:
0OP_RETURN: up to 40 byte (80 byte, 200 byte)
1
2scriptSig: [signature]
3scriptPubKey 0: [op_return] [payload]
4scriptPubKey 1: [public_key] [op_checksig]
5
6Bare multisig: up to 62 byte, embedded as public key, 31 byte each, with n = [2, 3]
7
8scriptSig: [op_0] [signature]
9scriptPubKey: [op_1] [public_key] [payload] ( [payload] ) [op_n] [op_checkmultisig]
10
11P2SH: up to 434 byte, embedded as public key, 31 byte each
12
13Transaction 1 (commit):
14scriptSig: [signature]
15scriptPubKey: [op_hash160] [script_hash] [op_equal]
16
17Transaction 2 (reveal) with n = [2, 15]:
18scriptSig: [op_0] [signature] [op_1] [public_key] [payload] ( ... ) [op_n] [op_checkmultisig]
19scriptPubKey: [public_key] [op_checksig]
no unspendable outputs
Better to determine what the cost for creating them is due to coins lost to the dust limit. Also calculate w/o dust limit, as lots of hashing power doesn’t follow it.
if the payload size surpasses capacity, then multiple transactions are created
Why multiple transactions? Why not have multiple outputs instead?
Why multiple transactions? Why not have multiple outputs instead?
Only one OP_RETURN output per transaction?
Also calculate w/o dust limit, …
I have the impression we’re drifting a bit off topic, but since this is about defining new defaults, I think other defaults should be respected, such as the dust rule.
Why multiple transactions? Why not have multiple outputs instead?
You’re right. It was mentioned in particular for OP_RETURN and this might be misleading. In the case of bare multisig or script hash transactions, using multiple outputs is prefered of course. Still worth to note: the overhead of creating a new transaction is actually only 10 byte, assuming outputs are redeemed, because each additional output requires an additional input to reclaim sooner or later.
I have the impression we’re drifting a bit off topic, but since this is about defining new defaults, I think other defaults should be respected, such as the dust rule.
I’m interested in what the economics are of people doing proof-of-publication, not about “respecting” any particular rule.
Similarly assuming outputs are redeemed is not a good assumption; what’s the underlying economics?
@petertodd: don’t get me wrong, I fully agree and think the economics and inceive aspects should dictate the decision whether the size of OP_RETURN should be increased or not. This is also the reason why I created the chart - which might not be 100 % accurate, but close to. One can start to make assumptions about miners who don’t “respect” the dust rule, but this is difficult to messure and there are a few other aspects as well, e.g.:
My previous tests showed (with sample sizes of less than thousand transactions) the acceptance of OP_RETURN increased significantly over the year, but bare multisig is still faster.
The bottom line:
Those additional factors support an increase in size, whether it’s based on the assumption that it’s more profitable to ignore the dust rule, to create unspent outputs or to include the fact that not every miner mines data transactions.
It can further be determined that the size should be at least 80 byte to stay competitive against other data embedding mechanisms.
I think it’s more important to move this check to policy.o (something similar to https://github.com/luke-jr/bitcoin/commit/581ffd7efe0f1c18f8f986005e4df3f5371fdc36) than to discuss the “perfect” defaults when they will always really be arbitrary. There’s no guarantee that the people defending 80 won’t be defending 120 in the future…
But it is true that people using bare multisig for data store is even worse, so I won’t oppose this PR (nor #5231, even though that’s another thing that should be only used in policy.o).
bitcoind -policy=custom as shown in https://github.com/jtimon/bitcoin/commit/f71f691a81ffc74d7773a2d8d6800b836ba03182 (related to #5595 and #5696 among other PRs).
Hopefully that would reduce these endless policy discussions…
The problem here is that most miners don’t care and are fine with whatever the default policy is as long as it doesn’t increase their orphan rate.
We all understand miners should be good citizens and vote, but they just can’t be bothered (just like with “real” democracy).
-datacarriersize it’s specially surprising that the discussions continue.
So some people don’t want the standard policy to be less important, but rather than it keeps being as important but it is the way they like it. This is very annoying and doesn’t scale to many different opinions.
-datacarriersize to have any practical relevance, you’ll have to make it mandatory.
There is also a matter of driving competent design rather than lazy first thing that works. E.g. In stealth addresses the early proposals use highly inefficient single ECDH point per output instead of simply pooling them. Network behavior is one of the few bits of friction driving good technical design rather than “move fast, break things, and force everyone else onto my way of doing thing rather than discussing the design in public”. No one wants to be an outright gatekeeper, but the network is a shared resource and it’s perfectly reasonable node behavior to be stingy about the perpetual storage impact of the transactions they’re willing to process, especially when it comes to neutral technical criteria like the amount of network irrelevant data stuffed in transactions.
There is also a very clear pattern we’ve seen in the past where people take anything the system lets them do as strong evidence that they have a irrevocable right to use the system in that way, and that their only responsibility– and if their usage harms the system it’s the responsibility of the system to not permit it. This is especially concerning since external data that isn’t necessary for the operation of the system comes with a liability that people may try ordering nodes or miners to censor the data (e.g. when it’s being used as a botnet control channel acting as a high bandwidth channel for “illegal information”). For mitigating these risks it’s optimal if transactions seem as uniform and indistinguishable as reasonably possible.
… surprising that the discussions continues.
It’s similar to other policy decisions such as default fees or permitting bare multisig. maxdatacarriersize is certainly a step in the right direction, but as long as there is a relation between default values and the actual behavior of the network, it continues to have relevance and any decision or non-decision in this context has consequences.
There is also a matter of driving competent design rather than lazy first thing that works.
The intend to push innovation, so to speak, is great and I like it.
There is also a very clear pattern we’ve seen in the past where people take anything the system lets them do as strong evidence that they have a irrevocable right to use the system in that way, and that their only responsibility– and if their usage harms the system it’s the responsibility of the system to not permit it.
True, this is the case, but this is related to the point I was trying to make earlier: OP_RETURN in it’s current form is not really competitive against other data carrying mechanisms such as using bare multisig, and this abuse probably won’t stop unless OP_RETURN gains an edge. The alternative, making bare multisig non-standard as proposed by Luke, may lead to even more harmful results, namely switching back to the creation of non-redeemable outputs, which is also cheaper in terms of effective transaction size/payload size for larger payloads.
@dexX7 those are good points but imagine we do both: make bare multisig non-standard and change MAX_OP_RETURN_RELAY from 40 to 80 or 999999. Maybe we can also have different default constants for relay and mining. Will that make standard policy discussions end? I don’t think so. That’s why I prefer to focus on making policy code behavior more easily configurable rather than dedicate an unbound amount of time to get to the “objectively best policy”, which we know doesn’t exist. People have to be aware that the standard policy is conservative, probably over-conservative; and deal with it by having their own policies. Having a standard policy being too permissive kind of defeats the purpose of having a standard policy in the first place. So I think I agree with @luke-jr there (but I will rephrase it): if having parts of the standard policy that people who care about policy don’t like, and that causes people who care about policies implement or configure their own policies….maybe it is good to have “unpopular” things on the standard policy. @genjix I don’t know about @gmaxwell’s solution but the maximum number of OP_RETURN outputs in the standard policy can be made configurable too.
I’d just rather not create more policy-specific command-line options until the policy code is isolated. I wish people had less time for discussing the “correct” standard policy and more time to review and ack a trivial PRs like #5696 (sorry for linking that again, but it just comes to mind every time I comment on anything related to policy).
I will just use normal Bitcoin addresses and avoid all this headache. And I’ll be sure to include false positives so they can’t be blacklisted or detected with certainty.
All I hear is blaa blaa about shaping Bitcoin according to how you think it should be used rather than how people are using it driven by their needs. What kind of arbitrary limit is one OP_RETURN per tx for?
Bitcoin is not and never will be a payments system. It is money, or store of value and the blockchain is a multi-use tool for many applications and it should be embraced totally in that vein rather than having small minds and trying to keep it pulled down in some metaphorical stone age. Unencumber the protocol and let people be free to create.
You can create a decentralised web, advanced financial instruments, stocks/shares/futures/options, distributed markets, publishing systems and much more… if you’re that concerned about resources then you should vehemently oppose increasing/removing the block size limit. But many people are too driven by a view of what they already know trying to force Bitcoin to be that, rather than what it could be. Bitcoin is not credit cards and will always be a rubbish payments system, so long as it remains decentralised.
@jtimon: I fully agree, moving towards user chosen policies is highly valuable, and I also support a more restrictive approach in general, but it doesn’t seem to be an either-or choice. While new default values very likely won’t stop similar discussions, I do believe it can encourage or discourage certain behavior.
The reduction of OP_RETURN to 40 byte was based on the assumption that:
80, as the standard line length, is almost asking for “insert your graffiti message here”.
As it turned out, this backfired with services such as cryptograffiti.info or roughly 210k out of 260k bare multisig outputs that can be linked to data storage, whereby the number of all OP_RETURN uses on mainnet is still less than 20k. It’s probably only a matter of time until more libraries such as blockpop or Stone appear, which abstract data storage and decouple data from the underlying transport layer, which is likely going to be chosen based on metrics such as transaction costs and miner adoption.
I have very mixed feelings about it in general, but getting the inceives right and minimize disadvantages of cost per payload and a slightly higher confirmation delay of using OP_RETURN compared to alternatives seems reasonable.
80, as the standard line length, is almost asking for “insert your graffiti message here”.
Yes, that’s a quote from me. I was afraid that OP_RETURN would be used en-masse to send messages. It has been standard for quite a while now and that hasn’t happened, so I’m okay with increasing the size to 80 (as I’ve already posted earlier above).
Edit: Also this has enough ACKs, no clue why this is still open.
On Mon, Feb 02, 2015 at 08:43:08PM -0800, Gregory Maxwell wrote:
@genjix PeterTodd specced out an alternative that can just put many prefixes in a single output.
That alternative compromises on privacy for no good reason; I specced it only because you asked me too and said you’d do a pull-req with the necessary op-ret changes to make it work. You haven’t done that, so I’m assuming you’re not interested. In the meantime, I’d much rather use the version that doesn’t have silly compromises.
0.10?
Some empirical data from the coinsecrets.org database to contribute to the discussion. So far there have been 16154 OP_RETURNs on mainnet, as of block 341912.
Breakdown by protocol that we can identify: 4877 Open Assets 3015 Proof of Existence 1933 CoinSpark 600 BlockSign 5729 (other)
So for now the two main identifiable use cases are colored coins and hashing a document to timestamp its existence. There are still far fewer OP_RETURNs than multisigs used by Counterparty.
Average OP_RETURNs per block (grouped by 10,000 blocks): 20xxxx 0.0000 21xxxx 0.0000 22xxxx 0.0002 23xxxx 0.0000 24xxxx 0.0029 25xxxx 0.0127 26xxxx 0.0134 27xxxx 0.0195 28xxxx 0.0274 29xxxx 0.0332 30xxxx 0.1520 31xxxx 0.2667 32xxxx 0.4258 33xxxx 0.5170 34xxxx 0.7559
So there is clear growth but it’s not showing signs of being exponential.
Proof of existence doesn’t need op_return, seehttps://github.com/Blockstream/contracthashtool . In fact, there’s a form of hard to censor colored coins that uses a similar technique instead of putting data in the chain (I believe the most used name is “committed assets”.
If this gets into 0.10 I will be jealous. Why this can go in but not my commit removing the miner’s hashmeter, for example?
On 4 February 2015 07:40:13 GMT-08:00, “Jorge Timón” notifications@github.com wrote:
Proof of existence doesn’t need op_return, seehttps://github.com/Blockstream/contracthashtool .
That technique is quite dangerous to use, as it creates coins unrecoverable by your wallet seed.
If this gets into 0.10 I will be jealous. Why this can go in but not my commit removing the miner’s hashmeter, for example?
Because more people cared about this.
@jtimon I think the reasoning for adding this patch to 0.10 is much more straight forward: it limits “damage” being done by meta protocols that currently use more wasteful and bloaty encoding. Allowing them to move to OP_RETURN encoding sooner is better for everyone.
Having this go into 0.10 now would mean the eco system moves to relaying 80 opreturns faster than if we wait for 0.11 which might be 6 months away. Let’s face it, wide deployment of a new version takes considerable time after release, so sooner is better for everyone. Given that the change is uncontroversial technically speaking, I personally think it should go into 0.10 now.
@petertodd I didn’t create a PR for this because it’s a straightforward cherry-pick, and since all merges are done manually anyhow, it doesn’t create any more or less work for the maintainers to pull.
On 4 February 2015 10:44:55 GMT-08:00, Gregory Maxwell notifications@github.com wrote:
@petertodd The contract hash tool works just as well on DSA nonces.
Mind uploading an implementation then? Also last I checked with you that crypto was uncertain; a concern repeated by some non-Bitcoin cryptographers I ran the idea past.