Remove arbitrary restrictions on OP

petertodd commented at 1:33 am on July 23, 2023: contributor

Any number or size of data-carrying OP_RETURN outputs are allowed, and the -datacarrier option is removed. For those who want to limit data carrying outputs, -datacarriersize is still supported, and has the functionality of applying the specified data carrier limit as well as limiting the number of data carrying OP_RETURN outputs to one. If -datacarriersize=0 is set, no data carrying output is allowed.

Rational: there’s lots of ways for people to publish data in the Bitcoin chain, and lots of data has been published. There’s no reason for us to place artificial limits on this particular method.

Replaces #27261

Fix off-by-one error in data carrier size limit

Previously the OP_RETURN was counted as part of the size limit; the OP_RETURN
opcode itself is not data.

7dec186b48

DrahtBot commented at 1:33 am on July 23, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept NACK	luke-jr, recursive-rat4, Sjors, 1ma, RobinLinus, Retropex, ChrisMartl, Sun0fABeach, jesterhodl, davidlick, cbergqvist, HenrikJannsen, imacfan, ns-xvrn, dimitaracev, BBezaire, gzuuus
Concept ACK	Ayms, ChristopherA, fiatjaf, hsjoberg, vicariousdrama, zndtoshi, itme-brain, dangershony

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#28132 (policy: Enable full-rbf by default by petertodd)
#27832 (doc: Clarify -datacarriersize, add -datacarriersize=2 tests by MarcoFalke)
#26525 (Remove -mempoolfullrbf option by BitcoinErrorLog)
#26291 (Update MANDATORY_SCRIPT_VERIFY_FLAGS by ajtowns)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

luke-jr commented at 2:15 am on July 23, 2023: member

Concept NACK, the spam filters should be fixed, not removed or relaxed.

DrahtBot added the label CI failed on Jul 23, 2023

recursive-rat4 commented at 10:24 am on July 23, 2023: none

NACK until the data is discardable and the block size is increased.

bitcoin deleted a comment on Jul 23, 2023

Ayms commented at 11:32 am on July 23, 2023: none

Concept ACK bitcoin surpasses eth in every domain except storage Current storage methods need two txs and many deviant methods were used in the past to store things in bitcoin, as we saw recently also

petertodd force-pushed on Jul 23, 2023

luke-jr commented at 4:07 pm on July 23, 2023: member

except storage

Blockchains don’t do storage. There’s a tool for that called ext4, and BitTorrent for distribution.

ChristopherA commented at 5:51 pm on July 23, 2023: none

Concept ACK. Current limits on op_return force people to use hacks to store data hiding in op_codes. Using op_return is more `honest’. In my own case, current op_return limits keep me from posting a signed hash plus some metadata (<256 bytes) and thus I must use Taproot tricks instead.

Ayms commented at 10:56 pm on July 23, 2023: none

@luke-jr the idea is not to store a video on bitcoin but indeed a reference to it like bittorrent Now if someone wants to store things in a full block nobody can avoid this Then this change just make things easier to avoid unwanted practices

MarcoFalke commented at 7:58 am on July 27, 2023: member

From CI:

0test/functional/mempool_datacarrier.py:31: error: Bracketed expression "[...]" is not valid as a type  [valid-type]
1test/functional/mempool_datacarrier.py:31: note: Did you mean "List[...]"?
2Found 1 error in 1 file (checked 269 source files)
3^---- failure generated from lint-python.py

in src/script/standard.h:39 in c7f2ab0f56 outdated

31@@ -32,12 +32,6 @@ class CScriptID : public BaseHash<uint160>
32     explicit CScriptID(const ScriptHash& in);
33 };
34 
35-/**
36- * Default setting for -datacarriersize. 80 bytes of data, +1 for OP_RETURN,
37- * +2 for the pushdata opcodes.
38- */
39-static const unsigned int MAX_OP_RETURN_RELAY = 83;

Sjors commented at 11:21 am on July 28, 2023:

Dropping a constant but then hardcoding it all over the place with no historical context is not a good idea. That’s another reason to just keep the default for now.

Ayms commented at 2:12 pm on July 28, 2023:

Can you please explain why ?

Sjors commented at 10:57 am on July 29, 2023:

Because future developers won’t (easily) know where the number comes from.

petertodd commented at 11:56 am on July 29, 2023:

Dropping a constant but then hardcoding it all over the place

Where have I hardcoded it? I deliberately used a different constant in some of the updated tests to make clear that after this change, it’s just a number. In the others - just 4 lines - I can of course just add a comment in those few lines explaining what the number is.

Sjors commented at 12:40 pm on July 29, 2023:

In transaction_tests it’s either the same number of +-1 which is confusing (can be solved with a comment), but I see in other places you made it 100 which is indeed more clear.

petertodd commented at 4:33 pm on July 29, 2023:

I reworked those unit tests with better comments and clearer constants, and expanded the test coverage a bit.

Sjors changes_requested

Sjors commented at 11:21 am on July 28, 2023: member

Concept NACK unless you keep the default at MAX_OP_RETURN_RELAY. Depending on adoption we can always drop that default later. No need to make pull requests more controversial than necessary.

petertodd force-pushed on Jul 29, 2023

petertodd commented at 11:59 am on July 29, 2023: contributor

Concept NACK unless you keep the default at MAX_OP_RETURN_RELAY. Depending on adoption we can always drop that default later. No need to make pull requests more controversial than necessary. @ChristopherA is paying me $1000 for 10 hours work to do this pull-req. Since the other sub-change got mired in controversy, I decided to meet my client requirement in a time-efficient manner. Obviously, I can’t promise to him that this will actually get merged. But I can promise to him that I’ll do a mergable pull-req in the time allotted.

So frankly, at this point I don’t care to split it up into two pull-reqs for such a minor reason. If someone else wants to, they can take this open-source licensed code and do it themselves.

petertodd force-pushed on Jul 29, 2023

petertodd commented at 12:06 pm on July 29, 2023: contributor

From CI:

0test/functional/mempool_datacarrier.py:31: error: Bracketed expression "[...]" is not valid as a type  [valid-type]
1test/functional/mempool_datacarrier.py:31: note: Did you mean "List[...]"?
2Found 1 error in 1 file (checked 269 source files)
3^---- failure generated from lint-python.py

From CI:

0test/functional/mempool_datacarrier.py:31: error: Bracketed expression "[...]" is not valid as a type  [valid-type]
1test/functional/mempool_datacarrier.py:31: note: Did you mean "List[...]"?
2Found 1 error in 1 file (checked 269 source files)
3^---- failure generated from lint-python.py

Hmm, I changed it to List[int], which still isn’t working. Anyone with more Python knowledge know what it should be? https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html suggests that list[int] should be valid. But the lint still complained about that too.

willcl-ark commented at 12:15 pm on July 29, 2023: contributor

I’m on mobile and didn’t check the pull, but do you need to from typing import List too?

Sjors commented at 12:49 pm on July 29, 2023: member

So frankly, at this point I don’t care to split it up into two pull-reqs for such a minor reason.

Making a significant change to default policy is not a minor reason.

is paying me $1000 for 10 hours work to do this pull-req

That’s great, but others have to spend hours reviewing this stuff, plus occasionally having to deal with the brigading. I’m more likely to do that for a change that keeps the default, but I’m not the only one who can review of course.

petertodd commented at 3:56 pm on July 29, 2023: contributor

So frankly, at this point I don’t care to split it up into two pull-reqs for such a minor reason.

Making a significant change to default policy is not a minor reason.

This is not a significant change. Large amounts of data are already published in the bitcoin blockchain in a variety of ways. Heck, -permitbaremultisig is still allowed by default, and people still use it to publish data in the form of spendable and even unspendable outputs. And unlike OP_Return outputs, bare multisig outputs have caused issued due to Bitcoin’s broken sigop counting scheme; the OP_Return mechanism was carefully designed to avoid this problem years ago.

This proposed change simply cleans up a bit of remaining standardness paternalism; we’ve gotten rid of essentially all other standardness paternalism. I can’t think of any standardness mechanism other than OP_Return and first-seen (which I’m arguing to remove by default in #28132) that doesn’t have a strong technical reason to exist.

is paying me $1000 for 10 hours work to do this pull-req

That’s great, but others have to spend hours reviewing this stuff, plus occasionally having to deal with the brigading. I’m more likely to do that for a change that keeps the default, but I’m not the only one who can review of course.

As I explain above, this is not a significant change and should not require hours of technical review.

petertodd force-pushed on Jul 29, 2023

Remove arbitrary restrictions on OP_RETURN by default

Any number or size of data-carrying OP_RETURN outputs are allowed, and the
`-datacarrier` option is removed. For those who want to limit data carrying
outputs, `-datacarriersize` is still supported, and has the functionality of
applying the specified data carrier limit as well as limiting the number of
data carrying OP_RETURN outputs to one. If `-datacarriersize=0` is set, no data
carrying output is allowed.

Rational: there's lots of ways for people to publish data in the Bitcoin chain,
and lots of data has been published.  There's no reason for us to place
artificial limits on this particular method.

50fc9f697b

petertodd force-pushed on Jul 29, 2023

petertodd commented at 4:45 pm on July 29, 2023: contributor

@willcl-ark

I’m on mobile and didn’t check the pull, but do you need to from typing import List too?

Thanks! I believe that fixed the issue.

Sjors commented at 6:18 pm on July 29, 2023: member

@ChristopherA wrote:

In my own case, current op_return limits keep me from posting a signed hash plus some metadata (<256 bytes) and thus I must use Taproot tricks instead. @petertodd wrote:

Large amounts of data are already published in the bitcoin blockchain in a variety of ways.

This seems like something to propose on the bitcoin-dev mailinglist, like it was ten years ago. You could recap the original arguments for the 80 byte limit, that (since SegWit?) inscriptions and similar methods already allow adding more. Also remind the reader what the effective size limit will be without that limit and how that compares to inscriptions.

Mention a few use cases that require more than 80 bytes (at the time IIRC nobody did, the discussion was only between 40 and 80).

Can it be used to pin transactions in a way that unupgraded nodes (including poorly maintained alternative implementations that are potentially used in second layer protocols) wouldn’t notice?

remaining standardness paternalism

You could add that argument too.

DrahtBot removed the label CI failed on Jul 29, 2023

Ayms commented at 11:39 am on July 30, 2023: none

As different people explained this change is really needed and really necessary for the future of bitcoin Remember that some nacking this did store in addresses It s too easy to store anywhere you want and even more easy if you collude with a miner Regarding the 80 B number I don t understand where it comes from you can t even store a signature and a hash with this

I am on mobile too so sorry for the typos strange to see how shxtty is github on mobile

petertodd commented at 4:01 pm on July 30, 2023: contributor

@ChristopherA wrote:

In my own case, current op_return limits keep me from posting a signed hash plus some metadata (<256 bytes) and thus I must use Taproot tricks instead.

@petertodd wrote:

Large amounts of data are already published in the bitcoin blockchain in a variety of ways.

This seems like something to propose on the bitcoin-dev mailinglist, like it was ten years ago. You could recap the original arguments for the 80 byte limit, that (since SegWit?) inscriptions and similar methods already allow adding more. Also remind the reader what the effective size limit will be without that limit and how that compares to inscriptions.

Mention a few use cases that require more than 80 bytes (at the time IIRC nobody did, the discussion was only between 40 and 80). @ChristopherA You’re welcome to respond to this, here or on the mailing list.

Personally I think the continued existence of so many other data publications mechanisms, including clearly problematic ones like UTXO-creating bare-multisig, puts the burden of proof on arguing that we should have an arbitrary 80 byte limit.

Can it be used to pin transactions in a way that unupgraded nodes (including poorly maintained alternative implementations that are potentially used in second layer protocols) wouldn’t notice?

I don’t consider that to be a valid complaint. Any difference in relay policy can do that. We do not bring up that objection every single time we choose to tweak relay policy. Protocols that are so fragile that they can’t tolerate minor differences in relay policy are simply broken and should be fixed.

Sjors commented at 10:21 am on July 31, 2023: member

puts the burden of proof on arguing that we should have an arbitrary 80 byte limit.

Perhaps. But that’s not an argument to catch them by surprise.

Any difference in relay policy can do that.

I’m not sure if that’s true. The introduction of new soft forks like Taproot do allow this to happen, but that’s not a minor tweak and it came with significant benefit.

In any case you can just state the above on the mailinglist and see if anyone disagrees. Other proposed policy tweaks, like package relay, have first been discussed on the mailinglist too.

Sjors commented at 11:50 am on July 31, 2023: member

Even assuming pinning was an issue, it wouldn’t be a blocker. It could simply be released at the same time as some other policy change (package relay, #27677, v3 transactions, the next soft fork, etc). Or even just with sufficient heads-up.

Ayms commented at 6:32 pm on August 1, 2023: none

Not on mobile any longer, then I read the 10 years article, counterparty was right, we are 10 years late, then many switched on eth The 40 or 80B limit was based on storing a hash or two, surprisingly nobody thought at that time that you must store a signed hash as @ChristopherA and myself wants to do and probably many others, plus some metadata, which is not possible today unless you know a miner, storing a hash or two is of no use, or just leads to a very centralized system, like eth sidechains/tokens, or some stupid system like Tether storing in bitcoin unprovable things some years ago There are plenty of use cases, not putting the links to my NFT or Swap proposals because nobody will read them So it’s surprising again to see how some (who again did store in addresses in the past @luke-jr ) are against this Imposing a limit is of no use also as stated by @petertodd, like many I can easily bypass whatever limits, needing two txs instead of one, flooding the network with useless txs, unless I store in addresses, then …. Ten years…

1ma commented at 4:42 pm on August 2, 2023: none

So there’s this thing called AWS S3… Bitcoin is about moving sats around, not freeriding on a global network of voluntary node runners to store your arbitrary data for free.

Enacting this change of policy would be reason enough for me to stop updating Bitcoin Core forever. If anything node runners are asking for ways to filter the recent tidal wave of spam while it is still unconfirmed, not surrender to it.

Concept NACK.

petertodd commented at 11:45 am on August 3, 2023: contributor

Even assuming pinning was an issue, it wouldn’t be a blocker. It could simply be released at the same time as some other policy change (package relay, #27677, v3 transactions, the next soft fork, etc). Or even just with sufficient heads-up.

I’ve sent an email to the bitcoin-dev mailing list announcing this proposed change: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2023-August/021840.html

petertodd commented at 11:49 am on August 3, 2023: contributor

Enacting this change of policy would be reason enough for me to stop updating Bitcoin Core forever. If anything node runners are asking for ways to filter the recent tidal wave of spam while it is still unconfirmed, not surrender to it.

FYI if merged, this pull-req would allow you to continue to limit or block data-carrying OP_Return outputs by setting -datacarriersize=n or -datacarriersize=0 respectively. Of course, if a non-trivial minority of nodes and miners choose to relay and mine such transactions, you blocking them will have no impact as they will be propagated and mined anyway.

RobinLinus commented at 12:23 pm on August 3, 2023: none

Concept NACK

This will most likely increase the spam problems significantly.

petertodd commented at 12:50 pm on August 3, 2023: contributor

Concept NACK

This will most likely increase the spam problems significantly.

Since OP_Return outputs to not benefit from the segwit discount, publishing data via them is about 4x more expensive than publishing data in witness space. Eg via the taproot mechanism used by inscriptions.

Why do you believe that expanding a publication mechanism that is 4x more expensive than the heavily used taproot witness mechanism will “increase the spam problems significantly”?

in src/init.cpp:592 in 50fc9f697b

588@@ -589,8 +589,7 @@ void SetupServerArgs(ArgsManager& argsman)
589     argsman.AddArg("-dustrelayfee=<amt>", strprintf("Fee rate (in %s/kvB) used to define dust, the value of an output such that it will cost more than its value in fees at this fee rate to spend it. (default: %s)", CURRENCY_UNIT, FormatMoney(DUST_RELAY_TX_FEE)), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::NODE_RELAY);
590     argsman.AddArg("-acceptstalefeeestimates", strprintf("Read fee estimates even if they are stale (%sdefault: %u) fee estimates are considered stale if they are %s hours old", "regtest only; ", DEFAULT_ACCEPT_STALE_FEE_ESTIMATES, Ticks<std::chrono::hours>(MAX_FILE_AGE)), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
591     argsman.AddArg("-bytespersigop", strprintf("Equivalent bytes per sigop in transactions for relay and mining (default: %u)", DEFAULT_BYTES_PER_SIGOP), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
592-    argsman.AddArg("-datacarrier", strprintf("Relay and mine data carrier transactions (default: %u)", DEFAULT_ACCEPT_DATACARRIER), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);

MarcoFalke commented at 12:56 pm on August 3, 2023:

Any reason to remove this option? Seems fine to keep it as an alias to completely disable any OP_RETURN. I know it may be redundant with the other option, but for some reason most other devs and users like the UX of it, similar to -regtest and -chain=regtest, or other options.

petertodd commented at 1:04 pm on August 3, 2023:

I don’t think the redundancy is warranted going forwards.
Re: backwards compatibility, I expect that only a very small number of people have set -datacarrier=0, so I’m fine with those few people having to set -datacarriersize=0.

Though if there is strong demand, I could make -datacarrier=0 act as an override on -datacarriersize, forcing the latter to zero.

MarcoFalke commented at 1:10 pm on August 3, 2023:

I think you also changed -datacarriersize to always be +1, so setting it to zero has a different meaning.

The cost seems low to keep the meaning and features the same, so personally I think it is fine to keep them the same.

In any case, release notes are needed

for all removed features,
for all changed meanings,
for all lifted restrictions, and
for all default value changes.

petertodd commented at 1:34 pm on August 3, 2023:

I think you also changed -datacarriersize to always be +1, so setting it to zero has a different meaning.

The cost seems low to keep the meaning and features the same, so personally I think it is fine to keep them the same.

The difference in meaning here is what I would consider to be a bug fix. Previously these options also applied to non-data-carrying OP_Return outputs, which doesn’t make any sense. Anyone actually using -datacarriersize would just increment the amount accepted by 1 byte, which is insignificant.

In any case, release notes are needed

* for all removed features,

* for all changed meanings,

* for all lifted restrictions, and

* for all default value changes.

Sure, I can write those. I take it the mechanism used by this pull-req is correct? https://github.com/bitcoin/bitcoin/pull/27757/files

1ma commented at 1:03 pm on August 3, 2023: none

FYI if merged, this pull-req would allow you to continue to limit or block data-carrying OP_Return outputs by setting -datacarriersize=n or -datacarriersize=0 respectively. Of course, if a non-trivial minority of nodes and miners choose to relay and mine such transactions, you blocking them will have no impact as they will be propagated and mined anyway.

I understand that, it’s a matter of trust. What I mean is that if the Bitcoin developers end up sanctioning such an obvious sabotage on the system I won’t trust them as a group to continue developing the reference implementation.

petertodd commented at 1:07 pm on August 3, 2023: contributor

I understand that, it’s a matter of trust. What I mean is that if the Bitcoin developers end up sanctioning such an obvious sabotage on the system I won’t trust them as a group to continue developing the reference implementation.

We still “sanction” even more “obvious sabotage” by the fact that bare multisig outputs are relayed by default. If you have a complaint, I’d suggest you start there rather than on OP_Return, a mechanism designed specifically for low impact and vigorously discussed years ago.

1ma commented at 1:26 pm on August 3, 2023: none

Yeah, I know all about it. To this day Bitcoin Core still relays bare multisig TXs by default because back when it was considered making them non-standard Mike Hearn weighted in with a baseless objection: #5231

Now that you mention it maybe it would be a good time to reconsider that again, though that belongs on another PR.

instagibbs commented at 2:29 pm on August 3, 2023: member

Can it be used to pin transactions in a way that unupgraded nodes (including poorly maintained alternative implementations that are potentially used in second layer protocols) wouldn’t notice?

I don’t see how this would result in pinning. Pinning is of two general flavors:

economic/rule#3 pinning: costly to RBF even though miner incentive would be to pick it up
package limit pinning: ancestor/descendant size/count limits are hit by adversary, so you cannot CPFP

I’m not sure how this would result in either of these cases

instagibbs commented at 2:58 pm on August 3, 2023: member

This proposed change simply cleans up a bit of remaining standardness paternalism; we’ve gotten rid of essentially all other standardness paternalism. I can’t think of any standardness mechanism other than OP_Return and first-seen (which I’m arguing to remove by default in #28132) that doesn’t have a strong technical reason to exist.

I’m not going to war for this change but this strikes me as correct. I classified the OP_RETURN limit as the last remaining “paternalism” restriction in the codebase, since you can’t even over-write how many datacarrier outputs are allowed via config.

Wrote about policies here: https://gist.github.com/instagibbs/ee32be0126ec132213205b25b80fb3e8

RobinLinus commented at 2:59 pm on August 3, 2023: none

Why do you believe that expanding a publication mechanism that is 4x more expensive than the heavily used taproot witness mechanism will “increase the spam problems significantly”? @ChristopherA just said that he’s paying you to push this because it makes it cheaper and easier for him to write his non-bitcoin stuff into the chain.

DrahtBot added the label Needs rebase on Aug 3, 2023

DrahtBot commented at 5:12 pm on August 3, 2023: contributor

🐙 This pull request conflicts with the target branch and needs rebase.

ChristopherA commented at 9:10 pm on August 3, 2023: none

Why do you believe that expanding a publication mechanism that is 4x more expensive than the heavily used taproot witness mechanism will “increase the spam problems significantly”?

@ChristopherA just said that he’s paying you to push this because it makes it cheaper and easier for him to write his non-bitcoin stuff into the chain.

Some clarification: I said I would contribute $1K toward a bounty for a real PR. Others said they might contribute to that bounty as well. My real goal was to have a legitimate discussion, which is what we have been having here. It is a discussion in PRs, with working code, that is how change really happens in bitcoin.

My goal is not “non-bitcoin stuff”. My specific challenge is that Lightning and Nostr use public keys in a persistent way that are not rotatable, which is a bad cryptographic practice. By committing to a rotated key and some other cryptographic in advance in L1, they could be. I believe this could be as small as a signed hash and a little metadata. However, this is larger than 80-byte limit in OP_RETURN. I can now put it into an inscription-style transaction and even get a 4x discount, or use bare multisig, or use some other trick.

However, I’d prefer not to use a “trick”, and use OP_RETURN as it is easy to identify, was explicitly created for this purpose, is not a hack, and nodes can easily ignore them. Given that the OP_RETURN limit is a relic of is_standard, and as @instagibbs said is “the last remaining ‘paternalism’ restriction in the codebase”, and there are multiple ways to do what I’m seeking in less “honest” ways, so why have this legacy limit at all? Thus my reasoning to contribute to the bounty for this PR.

If @petertodd doesn’t have time to continue to continue to support this PR, I’m glad to contribute toward bounties (hopefully with others) for others to keep it active.

– Christopher Allen, Blockchain Commons

ChristopherA commented at 9:17 pm on August 3, 2023: none

A question to those who object to this PR, would any of you change your mind if a new is_standard was simply a little larger? Maybe a modest 256 or 512 bytes?

Though I believe there are other reasons OP_RETURN shouldn’t be limited, if advancing a modest limit is an option, it is at least a way for devs who “want to do the right thing” rather than enormous inscriptions and other hacks, I would support it.

– Christopher Allen

1ma commented at 11:26 pm on August 3, 2023: none

My goal is not “non-bitcoin stuff”. My specific challenge is that Lightning and Nostr use public keys in a persistent way that are not rotatable, which is a bad cryptographic practice. By committing to a rotated key and some other cryptographic in advance in L1, they could be. I believe this could be as small as a signed hash and a little metadata. @ChristopherA can you clarify why is it imperative for this use case to store that data on the Bitcoin blockchain?

To give you some insight into my point of view: I’m a Bitcoin user and full node runner and my primary concern is that running a full node continues to be affordable for the rest of the 21st century, as it’s necessary to maintain a global network of independent validators & copies of the blockchain. Storing non-transactional data in the blockchain makes this arbitrarily more difficult. Ceteris paribus if we encourage using the Bitcoin blockchain as a storage medium there will be much less full nodes in 2070 than otherwise. Personally I’d like to see the devs working on new standardness rules to fight the spam, not removing the existing ones.

So I hope you’ll understand why I don’t agree with removing -datacarrier nor increasing the default -datacarriersize, but I’m willing to live by the status quo. I’d like to ask you to try to work around the 80 byte limit if at all possible, maybe with a more compact hash and metadata encoding scheme.

RobinLinus commented at 4:26 am on August 4, 2023: none

I can now put it into an inscription-style transaction and even get a 4x discount

This is not true. Having to use two transactions leads to about 150 bytes of overhead. Removing the spam protection for op_return outputs effectively results in a discount for small inscriptions.

Furthermore, you’re saying yourself that you want this change because it makes it easier for you to use the chain for contentious non-bitcoin use cases such as inscribing Nostr keys. However, the reason why you currently have to use hacks to do that is simply because others just don’t want to have to download and store your data to use Bitcoin. The fact that we can’t prevent these kind of spam attacks is not a justification to make the problem worse.

Also I wonder why you don’t use torrents to store your data?

Retropex commented at 11:30 am on August 4, 2023: none

Concept NACK.

We already have enough problems with inscriptions (That the developer still seems to ignore). In addition, why delete the -datacarrier option? This is an important option that gives nodes runner the choice, you could have simply removed the default limit.

Instead, focus on giving nodes runner the choice by implementing more options to customize the mempool policy.

iBeizsley commented at 12:06 pm on August 4, 2023: none

The 40 or 80B limit was based on storing a hash or two, surprisingly nobody thought at that time that you must store a signed hash as @ChristopherA and myself wants to do

Any hash you include in an op return is a signed hash. It’s signed by the keys controlling the inputs.

ChrisMartl commented at 12:19 pm on August 4, 2023: none

Concept NACK.

Bitcoin system faces the struggles of control taking; reduction of decentralization into an increment of centralization.

Some interested groups for that vision are striving to become “the” nodes of Bitcoin, uniting again mining and storage activities while wiping out the non-mining nodes via equipment and operation cost increment; one of that, is increasing the storage cost due to insertion of arbitrary data.

Sun0fABeach commented at 1:14 pm on August 4, 2023: none

Rational: there’s lots of ways for people to publish data in the Bitcoin chain, and lots of data has been published. There’s no reason for us to place artificial limits on this particular method.

Concept NACK

There is plenty of reasons to place limits on any method that abuses the timechain as data storage. Any relaxation of the default settings validates existing spam and is unacceptable. Give node runners strong defaults and options to mitigate current spam strategies before doing any of these tweaks.

luke-jr commented at 1:23 pm on August 4, 2023: member

the idea is not to store a video on bitcoin but indeed a reference to it like bittorrent

You can already do that within the current datacarriersize. Removing the limit has no purpose other than gratuitious abuse.

As different people explained this change is really needed and really necessary for the future of bitcoin

This is completely false and nonsensical. It does literally nothing good for Bitcoin, only harms Bitcoin.

Regarding the 80 B number I don t understand where it comes from you can t even store a signature and a hash with this

The transaction signature is somewhere else already. You don’t need it here. A hash is more than sufficient. The logical limit would be 40 bytes.

(who again did store in addresses in the past @luke-jr )

Are you slandering me? Get this crap off github.

Though I believe there are other reasons OP_RETURN shouldn’t be limited, if advancing a modest limit is an option, it is at least a way for devs who “want to do the right thing” rather than enormous inscriptions and other hacks, I would support it.

OP_RETURN abuse is NOT the right thing. The right thing is never putting data on Bitcoin at all. OP_RETURN over 40 bytes is doing the WRONG thing.

fiatjaf commented at 1:58 pm on August 4, 2023: none

Concept ACK.

https://stampchain.io/ is a thing and it’s still growing. These OP_RETURN limits are looking more and more like a legacy broken compromise.

jesterhodl commented at 3:49 pm on August 4, 2023: none

As a Bitcoin Node runner, Lightning Node operator, Nostr Relay operator and Nostr user, I recognize the problem @ChristopherA is looking to alleviate.

I recognize the problem is in the realm of authentication where cryptographic primitives and an immutable storage could be one option to pursue for a potential solution.

As a Bitcoiner I Concept NACK because:

Timechain is for monetary data, other uses should forever stay as tolerable, low volume noise, or it will decrease our capacity to forever keep optimizing for the monetary use case
authentication and general purpose storage is outside of Bitcoin
it is conducive to the spam problem
increases storage requirements and tilts towards centralization
as we’ve seen with Taproot, unforeseen consequences of interventions (Iatrogenics) is something we missed in the past. Perhaps you can trial this elsewhere first?

rot13maxi commented at 4:03 pm on August 4, 2023: none

For readers of the thread who are not familiar with the STAMPS community, people are going around the OP_RETURN limit and doing things like embedding data in bare multisig: https://mempool.space/tx/1c61f2d4000e1bc2ce27ec24e4fcbbb376c2bd19641e071403e68d84d48eea8a This creates many unprunable outputs and takes up more blockspace, so its bad for block capacity and utxoset bloat.

RobinLinus commented at 4:27 pm on August 4, 2023: none

This creates many unprunable outputs

This is their pitch though “Unprunable UTXO Art” . They are intentionally spamming the chain. Relaxing bitcoin’s spam mitigations doesn’t help against that.

1ma commented at 4:46 pm on August 4, 2023: none

For readers of the thread who are not familiar with the STAMPS community, people are going around the OP_RETURN limit and doing things like embedding data in bare multisig: https://mempool.space/tx/1c61f2d4000e1bc2ce27ec24e4fcbbb376c2bd19641e071403e68d84d48eea8a This creates many unprunable outputs and takes up more blockspace, so its bad for block capacity and utxoset bloat.

The stamps “community” can only route around OP_RETURN because back in 2014 Mike Hearn derailed the discussion to make them non-standard, but as I said yesterday it’s never too late to reopen that discussion and start discarding them by default. #5231 (comment)

Bitcoin Core is able to discard and not relay unconfirmed stamp transactions since March 2014: #3939. For readers of the thread who are not familiar, you can stop relaying stamps with your node by setting the permitbaremultisig=0 flag in bitcoin.conf.

I feel very strongly that the way to deal with the spam is not caving in to the spammers, but nudge them into using the blockchain in less damaging ways. In the case of stamps they should be nudged over to OP_RETURN by making bare multisig non-standard (nobody used bare multisig for multisig even in 2014).

RicYashiroLee commented at 5:00 pm on August 4, 2023: none

IMV, this is just another massive example of altcoining mindset, of many wishful thinking ideas detrimental to Bitcoin Full Node Operators, and only giving, as a direct consequence, more power to a central hub of Core Devs who are in high risk of conflict of interests, as he is included in such risk group of service providers and centralizers, ‘scaling’ business oriented. Full Node operators, on the contrary, are the only true decentralizers and the only ones to decide autonomously to restrict even further, op_return Trojan horse in Bitcoin, a gate for massive arbitrary hash-anchoring to Bitcoin, NOT core to its primary scope. It is not because a weak door in a house does not stop all burglars, that we should then take the door out and leave the house wide open…

RicYashiroLee commented at 5:25 pm on August 4, 2023: none

For readers of the thread who are not familiar with the STAMPS community, people are going around the OP_RETURN limit and doing things like embedding data in bare multisig: https://mempool.space/tx/1c61f2d4000e1bc2ce27ec24e4fcbbb376c2bd19641e071403e68d84d48eea8a This creates many unprunable outputs and takes up more blockspace, so its bad for block capacity and utxoset bloat.

The key word IMV is making things CUMBERSOME for any out of scope use of Bitcoin. Never more relaxed. And enable Node operators the decision of what to support and not, by having the full nodes soware with such options, the more the better decentralized Bitcoin becomes. Core=centralized. The smaller a CORE that allows Bitcoin to run exclusively for its primary scope, the more decentralized Bitcoin becomes. The idea of CORE Devs deciding always on node operators behalf is to me obviously damaging to Bitcoin. Devs don’t decide, they as service providers, provide options to the clients of Bitcoin hopefully node operators if they are able to run a tiny full node if they so wish by unsupporting all superfluous, and not an overweighted node.

rot13maxi commented at 5:44 pm on August 4, 2023: none

If you believe that core shouldnt be making policy decisions on behalf of users, then shouldnt the default be that there is no default policy and that users can set it to whatever they want? Thats what this PR would do (for this policy item).

On Fri, Aug 4, 2023 at 1:25 PM, Richard Yashiro Lee @.***(mailto:On Fri, Aug 4, 2023 at 1:25 PM, Richard Yashiro Lee «a href=)> wrote:

For readers of the thread who are not familiar with the STAMPS community, people are going around the OP_RETURN limit and doing things like embedding data in bare multisig: https://mempool.space/tx/1c61f2d4000e1bc2ce27ec24e4fcbbb376c2bd19641e071403e68d84d48eea8a This creates many unprunable outputs and takes up more blockspace, so its bad for block capacity and utxoset bloat.

The key word IMV is making things cumbersom for any out of scope use of Bitcoin, not more relaxed. And enable Node operators the decision of what to support and not, by having the full nodes soware with such options, the more the better decentralized Bitcoin becomes. Core=centralized. The smaller a CORE that allows Bitcoin to run exclusively for its primary scope, the more decentralized Bitcoin becomes. The idea of CORE Devs deciding always on node operators behalf is to me obviously damaging to Bitcoin. Devs don’t decide, they as service providers, provide options to the clients of Bitcoin hopefully node operators if they are able to run a tiny full node if they so wish by unsupporting all superfluous, and not an overweighted node.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

RicYashiroLee commented at 5:51 pm on August 4, 2023: none

If you believe that core shouldnt be making policy decisions on behalf of users, then shouldnt the default be that there is no default policy and that users can set it to whatever they want? Thats what this PR would do (for this policy item). … On Fri, Aug 4, 2023 at 1:25 PM, Richard Yashiro Lee @.(mailto:On Fri, Aug 4, 2023 at 1:25 PM, Richard Yashiro Lee «a href=)> wrote: > For readers of the thread who are not familiar with the STAMPS community, people are going around the OP_RETURN limit and doing things like embedding data in bare multisig: https://mempool.space/tx/1c61f2d4000e1bc2ce27ec24e4fcbbb376c2bd19641e071403e68d84d48eea8a This creates many unprunable outputs and takes up more blockspace, so its bad for block capacity and utxoset bloat. The key word IMV is making things cumbersom for any out of scope use of Bitcoin, not more relaxed. And enable Node operators the decision of what to support and not, by having the full nodes soware with such options, the more the better decentralized Bitcoin becomes. Core=centralized. The smaller a CORE that allows Bitcoin to run exclusively for its primary scope, the more decentralized Bitcoin becomes. The idea of CORE Devs deciding always on node operators behalf is to me obviously damaging to Bitcoin. Devs don’t decide, they as service providers, provide options to the clients of Bitcoin hopefully node operators if they are able to run a tiny full node if they so wish by unsupporting all superfluous, and not an overweighted node. — Reply to this email directly, [view it on GitHub](#28130 (comment)), or unsubscribe. You are receiving this because you commented.Message ID: @.>

This goes to the fundamental idea of what CORE means in Bitcoin. As I see it: whatever smallest code and setup which allows Bitcoin to fulfill its basic scope = CORE. All the rest is ALTCOINING, promoted by wishful thinkers and a central hub of ‘Core’ Devs =service providers, not stock owners of Bitcoin (watch out for the SEC).

RicYashiroLee commented at 5:59 pm on August 4, 2023: none

That is a twisted logic. It is not because it is already bad that anyone is centrally allowed to make it worse.

nikicat commented at 6:01 pm on August 4, 2023: none

The individuals opposing this PR, believing they protect Bitcoin from centralization, ironically desire to dictate to all other people how they should and should not use Bitcoin. This type of standardness paternalism could eventually prompt the creation of other ways to include non-standard transactions in the blockchain, like a transaction marketplace for miners, which would totally eliminate standardness checks as a way to regulate Bitcoin.

RicYashiroLee commented at 6:09 pm on August 4, 2023: none

I understand that, it’s a matter of trust. What I mean is that if the Bitcoin developers end up sanctioning such an obvious sabotage on the system I won’t trust them as a group to continue developing the reference implementation.

We still “sanction” even more “obvious sabotage” by the fact that bare multisig outputs are relayed by default. If you have a complaint, I’d suggest you start there rather than on OP_Return, a mechanism designed specifically for low impact and vigorously discussed years ago.

IMV Op_return was an altcoining experiment at Bitcoin’s inception, when Satoshi was still figuring out all possible uses of such type of sui generis supposedly permanently tiny DB called Bitcoin. Just that, Bitcoin DB is already assigned to a specific scope, all arbitrary other uses are NOT for its full nodes to compute/validate neither store.

RicYashiroLee commented at 6:25 pm on August 4, 2023: none

The individuals opposing this PR, believing they protect Bitcoin from centralization, ironically desire to dictate to all other people how they should and should not use Bitcoin. This type of standardness paternalism could eventually prompt the creation of other ways to include non-standard transactions in the blockchain, like a transaction marketplace for miners, which would totally eliminate standardness checks as a way to regulate Bitcoin.

Every supposedly weakness of Bitcoin (IMV any existing were created by previous altcoining, like this is now another example of) has to be dealt individually. It is NOT by creating even more weaknesses somewhere else in it, that we solve other.

iBeizsley commented at 6:27 pm on August 4, 2023: none

If you believe that core shouldnt be making policy decisions on behalf of users, then shouldnt the default be that there is no default policy and that users can set it to whatever they want? Thats what this PR would do (for this policy item).

A value of infinity is still a value. You’re suggesting a change from current default policy. You’re making a policy decision on behalf of others. Is the current state ideal? Maybe, maybe not. But it’s the state we have. To change from it, we should have a good reason.

It should be configurable. We already have configuration options (one of which this PR removes). If a change is needed, it’s just to add a configuration option for the remaining hardcoded values.

RicYashiroLee commented at 6:33 pm on August 4, 2023: none

Concept NACK unless you keep the default at MAX_OP_RETURN_RELAY. Depending on adoption we can always drop that default later. No need to make pull requests more controversial than necessary.

@ChristopherA is paying me $1000 for 10 hours work to do this pull-req. Since the other sub-change got mired in controversy, I decided to meet my client requirement in a time-efficient manner. Obviously, I can’t promise to him that this will actually get merged. But I can promise to him that I’ll do a mergable pull-req in the time allotted.

So frankly, at this point I don’t care to split it up into two pull-reqs for such a minor reason. If someone else wants to, they can take this open-source licensed code and do it themselves.

Core Devs are a centralized group in high risk of conflict of interests as we can see per this message, presenting a huge risk for the deterioration of Bitcoin’s decentralization and therefore its BIGGEST existencial direct threat IMV. Centralizing decision in ‘Core’ devs has nothing to do with what Bitcoin is about. This centralizatijn risk is only eliminated when full nodes software allows as many options as possible for node operators (not core devs) to decide what to (un)support.

Retropex commented at 6:40 pm on August 4, 2023: none

Bitcoin Core is able to discard and not relay unconfirmed stamp transactions since March 2014 @1ma I think it’s time to put this back on the table. #28217

davidlick commented at 7:30 pm on August 4, 2023: none

Concept NACK

As a full node runner it’s not acceptable to me to remove a data limit. Data limits are critical to making running a node accessible to as many as possible.

RobinLinus commented at 7:56 pm on August 4, 2023: none

individuals opposing this PR […] dictate to all other people how they should and should not use Bitcoin

That’s incorrect. Everyone clearly agrees that we want to use Bitcoin for transactions. In contrast, many people don’t want to use it for storing arbitrary data, because that reduces the scalability of transactions. Hacking non-transaction data into the chain “dictates to all other people” that everyone has to download and store this data forever, which harms all users who want to use Bitcoin for its intended purpose.

1ma commented at 8:09 pm on August 4, 2023: none

The individuals opposing this PR, believing they protect Bitcoin from centralization, ironically desire to dictate to all other people how they should and should not use Bitcoin.

If you believe that core shouldnt be making policy decisions on behalf of users, then shouldnt the default be that there is no default policy and that users can set it to whatever they want? Thats what this PR would do (for this policy item).

Personally I’d like that the developers chose a default mempool policy with a very hard stance on spam to try not to burden more than necessary the computing resources volunteered by node runners. And then provide a wide range of options to customize these defaults.

Currently they don’t seem very keen to do neither of these.

luke-jr commented at 8:10 pm on August 4, 2023: member

If you believe that core shouldnt be making policy decisions on behalf of users, then shouldnt the default be that there is no default policy and that users can set it to whatever they want? Thats what this PR would do (for this policy item).

There is inherently ALWAYS a policy. “No default policy” would mean users are forced to make a decision (that was #).

This PR would make the default policy actively HARMFUL to Bitcoin (more than it already is). If you want a simpler default, the obvious choice would be to relay NOTHING.

The individuals opposing this PR, believing they protect Bitcoin from centralization, ironically desire to dictate to all other people how they should and should not use Bitcoin.

No, we’re not suggesting removing the options. This is a straw-man ad-hominem.

This type of standardness paternalism could eventually prompt the creation of other ways to include non-standard transactions in the blockchain, like a transaction marketplace for miners,

If miners actively bypass the community’s efforts to keep Bitcoin safe from these attacks, that’s a reason to change to a PoW algorithm that isn’t compromised, not a reason to give miners control of Bitcoin.

IMV Op_return was an altcoining experiment at Bitcoin’s inception, when Satoshi was …

OP_RETURN is not intended for data, and using it for data at all is an abuse that originated long after Satoshi left the project.

Hacking non-transaction data into the chain “dictates to all other people” that everyone has to download and store this data forever, which harms all users who want to use Bitcoin for its intended purpose.

100%. Nobody is telling spammers they can’t store or distribute data. Nobody is telling others they can’t assist the spammers in that storage and distribution. We just don’t want to be literally forced to do it against our will.

cbergqvist commented at 9:21 pm on August 4, 2023: none

Concept NACK @ChristopherA wrote:

My goal is not “non-bitcoin stuff”. My specific challenge is that Lightning and Nostr use public keys in a persistent way that are not rotatable, which is a bad cryptographic practice. By committing to a rotated key and some other cryptographic in advance in L1, they could be. I believe this could be as small as a signed hash and a little metadata. However, this is larger than 80-byte limit in OP_RETURN. I can now put it into an inscription-style transaction and even get a 4x discount, or use bare multisig, or use some other trick.

No need to increase the limit, just build something akin to @petertodd’s OpenTimestamps to store a Merkle root hash and provide the rest of the data through other channels. Use the 80 OP_RETURN bytes for that one SHA256 hash (32 bytes), unique service identifier (8 bytes?), maybe use the remaining (40) bytes for some kind of bloom filter data so one can rule out certain entries quickly when doing lookups?

HenrikJannsen commented at 10:52 pm on August 4, 2023: none

Concept NACK

I also do not see any reason why 80 bytes is not enough for the use cases which makes sense.

Making life for spammers easier is not our task. Better put effort into how to make it harder and more inconvenient for them to spam the blockchain. E.g. by making node options to block abusive behavior more popular (like using permitbaremultisig=0 as mentioned here #28130 (comment)). Their “business models” (better call it scam schemes) will suffer if their success rate goes down.

hsjoberg commented at 11:22 pm on August 4, 2023: contributor

Concept ACK.

It’s absurd to keep such strict size limitations when ordinal inscriptions are abusing witness space and witness discount anyway.

imacfan commented at 2:15 am on August 5, 2023: none

Concept NACK As a full node runner it’s not acceptable to me to remove a data limit. Data limits are critical to making running a node accessible to as many as possible.

ns-xvrn commented at 2:29 am on August 5, 2023: none

Concept NACK

Just because a protocol has unwanted steganography attacks, instead of deterring that - you’re saying allow it with arms wide open. Totally unacceptable, this is not what Bitcoin protocol is for, as a full node operator - I don’t want it.

vicariousdrama commented at 3:17 am on August 5, 2023: none

Concept ACK Primarily for same reasons outlined by @ChristopherA. Allows for including signature by a third party other than that signing the keys controlling the inputs. E.g., service provider. Promotes decentralization as otherwise users that want this feature must work with miners directly. May help reduce rate of UTXO growth that results from abusive outputs used in bare multisig.

whycorn commented at 3:21 am on August 5, 2023: none

Concept ACK Primarily for same reasons outlined by @ChristopherA. Allows for including signature by a third party other than that signing the keys controlling the inputs. E.g., service provider. Promotes decentralization as otherwise users that want this feature must work with miners directly. May help reduce rate of UTXO growth that results from abusive outputs used in bare multisig.

have you considered looking at this: https://github.com/bitcoin/bitcoin/pull/28217

ariard commented at 3:52 am on August 5, 2023: member

Can it be used to pin transactions in a way that unupgraded nodes (including poorly maintained alternative implementations that are potentially used in second layer protocols) wouldn’t notice? @Sjors What do you mean by introducing a pinning vector for second layers, like you would need SIGHASH_SINGLE tx (which is the case with LN anchor output) where the tx finalizer doesn’t commit to inputs/outputs with SIGHASH_ALL, which mean it could be pinned by a large range of tricks, such as adding OP_TRUE output and a chain of junk descendants, I think? Otherwise the OP_RETURN embedded data is part of the sig hash.

RicYashiroLee commented at 4:17 am on August 5, 2023: none

If you believe that core shouldnt be making policy decisions on behalf of users, then shouldnt the default be that there is no default policy and that users can set it to whatever they want? Thats what this PR would do (for this policy item).

There is inherently ALWAYS a policy. “No default policy” would mean users are forced to make a decision (that was #).

This PR would make the default policy actively HARMFUL to Bitcoin (more than it already is). If you want a simpler default, the obvious choice would be to relay NOTHING.

The individuals opposing this PR, believing they protect Bitcoin from centralization, ironically desire to dictate to all other people how they should and should not use Bitcoin.

No, we’re not suggesting removing the options. This is a straw-man ad-hominem.

This type of standardness paternalism could eventually prompt the creation of other ways to include non-standard transactions in the blockchain, like a transaction marketplace for miners,

If miners actively bypass the community’s efforts to keep Bitcoin safe from these attacks, that’s a reason to change to a PoW algorithm that isn’t compromised, not a reason to give miners control of Bitcoin.

IMV Op_return was an altcoining experiment at Bitcoin’s inception, when Satoshi was …

OP_RETURN is not intended for data, and using it for data at all is an abuse that originated long after Satoshi left the project.

Hacking non-transaction data into the chain “dictates to all other people” that everyone has to download and store this data forever, which harms all users who want to use Bitcoin for its intended purpose.

100%. Nobody is telling spammers they can’t store or distribute data. Nobody is telling others they can’t assist the spammers in that storage and distribution. We just don’t want to be literally forced to do it against our will.

200%

dpc commented at 4:22 am on August 5, 2023: none

OP_RETURN size limit is a “feel good” policy, that achieves nothing and just make things artificially more difficult in other areas. You’d think that Bitcoiners are going to be more rational than politicians that need to fool the public to get re-elected with such policies.

People were putting garbage and “spamming” the blockchain in the past, spamming is rampant currently and I have never ever seen a convincing explanation of a future strategy that could actually stop it. And no, ever increasing “spam filters” catching up with inventive spammers is not a strategy. Neither is absolutely crazy “punishing miners by changing PoW” ideas, and some other stuff I’ve seen in this thread.

AFAICT, the only anti-spam mechanism that actually ever worked is the cost of spamming. Just accept the reality that in a censorship resistant distributed system, you can’t stop someone who is willing to pay for publishing some data.

If you can’t prevent it, at least don’t make the consequences worse than they have to be. With OP_RETURN the garbage is clearly marked, so can be optimized around, which is a big harm reduction.

zndtoshi commented at 8:13 am on August 5, 2023: none

Did anyone calculate what would be the cost of placing ordinals inside a no-limit op_return output? If the costs would be higher (as I imagine) than putting them in the witness space (and taking advantage of the witness discount) then the merge of this proposal would not solve anything because people will just continue to use the cheapest way. For those reasons I’m actually in favor.. so ACK.

dimitaracev commented at 8:38 am on August 5, 2023: contributor

Concept NACK for the same reasons others have stated above.

HenrikJannsen commented at 11:13 am on August 5, 2023: none

Did anyone calculate what would be the cost of placing ordinals inside a no-limit op_return output? If the costs would be higher (as I imagine) than putting them in the witness space (and taking advantage of the witness discount) then the merge of this proposal would not solve anything because people will just continue to use the cheapest way. For those reasons I’m actually in favor.. so ACK.

It is not only the financial costs one need to consider, there are other costs. Complexity of implementation and risk for long term stability are important factors as well.

Adding risks to the ordinals business model by uncertainty of future support is one avenue which might be promising to counter those spam attacks. Ideas like https://twitter.com/hashtag/ordislow with delaying blocks containing ordinals might be worth considering in more detail. I don’t understand why Bitcoin devs are not more engaged in those discussions.

There is no reason why one would need more than 80 bytes in OP_RETURN. I guess 40 bytes would be enough as well. If ppl want to add metadata (3rd party signatures) they have to use their own network for providing that data and use the blockchain transaction as the timestamp for securing that external data. OpenTimestamp might be enough for most such use cases. Just because its hard to run your own p2p network does not justifify to abuse the Bitcoin blockchain as database.

Seems some people want to apply the fundamental mistakes of Ethereum into Bitcoin as a new attack vector.

zndtoshi commented at 12:01 pm on August 5, 2023: none

An option would be also to keep status quo and let node runners modify bitcoin.conf if they want to increase the op_return limit. Like the discussion with fullmempoolrbf.

zkfrio commented at 12:12 pm on August 5, 2023: none

Ideas like https://twitter.com/hashtag/ordislow with delaying blocks containing ordinals might be worth considering in more detail. I don’t understand why Bitcoin devs are not more engaged in those discussions.

Nobody wants to attack the network because of some transactions.

Retropex commented at 12:28 pm on August 5, 2023: none

@zkfrio Do you think that giving the choice of the mempool policy is attacking the network?

zkfrio commented at 12:44 pm on August 5, 2023: none

@zkfrio Do you think that giving the choice of the mempool policy is attacking the network?

I was responding to ‘ordislow’ related comment. It’s an organized attack that would not improve anything.

petertodd commented at 1:52 pm on August 5, 2023: contributor

@HenrikJannsen

Ideas like https://twitter.com/hashtag/ordislow with delaying blocks containing ordinals might be worth considering in more detail. I don’t understand why Bitcoin devs are not more engaged in those discussions.

Bitcoin devs haven’t engaged with those discussions because #ordislow is a dumb idea that is very unlikely to work; to the extent it does work, it will simply act to further centralize mining. #ordislow is fundamentally an attempt at censorship of blocks (information) we don’t like. Unless a very high percentage of nodes choose to run it, it’ll have no affect at all, because a moderately sized minority is sufficient to propagate blocks. And even in that circumstance, it’s very easy for interested parties to run non-ordislow nodes that peer with each other to ensure good propagation of ordinal-containing blocks. This of course is easier for large miners to do, which is why #ordislow is potentially a centralizing force for Bitcoin hash power.

A very similar situation is full-rbf: while only a small minority of nodes relay full-rbf replacements, in conjunction with a few dozen full-rbf peering nodes that’s more than enough to reliably propagate full-rbf replacements to the ~40% of hashing power that mines them.

luke-jr commented at 2:34 pm on August 5, 2023: member

Allows for including signature by a third party other than that signing the keys controlling the inputs. E.g., service provider.

🚩🚩🚩 Sounds like maybe this is tied to some kind of KYC nonsense?

There shouldn’t be a “service provider” to begin with. (And if you really need such a signature, you could put it INSIDE the data which is being hashed.)

People were putting garbage and “spamming” the blockchain in the past, spamming is rampant currently and I have never ever seen a convincing explanation of a future strategy that could actually stop it. And no, ever increasing “spam filters” catching up with inventive spammers is not a strategy.

Yes, it is, and has been largely successful.

BBezaire commented at 4:11 pm on August 5, 2023: none

Concept NACK

My node isn’t your personal hard drive. It’s for financial transactions, period.

We’re already forced to store ordinals for free, while miners profit greatly off priority fees. Such a change will only make things worse for node operators, and destroy Bitcoin’s credibility as a crypto currency.

This is a slap in the face to everything Satoshi worked towards.

zkfrio commented at 5:32 pm on August 5, 2023: none

This is a slap in the face to everything Satoshi worked towards.

I agree Satoshi would never use witness or bare multisig or OP_RETURN. They preferred coinbase. Unfortunately only miners can use it.

itme-brain commented at 5:38 pm on August 5, 2023: none

Concept ACK

The limit is nothing more than feel-good policy, people have been circumventing it forever.

Maybe if OP_RETURN never had an arbitrary 80 byte limit we would’ve never gotten JPEGs in tapscripts?

1ma commented at 8:13 pm on August 5, 2023: none

Concept ACK

The limit is nothing more than feel-good policy, people have been circumventing it forever.

Maybe if OP_RETURN never had an arbitrary 80 byte limit we would’ve never gotten JPEGs in tapscripts?

If datacarriersize is truly not effective in practice then surely there is no need for this PR at all, and it can be closed.

HenrikJannsen commented at 8:40 pm on August 5, 2023: none

Allows for including signature by a third party other than that signing the keys controlling the inputs. E.g., service provider.

🚩🚩🚩 Sounds like maybe this is tied to some kind of KYC nonsense?

There shouldn’t be a “service provider” to begin with. (And if you really need such a signature, you could put it INSIDE the data which is being hashed.)

Would be interesting to hear the concrete use cases those who propose that have in mind. Sounds a bit like a feature request from those who want to get “travel rule” on Bitcoin.

vicariousdrama commented at 8:55 pm on August 5, 2023: none

@whycorn

have you considered looking at this: #28217

Its under consideration but independent of this PR and feature. @luke-jr

🚩🚩🚩 Sounds like maybe this is tied to some kind of KYC nonsense?

There shouldn’t be a “service provider” to begin with. (And if you really need such a signature, you could put it INSIDE the data which is being hashed.)

Your emotion appears to get the better of you at times with use of the 🚩🚩🚩. This has nothing to do with KYC. Furthermore, service providers/intermediaries are welcome to use the Bitcoin network along with any other entity. If someone who has created a data set that is stored off chain wants to sign it for attestation and produce a hash and expose both signature and hash or a url to said data without associating to their own Bitcoin for privacy reasons, or prefer paying out of band in some other asset or even god forbid fiat to an intermediary, then they should be able to. Are you wanting to define specific constraints on a protocol within the data of an OP_RETURN, or just OP_RETURN itself? The arbitrary 80 character limit pushes towards centralization. Are you pro centralization?

iBeizsley commented at 9:15 pm on August 5, 2023: none

If someone who has created a data set that is stored off chain wants to sign it for attestation and produce a hash and expose both signature and hash or a url to said data without associating to their own Bitcoin for privacy reasons, or prefer paying out of band in some other asset or even god forbid fiat to an intermediary, then they should be able to. Are you wanting to define specific constraints on a protocol within the data of an OP_RETURN, or just OP_RETURN itself? The arbitrary 80 character limit pushes towards centralization. Are you pro centralization?

Can you explain what benefit you get from storing the signature on chain? What is the benefit of storing a signature for some data on chain, and the data off chain, over storing just hash(concat(signature, data)) on chain, and the signature and data off chain?

You don’t need blockchain for signatures. You don’t need blockchain to share data. The only benefit you get from utilizing Bitcoin’s blockchain in your protocol is the ability to reliably timestamp chunks of (signed) data. To do that, all you need is a hash.

vicariousdrama commented at 9:47 pm on August 5, 2023: none

If someone who has created a data set that is stored off chain wants to sign it for attestation and produce a hash and expose both signature and hash or a url to said data without associating to their own Bitcoin for privacy reasons, or prefer paying out of band in some other asset or even god forbid fiat to an intermediary, then they should be able to. Are you wanting to define specific constraints on a protocol within the data of an OP_RETURN, or just OP_RETURN itself? The arbitrary 80 character limit pushes towards centralization. Are you pro centralization?

Care to explain why the arbitrary-key signature cannot be part of the data itself, with the 32 byte hash in the op return merely timestamping that signature and the data it signs, with both the signature and the data off chain?

You don’t need blockchain for signatures. You don’t need blockchain to share data. The only benefit you get from utilizing Bitcoin’s blockchain in your protocol is the ability to reliably timestamp chunks of (signed) data. To do that, all you need is a hash.

An intermediary wants to create an OP_RETURN that includes

URI to a resource
hash of the target resource
some other metadata which could be a date of validity or signature of the creator of the resource

The metadata, time stamps, signatures in number 3 are independent of the signature of the private keys used for spending the inputs to make the transaction. That is the service providers keys, not necessarily the creator of the URI. I concur with you. The arbitrary key signature (number 3 above) can be part of the data at the target URI in this case.

The size of OP_RETURN should ideally be larger to accommodate both a hash, and a reference to something outside of Bitcoin as the smallest way to make that reference / consume the least space.

iBeizsley commented at 9:58 pm on August 5, 2023: none

The size of OP_RETURN should ideally be larger to accommodate both a hash, and a reference to something outside of Bitcoin as the smallest way to make that reference / consume the least space.

You can store 80 bytes in an op return. A SHA256 hash is 32 bytes.

Why do you need more than 48 bytes for a reference?

vicariousdrama commented at 10:16 pm on August 5, 2023: none

You can store 80 bytes in an op return. A SHA256 hash is 32 bytes.

Why do you need more than 48 bytes for a reference?

Varies depending on the actual URI

Assuming a SHA-1 hash of the following, but stored in binary form of 20 bytes 0e7364296be76d9ce5bc53dee6a15024d139760c

How should something like this be stored in 60 bytes https://github.com/PoodleLabs/PoodleLabs.BitcoinBinaryVerificationScripts

luke-jr commented at 10:20 pm on August 5, 2023: member

It shouldn’t be. Transaction keys are not URIs.

RicYashiroLee commented at 10:26 pm on August 5, 2023: none

The size of OP_RETURN should ideally be larger to accommodate both a hash, and a reference to something outside of Bitcoin as the smallest way to make that reference / consume the least space.

You can store 80 bytes in an op return. A SHA256 hash is 32 bytes.

Why do you need more than 48 bytes for a reference?

Mostly, Bitcoin’s TINY SCOPE has no space for unrelated non-primary garbage data, desincentivizes bitcoiners of running full nodes and implicitly hurt Bitcoin’s true decentralization through the only decentralizers, the node operators=bitcoiners=the clients of Bitcoin. IMV Core Devs and Miners are service providers only, their goal is often contrary to its decentralization, mostly they make money out of providing a service to bitcoiners and related companies profiting from Bitcoin.

iBeizsley commented at 10:47 pm on August 5, 2023: none

You can store 80 bytes in an op return. A SHA256 hash is 32 bytes. Why do you need more than 48 bytes for a reference?

Varies depending on the actual URI

Assuming a SHA-1 hash of the following, but stored in binary form of 20 bytes 0e7364296be76d9ce5bc53dee6a15024d139760c

How should something like this be stored in 60 bytes https://github.com/PoodleLabs/PoodleLabs.BitcoinBinaryVerificationScripts

Storing such a URL which you don’t control in an immutable ledger is extremely fragile, wasteful, and generally a bad idea.

The hash itself can act as a resource identifier, and with a 20 byte hash, the following format:

[2 byte type/tld][1 byte length][76*6 bit characters / 10 byte raw onion link] would (more than) cover every possible clearnet domain and every possible onion ‘domain’. You could also reference, say, every single npub at no additional cost; there’s a lot of wasted space in that format.

Even ignoring the argument about whether such identifiers should be stored on-chain (and I would argue not), I do not see any convincing technical reason you need more than the space already available to you.

printer-jam commented at 9:53 pm on August 6, 2023: none

The individuals opposing this PR, believing they protect Bitcoin from centralization, ironically desire to dictate to all other people how they should and should not use Bitcoin.

Why create the PR to change Bitcoin? You are not stopped from using it within the current rules, yet you want to dictate to node runners that they must now store more arbitrary data, and if they don’t agree they are the ones ‘dictating’ by opposing this PRs demands? Classic projection.

gzuuus commented at 9:24 am on August 7, 2023: none

Concept NACK. It is easy to see that bitcoin is neither an efficient nor convenient way to store data in the “cloud”, so why support the use of a feature that does not benefit and goes against of bitcoin’s value transaction and is inconvenient for node operators and users in general? Strongly agree with the above answers NACK

garlonicon commented at 4:43 pm on August 7, 2023: none

Rational: there’s lots of ways for people to publish data in the Bitcoin chain, and lots of data has been published.

When it comes to witness data, I can easily prune all of them by downgrading my node to be a non-witness node. How to achieve the same thing for large OP_RETURNs?

There’s no reason for us to place artificial limits on this particular method.

The reason is you will encourage people to downgrade the way they publish data. Witness limit is 4 MB for a reason: it is easy to prune (just disable witness), does not affect legacy nodes, and in fact makes their life easier (see what is the size of the infamous block 774628 from the perspective of a non-witness node).

Current storage methods need two txs and many deviant methods were used in the past to store things in bitcoin, as we saw recently also

Just use commitments. All typical transactions contain at least one signature. If you have a single OP_CHECKSIG anywhere, you need a signature. If you use P2TR, you have a signature in your spend-by-key path, or you could use OP_CHECKSIG inside TapScript.

So, if you have a signature, it contains two parts: r-value, and s-value. You want to store something? Then just tweak your r-value in your signature, and you will get the same result as if you would when putting 256-bit hash inside OP_RETURN. With no additional on-chain bytes, and no traces that could be used to censor your transaction. For all address types that support OP_CHECKSIG or equivalent. What is the reason to not use commitments?

luke-jr commented at 1:58 am on August 8, 2023: member

non-witness node

This isn’t a thing. Witness data is no more prunable than other data, to a full node.

elocremarc commented at 5:28 am on August 8, 2023: none

I hate to break it to you all. Bitcoin is primary a data layer and secondary a monetary layer. The data of transactions is worth spending to bribe miners for inclusion. However if some arbitrary data you disagree with gets inclusion and attestation and still follows the rules of consensus its a valid transaction. To censor any transaction that follows consensus is censorship.

Bitcoin is about uncensorable speech be it a monetary transaction or arbitrary data for someone willing to spend for inclusion. If the block utilization is low enough to allow spam then that will happen regardless of your opinion. If the block utilization is high enough from financial transactions then spam will be priced out. This is all about incentives of data inclusion nothing more.

0"Information does not just want to be free,
1it longs to be free.  Information expands to fill the available
2storage space."
3-- Cypherpunk Manifesto

Psifour commented at 6:02 am on August 8, 2023: none

Bitcoin is purely a layer for storage of data that, typically, represents financial transactions. We have witnessed for years that there is no reasonable way to prevent the storage of abitrary data (any system that allows user input of ‘random’ values should accept the inevitability of data storage, “the signal is the noise”).

The internet became the ubiquitous system that it is today not because some weak-willed soul decided for the users that they needed to be protected by limiting the power of their software stack, but instead by the raw memetic power of trolling on BBSs, porn, and attempting to subvert the control of others over our own lives. It is messy and dirty, but by trying to fight this we have held bitcoin back repeatedly as it inevitably results in users searching for (and finding) other means of accomplishing their objectives that are worse for the network overall. I’m happy to play cat-and-mouse games and pay my non-penalized witness fee rate if I can’t use OP_RETURN.

Let’s take the non-consensus shackles off and let bitcoin do what it does best using the mechanisms that were purpose built to solve these problems, or let’s keep pretending that endless ‘spam filter’ bandaids are at all a viable solution while we perpetually lean further from the permissionless cypherpunk Mecca that bitcoin can be.

natahala3 commented at 6:44 am on August 8, 2023: none

Cypherpunks write code. We know that someone has to write software to defend privacy, and since we can’t get privacy unless we all do, we’re going to write it. We publish our code so that our fellow Cypherpunks may practice and play with it. Our code is free for all to use, worldwide. We don’t much care if you don’t approve of the software we write. We know that software can’t be destroyed and that a widely dispersed system can’t be shut down.

dangershony commented at 7:17 am on August 8, 2023: contributor

Concept ACK

We’re building a specialized wallet and often struggle on where to keep pubkeys to recreate tapscripts, using opreturn makes it much simpler

ChrisMartl commented at 9:19 am on August 8, 2023: none

I hate to break it to you all. Bitcoin is primary a data layer and secondary a monetary layer. The data of transactions is worth spending to bribe miners for inclusion. However if some arbitrary data you disagree with gets inclusion and attestation and still follows the rules of consensus its a valid transaction. To censor any transaction that follows consensus is censorship.

Bitcoin is about uncensorable speech be it a monetary transaction or arbitrary data for someone willing to spend for inclusion. If the block utilization is low enough to allow spam then that will happen regardless of your opinion. If the block utilization is high enough from financial transactions then spam will be priced out. This is all about incentives of data inclusion nothing more.
0"Information does not just want to be free,
1it longs to be free.  Information expands to fill the available
2storage space."
3-- Cypherpunk Manifesto

First, Bitcoin is a network which realize a monetary layer through decentralization; and processes data in order to pursue that layer.

Since it is a network, there are psychical instances which make possible this network to work. Those instances are provide by individuals who have the legit right of private property. As such, private property owners have the right to control what is expressed or communicated on their property. If a private entity chooses to moderate content on its physical instance, it does so as an exercise of its property rights.

Nobody prevents you or force you to not use an alternative network which satisfies your wishes; the free market offers already options for you.

If Bitcoin participants should value something more than other, then it shall be “decentralization”; and decentralization of regular Bitcoin nodes. Decentralization provides more control to individuals over their properties and reduce the power of centralized entities in censoring information; like mining-node-entities.

It is clear, your view for Bitcoin is one which the decentralization of regular Bitcoin nodes is diminished towards more control and thus network centralization realized by mining-node-entities.

RicYashiroLee commented at 10:27 am on August 8, 2023: none

I hate to break it to you all. Bitcoin is primary a data layer and secondary a monetary layer. The data of transactions is worth spending to bribe miners for inclusion. However if some arbitrary data you disagree with gets inclusion and attestation and still follows the rules of consensus its a valid transaction. To censor any transaction that follows consensus is censorship.

Bitcoin is about uncensorable speech be it a monetary transaction or arbitrary data for someone willing to spend for inclusion. If the block utilization is low enough to allow spam then that will happen regardless of your opinion. If the block utilization is high enough from financial transactions then spam will be priced out. This is all about incentives of data inclusion nothing more.
0"Information does not just want to be free,
1it longs to be free.  Information expands to fill the available
2storage space."
3-- Cypherpunk Manifesto

The altcoiner in Bitcoin throws the confusing argument of out of the point ‘uncensorable’ word, to claim Bitcoin is an all purpose data dumping platform, when it is not. Its scope is very TINY, based on principles, the decentralization trilemma (the case here) and the auditability/transparency vs privacy on-chain. When those principles are not followed or understood, you are suffering from altcoining mindset (your case, throwing confusing unrelated words like ‘uncensorable speech’, no different from altcoiner’s ‘inventions/solutions’ on-chain (instead of off-chain where true ingenious innovation belongs, for Bitcoin). Bitcoin is not an altcoin because it MUST tyrannically (Satoshi’s word) seek continuously more decentralization, not less as you are proposing. Bitcoin is getting more and more centralized due to all the BIP mania already introduced and reason why all current weaknesses (Segwit and Taproot are examples) exist, and it is not by adding more of such altcoining mindset weaknesses that the scenario improves, it is instead about allowing node operators more (UN)support OPTIONS in FULL (CORE) NODE SOFTWARE (not less), so that what is CORE=CENTRAL does get decided ONLY by them, and NOT by CORE DEVS who are in obvious conflict of interests as service providers in Bitcoin, NOT the true clients of Bitcoin. HASH-ANCHORING (=stamping, inscriptions,etc) in Bitcoin is possible the easier scripts are allowed by its introduced ‘features’, but instead Satoshi would definitely wanted it to continuously become as CUMBERSOME as possible in all ingenious ways possible with minimum scripts allowed, I am sure he would figure out eventually, to keep Bitcoin’s SCOPE TYRANICALLY TINY as it was his intention (the only reason why Bitcoin has value and altcoins NOT). So, NOT make it even easier as you are proposing, throwing misplaced out-of-the-point ilogical intentionally confusing argumentation, against Bitcoin’s basic principles and core value proposition.

RicYashiroLee commented at 10:57 am on August 8, 2023: none

Bitcoin is purely a layer for storage of data that, typically, represents financial transactions. We have witnessed for years that there is no reasonable way to prevent the storage of abitrary data (any system that allows user input of ‘random’ values should accept the inevitability of data storage, “the signal is the noise”).

The internet became the ubiquitous system that it is today not because some weak-willed soul decided for the users that they needed to be protected by limiting the power of their software stack, but instead by the raw memetic power of trolling on BBSs, porn, and attempting to subvert the control of others over our own lives. It is messy and dirty, but by trying to fight this we have held bitcoin back repeatedly as it inevitably results in users searching for (and finding) other means of accomplishing their objectives that are worse for the network overall. I’m happy to play cat-and-mouse games and pay my non-penalized witness fee rate if I can’t use OP_RETURN.

Let’s take the non-consensus shackles off and let bitcoin do what it does best using the mechanisms that were purpose built to solve these problems, or let’s keep pretending that endless ‘spam filter’ bandaids are at all a viable solution while we perpetually lean further from the permissionless cypherpunk Mecca that bitcoin can be.

Bitcoin is NOT an all purpose DB. The reason why it has any value is exactly because it is NOT. CORE DEVS centralization of decision is a cancer in Bitcoin, as they are by nature in constant conflict of interests as service providers, they are NOT the clients of Bitcoin. The mechanisms of Bitcoin decentralization are corrupted the more power Core Devs have, the less power node operators have, which unfortunately has been the case, and IS SET BY CORE DEVS themselves in FULL NODE (CORE) SOFTWARE. Node operators are the only true decentralizers of Bitcoin, and the ones to decide what ‘features’ to VOLUNTARILY (UN)support in their full nodes (core software), NOT Core Devs with their non-stop BIP mania which only adds unwanted code complexity on-chain against Bitcoin basic principles and supposedly TINY SCOPE of its sui generis DB.

Retropex commented at 12:29 pm on August 8, 2023: none

Bitcoin: A Peer-to-Peer Electronic ~~Database~~ Cash System

iBeizsley commented at 1:08 pm on August 8, 2023: none

Bitcoin is primarily for data storage

No. This is an absurd take.

Bitcoin is an absolutely horrible solution for storing arbitrary data in a permissionless, uncensorable manner.

If you want to store data as such, use something like a DHT torrent. It’s a better solution, and doesn’t force others to unwillingly store your arbitrary data.

Bitcoin is a monetary network first, but it does also happen to be the best decentralised option we have for timestamping too. But to make use of that secondary function, you do not need to store arbitrary data on Bitcoin’s blockchain, as already discussed at length above.

bitcoin locked this on Aug 8, 2023

fanquake commented at 1:18 pm on August 10, 2023: member

I locked this thread, because the discussion here was clearly not achieving anything (other than spamming the thousands of subscribers to this repository).

I’d suggest that anyone who’s interested in continuing this discussion, take their thoughts to the mailing list: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2023-August/021840.html.

In the mean time, I think it’s unlikely that Bitcoin Core is going to make any changes in regards to OP_RETURN. For now, I’m going to close this PR. We can revisit this in future, depending on the results of further discussion.

fanquake closed this on Aug 10, 2023

Remove arbitrary restrictions on OP_RETURN by default #28130

Reviews

Conflicts