policy: uncap datacarrier by default #32406

pull instagibbs wants to merge 5 commits into bitcoin:master from instagibbs:2025-05-uncap_datacarrier changing 25 files +145 −93
  1. instagibbs commented at 3:41 pm on May 2, 2025: member

    Retains the -datacarrier* args, marks them as deprecated, and does not require another startup argument for multiple OP_RETURN outputs.

    If a user has set -datacarriersize the value is “budgeted” across all seen OP_RETURN output scriptPubKeys. In other words the total script bytes stays the same, but can be spread across any number of outputs. This is done to not introduce an additional argument to support multiple outputs.

    I do not advise people use the option with custom arguments and it is marked as deprecated to not mislead as a promise to offer it forever. The argument itself can be removed in some future release to clean up the code and minimize footguns for users.

  2. DrahtBot commented at 3:41 pm on May 2, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32406.

    Reviews

    See the guideline for information on the review process.

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #32421 (test: refactor: overhaul (w)txid determination for CTransaction objects by theStack)
    • #29954 (RPC: Return permitbaremultisig and maxdatacarriersize in getmempoolinfo by kristapsk)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label TX fees and policy on May 2, 2025
  4. bitcoin locked this on May 2, 2025
  5. ajtowns added the label Needs release note on May 2, 2025
  6. instagibbs force-pushed on May 2, 2025
  7. DrahtBot added the label CI failed on May 2, 2025
  8. instagibbs force-pushed on May 2, 2025
  9. instagibbs force-pushed on May 2, 2025
  10. DrahtBot removed the label CI failed on May 2, 2025
  11. bitcoin unlocked this on May 5, 2025
  12. TheCharlatan commented at 1:41 pm on May 5, 2025: contributor

    Concept ACK

    I think I’d prefer keeping the option until #32401 is addressed. Clearly there seems to be at least some demand for mining on charitable templates. If this is worth the additional code and maintenance is another discussion, and in the meantime I’d also support it being marked for future removal.

  13. wizkid057 commented at 1:56 pm on May 5, 2025: none
    Concept NACK Reasons outlined on mailing list and other PR
  14. in src/policy/policy.h:78 in 3ba7449f6c outdated
    72@@ -73,10 +73,9 @@ static constexpr unsigned int DEFAULT_DESCENDANT_SIZE_LIMIT_KVB{101};
    73 /** Default for -datacarrier */
    74 static const bool DEFAULT_ACCEPT_DATACARRIER = true;
    75 /**
    76- * Default setting for -datacarriersize. 80 bytes of data, +1 for OP_RETURN,
    77- * +2 for the pushdata opcodes.
    78+ * Default setting for -datacarriersize in vbytes.
    79  */
    80-static const unsigned int MAX_OP_RETURN_RELAY = 83;
    81+static const unsigned int MAX_OP_RETURN_RELAY = MAX_STANDARD_TX_WEIGHT / WITNESS_SCALE_FACTOR;
    


    ismaelsadeeq commented at 2:01 pm on May 5, 2025:

    Contrary to the PR title and description we still have a capacity of the maximum standard tx weight?

    So could this be clarified to indicate that we still maintain the standardness limit?


    “hypothetical” but what if we see in the future “transaction landscape has rendered this cap ineffective” as well?


    instagibbs commented at 3:02 pm on May 5, 2025:
    This should be a no-op at default due to MAX_STANDARD_TX_WEIGHT, but if it’s clearer to set it to int max or similar, I can do that

    jamesob commented at 6:47 pm on May 6, 2025:
    In a vacuum it might be nice as a std::optional<unsigned int>, but that probably causes too much downstream churn.
  15. BitcoinMechanic commented at 2:10 pm on May 5, 2025: none
    Concept NACK. Nodes have no incentive to become free relays between those who want to store arbitrary data and miners. Setting defaults to the opposite effect just results in distrust of Core and migration away from it (as we have witnessed over the last week - although of course not Sybil resistant, seems genuine.)
  16. pinheadmz commented at 2:13 pm on May 5, 2025: member

    @BitcoinMechanic

    no incentive

    Fee estimation and block propagation to name a few: https://groups.google.com/g/bitcoindev/c/d6ZO7gXGYbQ/m/3WVL60u6EQAJ

  17. BitcoinMechanic commented at 2:18 pm on May 5, 2025: none

    @BitcoinMechanic

    no incentive

    Fee estimation and block propagation to name a few: https://groups.google.com/g/bitcoindev/c/d6ZO7gXGYbQ/m/3WVL60u6EQAJ

    It does no harm to fee estimation or block propagation. Nodes can and do cache transactions they reject from their mempools making compact blocks just as quick to verify regardless of if some of their contents was filtered.

    As for fee estimation, it does not require knowledge of “the” mempool and there can never be such a thing.

    The efforts to design Bitcoin Core around the increased reliance on mempool homogeneity are misguided and a trend in the wrong direction.

    More high level - if nodes can configure their own mempool policies it obviously doesn’t break things and demonstrably never has.

  18. Sjors commented at 2:19 pm on May 5, 2025: member

    Concept ACK. This adds a deprecation step to #32359, which seems fine from a technical point of view, and was requested by regular contributors as well.

    It will re-invite the brigading when the actual code is removed, but it will be easier to point to earlier discussion.

    Code looks reasonable at first glance, when compared to #32359, but will re-review it.

    3ba7449f6c335026b752366c53f9c309f09e6c64 could be split between a commit that allows multiple outputs and one that switches the default.


    There’s no need to Concept N(ACK) this if all you’re going to do is repeat comments from #32359. They’re not votes. Any actual reviewer of this PR (including maintainers) can read those arguments there.

  19. Retropex commented at 2:20 pm on May 5, 2025: none

    Concept NACK.

    For the same reasons mentioned in #32359.

  20. in src/init.cpp:910 in 664ae315f4 outdated
    905@@ -906,6 +906,10 @@ bool AppInitParameterInteraction(const ArgsManager& args)
    906         InitWarning(_("Option '-checkpoints' is set but checkpoints were removed. This option has no effect."));
    907     }
    908 
    909+    if (args.IsArgSet("-datacarriersize") || args.IsArgSet("-datacarrier")) {
    910+        LogInfo("Warning: Options '-datacarrier' or '-datacarriersize' are set but are marked as deprecated. They will be removed in future versions.");
    


    polespinasa commented at 2:27 pm on May 5, 2025:

    I think this should be a warning- something like:

    0warnings.push_back(_("Warning: Options '-datacarrier' or '-datacarriersize' are set but are marked as deprecated. They will be removed in future versions."));
    

    See #31278


    instagibbs commented at 2:49 pm on May 5, 2025:
    functional tests were failing on shutdown when InitWarning was being triggered on an earlier version of my code. Is there a way to make the tests ok with it being set for specific tests?

    polespinasa commented at 3:32 pm on May 5, 2025:

    Oh right my bad should be InitWarning(_("Warning: ..."))!

    Anyway yes you should be able to do so. You have to specify the expected stderr like: https://github.com/bitcoin/bitcoin/blob/baa848b8d38187ce6b24a57cfadf1ea180209711/test/functional/wallet_create_tx.py#L72

    By default is set to an empty string: https://github.com/bitcoin/bitcoin/blob/baa848b8d38187ce6b24a57cfadf1ea180209711/test/functional/test_framework/test_framework.py#L607


    instagibbs commented at 11:55 am on May 6, 2025:
    still fumbling this. In mempool_datacarrier.py I’m not explicitly stopping or restarting the nodes but allowing test teardown to do it. Should I just be stopping the node myself with the optional arg?

    polespinasa commented at 12:26 pm on May 6, 2025:

    I just checked, when is auto-stopped at the end it stop all nodes by calling:

    0self.log.info("Stopping nodes")
    1            if self.nodes:
    2                self.stop_nodes()
    

    It uses stop_nodes() which doesn’t give any expected_stderr value:

    0    def stop_node(self, i, expected_stderr='', wait=0):
    1        """Stop a bitcoind test node"""
    2        self.nodes[i].stop_node(expected_stderr, wait=wait)
    3
    4    def stop_nodes(self, wait=0):
    5        """Stop multiple bitcoind test nodes"""
    6        for node in self.nodes:
    7            # Issue RPC to stop nodes
    8            node.stop_node(wait=wait, wait_until_stopped=False)
    

    Maybe could define a self.expected_stderr=[""]*self.nNodes at test_framework class and pass it to stop_node by default instead of an empty string. So you could override it on set_test_params(self)


    maflcko commented at 1:10 pm on May 6, 2025:

    Should I just be stopping the node myself with the optional arg?

    I’d say yes, if you go down the route. Not sure about adding implicit fields to the test framework for this.


    polespinasa commented at 1:22 pm on May 6, 2025:
    0InitWarning(_("Options '-datacarrier' or '-datacarriersize' are set but are marked as deprecated. They will be removed in future versions."));
    

    polespinasa commented at 1:28 pm on May 6, 2025:
    Code suggested here: #32406#pullrequestreview-2818267234

    maflcko commented at 7:04 pm on May 6, 2025:
    However, if some people prefer the option to not be deprecated (for whatever reason), it seems pretty minimal maintenance to keep it around without deprecation for now. There is no rush for this pull request, as the 30.x release is months away and also no rush to remove or deprecate the setting. Even if it is only for the extremely unlikely case to avoid having to undeprecate it later on (see also #32359 (comment)). The option has existed for many years, and if it were to be removed completely, it shouldn’t matter much if that were to happen at time N or time N+6mo.

    instagibbs commented at 12:41 pm on May 7, 2025:
    done
  21. nsvrn commented at 2:33 pm on May 5, 2025: contributor

    Concept NACK

    If fee estimation, block propagation etc has issues with mempool diversity specially for defense against spam/DoS then the position of not fixing that or making a future economic judgement based on theoretical assumptions is a fallacy. This broad position of Bitcoin Core can clearly increase spam and centralization regardless of economic incentives of miners which are all speculations and projections to modify how Bitcoin works instead of being realistic about the end goal of Bitcoin network. It’s disappointing where this is headed.

  22. in src/test/transaction_tests.cpp:889 in 3ba7449f6c outdated
    885@@ -888,21 +886,28 @@ BOOST_AUTO_TEST_CASE(test_IsStandard)
    886     t.vout[0].scriptPubKey = CScript() << OP_RETURN;
    887     CheckIsStandard(t);
    888 
    889-    // Only one TxoutType::NULL_DATA permitted in all cases
    890+    // Multiple TxoutType::NULL_DATA now permitted
    


    ismaelsadeeq commented at 2:33 pm on May 5, 2025:

    nit:

    0    // Multiple TxoutType::NULL_DATA are permitted
    

    instagibbs commented at 12:09 pm on May 6, 2025:
    done
  23. instagibbs commented at 2:36 pm on May 5, 2025: member

    FWIW I will not engage in meta discussion here. I will respond to any deficiencies in the code itself.

    https://github.com/bitcoin/bitcoin/commit/3ba7449f6c335026b752366c53f9c309f09e6c64 could be split between a commit that allows multiple outputs and one that switches the default.

    Touching tests twice is kind of annoying, but I can do if it makes code history easier and others agree.

  24. pinheadmz commented at 2:36 pm on May 5, 2025: member
    @nsvrn this PR specifically enables users to have a divergent mempool. I don’t understand the use case for that, but the proposal here addresses feedback on #32359 and adds a feature specifically for users who do not want their mempool to anticipate miner behavior.
  25. BitcoinMechanic commented at 2:36 pm on May 5, 2025: none
    @Sjors this is materially different from the other PR as it at least allows users the ability to configure. Not sure it’s appropriate to invoke same reasoning in both cases?
  26. in src/policy/policy.cpp:136 in 664ae315f4 outdated
    132@@ -137,17 +133,22 @@ bool IsStandardTx(const CTransaction& tx, const std::optional<unsigned>& max_dat
    133         }
    134     }
    135 
    136-    unsigned int nDataOut = 0;
    137+    unsigned int datacarrier_bytes_left = max_datacarrier_bytes.value_or(0);
    


    polespinasa commented at 2:40 pm on May 5, 2025:

    A personal opinion to read the code (non blocking)

    Maybe would be more clear to have it as a constant and override it if the option is set? If not, maybe a comment would be good.


    Psifour commented at 8:46 pm on May 5, 2025:

    agreed on this nit.

    It is not immediately apparent that max_datacarrier_bytes will be set to MAX_OP_RETURN_RELAY if no -datacarriersize flag is present.


    instagibbs commented at 12:00 pm on May 6, 2025:
    Documentation for this is in src/kernel/mempool_options.h, it can be further spelled out in policy.h for IsStandardTx if that helps?

    pithosian commented at 9:24 am on May 11, 2025:

    For additional context, in mempool_args.cpp::apply_args_man_options, we have this check:

    0if (argsman.GetBoolArg("-datacarrier", DEFAULT_ACCEPT_DATACARRIER)) {
    1    mempool_opts.max_datacarrier_bytes = argsman.GetIntArg("-datacarriersize", MAX_OP_RETURN_RELAY);
    2} else {
    3    mempool_opts.max_datacarrier_bytes = std::nullopt;
    4}
    

    GetIntArg can’t take nullopt as a default (it wants an integer), so the true path has to set to MAX_OP_RETURN_RELAY as the default, rather than setting nullopt with the false path setting 0. The false path could set zero, but this guard would still need to be here (foregoing a fair amount of extra churn to make max_datacarrier_bytes an integer, not an optional integer).

    I think this line is fine as-is, but were it to be clarified something as simple as the following could suffice:

    0unsigned int datacarrier_bytes_left = max_datacarrier_bytes.value_or(0 /* datacarrier=false */);
    

    stickies-v commented at 10:37 am on May 13, 2025:

    nit: no strong view, and not important, but my preference would be to just remove the std::optional altogether. Simplifies fn signature and defines what zero means in a single location.

     0diff --git a/src/kernel/mempool_options.h b/src/kernel/mempool_options.h
     1index d57dbb393f..4786aca67d 100644
     2--- a/src/kernel/mempool_options.h
     3+++ b/src/kernel/mempool_options.h
     4@@ -48,9 +48,9 @@ struct MemPoolOptions {
     5      * type is designated as TxoutType::NULL_DATA.
     6      *
     7      * Maximum size of TxoutType::NULL_DATA scripts that this node considers standard.
     8-     * If nullopt, any size is nonstandard.
     9+     * If zero, any size is nonstandard.
    10      */
    11-    std::optional<unsigned> max_datacarrier_bytes{DEFAULT_ACCEPT_DATACARRIER ? std::optional{MAX_OP_RETURN_RELAY} : std::nullopt};
    12+    unsigned int max_datacarrier_bytes{DEFAULT_ACCEPT_DATACARRIER ? MAX_OP_RETURN_RELAY : 0};
    13     bool permit_bare_multisig{DEFAULT_PERMIT_BAREMULTISIG};
    14     bool require_standard{true};
    15     bool persist_v1_dat{DEFAULT_PERSIST_V1_DAT};
    16diff --git a/src/node/mempool_args.cpp b/src/node/mempool_args.cpp
    17index 6dbba78381..c2bf1749a9 100644
    18--- a/src/node/mempool_args.cpp
    19+++ b/src/node/mempool_args.cpp
    20@@ -84,7 +84,7 @@ util::Result<void> ApplyArgsManOptions(const ArgsManager& argsman, const CChainP
    21     if (argsman.GetBoolArg("-datacarrier", DEFAULT_ACCEPT_DATACARRIER)) {
    22         mempool_opts.max_datacarrier_bytes = argsman.GetIntArg("-datacarriersize", MAX_OP_RETURN_RELAY);
    23     } else {
    24-        mempool_opts.max_datacarrier_bytes = std::nullopt;
    25+        mempool_opts.max_datacarrier_bytes = 0;
    26     }
    27 
    28     mempool_opts.require_standard = !argsman.GetBoolArg("-acceptnonstdtxn", DEFAULT_ACCEPT_NON_STD_TXN);
    29diff --git a/src/policy/policy.cpp b/src/policy/policy.cpp
    30index fdc2fdb9cd..51f5ae972e 100644
    31--- a/src/policy/policy.cpp
    32+++ b/src/policy/policy.cpp
    33@@ -96,7 +96,7 @@ bool IsStandard(const CScript& scriptPubKey, TxoutType& whichType)
    34     return true;
    35 }
    36 
    37-bool IsStandardTx(const CTransaction& tx, const std::optional<unsigned>& max_datacarrier_bytes, bool permit_bare_multisig, const CFeeRate& dust_relay_fee, std::string& reason)
    38+bool IsStandardTx(const CTransaction& tx, unsigned int max_datacarrier_bytes, bool permit_bare_multisig, const CFeeRate& dust_relay_fee, std::string& reason)
    39 {
    40     if (tx.version > TX_MAX_STANDARD_VERSION || tx.version < 1) {
    41         reason = "version";
    42@@ -133,7 +133,7 @@ bool IsStandardTx(const CTransaction& tx, const std::optional<unsigned>& max_dat
    43         }
    44     }
    45 
    46-    unsigned int datacarrier_bytes_left = max_datacarrier_bytes.value_or(0);
    47+    unsigned int datacarrier_bytes_left{max_datacarrier_bytes};
    48     TxoutType whichType;
    49     for (const CTxOut& txout : tx.vout) {
    50         if (!::IsStandard(txout.scriptPubKey, whichType)) {
    51diff --git a/src/policy/policy.h b/src/policy/policy.h
    52index e62efa02e3..82cb13d1e6 100644
    53--- a/src/policy/policy.h
    54+++ b/src/policy/policy.h
    55@@ -149,7 +149,7 @@ static constexpr decltype(CTransaction::version) TX_MAX_STANDARD_VERSION{3};
    56 * Check for standard transaction types
    57 * [@return](/bitcoin-bitcoin/contributor/return/) True if all outputs (scriptPubKeys) use only standard transaction forms
    58 */
    59-bool IsStandardTx(const CTransaction& tx, const std::optional<unsigned>& max_datacarrier_bytes, bool permit_bare_multisig, const CFeeRate& dust_relay_fee, std::string& reason);
    60+bool IsStandardTx(const CTransaction& tx, unsigned int max_datacarrier_bytes, bool permit_bare_multisig, const CFeeRate& dust_relay_fee, std::string& reason);
    61 /**
    62 * Check for standard transaction types
    63 * [@param](/bitcoin-bitcoin/contributor/param/)[in] mapInputs       Map of previous transactions that have outputs we're spending
    64diff --git a/src/test/fuzz/transaction.cpp b/src/test/fuzz/transaction.cpp
    65index c9eb11222f..f5cbacfb78 100644
    66--- a/src/test/fuzz/transaction.cpp
    67+++ b/src/test/fuzz/transaction.cpp
    68@@ -61,8 +61,8 @@ FUZZ_TARGET(transaction, .init = initialize_transaction)
    69 
    70     const CFeeRate dust_relay_fee{DUST_RELAY_TX_FEE};
    71     std::string reason;
    72-    const bool is_standard_with_permit_bare_multisig = IsStandardTx(tx, std::nullopt, /* permit_bare_multisig= */ true, dust_relay_fee, reason);
    73-    const bool is_standard_without_permit_bare_multisig = IsStandardTx(tx, std::nullopt, /* permit_bare_multisig= */ false, dust_relay_fee, reason);
    74+    const bool is_standard_with_permit_bare_multisig = IsStandardTx(tx, /*max_datacarrier_bytes=*/0, /* permit_bare_multisig= */ true, dust_relay_fee, reason);
    75+    const bool is_standard_without_permit_bare_multisig = IsStandardTx(tx, /*max_datacarrier_bytes=*/0, /* permit_bare_multisig= */ false, dust_relay_fee, reason);
    76     if (is_standard_without_permit_bare_multisig) {
    77         assert(is_standard_with_permit_bare_multisig);
    78     }
    79diff --git a/src/test/script_p2sh_tests.cpp b/src/test/script_p2sh_tests.cpp
    80index bb408b7b0f..68909dbb8a 100644
    81--- a/src/test/script_p2sh_tests.cpp
    82+++ b/src/test/script_p2sh_tests.cpp
    83@@ -21,13 +21,13 @@
    84 // Helpers:
    85 static bool IsStandardTx(const CTransaction& tx, bool permit_bare_multisig, std::string& reason)
    86 {
    87-    return IsStandardTx(tx, std::nullopt, permit_bare_multisig, CFeeRate{DUST_RELAY_TX_FEE}, reason);
    88+    return IsStandardTx(tx, /*max_datacarrier_bytes=*/0, permit_bare_multisig, CFeeRate{DUST_RELAY_TX_FEE}, reason);
    89 }
    90 
    91 static bool IsStandardTx(const CTransaction& tx, std::string& reason)
    92 {
    93-    return IsStandardTx(tx, std::nullopt, /*permit_bare_multisig=*/true, CFeeRate{DUST_RELAY_TX_FEE}, reason) &&
    94-           IsStandardTx(tx, std::nullopt, /*permit_bare_multisig=*/false, CFeeRate{DUST_RELAY_TX_FEE}, reason);
    95+    return IsStandardTx(tx, /*max_datacarrier_bytes=*/0, /*permit_bare_multisig=*/true, CFeeRate{DUST_RELAY_TX_FEE}, reason) &&
    96+           IsStandardTx(tx, /*max_datacarrier_bytes=*/0, /*permit_bare_multisig=*/false, CFeeRate{DUST_RELAY_TX_FEE}, reason);
    97 }
    98 
    99 static std::vector<unsigned char> Serialize(const CScript& s)
    

    instagibbs commented at 2:02 pm on May 13, 2025:

    that’s not the same behavior IIUC leaving as-is to not increase scope of review

     0diff --git a/test/functional/mempool_datacarrier.py b/test/functional/mempool_datacarrier.py
     1index 6347215b86..ce1d65a9ed 100755
     2--- a/test/functional/mempool_datacarrier.py
     3+++ b/test/functional/mempool_datacarrier.py
     4@@ -20,16 +20,17 @@ from random import randbytes
     5 # The historical maximum, now used to test coverage
     6 CUSTOM_DATACARRIER_ARG = 83
     7 
     8 class DataCarrierTest(BitcoinTestFramework):
     9     def set_test_params(self):
    10-        self.num_nodes = 4
    11+        self.num_nodes = 5
    12         self.extra_args = [
    13             [], # default is uncapped
    14             ["-datacarrier=0"], # no relay of datacarrier
    15             ["-datacarrier=1", f"-datacarriersize={CUSTOM_DATACARRIER_ARG}"],
    16             ["-datacarrier=1", "-datacarriersize=2"],
    17+            ["-datacarrier=1", "-datacarriersize=0"],
    18         ]
    19 
    20     def test_null_data_transaction(self, node: TestNode, data, success: bool) -> None:
    21         tx = self.wallet.create_self_transfer(fee_rate=0)["tx"]
    22         data = [] if data is None else [data]
    23@@ -73,10 +74,11 @@ class DataCarrierTest(BitcoinTestFramework):
    24         self.log.info("Testing a null data transaction with no data.")
    25         self.test_null_data_transaction(node=self.nodes[0], data=None, success=True)
    26         self.test_null_data_transaction(node=self.nodes[1], data=None, success=False)
    27         self.test_null_data_transaction(node=self.nodes[2], data=None, success=True)
    28         self.test_null_data_transaction(node=self.nodes[3], data=None, success=True)
    29+        self.test_null_data_transaction(node=self.nodes[4], data=None, success=True)
    30 
    31         self.log.info("Testing a null data transaction with zero bytes of data.")
    32         self.test_null_data_transaction(node=self.nodes[0], data=zero_bytes, success=True)
    33         self.test_null_data_transaction(node=self.nodes[1], data=zero_bytes, success=False)
    34         self.test_null_data_transaction(node=self.nodes[2], data=zero_bytes, success=True)
    
  27. BitcoinMechanic commented at 2:40 pm on May 5, 2025: none

    @nsvrn this PR specifically enables users to have a divergent mempool. I don’t understand the use case for that

    The use case for users maintaining control over their mempools is self evident.

  28. Sjors commented at 2:41 pm on May 5, 2025: member

    @BitcoinMechanic wrote:

    allows users the ability to configure

    It briefly retains it, typically deprecation is followed by removal one release later.

    So if you’re opposed to direct removal, then everyone will understand that you’re also opposed to deprecation, so there’s nothing new to say.

  29. in src/init.cpp:909 in 201b76101f outdated
    905@@ -906,6 +906,10 @@ bool AppInitParameterInteraction(const ArgsManager& args)
    906         InitWarning(_("Option '-checkpoints' is set but checkpoints were removed. This option has no effect."));
    907     }
    908 
    909+    if (args.IsArgSet("-datacarriersize") || args.IsArgSet("-datacarrier")) {
    


    ismaelsadeeq commented at 2:41 pm on May 5, 2025:

    From what I see from previous PR is that after deprecation; the feature will be removed in next release cycle, so maybe indicate that? #31278 (review)

    If we will eventually deprecate this then let’s indicate that.


    instagibbs commented at 3:06 pm on May 5, 2025:
    Personally, I’m fine with the option being sunset over a longer period. I’ll let other people weigh in on that as a possibility.

    Sjors commented at 9:16 am on May 6, 2025:
    I think “deprecated” is clear enough, no need to commit to a timeline.
  30. polespinasa commented at 2:43 pm on May 5, 2025: contributor

    cACK

    I like this approach, is far less problematic than #32359 Not going to add any more argument than the ones given there.

    Left two small non-blocking comments

  31. ismaelsadeeq commented at 3:00 pm on May 5, 2025: member

    Concept ACK on relaxing the maximum capacity of the OP_RETURN size to the current transaction standardness limit.

    1. This change will improve fee estimation in Bitcoin Core, as users will now be able to see the fee pressure from “data-carier” transactions that were previously broadcast out of band.

    2. It will also improve block propagation time and make mining fairer, as data carrier transactions below standardness limit will now be relayed in nodes’ mempools before being included in block templates. Nodes that already have these transactions can validate new blocks faster and relay them to peers and potentially other miners. If I understand correctly, Bitcoin’s incentives require miners to learn about new blocks as quickly as possible so this is a positive change.

    3. This could also potentially reduce the reliance on “out-of-band” services that collect data carrier transactions at high fees and possibly earn more revenue than other miners.

    However, I still believe that while Bitcoin Core’s transaction relay policy rules have significant advantages, they leave room for out-of-band services like those in point 3. As long as new innovations seek to utilize create transactions that violate those policies, we may continue to see these services persist.

    This leads us to repeat the same process of relaxing rules to address issues 1, 2, and 3.

    In my opinion, policy rules are an “easy way out” but limited solution. If we truly want to prevent issues 1, 2, and 3, those rules should be part of the consensus layer.


    On Deprecating this feature: I will like to see this option not being deprecated and let’s give users the ability to have a datacarier limit as they see fit; even though it is a footgun; we have a far worst option available i,e -blocksonly mode; in blocks only mode fee estimation is disabled. The work in #30157 allows a node to also detect when it’s mempool is roughly not in sync with miners and prevent serving fee rate estimate.

  32. moth-oss commented at 3:20 pm on May 5, 2025: none

    Concept NACK

    I’ve read the justifications but remain unconvinced the timing as well as the decision to altogether remove the option to configure is the right one.

    Increasing the default and giving node runners the option to change that is the right way to go. And a few versions layers maybe the discussion to remove it altogether can be restarted. Current PR is way too sudden. It seems we did not learn the lesson from the unintentional effect lifting the witness script size limit had on the quantity of arbitrary data getting stored in the block.

    Yes, I am aware it was possible to store data even before that, but it became economically lucrative since that specific change of removing the 10kb limit. Instead of capping that somehow, we are on a path to remove any and all data size configuration option the node runners currently have.

    Besides, removing this option doesn’t even help in a major way to remove utxo bloat because segwit discount will still be preferred for data above a certain size threshold.

    Earnestly urge the project maintainers to rethink this one and implement it in more gradual changes. Thank you.

  33. miketwenty1 commented at 4:33 pm on May 5, 2025: contributor

    Concept ACK.

    I believe this is a marginal improvement over #32359. It’s better to err on the side of a normal deprecation path with more knobs rather than fewer, especially since this knob (-datacarriersize) was previously available. Its removal has been a specific point of controversy, explicitly described by some as core compelling speech. However, this conceptual change will likely affect only a small percentage of previous NACKers, since the intent and outcome seem clear and unchanged imo. At best, it provides hypothetical hope that if the option proves beneficial during the deprecation period, it might not ultimately be removed.

  34. luke-jr commented at 6:46 pm on May 5, 2025: member
    Concept NACK. Already been over this on multiple PRs. Spamming it with immaterial minor differences won’t change the fundamental insanity and malice of the concept.
  35. 1440000bytes commented at 6:46 pm on May 5, 2025: none
    @moth-oss config options are not removed in this pull request.
  36. Psifour commented at 9:32 pm on May 5, 2025: none

    Concept ACK with 1 nit

    My issue with previous PR was largely due to removal vs deprecation of config options.

  37. in src/init.cpp:640 in 34c3ef7160 outdated
    634@@ -635,9 +635,9 @@ void SetupServerArgs(ArgsManager& argsman, bool can_listen_ipc)
    635     argsman.AddArg("-dustrelayfee=<amt>", strprintf("Fee rate (in %s/kvB) used to define dust, the value of an output such that it will cost more than its value in fees at this fee rate to spend it. (default: %s)", CURRENCY_UNIT, FormatMoney(DUST_RELAY_TX_FEE)), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::NODE_RELAY);
    636     argsman.AddArg("-acceptstalefeeestimates", strprintf("Read fee estimates even if they are stale (%sdefault: %u) fee estimates are considered stale if they are %s hours old", "regtest only; ", DEFAULT_ACCEPT_STALE_FEE_ESTIMATES, Ticks<std::chrono::hours>(MAX_FILE_AGE)), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
    637     argsman.AddArg("-bytespersigop", strprintf("Equivalent bytes per sigop in transactions for relay and mining (default: %u)", DEFAULT_BYTES_PER_SIGOP), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
    638-    argsman.AddArg("-datacarrier", strprintf("Relay and mine data carrier transactions (default: %u)", DEFAULT_ACCEPT_DATACARRIER), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
    639+    argsman.AddArg("-datacarrier", strprintf("(DEPRECATED) Relay and mine data carrier transactions (default: %u)", DEFAULT_ACCEPT_DATACARRIER), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
    640     argsman.AddArg("-datacarriersize",
    641-                   strprintf("Relay and mine transactions whose data-carrying raw scriptPubKey "
    642+                   strprintf("(DEPRECATED) Relay and mine transactions whose data-carrying raw scriptPubKey "
    


    achow101 commented at 11:08 pm on May 5, 2025:

    In 34c3ef7160a3e54980d74426f12237a2a674e0f2 “datacarrier: deprecate startup arguments for future removal”

    Perhaps this should mention that multiple outputs are allowed?


    instagibbs commented at 12:09 pm on May 6, 2025:
    done
  38. achow101 commented at 11:08 pm on May 5, 2025: member
    Needs a release note.
  39. petertodd commented at 11:56 pm on May 5, 2025: contributor

    ACK c164064c266c518588d9d9175f9b14140ee751b6

    Manually tested that the -datacarriersize limit does in fact match on a two OP_Return output transaction of the expected size, and that the resulting transaction was propagated to other nodes as expected.

    Of course, I still (weakly) prefer my version. But this is acceptable too.

  40. DrahtBot requested review from ismaelsadeeq on May 5, 2025
  41. DrahtBot requested review from TheCharlatan on May 5, 2025
  42. DrahtBot requested review from Sjors on May 5, 2025
  43. DrahtBot requested review from polespinasa on May 5, 2025
  44. DrahtBot requested review from Psifour on May 5, 2025
  45. instagibbs force-pushed on May 6, 2025
  46. in src/init.cpp:641 in 3b3fafdb1c outdated
    639+    argsman.AddArg("-datacarrier", strprintf("(DEPRECATED) Relay and mine data carrier transactions (default: %u)", DEFAULT_ACCEPT_DATACARRIER), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
    640     argsman.AddArg("-datacarriersize",
    641-                   strprintf("Relay and mine transactions whose data-carrying raw scriptPubKey "
    642-                             "is of this size or less (default: %u)",
    643+                   strprintf("(DEPRECATED) Relay and mine transactions whose data-carrying raw scriptPubKeys in aggregate"
    644+                             "are of this size or less, allowing multiple outputs (default: %u)",
    


    maflcko commented at 1:10 pm on May 6, 2025:

    aggregateare -> aggregate are

    (missing space, from the DrahtBot LLM linter)

  47. in test/functional/mempool_datacarrier.py:32 in 3b3fafdb1c outdated
    29-            ["-datacarrier=1", f"-datacarriersize={MAX_OP_RETURN_RELAY - 1}"],
    30+            [], # default is uncapped
    31+            ["-datacarrier=0"], # no relay of datacarrier
    32+            ["-datacarrier=1", f"-datacarriersize={CUSTOM_DATACARRIER_ARG}"],
    33             ["-datacarrier=1", "-datacarriersize=2"],
    34         ]
    


    polespinasa commented at 1:23 pm on May 6, 2025:
    0        self.expected_stderr = [
    1            "",  # node 0 has no deprecated options
    2            "Warning: Options '-datacarrier' or '-datacarriersize' are set but are marked as deprecated. They will be removed in future versions.",
    3            "Warning: Options '-datacarrier' or '-datacarriersize' are set but are marked as deprecated. They will be removed in future versions.",
    4            "Warning: Options '-datacarrier' or '-datacarriersize' are set but are marked as deprecated. They will be removed in future versions.",
    5        ]
    

    polespinasa commented at 1:24 pm on May 6, 2025:

    At the end of the test add:

    0        for i in range(self.num_nodes):
    1            self.stop_node(i, expected_stderr=self.expected_stderr[i])
    

    instagibbs commented at 12:41 pm on May 7, 2025:
    done
  48. polespinasa commented at 1:27 pm on May 6, 2025: contributor
    The code suggested uses the InitWarning and handles the errors on the failing test stoping the nodes manually
  49. Fiach-Dubh commented at 3:45 pm on May 6, 2025: none

    @moth-oss config options are not removed in this pull request.

    This config option is, however, marked for future removal and is being marketed for future removal by the author of the PR by being defined as “DEPRECATED”. I assume this will mean the removal of this toggle eventually when the controversy has died down and people aren’t paying attention.

    I believe this “DEPRECATED” definition is up for debate. Marking a toggle that has worked for my mempool policy and will work for my mempool policy in the future as “DEPRECATED” is FALSE & MISLEADING. It still works.

    On that basis alone, I’m going to have to concept NACK this PR.

  50. jamesob commented at 6:53 pm on May 6, 2025: contributor

    Concept ACK

    Leaving the knob in but changing its default is a good approach.

  51. in doc/release-notes-32406.md:3 in 3b3fafdb1c outdated
    0@@ -0,0 +1,4 @@
    1+- `-datacarriersize` default is now set to effectively uncapped limit, and the argument
    2+  is now used to limit the aggregate size of scriptPubKeys in a transaction's
    3+  OP_RETURN outputs. This allows multiple datacarrier output transactions while respecting
    


    ajtowns commented at 5:01 am on May 7, 2025:
    “multiple datacarrier outputs within a transaction”

    instagibbs commented at 12:41 pm on May 7, 2025:
    done
  52. 1440000bytes commented at 6:25 am on May 7, 2025: none
  53. instagibbs force-pushed on May 7, 2025
  54. instagibbs commented at 1:16 pm on May 7, 2025: member

    I believe this “DEPRECATED” definition is up for debate. Marking a toggle that has worked for my mempool policy and will work for my mempool policy in the future as “DEPRECATED” is FALSE & MISLEADING. It still works.

    Deprecation isn’t a value judgment but a statement that it will be removed in a future version.

  55. in src/init.cpp:910 in f4ae3126ea outdated
    905@@ -906,6 +906,10 @@ bool AppInitParameterInteraction(const ArgsManager& args)
    906         InitWarning(_("Option '-checkpoints' is set but checkpoints were removed. This option has no effect."));
    907     }
    908 
    909+    if (args.IsArgSet("-datacarriersize") || args.IsArgSet("-datacarrier")) {
    910+        InitWarning(_("Options '-datacarrier' or '-datacarriersize' are set but are marked as deprecated. They will be removed in a future version."));
    


    Sjors commented at 3:08 pm on May 7, 2025:

    f4ae3126ea9650c4da9aa80c0f8c5d12da84be2f: in the GUI this results in a popup every time you start the node.

    m_warnings could be used instead to treat it similar to the “This is a pre-release test build” message. Or we’d need a way to permanently dismiss a given warning string, which would be a GUI followup.

    For similar discussion, see #31916 (comment)


    polespinasa commented at 9:15 am on May 14, 2025:
    Nit: Will it be removed for sure? If there’s no clear consensus on removal, maybe is worth deleting the last part of the sentence. I’ve seen a lot of controversy bc of that the last two days.

    instagibbs commented at 1:06 pm on May 14, 2025:
    Directionally I want them marked for removal, even if it sits around for a few years. This can re-litigated with new information perhaps, but I don’t find it “honest” to not mark it as such unless the project changes direction.

    polespinasa commented at 1:37 pm on May 14, 2025:

    I agree, want it deleted too. Just added this comment because of the discussions some core members were having yesterday on twitter regarding what deprecation means bc people were getting crazy saying it means removal.

    But it’s just an observation, feel free to ignore.


    ajtowns commented at 3:10 am on May 15, 2025:

    Personally, I think it’s better to think of “deprecated” as meaning just “the devs recommend you don’t use this feature”.

    Despite recommending against using it, a feature might be left in the software ~forever, eg because it’s needed for backwards compatability (see gets() in C, deprecated in 1999, removed from the standard in 2011, but still usable with current compilers and included in glibc), or it might be removed in the next release, because everybody is willing and able to stop using it.

    Here, the reason to recommend not using it is because it will likely harm the efficiency of block reconstruction, slowing down block relay, and also make you use more bandwidth (requesting the tx then rejecting it, then receiving the tx again when it’s included in a block), for very little benefit – it won’t stop spam from getting into blocks, and likely won’t stop spam from getting into your mempool given other encoding methods are more efficient. I think the vast majority of developers contributing to core would recommend not setting this option to a lower value than the default, so I think marking it as “deprecated” is reasonable, even if there are a few developers who strongly disagree.

    Whether (or when) the option should also be removed rather than just deprecated depends on the trade-off between how many users continue to appreciate the feature, and how much effort it is to continue to support the code, and, perhaps, whether the presence of the feature is resulting in any harmful network effects. Given currently lots of people seem to want it, and maintaining it is very low effort, leaving the option in seems obviously sensible to me. YMMV, of course. And of course, even if it does get removed from core, if users actually want it, the code is open source so they can patch it back in or use a derivative that does that for them.


    polespinasa commented at 9:58 am on May 15, 2025:

    Personally, I think it’s better to think of “deprecated” as meaning just “the devs recommend you don’t use this feature”.

    Agree on all you said, that’s what I tried to argue in few words :)

  56. in test/functional/mempool_accept.py:342 in 062fbe7f89 outdated
    337+        )
    338+
    339+        # Multiple OP_RETURN and more than 83 bytes are standard since v30
    340+        tx = tx_from_hex(raw_tx_reference)
    341+        tx.vout.append(CTxOut(0, CScript([OP_RETURN, b'\xff'])))
    342+        tx.vout.append(CTxOut(0, CScript([OP_RETURN, b'\xff' * 10000])))
    


    Sjors commented at 3:17 pm on May 7, 2025:
    062fbe7f894136ee1f9d52c62d42bdf7b2cc1c0a: could comment here that both MAX_SCRIPT_ELEMENT_SIZE (520) and MAX_SCRIPT_SIZE (10,000) are only evaluated when spending an output, not when creating one. This is why 10K bytes fit here despite the 4 byte overhead from OP_RETURN, OP_PUSHDATA2 and two length bytes.

    instagibbs commented at 12:08 pm on May 8, 2025:
    I can add a note if I touch the PR again

    theStack commented at 10:59 pm on May 8, 2025:
    Seeing this constant also tricked me into thinking that the script size limit might play a role here. Agree that a comment would be nice, or maybe just bump it up to something much higher like e.g. 50000.

    instagibbs commented at 2:55 pm on May 9, 2025:
    added a comment and bumped the value
  57. in src/policy/policy.cpp:147 in 4426025fef outdated
    147-            nDataOut++;
    148-        else if ((whichType == TxoutType::MULTISIG) && (!permit_bare_multisig)) {
    149+        if (whichType == TxoutType::NULL_DATA) {
    150+            unsigned int size = txout.scriptPubKey.size();
    151+            if (size > datacarrier_bytes_left) {
    152+                reason = "datacarrier";
    


    Sjors commented at 3:33 pm on May 7, 2025:

    4426025fef23b014da8b8caab64239ba8f46a297: although this more specific error message is an improvement, there might be software out there that expects scriptpubkey. If so, it would be pointless for them to change their code for something that we’re deprecating anyway.

    I asked co-pilot to look (update: had to make it filter altcoins a few times):


    Got it! Ignoring the websites, here are the remaining results relevant to “multi-op-return,” excluding altcoins and websites:

    1. Mako:

      • Implements transaction validation logic to ensure only one OP_RETURN output is allowed.
    2. sBTC-Bridge:

      • Handles transaction rejection for multiple OP_RETURN outputs in policy checks.
    3. Timgates42’s Mako:

      • Another variant of multi-op-return validation logic within the Mako project.

    These repositories demonstrate clear implementations of the multi-op-return checks. Let me know if you’d like further details!


    Sjors commented at 3:47 pm on May 7, 2025:
    Make is an alternative node implementation, so it doesn’t use Bitcoin Core.

    ajtowns commented at 5:03 am on May 8, 2025:
    The sBTC-Bridge code linked will just return “unknown error” rather than a more informative message. “multi-op-return” isn’t appropriate here, the “datacarrier” error will trigger even for a single OP_RETURN that’s larger than the limit. Having “scriptpubkey” here doesn’t really seem much more helpful here than “unknown error”.

    Sjors commented at 6:43 am on May 8, 2025:

    (I looked for multi-op-return rather than scriptpubkey because it’s a more unique string)

    will just return “unknown error”

    Right, so I was only looking for clients that call our RPC and parse the message. It doesn’t matter what messages those applications emit themselves.


    instagibbs commented at 12:08 pm on May 8, 2025:
    Leaving as-is for now
  58. Sjors approved
  59. Sjors commented at 3:38 pm on May 7, 2025: member

    ACK 8c360d78b13fa7136db8d6e1e0af5d05f5141a64

    The GUI popup can be fixed in a followup (before v30).

  60. ajtowns commented at 4:59 am on May 8, 2025: contributor

    Deprecation isn’t a value judgment but a statement that it will be removed in a future version.

    Deprecation doesn’t even always mean it will actually be removed in a future version; see #32423 eg.

  61. mzumsande commented at 3:24 pm on May 8, 2025: contributor
    Concept ACK / Approach ACK, wrote the reason in the other PR
  62. theStack approved
  63. theStack commented at 11:01 pm on May 8, 2025: contributor
    Concept and code-review ACK 8c360d78b13fa7136db8d6e1e0af5d05f5141a64
  64. instagibbs force-pushed on May 9, 2025
  65. iicuriosity commented at 8:44 pm on May 10, 2025: none
    Concept NACK for the same reason as #32359. Why would you remove a feature with a valid use case?
  66. pithosian commented at 9:27 am on May 11, 2025: none

    Code ACK, conceptual NACK purely on option deprecation (comments here and in the mailing list; the justifications for deprecating/removing these options appear faulty to me).

    Edit to summarize the points already made, because there’s discussion going on here now, with some claiming arguments they’re making which have been addressed haven’t been, often repeating points they already made elsewhere without adding anything new:

    1. Any delay in a miner receiving a compact block because nodes along the shortest (time) path for that compact block to reach them are missing transactions in that block can be entirely removed. Compact block relay doesn’t need to be delayed by mempool filters. See mailing list.
    2. Globally consistent mempool policy is not possible, but you don’t need it for transactions with larger-than-80-byte OP_RETURNS to be relayed. You just need enough nodes with more permissive relay policy. Librerelay is already an option you could be pushing people to run, and this change would allow people to achieve the same using Core without deprecating and removing options noderunners have had for years to manage their own resources. Deprecation and removal of these flags achieves absolutely nothing (besides encouraging users to run alternative implementations).
    3. Downloading a transaction twice is not the same as downloading it once, and uploading it N times, even where N is 1. Upload bandwidth is not download bandwidth. On many residential networks, and particularly cellular, upload bandwidth is significantly lower. I can download 30x as much as I can upload in any given period. See linked comment.
    4. A ’new’ point I’ve only made away from the ML and GitHub: no, missing some transactions from your mempool doesn’t make all that much of a difference when setting fees. You still have a view of what’s getting confirmed, and a view of all the non-filtered transactions at the top of your own mempool. As somebody who has actually run a filtered mempool during periods where those filters were excluding many transactions being included within the next few blocks, it wasn’t difficult to set fees. Made other points in the above linked comment, but not yet relevant to the discussion going on in this PR.
  67. k98kurz commented at 4:54 pm on May 11, 2025: none

    @BitcoinMechanic It does no harm to fee estimation or block propagation. Nodes can and do cache transactions they reject from their mempools making compact blocks just as quick to verify regardless of if some of their contents was filtered.

    This is the first I have heard about this caching of non-standard transactions. If this is indeed the case, then the small-block-propagation-performance-degradation argument is largely invalidated, but this requires that the cache be sufficiently large and long-lived that the cache + mempool will effectively match a mempool without the standardness restrictions. What are the characteristics of this caching behavior, and does it actually prevent degradation of small block propagation in practice?

  68. liviu-liviu commented at 7:11 pm on May 11, 2025: none
    Concept NACK We should create more (and possibly easier) avenues that help node operators express their will. This PR is moving us in the opposite direction.
  69. in src/init.cpp:638 in 94ad73ff33 outdated
    634@@ -635,9 +635,9 @@ void SetupServerArgs(ArgsManager& argsman, bool can_listen_ipc)
    635     argsman.AddArg("-dustrelayfee=<amt>", strprintf("Fee rate (in %s/kvB) used to define dust, the value of an output such that it will cost more than its value in fees at this fee rate to spend it. (default: %s)", CURRENCY_UNIT, FormatMoney(DUST_RELAY_TX_FEE)), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::NODE_RELAY);
    636     argsman.AddArg("-acceptstalefeeestimates", strprintf("Read fee estimates even if they are stale (%sdefault: %u) fee estimates are considered stale if they are %s hours old", "regtest only; ", DEFAULT_ACCEPT_STALE_FEE_ESTIMATES, Ticks<std::chrono::hours>(MAX_FILE_AGE)), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
    637     argsman.AddArg("-bytespersigop", strprintf("Equivalent bytes per sigop in transactions for relay and mining (default: %u)", DEFAULT_BYTES_PER_SIGOP), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
    638-    argsman.AddArg("-datacarrier", strprintf("Relay and mine data carrier transactions (default: %u)", DEFAULT_ACCEPT_DATACARRIER), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
    639+    argsman.AddArg("-datacarrier", strprintf("(DEPRECATED) Relay and mine data carrier transactions (default: %u)", DEFAULT_ACCEPT_DATACARRIER), ArgsManager::ALLOW_ANY, OptionsCategory::NODE_RELAY);
    


    ajtowns commented at 7:39 am on May 12, 2025:

    I’d suggest combining this commit with the “Init: warn if set” one.

    I’m -0 on deprecating these; I’m not particularly bothered if they are marked as deprecated, but it doesn’t seem necessary to me, and at least for now there seem to be a bunch of users who like the option.


    instagibbs commented at 2:13 pm on May 13, 2025:
    combined commits, still leaning towards a long deprecation cycle to not mislead longer term
  70. in doc/release-notes-32406.md:4 in c164064c26 outdated
    0@@ -0,0 +1,4 @@
    1+- `-datacarriersize` default is now set to effectively uncapped limit, and the argument
    2+  is now used to limit the aggregate size of scriptPubKeys in a transaction's
    3+  OP_RETURN outputs. This allows multiple datacarrier outputs within a transaction while respecting
    4+  the set limit. Both `-datacarrier` and `-datacarriersize` are marked as deprecated. (#32406)
    


    ajtowns commented at 9:00 am on May 12, 2025:

    I’d probably have phrased this more like:

    Updated Settings

    • -datacarriersize is increased to 100,000 which effectively uncaps the limit (as the maximum transaction size limit will be hit first). It can be overridden with -datacarriersize=83 to revert to the limit enforced in previous versions. Both -datacarrier and -datacarriersize options have been marked as deprecated and are expected to be removed in a future release. (#32406)
    • Multiple data carrier (OP_RETURN) outputs in a transaction are now permitted for relay and mining. The -datacarriersize limit applies to the aggregate size of the scriptPubKeys across all such outputs in a transaction. (#32406)

    Feel free to take or ignore as you like.


    instagibbs commented at 2:12 pm on May 13, 2025:
    taken with minor modifications
  71. instagibbs commented at 1:25 pm on May 12, 2025: member
    @k98kurz It’s not generally true, that buffer is limited to 100 txns total, with no ratelimiting whatsoever. Please take the time to do some background reading: https://delvingbitcoin.org/t/stats-on-compact-block-reconstructions/1052
  72. Sjors commented at 1:33 pm on May 12, 2025: member

    re-ACK c164064c266c518588d9d9175f9b14140ee751b6

    Since my last review it adds MAX_SCRIPT_ELEMENT_SIZE to the test and uses a larger payload for it.

  73. 1ma commented at 1:52 pm on May 12, 2025: none
    Concept NACK, for the same reasons stated in #32359 and #28130
  74. ajtowns approved
  75. ajtowns commented at 1:54 pm on May 12, 2025: contributor

    ACK c164064c266c518588d9d9175f9b14140ee751b6

    I believe this PR makes the following changes:

    • allows multiple data carrier outputs in a single transaction (removing multi-op-return standardness failures)
    • puts multiple data carrier outputs under the same limit (giving a new datacarrier standardness failure when the limit is exceeded instead of scriptpubkey)
    • increases the limit’s default from 83 to 100,000, making the limit irrelevant
    • deprecates the two datacarrier settings
    • simplifies some tests (that would otherwise fail due to the warning about setting the deprecated option)
    • adds some tests for the new multi-datacarrier behaviour

    I think I’d slightly favour not deprecating the options, and only increasing the limit to cover the real use cases we know of, so perhaps 160 bytes. I’m not sure I’d even rate those as a nit, though.

    I don’t see any issue with allowing multiple op-returns – only using one where possible is already encouraged via it being cheaper in fees, and there’s otherwise no significant impact on nodes that I can see.

    I don’t think this PR will have any impact on fee estimation (the only way that would occur is if a large majority of the mempool were transactions with multiple OP_RETURN outputs or an OP_RETURN output larger than 83 bytes, and that’s not the case).

    I think the impact of this PR on block relay is likely to also be very small; probably slightly negative in the short term, as more blocks are mined with txs with more/larger OP_RETURNs before many nodes in the network include such txs in their mempool, and slightly positive in the long term, as more nodes in the network do include those txs in their mempools.

    Based on citrea’s Clementine design, I think this will slightly lower how much the utxo set gets spammed – trying to get every bit of data published on the blockchain moved to the witness or reduced to a commitment still seems possible to me, but given the complexity of BitVM-style protocols and bridges in general, I suspect it’s unlikely that developers of those protocols will be willing to spend much time shaving off bytes here and there like we might otherwise like. Given that, it seems better to make the easiest solution one that also minimises long term harm, and leave the incentive for shaving off bytes to just be minimising fees.

  76. instagibbs commented at 2:28 pm on May 12, 2025: member
    @1ma @iicuriosity I could not find your comment with justification in the other PR. Please edit and add a link to your concept NACK
  77. theStack approved
  78. theStack commented at 2:49 pm on May 12, 2025: contributor

    re-ACK c164064c266c518588d9d9175f9b14140ee751b6

    (test-only change since my previous ACK; as per git diff 8c360d78 c164064c, the review comments #32406 (review) ff. were addressed)

  79. aviv57 commented at 4:41 pm on May 12, 2025: none

    Concept ACK, if this won’t be merged, actors will mine their non-standard transactions by contacting the miners directly and not via the p2p network.

    in this way anyone willing to change their own mempool policy is still able to do it

  80. chrisguida commented at 4:41 pm on May 12, 2025: none
    Concept NACK, same rationale as #32359 (comment)
  81. instagibbs commented at 5:41 pm on May 12, 2025: member
    @donaldevinev1 (just in case this isn’t an LLM review) there are no quadratics introduced here, downstream parsers doing bizarre things is not in scope for this PR.
  82. xstoicunicornx commented at 6:20 pm on May 12, 2025: none

    Bitcoin’s success hinges on its decentralization and accessibility.

    Moreover, out-of-band services that cater to data-carrier transactions are not inherently harmful to the network. They exist because Bitcoin was intentionally not designed as a general-purpose data storage layer, but rather as a secure payments system. Private markets offering these services reflect a willingness to pay for non-standard transactions, and this market demand does not justify altering Bitcoin’s core design to subsidize these use cases.

    These two statements are incongruent with each other, as out-of-band services directly undermine the decentralization and accessibility of the network by eroding the transparency of the fee market as well as miner rewards.

  83. instagibbs commented at 6:26 pm on May 12, 2025: member
    @xstoicunicornx It’s LLM spam
  84. moth-oss commented at 6:57 pm on May 12, 2025: none

    @moth-oss config options are not removed in this pull request.

    It is deprecating the option though. I don’t think deprecating it now is the right move. Increase the default (ideally not disabled/max) and see how things go before making the decision to deprecate it.

  85. willcl-ark commented at 7:08 pm on May 12, 2025: member

    Reminder that this project’s current moderation guidelines describe what we expect of contributors to this repository.

    To keep it productive, especially on threads like these please:

    • Stay on-topic: discuss the patch, not Bitcoin in general.
    • Critique ideas, not motives.
    • Contribute new signal: avoid repeating arguments already made earlier (these will start being marked as duplicate, to make the thread easier to navigate).

    Note that this repository is not a general bitcoin discussion board. It’s one step removed from general discussions as contributors here discuss changes via commenting on new revisions of the source code. For protocol-wide chat use the mailing list, Delving Bitcoin, Reddit, X, etc. There are now many conversations taking place on this very topic :)

    If you think your comment has been moderated incorrectly, edit your comment (or not) and raise your concern in bitcoin-core/meta.

  86. 1440000bytes commented at 7:20 pm on May 12, 2025: none

    @moth-oss config options are not removed in this pull request.

    It is deprecating the option though. I don’t think deprecating it now is the right move. Increase the default (ideally not disabled/max) and see how things go before making the decision to deprecate it.

    Deprecation notice can last for years: #32406 (comment)

  87. moonbootspleb commented at 7:25 pm on May 12, 2025: none

    Concept NACK.

    While appearing to be a compromise, this PR uniquely does not achieve even partial success of the desired outcomes of either faction as stated in #32359 and as such is ineffective.

    You might say this is the nature of a compromise, but I find this “splitting the difference” strategy to be uniquely harmful in its result.

    This PR achieves none of the stated benefits of #32359 which were:

    • better fee estimation/ unification of mempool state
    • reduction of harm through the use of OP_RETURN rather than witness storage
    • reduction of mining centralization through incentivizing use of “honest” data use case

    It also substantially changes mempool filter behavior such that nodes with default configurations will have a significant relay change.

    The ultimate result of this particular PR will be:

    • further fracturing of the mempool policies
    • further mining centralization as a means of reducing cost of non-standard transactions
    • fee estimation continuing to be limited in scope
    • continued use of witness data field for arbitrary data incentivized by lower cost from witness discount

    For these reasons, I think it is unwise to handle these changes piecemeal and would advocate for closing this PR and revisiting mempool transaction standardness policy when transaction consensus cleanup has been successfully achieved. This makes the most sense since proponents of both sides of this argument say they don’t want arbitrary data in the blockchain and that mempool policy should be tightly coupled with consensus policy.

  88. bigshiny90 commented at 7:27 pm on May 12, 2025: none

    @moth-oss config options are not removed in this pull request.

    It is deprecating the option though. I don’t think deprecating it now is the right move. Increase the default (ideally not disabled/max) and see how things go before making the decision to deprecate it.

    Deprecation notice can last for years: #32406 (comment)

    Deprecation is a clear signal that a feature will be removed in the future. Having an unspecified timeline till deprecation isn’t not an argument against OP’s point.

    if the intention is that a feature may or may not be removed in the future, than deprecating it now is incorrect.

  89. n4HeVQSGqDeEu6 commented at 7:31 pm on May 12, 2025: none

    Concept NACK.

    There clearly is no consensus, moving forward now will completely destroy the core reputation.

  90. k98kurz commented at 8:05 pm on May 12, 2025: none

    @BootsStribling

    This PR achieves none of the stated benefits of #32359 which were:

    • reduction of harm through the use of OP_RETURN rather than witness storage

    The ultimate result of this particular PR will be:

    • continued use of witness data field for arbitrary data incentivized by lower cost from witness discount

    The harm reduction argument is that OP_RETURN use allows L2 users an alternative to stuffing arbitrary data into non-provably unspendable outputs that bloat the UTXO set. The witness data bloat was the focus of a PR from several years ago by luke-jr that added additional data-carrier filters, not this or the related PR by petertodd. The origin of this is an L2/sidechain use case that requires 144 bytes of ZKP data in transaction outputs – they cannot rely upon stuffing it into witness data because so many nodes already filter out such transactions. The argument is that the current OP_RETURN filter incentivizes more harmful UTXO bloat compared to larger OP_RETURN outputs which can be dropped from the UTXO set.

    I find that argument somewhat persuasive; however, after reading more arguments and giving it a bit more thought, concept NACK for deprecating the option instead of just changing the default configuration to fit the specific use case that started this whole controversy. From a UX design perspective, software should have sane defaults but not prevent power users from tuning, experimenting, and/or breaking their own systems.

  91. ariel-lore commented at 9:33 pm on May 12, 2025: none

    Concept NACK

    Deprecating the feature is an indication that the feature could be removed in the future without discussion. It is not different enough to just removing the feature entirely today, thus is isn’t a compromise. Some are saying that features can stay deprecated for years but that’s a false equivalence because those features are not as controversial as the OP_RETURN limit.

    Increasing the default OP_RETURN limit to 100,000 is also contentious and should not be merged without further discussion on the mailing list. Forcing node runners to do extra setup work to limit data carrier transactions is against the preference of a significant group of node runners. Increasing the default limit still incentivizes them to migrate to alternative node implementations that don’t require extra setup steps. Keeping the default OP_RETURN limit at 80 bytes or a somewhat sensible 144 bytes is a better compromise. Data usage above 144 bytes is less expensive in witness data anyway because of the discount so no reason to raise the default limit above 144 bytes.

  92. gmaxwell commented at 1:49 am on May 13, 2025: contributor

    utACK while I’m doubtful my comment should carry much weight, I’m wary of the public presentation weight up vacuous opposition, which ought count for nothing, against an absence of positive comments (because the choice is obvious enough that little need be said)… and the most effective thing I can do on that point is to get counted.

    I have a slight preference for the closed PR that removes it completely. This is primarily because removing it completely avoids some predictable meta-abuse (e.g. the same people who deceived others about this PR existing, claiming they narrowly saved the day when no great doom happens). Also I have some small concern that if/when the contributors decide total removal would be best that they’ll refrain from doing so out of a justified fear of another public abuse campaign, which could be avoided by just doing it now. That said, since these aren’t points that go to the substance of the change itself, the other PR is currently closed, I don’t think they matter much and I wouldn’t fault someone for saying such reasoning should be completely disregarded. I’ll refrain for further meta-points, though I think they are worth at least some consideration because they are akin to technical debt created by the proposal (but they are political rather than technical). As far as actually when the settings are removed after this PR, if ever, I think that is mostly just the judgement of the maintainers in terms of combinatorics testing cost and codebase cleanliness.

    I am November Alpha Charlie Kilo (drat bot!) on suggestions that leave the limit slightly increased for some user or another (as mentioned by @ariel-lore @ajtowns and the later @k98kurz comment) unless they come with a credible argument that ~all the hashpower will enforce the same limit. Without that, a increased limit but still less than what many miners will do leaves the harmful gap between relay and mining policy. I think it would be an obviously poor choice that undermines much of the principled argument for making the change at all. (and to be clear, I would agree that all substantive discrepancies ought to be closed either by convincing miners to change behavior or removing the restrictions, and if a discrepant rule is hiding a DOS vulnerability it must be fixed either in code or consensus rules). If there were a serious argument that progress could be made getting most hashpower to enforce some other limit, then I would agree with trying to achieve that. My understanding however is that there is no real prospect of that.

    Turning to the substance above, @k98kurz advances an argument that rejected transactions are cached as a reason relay/mining mismatches don’t hurt block propagation. @instagibbs provides a technically correct response that the caching is insignificant and demonstratively doesn’t avoid the problem, but I think it’s important to give a broader answer in the context of this PR and the default relay policy of Bitcoin Core: If the transaction is never relayed to you because all your peers dropped it then no amount of caching can help.

    Caching, when it is successful, reduces the bandwidth inflation from transmitting the txn twice (once to just throw it out, the second when it shows up in a block) and it may diminish the block relay harm caused by a small number of mining-inconsistent nodes. Improving that caching might help a little but especially in terms of widely deployed policy it can’t help much because nodes simply won’t see the transactions. Steel-manning the opposition a little: It could be argued that block relay could be improved to be less sensitive to missed transactions. I agree that it could and should, but doing so is very technically complex, unlikely to be accomplished safely in the social climate exhibited in these PRs, and that even at it’s best is inherently inferior to already having the transaction and having already validated it. The best improvements we know of also trade more bandwidth usage to diminish the huge round trip delays, so keeping the discrepancy as small as can be is important to them. Some rare discrepancy is unavoidable, but filtering transactions that will get mined is an entirely unforced error.

    [And as co-designer of the block relay mechanism: the extras pool is intended to deal with replacement races– when a new replacement comes in for a little while it’s anyone’s guess which one a just found block will have. Depending on the ordering the miner saw the transactions a block could have any version, even when replacement would be denied by the replacement rules. Most of the time in the Bitcoin p2p network we took an assumption that relay and mining policy would not have any persistent substantive differences.]

    The comment of @BootsStribling appears to be disconnected from reality in a manner characteristic of LLM hallucinations. It takes a number of positions which would be interesting if they were true or substantiated in any way by the comment. I don’t think there is any serious dispute that this PR will not make the default behavior more consistent with what gets mined.

    As far as fee estimation goes, the actual estimator is designed to be robust against missed transactions (they’re basically ignored)– but I know that I and many other people I know look at an actual mempool to make decisions on what to bid against… and having transactions unknown to us that will likely show up in the next block makes it harder to make a good decision. Again, I don’t think there is any serious dispute on this point.

    There are a number of other easily addressed fallacious points I’ve seen raised in opposition to this elsewhere, but since they haven’t (yet) been raised here I won’t (yet) readdress them, I don’t think they’re useful even as steelmanning the opposition because none of my counters on those have been countered. If anyone has any interest they can go follow my recent reddit traffic (u/nullc) which has been ~exclusively on this subject.

    Cheers.

    [And if anyone wants to respond to me but can’t do so here for whatever reason, please reach out directly… my email is available to anyone!]

  93. in src/policy/policy.cpp:150 in 4426025fef outdated
    150+            unsigned int size = txout.scriptPubKey.size();
    151+            if (size > datacarrier_bytes_left) {
    152+                reason = "datacarrier";
    153+                return false;
    154+            }
    155+            datacarrier_bytes_left -= size;
    


    Kixunil commented at 2:15 am on May 13, 2025:
    This doesn’t account for the length of serialization of value (8B). So after this change if one wants to keep the 80B limit, the transaction can actually increase by a lot more (80*11 if I count correctly). While I find the arguments for keeping the limit unconvincing this seems to not follow the intention. (But it is true that the amount cannot contain arbitrary data.)

    ajtowns commented at 9:38 am on May 13, 2025:

    This is only a limit on the arbitrary data you’re encoding in OP_RETURN scriptPubKeys, not a limit on the overall transaction size.

    If you want to encode additional data, there are three approaches: extend an existing PUSH with extra data, add an additional PUSH, or add an additional output. These have an overhead of roughly 0, 1 and 2 with respect to this limit, and an overhead of 0, 1 and 10 vbytes wrt transaction weight with a corresponding impact on fees needed. If you did indeed have 83 OP_RETURN outputs, then a datacarriersize=83 limit would mean that you’d spent your entire datacarrier budget on that overhead, without actually including any data. It would also only add 830vb to your transaction, which is about the same as adding 5 or 6 p2pkh inputs to your transaction.


    Kixunil commented at 12:26 pm on May 13, 2025:

    If you did indeed have 83 OP_RETURN outputs, then a datacarriersize=83 limit would mean that you’d spent your entire datacarrier budget on that overhead, without actually including any data.

    Yes, but isn’t it the entire point of the parameter? “to not flood the chain with garbage”

    I personally don’t care about it, just trying to prevent the brigaders claiming “Core snuck in a backdoor that can blow up the data size”.


    instagibbs commented at 2:14 pm on May 13, 2025:
    The options is ostensibly to stop people from data packing, not vbytes usage per se. The smaller the payload, the more overhead they are incuring.

    Kixunil commented at 9:53 pm on May 13, 2025:
    OK, I don’t really care. Especially since the exact people who are spamming the PR did not even answer this, I think they don’t care.

    luke-jr commented at 10:56 pm on May 13, 2025:
    This PR shouldn’t be merged at all, but obviously to even be considered bugs like this would have to be fixed. Don’t forget the scriptPubKey lengths.

    Kixunil commented at 3:45 pm on May 14, 2025:

    Good that finally someone commented. I think that this should go in line with the wishes of limit-promoters so if adding zero amount is undesirable it should be changed.

    Don’t forget the scriptPubKey lengths.

    IIUC these are accounted for by the size method.


    instagibbs commented at 4:01 pm on May 14, 2025:
    @Kixunil the serialized size isn’t included, but it was also not included in master. You could say it’s a bug in the current code as well, but my intention here is to do no worse from payload perspective.
  94. Kixunil commented at 2:24 am on May 13, 2025: none

    Concept ACK, the code looks sensible from brief look, though it’s unclear what the intention of the parameter was.

    I’d also like to ask the opposition to address the numerous arguments about the limit driving centralization and causing long-term storage problems as well as causing the size of the chain to increase in the mailinglist. I haven’t seen a single person attempting to do that yet.

  95. 1440000bytes commented at 5:51 am on May 13, 2025: none

    @FractalEncrypt you can use python to create a transaction with multiple OP_RETURN, fundrawtransaction to add inputs, signrawtransactionwithwallet to sign and sendrawtransaction to broadcast it later. It will be listed in the output for getrawmempool once broadcasted.

     0from bitcoin.core import CTransaction, CTxOut, CScript, b2x
     1from bitcoin.core.script import OP_RETURN
     2
     3inputs = [] 
     4outputs = [
     5    CTxOut(0, CScript([OP_RETURN, b'deadbeef'])),
     6    CTxOut(0, CScript([OP_RETURN, b'cafebabe']))
     7]
     8
     9tx = CTransaction(inputs, outputs)
    10
    11raw_tx = b2x(tx.serialize())
    12print(raw_tx)
    

    Example tx:

    0010000000001019eee1ba28a55abf31bd52501515d5c701478ea7edc27ba0b5e8a299e61bb81260000000000fdffffff0300000000000000000a6a08646561646265656600000000000000000a6a086361666562616265c0eb052a010000002251203e9d78540e0bb81dbb409b0ef517c11e2cb94d1ffab46ace3bcc45ccee39a1e80247304402205ae327bf416966d9e1567d4c141a7e408ed53e2863c2400023345bc3ce7d0d35022067f00d09a499e4282d33598c76f0bfa3d34b3c52810a5e39f61d100a607499ed0121020ead75a237556f4986a78e815700c3b4215679ffca5f25539f06ef38b766282900000000
    
     0{
     1  "txid": "3d0f6a1d4b0854968b9af9f8d8f477271adec96e85266e9c49ef08eec35970db",
     2  "hash": "c2765e5146a7cc07a81294ebea71c57b30b3083947460d7412f2f00115fe62a2",
     3  "version": 1,
     4  "size": 241,
     5  "vsize": 160,
     6  "weight": 637,
     7  "locktime": 0,
     8  "vin": [
     9    {
    10      "txid": "2681bb619e298a5e0bba27dc7eea7814705c5d510125d51bf3ab558aa21bee9e",
    11      "vout": 0,
    12      "scriptSig": {
    13        "asm": "",
    14        "hex": ""
    15      },
    16      "txinwitness": [
    17        "304402205ae327bf416966d9e1567d4c141a7e408ed53e2863c2400023345bc3ce7d0d35022067f00d09a499e4282d33598c76f0bfa3d34b3c52810a5e39f61d100a607499ed01",
    18        "020ead75a237556f4986a78e815700c3b4215679ffca5f25539f06ef38b7662829"
    19      ],
    20      "sequence": 4294967293
    21    }
    22  ],
    23  "vout": [
    24    {
    25      "value": 0.00000000,
    26      "n": 0,
    27      "scriptPubKey": {
    28        "asm": "OP_RETURN 6465616462656566",
    29        "desc": "raw(6a086465616462656566)#2yjnw38t",
    30        "hex": "6a086465616462656566",
    31        "type": "nulldata"
    32      }
    33    },
    34    {
    35      "value": 0.00000000,
    36      "n": 1,
    37      "scriptPubKey": {
    38        "asm": "OP_RETURN 6361666562616265",
    39        "desc": "raw(6a086361666562616265)#kgaw0u9z",
    40        "hex": "6a086361666562616265",
    41        "type": "nulldata"
    42      }
    43    },
    44    {
    45      "value": 49.99998400,
    46      "n": 2,
    47      "scriptPubKey": {
    48        "asm": "1 3e9d78540e0bb81dbb409b0ef517c11e2cb94d1ffab46ace3bcc45ccee39a1e8",
    49        "desc": "rawtr(3e9d78540e0bb81dbb409b0ef517c11e2cb94d1ffab46ace3bcc45ccee39a1e8)#2q3qw2vr",
    50        "hex": "51203e9d78540e0bb81dbb409b0ef517c11e2cb94d1ffab46ace3bcc45ccee39a1e8",
    51        "address": "bcrt1p86whs4qwpwupmw6qnv80297prcktjngll26x4n3me3zuem3e585qf6mmjg",
    52        "type": "witness_v1_taproot"
    53      }
    54    }
    55  ]
    56}
    
  96. delta1 commented at 6:07 am on May 13, 2025: none
    Concept ACK
  97. in test/functional/mempool_datacarrier.py:56 in c164064c26 outdated
    56+
    57+        # If it is custom set to 83, the historical value,
    58+        # only 80 bytes are used for data (+1 for OP_RETURN, +2 for the pushdata opcodes).
    59+        custom_size_data = randbytes(CUSTOM_DATACARRIER_ARG - 3)
    60+        too_long_data = randbytes(CUSTOM_DATACARRIER_ARG - 2)
    61+        extremely_long_data = randbytes(99800)
    


    stickies-v commented at 10:52 am on May 13, 2025:

    nit: would prefer avoiding the literal by keeping MAX_OP_RETURN_RELAY, or at least document why you picked 99800.

     0diff --git a/test/functional/mempool_datacarrier.py b/test/functional/mempool_datacarrier.py
     1index 6347215b86..8eab4ce6b2 100755
     2--- a/test/functional/mempool_datacarrier.py
     3+++ b/test/functional/mempool_datacarrier.py
     4@@ -5,6 +5,7 @@
     5 """Test datacarrier functionality"""
     6 from test_framework.messages import (
     7     CTxOut,
     8+    MAX_OP_RETURN_RELAY,
     9 )
    10 from test_framework.script import (
    11     CScript,
    12@@ -53,7 +54,7 @@ class DataCarrierTest(BitcoinTestFramework):
    13         # only 80 bytes are used for data (+1 for OP_RETURN, +2 for the pushdata opcodes).
    14         custom_size_data = randbytes(CUSTOM_DATACARRIER_ARG - 3)
    15         too_long_data = randbytes(CUSTOM_DATACARRIER_ARG - 2)
    16-        extremely_long_data = randbytes(99800)
    17+        extremely_long_data = randbytes(MAX_OP_RETURN_RELAY - 200)
    18         one_byte = randbytes(1)
    19         zero_bytes = randbytes(0)
    20 
    21diff --git a/test/functional/test_framework/messages.py b/test/functional/test_framework/messages.py
    22index 007a8499c1..5549d8f753 100755
    23--- a/test/functional/test_framework/messages.py
    24+++ b/test/functional/test_framework/messages.py
    25@@ -73,6 +73,11 @@ WITNESS_SCALE_FACTOR = 4
    26 DEFAULT_ANCESTOR_LIMIT = 25    # default max number of in-mempool ancestors
    27 DEFAULT_DESCENDANT_LIMIT = 25  # default max number of in-mempool descendants
    28 
    29+
    30+# Default setting for -datacarriersize.
    31+MAX_OP_RETURN_RELAY = 100_000
    32+
    33+
    34 DEFAULT_MEMPOOL_EXPIRY_HOURS = 336  # hours
    35 
    36 MAGIC_BYTES = {
    

    instagibbs commented at 2:12 pm on May 13, 2025:
    taken
  98. in test/functional/mining_basic.py:190 in c164064c26 outdated
    185@@ -186,6 +186,9 @@ def test_blockmintxfee_parameter(self):
    186             assert tx_below_min_feerate['txid'] not in block_template_txids
    187             assert tx_below_min_feerate['txid'] not in block_txids
    188 
    189+            # Restart node to clear mempool for the next test
    190+            self.restart_node(0)
    


    stickies-v commented at 11:05 am on May 13, 2025:

    nit: I think it’s strange to let one test handle another test’s pre’s, would just keep this in test_block_max_weight?

     0diff --git a/test/functional/mining_basic.py b/test/functional/mining_basic.py
     1index 21451a9aaf..415a370b06 100755
     2--- a/test/functional/mining_basic.py
     3+++ b/test/functional/mining_basic.py
     4@@ -186,9 +186,6 @@ class MiningTest(BitcoinTestFramework):
     5             assert tx_below_min_feerate['txid'] not in block_template_txids
     6             assert tx_below_min_feerate['txid'] not in block_txids
     7 
     8-            # Restart node to clear mempool for the next test
     9-            self.restart_node(0)
    10-
    11     def test_timewarp(self):
    12         self.log.info("Test timewarp attack mitigation (BIP94)")
    13         node = self.nodes[0]
    14@@ -287,6 +284,7 @@ class MiningTest(BitcoinTestFramework):
    15         HIGH_FEERATE = Decimal("0.0003")
    16 
    17         # Ensure the mempool is empty
    18+        self.restart_node(0)
    19         assert_equal(len(self.nodes[0].getrawmempool()), 0)
    20 
    21         # Generate UTXOs and send 10 large transactions with a high fee rate
    

    instagibbs commented at 2:12 pm on May 13, 2025:
    style difference I guess? we assert the invariant we expect and I like subtests cleaning up their state in general for mempool stuff. Keeping as is.
  99. in test/functional/mempool_accept.py:339 in c164064c26 outdated
    334+        self.check_mempool_result(
    335+            result_expected=[{'txid': tx.rehash(), 'allowed': False, 'reject-reason': 'scriptpubkey'}],
    336+            rawtxs=[tx.serialize().hex()],
    337+        )
    338+
    339+        # Multiple OP_RETURN and more than 83 bytes, even if over MAX_SCRIPT_ELEMENT_SIZE
    


    stickies-v commented at 11:11 am on May 13, 2025:
    nit: would make sense to actually use test_framework.script.MAX_SCRIPT_ELEMENT_SIZE here?

    instagibbs commented at 2:12 pm on May 13, 2025:
    No, these are different concepts entirely. @Sjors asked for a comment and thought it wouldn’t hurt. Leaving as is or removing this if it adds to confusion. Thoughts?

    stickies-v commented at 3:05 pm on May 13, 2025:
    I just mean adding something like assert_greater_than(len(b'\xff' * 5_000), MAX_SCRIPT_ELEMENT_SIZE). Adding a docstring that is not actually enforced in tests is confusing and fragile imo. Unless this is already tested in some other way I’m missing? If it shouldn’t be tested, it probably shouldn’t be documented either?
  100. stickies-v approved
  101. stickies-v commented at 11:34 am on May 13, 2025: contributor

    ACK c164064c266c518588d9d9175f9b14140ee751b6. I’ve left a few nits, but nothing blocking, especially in a PR with this much activity.

    With the new default, I think users disabling or lowering datacarriersize are mostly reducing the performance of their own node, so I am in support of removing it completely. However, some people seem really keen on continuing to be able to do so, so pragmatically I think the approach taken in this PR is sensible. Changing the default is what’s really important here.

  102. juanitoddd commented at 1:48 pm on May 13, 2025: none

    Concept NACK

    There should be freedom and sovereignty on what users relay on their nodes mempool. To marked them as deprecated sets the precedent that in the future node runners wont be able to configure this.

  103. policy: uncap datacarrier by default
    Datacarrier output script sizes and output counts are now
    uncapped by default.
    
    To avoid introducing another startup argument, we modify the
    OP_RETURN accounting to "budget" the spk sizes.
    
    If a user has set a custom default, this results in that
    budget being spent over the sum of all OP_RETURN outputs'
    scripts in the transaction, no longer capping the number
    of OP_RETURN outputs themselves. This should allow a
    superset of current behavior while respecting the passed
    argument in terms of total arbitrary data storage.
    
    Co-authored-by: Anthony Towns <aj@erisian.com.au>
    496c5c2e92
  104. instagibbs force-pushed on May 13, 2025
  105. instagibbs force-pushed on May 13, 2025
  106. instagibbs commented at 2:21 pm on May 13, 2025: member
    Thanks to everyone who gave thoughtful review, I hopefully addressed all code-related questions, if not let me know.
  107. michaelfolkson commented at 2:45 pm on May 13, 2025: none

    Concept ACK for this and Concept ACK for #32359 unless @luke-jr/Knots argues convincingly that the removal of these config options in Core makes maintaining them in Knots an excessive amount of work.

    The objectives of Core’s mempool policy and Knots mempool policy are totally different. Knots is trying to filter what it defines as spam and Core isn’t. I don’t know if it makes sense for Core to try catering to Knots’ objectives when they clash with Core’s objectives. It makes more sense for Core to direct users who want to attempt to filter spam-like transactions to run Knots instead.

    That might be anathema to some but mempool policy isn’t consensus critical and consensus compatible forks like Knots are there for those who strongly disagree with the merge decisions or the direction of travel of Core on non-consensus changes. I don’t think anyone needs or wants this kind of discussion every time Core makes a change to mempool policy.

    Anyway, my two cents for what it is worth.

  108. test: remove unnecessary -datacarriersize args from tests af9f354c5a
  109. datacarrier: deprecate startup arguments for future removal 1ade56661f
  110. Add more OP_RETURN mempool acceptance functional tests
    Credit: Sjors Provoost and Antoine Poinsot
    1c4bf434db
  111. add release note for datacarriersize default change 35bcd8eed3
  112. instagibbs force-pushed on May 13, 2025
  113. Sjors commented at 3:37 pm on May 13, 2025: member

    re-ACK 35bcd8eed38130445aef5ebe217ab42248fa6f18

    Smaller code code changes can be left to followups, as it’s tedious to keep track of actual code review comments in this PR due to continued brigading.

    unless @luke-jr/Knots argues convincingly that the removal of these config options in Core makes maintaining them in Knots an excessive amount of work

    It really shouldn’t be. After this PR it’s a matter of changing the default. After we actually drop the deprecated option, it would be a matter of reverting that one commit.

    I don’t know if it makes sense for Core to try catering to Knots’ objectives

    In general Bitcoin Core provides very limited catering to forks, whether that’s altcoins, modified implementations or even just experimental features. We do have e.g. CLIENT_NAME "Bitcoin Core" that’s easy to override, and a CI flag SKIP_BRANCH_PUSH which replaced a hardcoded gui repository name. But those require some convincing to include, as I’ve experienced myself with #29274.

    It probably does not help (in motivating reviewers) if your fork’s main marketing message is to discredit upstream. @gmaxwell wrote:

    I have a slight preference for the closed PR that removes it completely.

    So do I conceptually, but at this point this PR has had more code review and some subsequent improvements, so I’d rather not switch back.

  114. bytejedi commented at 3:51 pm on May 13, 2025: none
    Bitcoin is a digital currency that OP_RETURN should NOT EXIST ever since the beginning. You all turning bitcoin to shitcoin. Look what you all did these years, just fix bugs and CVEs please.
  115. stickies-v approved
  116. stickies-v commented at 4:00 pm on May 13, 2025: contributor

    re-ACK 35bcd8eed38130445aef5ebe217ab42248fa6f18

    No changes except addressing review feedback:

    • squashing commits
    • improving release notes
    • test style changes

    Smaller code code changes can be left to followups, as it’s tedious to keep track of actual code review comments in this PR due to continued brigading.

    Agreed. Leaving nits here just for reference but they shouldn’t block progress.

  117. theStack approved
  118. theStack commented at 4:57 pm on May 13, 2025: contributor
    re-ACK 35bcd8eed38130445aef5ebe217ab42248fa6f18
  119. maflcko removed the label Needs release note on May 13, 2025
  120. in doc/release-notes-32406.md:3 in 35bcd8eed3
    0@@ -0,0 +1,3 @@
    1+- `-datacarriersize` is increased to 100,000 which effectively uncaps the limit (as the maximum transaction size limit will be hit first). It can be overridden with -datacarriersize=83 to revert to the limit enforced in previous versions. Both `-datacarrier` and `-datacarriersize` options have been marked as deprecated and are expected to be removed in a future release. (#32406)
    2+
    3+- Multiple data carrier (OP_RETURN) outputs in a transaction are now permitted for relay and mining. The `-datacarriersize` limit applies to the aggregate size of the scriptPubKeys across all such outputs in a transaction, not including the scriptPubKey size itself. (#32406)
    


    vostrnad commented at 10:23 pm on May 13, 2025:

    nit: “scriptPubKey” instead of “script” is needlessly specific when we’re already talking about outputs.

    0- Multiple data carrier (OP_RETURN) outputs in a transaction are now permitted for relay and mining. The `-datacarriersize` limit applies to the aggregate size of the scripts across all such outputs in a transaction, not including the script size prefix itself. (#32406)
    
  121. in test/functional/mempool_accept.py:365 in 35bcd8eed3
    361+        )
    362+
    363+        self.log.info("A transaction with an OP_RETURN output that bumps into the max standardness tx size.")
    364+        tx = tx_from_hex(raw_tx_reference)
    365+        tx.vout[0].scriptPubKey = CScript([OP_RETURN])
    366+        data_len = int(MAX_STANDARD_TX_WEIGHT / 4) - tx.get_vsize() - 5 - 4  # -5 for PUSHDATA4 and -4 for script size
    


    vostrnad commented at 10:23 pm on May 13, 2025:

    nit

    0        data_len = int(MAX_STANDARD_TX_WEIGHT / WITNESS_SCALE_FACTOR) - tx.get_vsize() - 5 - 4  # -5 for PUSHDATA4 and -4 for script size
    
  122. in test/functional/mempool_accept.py:367 in 35bcd8eed3
    363+        self.log.info("A transaction with an OP_RETURN output that bumps into the max standardness tx size.")
    364+        tx = tx_from_hex(raw_tx_reference)
    365+        tx.vout[0].scriptPubKey = CScript([OP_RETURN])
    366+        data_len = int(MAX_STANDARD_TX_WEIGHT / 4) - tx.get_vsize() - 5 - 4  # -5 for PUSHDATA4 and -4 for script size
    367+        tx.vout[0].scriptPubKey = CScript([OP_RETURN, b"\xff" * (data_len)])
    368+        assert_equal(tx.get_vsize(), int(MAX_STANDARD_TX_WEIGHT / 4))
    


    vostrnad commented at 10:23 pm on May 13, 2025:
    nit: This might be cleaner with tx.get_weight, avoids the division by 4. (same on line 373)
  123. in test/functional/test_framework/mempool_util.py:45 in 35bcd8eed3
    40@@ -41,7 +41,7 @@ def fill_mempool(test_framework, node, *, tx_sync_fun=None):
    41     """Fill mempool until eviction.
    42 
    43     Allows for simpler testing of scenarios with floating mempoolminfee > minrelay
    44-    Requires -datacarriersize=100000 and -maxmempool=5 and assumes -minrelaytxfee
    45+    Requires -maxmempool=5 and assumes -minrelaytxfee
    46     is 1 sat/vbyte.
    


    vostrnad commented at 10:24 pm on May 13, 2025:
    nit: These two comment lines can now be merged.
  124. vostrnad commented at 10:24 pm on May 13, 2025: none

    utACK 35bcd8eed38130445aef5ebe217ab42248fa6f18

    All nits that can be done in a follow-up.

  125. 1440000bytes commented at 10:57 pm on May 13, 2025: none

    ACK https://github.com/bitcoin/bitcoin/pull/32406/commits/35bcd8eed38130445aef5ebe217ab42248fa6f18

    I have tested this branch with transactions having multiple OP_RETURN. Steps are shared in this comment marked as off-topic: #32406 (comment)

    I am interested to see the usage of multiple OP_RETURN after v30 if this pull request gets merged. Written a post about their usage in coinjoin and other thoughts are shared on delving.

    Last review: #32406 (comment)

  126. jmatcho commented at 0:37 am on May 14, 2025: none
    Concept NACK for changing an existing default and deprecating an existing feature that take control away from the people’s nodes.
  127. bigshiny90 commented at 2:25 am on May 14, 2025: none

    Concept NACK

    there is no clear need for this change. Use cases expressed are minor, with only hope of future usage. Expressed use cases can be accommodated in a simpler way with a minor size change. If Core would like this op return change for the future, do the work and convince node runners to change their defaults. Arguments for block propagation speed and fee estimates, though technically sound, are potentially minor compared to forcing the default size to max. Slow this down and move towards this goal if this is what you ultimately think is best, but gradually, allowing node runners to respond instead of being forced.

    Definitely do not deprecate

  128. BitcoinMechanic commented at 2:28 am on May 14, 2025: none
    Did the NACK/ACK bot break? Doesn’t seem to be updating? @DrahtBot
  129. maflcko commented at 5:29 am on May 14, 2025: member

    Did the NACK/ACK bot break? Doesn’t seem to be updating? @DrahtBot

    This is a known issue. See https://github.com/maflcko/DrahtBot/issues/51. edit: fixed

  130. DrahtBot requested review from ajtowns on May 14, 2025
  131. DrahtBot requested review from mzumsande on May 14, 2025
  132. DrahtBot requested review from jamesob on May 14, 2025
  133. DrahtBot requested review from Kixunil on May 14, 2025
  134. DrahtBot requested review from polespinasa on May 14, 2025
  135. delta1 commented at 8:37 am on May 14, 2025: none
    @gmaxwell drahtbot has categorized your comment as nack instead of ack
  136. polespinasa commented at 9:32 am on May 14, 2025: contributor

    code review ACK 35bcd8eed3

    left one small comment to avoid further discussions…

  137. maflcko commented at 9:38 am on May 14, 2025: member

    drahtbot has categorized your comment as nack instead of ack

    When a comment contains both ack and nack (in uppercase), the bot just picks one. Obviously, it is off-topic to discuss here, but I think there is a misunderstanding what the goal of the summary comment by the bot is. The goal is not to have everyone “register” their “vote”. It is simply a best-effort overview to have a clickable link to every reviewer’s most recent review comment. The technical content of those review comments is what matters, not how many are registered in what category. This is also explained in the comment above the table: “See the guideline for information on the review process.”

  138. ajtowns commented at 7:52 am on May 16, 2025: contributor

    ACK 35bcd8eed38130445aef5ebe217ab42248fa6f18 – only minor tweaks since previous review

    I wrote:

    I don’t think this PR will have any impact on fee estimation (the only way that would occur is if a large majority of the mempool were transactions with multiple OP_RETURN outputs or an OP_RETURN output larger than 83 bytes, and that’s not the case).

    gmaxwell wrote:

    As far as fee estimation goes, the actual estimator is designed to be robust against missed transactions (they’re basically ignored)– but I know that I and many other people I know look at an actual mempool to make decisions on what to bid against… and having transactions unknown to us that will likely show up in the next block makes it harder to make a good decision.

    Prior to this discussion, there were only 61 txs included in blocks in the prior ~11 months (blocks 846,000 to 894,000) that this PR would allow to be relayed (using up ~0.004% of blockspace during that period). Since this discussion started, that’s increased, eg to ~0.16% with ~11,800 txs (in blocks 894,000 to 896,528), though that rises to 3.1% of blockspace if you only count the blocks that actually included such txs. Those levels seem pretty easy to cope with, no matter how you calculate fees. (Only 3 of the 61 txs had multiple OP_RETURNs, 33 had a single OP_RETURN scriptPubKey between 84 and 160 bytes, and there were an additional six OP_RETURNs in txs larger than 100kvB which will still be non-standard if this PR is merged, one of which was also 0 fees)

    gmaxwell also wrote:

    I would agree that all substantive discrepancies ought to be closed either by convincing miners to change behavior or removing the restrictions,

    I don’t disagree with this in principle, but I don’t think the level of non-standard OP_RETURNs that we were seeing prior to this topic being raised for discussion (ie 61 txs over a period of 48,000 blocks) should be counted as a “substantive” discrepency.

    If it were to be counted, I think that’s a goal that’s pretty unlikely to be achievable – eg Mara’s slipstream has a transaction privacy policy (“Before they are mined, yes. Your transactions are not broadcasted to the Bitcoin network until they have received at least one confirmation”) which is advertised as being particularly useful to prevent token mints being sniped. Violations of other standardness rules (particularly tx size/weight) or using replace-by-feerate policies versus replace-by-fee ones also seem likely to remain sources of occasional discrepencies.

  139. earonesty commented at 5:13 pm on May 16, 2025: none
    makes more sense to change the default to be larger or whatever not deprecate the option. have we been seeing a lot of non-standard transactions with large op_returns? 61 out of 48,000 is kind of small and probably anomalous. this pull request should come with some evidence that this is necessary to keep standardness in line with usage
  140. BitcoinMechanic commented at 8:13 pm on May 16, 2025: none
    It seems if you want people to actually use this much OP RETURN data by relaxing the filter it’d make sense to simultaneously prevent “inscriptions” with this https://github.com/bitcoin/bitcoin/pull/28408
  141. 1440000bytes commented at 10:00 am on May 18, 2025: none

    It seems if you want people to actually use this much OP RETURN data by relaxing the filter it’d make sense to simultaneously prevent “inscriptions” with this #28408

    You can ACK this pull request if you agree with the changes and open a new issue or pull request for #28408

  142. blockdyor commented at 10:24 am on May 18, 2025: none

    Concept NACK

    Nodes shouldn’t relay arbitrary data on the network. Removing the cap on the default datacarriersize makes that easier. We’ve already seen a clear signal that people don’t want this, with a noticeable migration from Bitcoin Core to Bitcoin Knots over the past three weeks. And since [PR #32359](https://github.com/bitcoin/bitcoin/pull/32359#event-17618932800) was closed by @glozow a few days ago, I’d expect this one to be closed too.

  143. instagibbs commented at 5:54 pm on May 19, 2025: member
    @blockdyor Unclear to me why the other PR being closed in favor of this one implies this one should be closed. Going to keep this open, thanks!
  144. douglaz commented at 7:34 pm on May 22, 2025: none

    We’ve already seen a clear signal that people don’t want this, with a noticeable migration from Bitcoin Core to Bitcoin Knots over the past three weeks.

    It’s clear we’ve agreed to disagree: people who prefer arbitrary filters will run Bitcoin Knots, and people who prefer harmless, consensus-valid transactions will run Bitcoin Core.

  145. in src/policy/policy.cpp:163 in 35bcd8eed3
    159@@ -159,12 +160,6 @@ bool IsStandardTx(const CTransaction& tx, const std::optional<unsigned>& max_dat
    160         return false;
    161     }
    162 
    163-    // only one OP_RETURN txout is permitted
    164-    if (nDataOut > 1) {
    


    mrberlinorg commented at 8:52 am on May 23, 2025:

    Removing the check for nDataOut > 1 would allow multiple OP_RETURN outputs in a single transaction, which goes against the standard behavior of the Bitcoin protocol. This could introduce several issues:

    Non-standard transactions: Multiple OP_RETURN outputs in a transaction are considered non-standard. Allowing them could lead to network inconsistencies, as some nodes might not be able to properly process or relay these transactions.

    Network congestion: More OP_RETURN outputs would increase transaction size, reducing the number of transactions that can be included in each block, which could lead to increased network congestion.

    Compatibility problems: Non-standard transactions could cause compatibility issues with other nodes, wallets, and services that expect standard behavior.

    Maintaining this check ensures that Bitcoin transactions remain within the protocol’s standards, preventing these potential issues and maintaining network stability.


    willcl-ark commented at 9:27 am on May 23, 2025:

    A gentle reminder not to post AI-generated comments like this again here to avoid being banned.

    multiple OP_RETURN outputs in a single transaction, which goes against the standard behavior of the Bitcoin protocol.

    There are no protocol rules defining how many OP_RETURN outputs can be included in a transaction.

    Non-standard transactions: Multiple OP_RETURN outputs in a transaction are considered non-standard. Allowing them could lead to network inconsistencies, as some nodes might not be able to properly process or relay these transactions.

    Full nodes must be able to process all consensus-valid transactions, or else they will fork themselves off the network. You will be glad to learn that improving network inconsistencies due to improper transaction relay is the motivation of this very PR!

    Network congestion: More OP_RETURN outputs would increase transaction size, reducing the number of transactions that can be included in each block, which could lead to increased network congestion.

    This is correct insomuch as the more data that is stored in OP_RETURN(s) , vs extra unspendable outputs for example, the smaller the number of bytes the witness discount is applied to in a transaction. This makes transactions storing data in OP_RETURN “appear larger” to the block weight calculation, therefore allowing fewer of them per block.

    So whilst it does not “increase transaction size” vs an alternative style of data-carrying transaction (extra unspendable outputs), it does mean that fewer of them (i.e. less data) will fit in a block.

    Compatibility problems: Non-standard transactions could cause compatibility issues with other nodes, wallets, and services that expect standard behavior.

    Non-standard transactions are certainly compatible with “other nodes and wallets”; they are found in almost 100% of blocks today. What “compatibility issues” are you referring to here exactly?

  146. waketraindev commented at 8:32 pm on May 23, 2025: none
    Concept ACK. Tested on top of current master, no issues observed. Looks good to me.

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-05-24 00:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me