Drop script_pub_key arg from createNewBlock #31318

pull Sjors wants to merge 3 commits into bitcoin:master from Sjors:2024/11/no-script_pub_key changing 21 files +123 −77
  1. Sjors commented at 3:31 pm on November 18, 2024: member

    Providing a script for the coinbase transaction is only done in test code and for (unoptimized) CPU solo mining.

    Production miners use the getblocktemplate RPC which omits the coinbase transaction entirely from its block template, leaving it to external (pool) software to construct it.

    This commit removes the script_pub_key argument from createNewBlock() in the Mining interface.

    A coinbase script can still be passed via BlockCreateOptions instead. Tests are modified to do so.

  2. DrahtBot commented at 3:31 pm on November 18, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31318.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK TheCharlatan, ryanofsky

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31283 (Add waitNext() to BlockTemplate interface by Sjors)
    • #31196 (Prune mining interface by Sjors)
    • #31112 (Improve parallel script validation error debug logging by sipa)
    • #30391 (BlockAssembler: return selected packages virtual size and fee by ismaelsadeeq)
    • #30157 (Fee Estimation via Fee rate Forecasters by ismaelsadeeq)
    • #29415 (Broadcast own transactions only via short-lived Tor or I2P connections by vasild)
    • #28676 ([WIP] Cluster mempool implementation by sdaftuar)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. Sjors commented at 3:37 pm on November 18, 2024: member
    The PR was prompted because I noticed in #31283 I had to needlessly pass script_pub_key around.
  4. Sjors force-pushed on Nov 18, 2024
  5. DrahtBot commented at 3:39 pm on November 18, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33145905317

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  6. DrahtBot added the label CI failed on Nov 18, 2024
  7. in src/node/types.h:50 in 523bde0f44 outdated
    43@@ -43,6 +44,10 @@ struct BlockCreateOptions {
    44      * transaction outputs.
    45      */
    46     size_t coinbase_output_max_additional_sigops{400};
    47+    /**
    48+     * Script to put in the coinbase transaction.
    49+     */
    50+    std::optional<CScript> coinbase_script{};
    


    maflcko commented at 3:41 pm on November 18, 2024:

    not sure about making this optional. Even if this is a dummy, it would be good to spell it out.

    Using std::optional here is just making code more complex, adding two lines of code in one place to save one line of code in another place.

    Also, it may be better to call mention output script in the name.


    Sjors commented at 3:50 pm on November 18, 2024:

    In production this is always a dummy. CreateNewBlock() sets the script to OP_TRUE and then the getblocktemplate RPC omits the whole coinbase transaction.

    It’s not safe for the caller to set an arbitrary dummy, because IIUC if it’s larger than coinbase_max_additional_weight the TestBlockValidity internal check for getblocktemplate will spuriously fail. Which is why I think it’s better for CreateNewBlock to decide what the dummy should be.

    mention output script in the name

    That makes sense.


    ryanofsky commented at 6:50 pm on November 19, 2024:

    re: #31318 (review)

    Would it make sense to declare the field as CScript coinbase_output_script{CScript{} << OP_TRUE} instead of std::optional<CScript> coinbase_output_script{}; just to be clearer about what the default value is? Or maybe just add documentation to say what happens if this is not set?

    I think the drawback of using std::optional here is that it’s not really clear what happens if the option is not set. It’s also not clear when you would want to set the option here, if ever.


    Sjors commented at 8:00 pm on November 19, 2024:

    I expanded the comment to explain things a bit more.

    I think you should not set this option unless you care what the coinbase output is. For example the block_assemble benchmark uses P2WSH_OP_TRUE. I’m guessing that’s because it wants to look at the performance related to spending from a segwit coinbase.

  8. Sjors force-pushed on Nov 18, 2024
  9. Sjors commented at 4:11 pm on November 18, 2024: member
    Renamed to coinbase_output_script, added warning and discouragement, fixed fuzz binary build.
  10. Sjors force-pushed on Nov 19, 2024
  11. Sjors commented at 10:43 am on November 19, 2024: member
    Forgot to modify mining.capnp.
  12. DrahtBot removed the label CI failed on Nov 19, 2024
  13. TheCharlatan commented at 1:00 pm on November 19, 2024: contributor
    I’m not sure why this is really an improvement. Can you elaborate on why no external user would want to pass in a script pubkey directly, instead of manually manipulating the coinbase in the block template?
  14. Sjors commented at 1:56 pm on November 19, 2024: member

    Afaik the only way external users make blocks today is the getblocktemplate RPC, which drops the coinbase transaction from the template it returns.

    Similarly in submitSolution, used by Stratum v2, we expect to receive the full coinbase transaction and replace whatever was in the CBlockTemplate.

    Unless you’re solo mining, the pool is going to add its own outputs, manipulate the extra nonce, add its own name, etc. Even with a solo pool (a weird concept) you would configure the payout address in that software, not in the node.

    There is a way in Stratum v2 to specify a list of outputs that you want in the coinbase. In my current implementation we use this to include the witness commitment, but nothing else.

    In the longer run we could expand the interface to specify more outputs.

    The special case for that is a single payout address that takes all the money from the block. That could be used for solo mining with a single small ASIC that doesn’t need to grind the extra nonce (up to about 100 TH/s with BIP320). But that’s fairly niche use case. As soon as you use Stratum v1 / v2 software you typically pass a payout address to the the pool software, not to the node.

  15. ryanofsky approved
  16. ryanofsky commented at 6:54 pm on November 19, 2024: contributor
    Concept ACK . It seems like a nice improvement to not require specifying an option that won’t typically be used. But I’m a little hazy on the details of this. I think better documentation could help. Left a suggestion and some questions below.
  17. Sjors force-pushed on Nov 19, 2024
  18. Sjors commented at 7:49 pm on November 19, 2024: member

    Rebased to avoid spurious CI failure.

    I’ll take a look later if for any of the tests I can drop their custom coinbase output.

  19. TheCharlatan commented at 9:37 pm on November 19, 2024: contributor

    Re #31318 (comment)

    Thank you for spelling this out @Sjors. I also like ryanofsky’s idea of adding the OP_TRUE script as the default option instead of making it a std::optional.

    Concept ACK

  20. DrahtBot added the label Needs rebase on Nov 20, 2024
  21. in src/node/miner.cpp:126 in 42f880aeb0 outdated
    119@@ -120,6 +120,12 @@ std::unique_ptr<CBlockTemplate> BlockAssembler::CreateNewBlock(const CScript& sc
    120     pblocktemplate->vTxFees.push_back(-1); // updated at end
    121     pblocktemplate->vTxSigOpsCost.push_back(-1); // updated at end
    122 
    123+    // Use dummy coinbase output script if none is provided
    124+    CScript coinbase_output_script{CScript() << OP_TRUE};
    125+    if (m_options.coinbase_output_script) {
    126+        coinbase_output_script = m_options.coinbase_output_script.value();
    


    ryanofsky commented at 2:43 am on November 21, 2024:

    In commit “Drop script_pub_key arg from createNewBlock” (42f880aeb080070a42229d0bccd517100a3fe7fb)

    Would suggest using ‘*’ instead of .value() since this code isn’t intending to trigger the std::bad_optional_access exception, and to make it more obvious a dereference is happening.


    Sjors commented at 9:30 am on November 21, 2024:
    I changed coinbase_output_script to no longer be std::optional.
  22. in src/node/miner.cpp:109 in 42f880aeb0 outdated
    105@@ -106,7 +106,7 @@ void BlockAssembler::resetBlock()
    106     nFees = 0;
    107 }
    108 
    109-std::unique_ptr<CBlockTemplate> BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn)
    110+std::unique_ptr<CBlockTemplate> BlockAssembler::CreateNewBlock()
    


    ryanofsky commented at 2:55 am on November 21, 2024:

    In commit “Drop script_pub_key arg from createNewBlock” (42f880aeb080070a42229d0bccd517100a3fe7fb)

    Would it make sense to change the BlockAssembler::CreateNewBlock() signature in a separate followup commit? The test changes in this commit vastly outnumber the non-test changes, which I think make it more likely a bug in the non-test code could be missed.

    Maybe this there could be two overloads initially:

    0std::unique_ptr<CBlockTemplate> CreateNewBlock()
    1{
    2    return CreateNewBlock(m_options.coinbase_output_script ? *m_options.coinbase_output_script : CScript{} << OP_TRUE);
    3}
    4
    5std::unique_ptr<CBlockTemplate> CreateNewBlock(const CScript& scriptPubKeyIn);
    

    So test code in could updated in a separate the commit and then the overload could be deleted.


    Sjors commented at 9:31 am on November 21, 2024:
    I made an overload, though slightly differently.
  23. ryanofsky approved
  24. ryanofsky commented at 2:59 am on November 21, 2024: contributor
    Code review ACK 1dfc54d87c7b35cc93cfb81cc2a46c9e8eae7194. Left a style suggestion, and a suggestion to split up the first commit, if we want to do that. But these can be ignored and this all looks good as-is. Makes a lot of sense to not make a parameter which should only be used for testing a required argument.
  25. DrahtBot requested review from TheCharlatan on Nov 21, 2024
  26. Sjors force-pushed on Nov 21, 2024
  27. Sjors commented at 9:30 am on November 21, 2024: member

    Rebased after #31122.

    I changed coinbase_output_script from std::optional to having a default of OP_TRUE.

    I split the main commit into three parts:

    1. Remove scriptPubKeyIn arg from CreateNewBlock, using a temporary overload to avoid test changes
    2. Dropping the temp overload and change the tests
    3. Change the interface
  28. in src/node/types.h:55 in b35b23d823 outdated
    50+     *
    51+     * Should only be used for tests, when the default doesn't suffice.
    52+     *
    53+     * Note that higher level code like the getblocktemplate RPC may omit the
    54+     * coinbase transaction entirely. It's instead constructed by pool software
    55+     * using fields like coinbaseaux, coinbasevalue, coinbaseaux and
    


    TheCharlatan commented at 9:37 am on November 21, 2024:
    Nit: Is there one coinbaseaux too many here?
  29. DrahtBot removed the label Needs rebase on Nov 21, 2024
  30. in src/rpc/mining.cpp:817 in b35b23d823 outdated
    813@@ -814,11 +814,9 @@ static RPCHelpMan getblocktemplate()
    814         time_start = GetTime();
    815 
    816         // Create new block
    817-        CScript scriptDummy = CScript() << OP_TRUE;
    818-        block_template = miner.createNewBlock(scriptDummy);
    819+        block_template = miner.createNewBlock(CScript() << OP_TRUE);
    


    TheCharlatan commented at 9:42 am on November 21, 2024:

    In commit 59e75470591ce5a36500b649c987898a0c534d20:

    Nit: I think we just directly call the new function here?


    Sjors commented at 9:53 am on November 21, 2024:
    This change was premature since the interface is changed in a later commit. It also doesn’t do anything, so I’m dropping it.
  31. Sjors force-pushed on Nov 21, 2024
  32. Sjors commented at 9:44 am on November 21, 2024: member
    I also split out the renames in 55f2bb7a96806d33e475122f4d8f54b117838afe. They were an accidental find & replace, but useful to be consistent with the new BlockCreateOptions field.
  33. Sjors force-pushed on Nov 21, 2024
  34. Sjors force-pushed on Nov 21, 2024
  35. Sjors commented at 10:01 am on November 21, 2024: member
    I re-combined d31e5f9d9986ce1e80b014a1585ab0e944ebcc86 and d1972d751fa882960375050bfa7fc6f4043e0be4 into 81a2587015dbf3eb129e4c35b198e56aee70191e so that the interface change is done along with the change to CreateNewBlock. This seems more clear than having them separate.
  36. in src/test/fuzz/process_message.cpp:50 in 81a2587015 outdated
    45@@ -44,8 +46,10 @@ void initialize_process_message()
    46             /*chain_type=*/ChainType::REGTEST,
    47             {.extra_args = {"-txreconciliation"}});
    48     g_setup = testing_setup.get();
    49+    BlockAssembler::Options options;
    50+    options.coinbase_output_script = CScript() << OP_TRUE;
    


    TheCharlatan commented at 1:52 pm on November 21, 2024:
    I would squash the last commit, then you can also drop the using node::BlockAssembler declaration entirely by just calling MineBlock(g_setup->m_node, {}).
  37. rpc: rename coinbase_script to coinbase_output_script d0fbff96bf
  38. Drop script_pub_key arg from createNewBlock
    Providing a script for the coinbase transaction is only done in test code
    and for CPU solo mining.
    
    Production miners use the getblocktemplate RPC which omits the coinbase
    transaction entirely from its block template, leaving it to external (pool)
    software to construct it.
    
    A coinbase script can still be passed via BlockCreateOptions instead.
    
    A temporary overload is added so that the test can be modified in the
    next commit.
    1e32ab7a8b
  39. Sjors force-pushed on Nov 21, 2024
  40. Sjors force-pushed on Nov 21, 2024
  41. test: drop scriptPubKeyIn arg from CreateNewBlock
    This removes the temporary overload added in the previous commit.
    
    Also drop unneeded custom coinbase output scripts.
    027ce3e963
  42. Sjors force-pushed on Nov 21, 2024
  43. Sjors commented at 2:30 pm on November 21, 2024: member

    Squashed the last two commits.

    I dropped a few spurious (...)coinbase(_output)_script_pub_key renames from the test commit.

    And dropped another unneeded OP_TRUE from tx_pool.

  44. TheCharlatan approved
  45. TheCharlatan commented at 9:39 pm on November 21, 2024: contributor
    ACK 027ce3e9634b2f47c508899942d05690572de516
  46. DrahtBot requested review from ryanofsky on Nov 21, 2024
  47. ryanofsky approved
  48. ryanofsky commented at 2:39 pm on November 22, 2024: contributor
    Code review ACK 027ce3e9634b2f47c508899942d05690572de516. Since last review, this was rebased to avoid conflicts, and option is now a plain cscript instead of an optional<cscript>, and the commits are split up and there are some minor changes in tests (renames, whitespace, dropping redundant assignments. This is pretty easy to review now that the main commit is so small and changes are split up
  49. Sjors referenced this in commit a8da99a25e on Nov 29, 2024
  50. Sjors commented at 4:53 pm on November 29, 2024: member
    This PR incidentally “fixes” the IPC bug discovered in https://github.com/Sjors/bitcoin/issues/71
  51. Sjors referenced this in commit 9634b8ae00 on Dec 2, 2024
  52. Sjors referenced this in commit 392967375b on Dec 2, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-03 18:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me