Add waitNext() to BlockTemplate interface #31283

pull Sjors wants to merge 2 commits into bitcoin:master from Sjors:2024/11/wait-next changing 4 files +209 −79
  1. Sjors commented at 12:30 pm on November 13, 2024: member

    Alternative approach to #31003, suggested in #31109 (comment).

    This PR introduces waitNext(). It waits for either the tip to update or for fees at the top of the mempool to rise sufficiently. It then returns a new template, with which the caller can rinse and repeat.

  2. DrahtBot commented at 12:30 pm on November 13, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31283.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31318 (Drop script_pub_key arg from createNewBlock by Sjors)
    • #31288 (Add destroy to BlockTemplate schema by Sjors)
    • #31197 (refactor: mining interface 30955 followups by Sjors)
    • #31122 (cluster mempool: Implement changeset interface for mempool by sdaftuar)
    • #30391 (BlockAssembler: return selected packages virtual size and fee by ismaelsadeeq)
    • #30157 (Fee Estimation via Fee rate Forecasters by ismaelsadeeq)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. in src/node/interfaces.cpp:1010 in f955b8e406 outdated
    993+        }
    994+
    995+        return block_template;
    996+    }
    997+
    998+    const CScript m_script_pub_key;
    


    Sjors commented at 12:34 pm on November 13, 2024:
    Separate from this PR, I’m looking into whether we can avoid passing script_pub_key around. Afaik it’s only used by tests and for solo CPU mining.

    Sjors commented at 3:33 pm on November 18, 2024:
    #31318 drops this argument. It’s only slightly easier to merge that first, so I’ll leave this PR open.
  4. in src/node/interfaces.cpp:943 in f955b8e406 outdated
    937@@ -938,9 +938,71 @@ class BlockTemplateImpl : public BlockTemplate
    938         return chainman().ProcessNewBlock(block_ptr, /*force_processing=*/true, /*min_pow_checked=*/true, /*new_block=*/nullptr);
    939     }
    940 
    941+    std::unique_ptr<BlockTemplate> waitNext(CAmount fee_threshold, MillisecondsDouble timeout) override
    942+    {
    943+        if (timeout > std::chrono::years{100}) timeout = std::chrono::years{100}; // Upper bound to avoid UB in std::chrono
    


    Sjors commented at 12:37 pm on November 13, 2024:
    I still need to study #31003 (review) in more detail to see if this code isn’t making similar mistakes.

    Sjors commented at 1:30 pm on November 14, 2024:

    I dropped this line.

    Additionally this PR does not make use of last_mempool_update, since it’s not very useful in a typical production setting where something in the mempool updates all the time.

    I think the race condition mentioned there is not an issue here because we don’t pass in the current fees via argument, but instead derive them from the original template.

    I did forget to update now at the end of the loop, which I fixed.

  5. Sjors commented at 7:46 pm on November 13, 2024: member
    I pushed a commit refactoring miner_tests to use the Mining interface. I plan to expand those tests to cover waitNext().
  6. in src/test/miner_tests.cpp:70 in a316ba58cb outdated
    71-BlockAssembler MinerTestingSetup::AssemblerForTest(CTxMemPool& tx_mempool)
    72-{
    73-    BlockAssembler::Options options;
    74-
    75-    options.nBlockMaxWeight = MAX_BLOCK_WEIGHT;
    76-    options.blockMinFeeRate = blockMinFeeRate;
    


    Sjors commented at 7:47 pm on November 13, 2024:
    a316ba58cba5df8aadb05737959b97eeb6a350b4: I think dropping this doesn’t matter?
  7. in src/test/miner_tests.cpp:662 in a316ba58cb outdated
    684         }
    685-        std::shared_ptr<const CBlock> shared_pblock = std::make_shared<const CBlock>(*pblock);
    686-        BOOST_CHECK(Assert(m_node.chainman)->ProcessNewBlock(shared_pblock, true, true, nullptr));
    687-        pblock->hashPrevBlock = pblock->GetHash();
    688+        std::shared_ptr<const CBlock> shared_pblock = std::make_shared<const CBlock>(block);
    689+        BOOST_REQUIRE(Assert(m_node.chainman)->ProcessNewBlock(shared_pblock, /*force_processing=*/true, /*min_pow_checked=*/true, nullptr));
    


    Sjors commented at 7:51 pm on November 13, 2024:
    a316ba58cba5df8aadb05737959b97eeb6a350b4: I’m tempted to use miner.submitSolution() here.

    Sjors commented at 12:16 pm on November 14, 2024:
    Done
  8. DrahtBot added the label CI failed on Nov 13, 2024
  9. DrahtBot commented at 8:24 pm on November 13, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/32946689392

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  10. Sjors force-pushed on Nov 13, 2024
  11. DrahtBot removed the label CI failed on Nov 13, 2024
  12. Sjors force-pushed on Nov 14, 2024
  13. Sjors commented at 12:17 pm on November 14, 2024: member
    I made the test code alternate calls between Chainman’s ProcessNewBlock and submitSolution via the Mining interface. The latter approach is much simpler, but both are used in production code, so should be covered.
  14. Sjors force-pushed on Nov 14, 2024
  15. Sjors force-pushed on Nov 14, 2024
  16. Sjors force-pushed on Nov 14, 2024
  17. Sjors marked this as ready for review on Nov 14, 2024
  18. Sjors commented at 1:05 pm on November 14, 2024: member
    I added test coverage for waitNext().
  19. Sjors force-pushed on Nov 14, 2024
  20. Sjors force-pushed on Nov 14, 2024
  21. TheCharlatan commented at 1:18 pm on November 14, 2024: contributor

    In commit f9200f6c65b5a97a4842cc1ec34e688ec13bffcd

    “Use createNewBlock via the interface instead of calling Chainman’s CreateNewBlock.”

    Do you mean “Use createNewBlock via the interface instead of calling CreateNewBlock through the BlockAssembler directly”? There is no CreateNewBlock in the chainman.

  22. Sjors force-pushed on Nov 14, 2024
  23. Sjors commented at 1:31 pm on November 14, 2024: member
    Made some adjustments based on #31283 (review)
  24. test: use Mining interface in miner_tests
    Alternate calls between Chainman's ProcessNewBlock and submitSolution. The Chainman code path is used in production, so it's important to keep test coverage for it.
    
    Use createNewBlock via the interface instead of using the BlockAssembler directly. The latter is always called via the interface in production code.
    b2f8483daa
  25. Sjors force-pushed on Nov 14, 2024
  26. Sjors commented at 1:33 pm on November 14, 2024: member
    @TheCharlatan I adjusted the commit message
  27. Add waitNext() to BlockTemplate interface fdfab036a1
  28. in src/node/interfaces.cpp:999 in e5a9b73210 outdated
    987+                CAmount new_fees = 0;
    988+                for (CAmount fee : block_template->m_block_template->vTxFees) {
    989+                    // Skip coinbase
    990+                    if (fee < 0) continue;
    991+                    new_fees += fee;
    992+                    if (new_fees >= current_fees + fee_threshold) return block_template;
    


    Sjors commented at 10:55 am on November 15, 2024:
    A fee_threshold of 0 combined with a timeout of 0 will immediately return a new template. This is useful for the Template Provider, because it needs to unconditionally send new templates to all connected clients if fees increased sufficiently for one its connected clients. (at least until cluster mempool makes the fee calculation trivially cheap)
  29. Sjors force-pushed on Nov 15, 2024
  30. Sjors commented at 5:37 pm on November 15, 2024: member
    Added early return logic. Similar to #31297 this would be less tedious if notifications().m_tip_block was set to the tip earlier.
  31. Sjors commented at 1:33 pm on November 19, 2024: member
    Marking this draft, let’s review the early return and m_tip_block block changes in #31297 first.
  32. Sjors marked this as a draft on Nov 19, 2024
  33. Sjors commented at 7:11 pm on November 19, 2024: member

    #31297 is on hold, as it’s probably unnecessary.

    Please review #31325 first, so I can decide whether to build this PR on top of it.

  34. DrahtBot added the label Needs rebase on Nov 20, 2024
  35. DrahtBot commented at 2:48 pm on November 20, 2024: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-21 06:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me