Stratum v2 via IPC Mining Interface tracking issue #31098

issue Sjors openend this issue on October 16, 2024
  1. Sjors commented at 12:43 pm on October 16, 2024: member

    After much discussion in #29432 it appears there’s currently little support for directly supporting Stratum v2 in Bitcoin Core, with most contributors favouring the creation of a Mining IPC interface that external applications can use.

    The only such application currently is the Template Provider implemented by me in https://github.com/Sjors/bitcoin/pull/48. This also serves to illustrate how the interface is used and that it’s complete. The folks working on SRI should be able to build a Template Provider as well, once the interface is complete and shipped in a release.

    Needed for (hopefully) v29

    To complete the interface:

    For multiprocess release binaries

    Libmultiprocess bugfixes

    Can wait for later releases

    • #30437
    • #31002 DATUM developers are currently uninterested in taking advantage of the more performant interface, preferring to use the getblocktemplate RPC instead, despite an earlier concept ACK on Stratum v2 support #27854 (comment).
    • new interface methods:
      • createFutureBlock() for faster work switching, see #31109
      • waitNewPowBlock() for SPV mining, see #31109
      • verifyBlock(CBlock block, bool allow_rollback, bool check_pow, uint256 target)
        • for Stratum v2 pools to verify a proposed block, ignoring PoW
        • can also be used to verify weak blocks in P2pool like systems
        • can use the mempool to accelerate validation, but doesn’t update it
        • (maybe) can optionally roll back, to inspect older templates

    Current Template Provider users

    As a stop-gap measure I plan to maintain https://github.com/Sjors/bitcoin/pull/68 which is a continuation of #29432, since people are currently using that. Once the Mining interface makes it into a release, along with multiprocess enabled Guix binaries, I plan to drop that in favour of https://github.com/Sjors/bitcoin/pull/48.

    Additionally I plan to convert the latter from a patch set on top of Bitcoin Core into a standalone application, to make it easier for others to contribute. That should make it less dependent on my own effort. The Bitcoin Core project could decide in the future to “adopt” this project, but that is entirely uncertain and most contributors seems averse to this idea.

  2. TheBlueMatt commented at 1:35 pm on October 16, 2024: contributor
    I’m really quite confused by this. Creating a “template IPC” interface is not substantially different than creating any other protocol for providing templates. What is the advantage for Bitcoin core to provide something proprietary over a standard that matches what consumers of that interface would want?
  3. TheBlueMatt commented at 2:27 pm on October 16, 2024: contributor

    My understanding of the conversation previously was (a) a desire to avoid a ton of TCP logic (which will largely be addressed by upcoming work to remove libevent’s HTTP server and presumably then unify the Connman logic across any TCP listeners), (b) a desire to use this as a way to get more multiprocess in Core (cool, but this is unrelated to whether Sv2 is supported in Core). Neither of those precludes or leads to the conclusion that “there’s currently little support for directly supporting Stratum v2 in Bitcoin Core”.

    I’d like to better understand the logic for building a new protocol (obligatory XKCD 927 referehce) here and what design considerations are going into that protocol design. Was consideration given to the goals of allowing regulated institutions to use third-party template providers with minimal restrictions? Presumably this new protocol needs a BIP and should go through normal protocol review.

    It’s also my understanding the the IPC logic in Bitcoin Core is not intended as a public API, and thus users of it are expected to exclusively be in Bitcoin Core, I suppose that is changing here?

  4. Sjors commented at 4:03 pm on October 16, 2024: member

    To start with the last point:

    It’s also my understanding the the IPC logic in Bitcoin Core is not intended as a public API, and thus users of it are expected to exclusively be in Bitcoin Core, I suppose that is changing here?

    In general IPC interfaces are not intended to be public stable, but the mining interface is an exception. @josibake is working on a proof-of-concept Rust demo that uses it, which should be fairly easy to use in SRI.

    Creating a “template IPC” interface is not substantially different than creating any other protocol for providing templates.

    The mining interface in interfaces/mining.h, node/interfaces.cpp and mining.capnp is orders of magnitude less code than what’s currently in https://github.com/Sjors/bitcoin/pull/68. That would be still be true after a refactor of Sv2Connman. It also moves all DoS-resistance responsibility to the user of this trusted interface.

    Neither of those precludes or leads to the conclusion that “there’s currently little support for directly supporting Stratum v2 in Bitcoin Core”.

    There are several concept NACKs on #29432 and earlier versions of it. There were a few concept ACKs from experienced developers on the first version in #29432, but they haven’t reviewed this work. The pace of review on the sub PRs for noise encryption #29346 / https://github.com/Sjors/bitcoin/pull/66, transport #30315 / https://github.com/Sjors/bitcoin/pull/67, connection management https://github.com/Sjors/bitcoin/pull/67 and the template provider application https://github.com/Sjors/bitcoin/pull/49 is glacial, it would take years at this pace to get them merged.

    In contrast review on the multiprocess approach has been fast.

    design considerations are going into that protocol design

    The Mining interface was designed by refactoring https://github.com/Sjors/bitcoin/pull/68 to use it. Additionally there’s a few methods that the RPC uses internally. There’s not much more to it.

    Was consideration given to the goals of allowing regulated institutions to use third-party template providers with minimal restrictions?

    Nothing changes here. Anyone can run bitcoin-node and connect a template provider application to it that listens on the public internet, either the one I wrote or a new one.

    Presumably this new protocol needs a BIP and should go through normal protocol review.

    I think you’re assuming here that this IPC is directly exposed to the untrusted internet? It’s really only supposed to be used by a trusted application that in turn can accept connections from the outside.

  5. TheBlueMatt commented at 5:04 pm on October 16, 2024: contributor

    The mining interface in interfaces/mining.h, node/interfaces.cpp and mining.capnp is orders of magnitude less code than what’s currently in https://github.com/Sjors/bitcoin/pull/68. That would be still be true after a refactor of Sv2Connman.

    I’m curious where the bulk of that is. It should be basically identical in terms of total volume (framing and such all has to happen either way, even if its auto-generated code when using capnproto), the protocol would be ~identical in function, and IPC vs TCP should be fairly similar (I assume its a socket interface either way). That just leaves the encryption on top which IIRC wasn’t a huge volume of code.

    It also moves all DoS-resistance responsibility to the user of this trusted interface.

    I’m curious what the worry is here - the protocol is almost entirely push-only, do we anticipate DoS is a material risk there? If there are material concerns here (eg too-large-coinbase-tx signaling pushing transactions out of the template attacks) they’ll simply not be addressed at all. The nice thing about having things in an active project is that people may actually look at it.

    There are several concept NACKs on #29432 and earlier versions of it. There were a few concept ACKs from experienced developers on the first version in #29432, but they haven’t reviewed this work. The pace of review on the sub PRs for noise encryption #29346 / https://github.com/Sjors/bitcoin/pull/66, transport #30315 / https://github.com/Sjors/bitcoin/pull/67, connection management https://github.com/Sjors/bitcoin/pull/67 and the template provider application https://github.com/Sjors/bitcoin/pull/49 is glacial, it would take years at this pace to get them merged.

    I understand things may be slow, but I’m curious why the conclusion there is that instead we should build a new protocol that accomplishes substantially the same thing but with fewer features. If the concern is that things like the encryption layer are moving too slow and we want to get something out it would be perfectly reasonable to ship a shorter-term patchset that doesn’t support that and consider adding support for it later. Same goes for TCP, even, we could consider adding a named-socket-only version that does the same framing but based on named sockets to get something available and slowly add the full protocol parts at a later time based on use.

    In contrast review on the multiprocess approach has been fast.

    Yes, developers have a tendency to prefer their own NIH thing, but that is fairly weak justification.

    Nothing changes here. Anyone can run bitcoin-node and connect a template provider application to it that listens on the public internet, either the one I wrote or a new one.

    That is a very substantial change - requiring an external application makes the value proposition of a common protocol that miners can connect to third-party nodes for templates much, much weaker. “Just turn this option on in bitcoin.conf” is a very different story from “also figure out how to reliably operate some unmaintained software that someone posted on github five years ago” (which is the current state of much of the solo-mining world).

    The Mining interface was designed by refactoring https://github.com/Sjors/bitcoin/pull/68 to use it. Additionally there’s a few methods that the RPC uses internally. There’s not much more to it.

    Sure, and there’s no much more to the Sv2 mining interface either :). Either way, I’m still kinda confused why it was decided to NIH a new protocol here.

    I think you’re assuming here that this IPC is directly exposed to the untrusted internet? It’s really only supposed to be used by a trusted application that in turn can accept connections from the outside.

    No, I am not. Exposing work coming out of Bitcoin Core is possibly the most important API from Bitcoin Core (outside of P2P and maybe sendrawtransaction) for the broader Bitcoin ecosystem. Defining a new API to do it haphazardly seems incredibly strange to me.

  6. TheBlueMatt commented at 5:06 pm on October 16, 2024: contributor
    Its probably worth pointing out that with new MEV-extraction things coming out these days Bitcoin Core will not be the only server of work, which means we also need to define new mining work-providing protocols in terms of the server-side behavior as well (not just “whatever Bitcoin Core does”, and not just whatever goes in its docs).
  7. Sjors commented at 8:31 am on October 17, 2024: member

    I’m curious where the bulk of that is.

    It’s literally in https://github.com/Sjors/bitcoin/pull/68, split between https://github.com/Sjors/bitcoin/pull/66, https://github.com/Sjors/bitcoin/pull/67, https://github.com/Sjors/bitcoin/pull/50 and https://github.com/Sjors/bitcoin/pull/49 in that order. I’ve already refactored that code to use the interface, just not via IPC, so it’s all additional. The first three PRs don’t use the interface at all.

    framing and such all has to happen either way, even if its auto-generated code when using capnproto

    I think you misunderstand the intended setup: the idea is to have a bitcoin-node application to which you connect via unix socket a bitcoin-stratum Template Provider application (https://github.com/Sjors/bitcoin/pull/68 or something based on SRI). That application then listens on the stratum port, uses noise encryption, etc.

    So it’s not either, or, it’s both. But the Bitcoin Core project would only have to maintain the cap&proto stuff - which it can use for other things, like separating the GUI from the node.

    do we anticipate DoS is a material risk there

    One obvious DoS issue is that as clients receive multiple templates the node holds on to transaction references even as the mempool drops them. I still have to open a followup issue to discuss that. One suggested solution is to have the node refuse to make new templates are some time, or give the client a way to measure memory footprint so it can choose which older templates to drop. That doesn’t have to be fully resolved on the Bitcoin Core end: it sees bitcoin-stratum as a trusted process, it will have to deal with the problem.

    More generally many historical bugs were in p2p code. Adding more p2p functionality adds more unknown unknowns.

    I understand things may be slow, but I’m curious why the conclusion there is that instead we should build a new protocol that accomplishes substantially the same thing but with fewer features.

    Hopefully I clarified above that this doesn’t replace stratum v2, it’s a layer in between. I’m able to reuse the same code for both the integrated and via-IPC Template Provider, so there’s no loss in functionality. The only big difference is division of responsibility for maintaining the functionality.

    The nice thing about having things in an active project is that people may actually look at it.

    That is a very substantial change - requiring an external application makes the value proposition of a common protocol that miners can connect to third-party nodes for templates much, much weaker.

    I agree with both statements, but I haven’t been able to convince others to take on that burden.

    we also need to define new mining work-providing protocols in terms of the server-side behavior as well

    I think Stratum v2 should remain the standard for that.

  8. TheBlueMatt commented at 1:15 pm on October 17, 2024: contributor

    It’s literally in https://github.com/Sjors/bitcoin/pull/68, split between https://github.com/Sjors/bitcoin/pull/66, https://github.com/Sjors/bitcoin/pull/67, https://github.com/Sjors/bitcoin/pull/50 and https://github.com/Sjors/bitcoin/pull/49 in that order. I’ve already refactored that code to use the interface, just not via IPC, so it’s all additional. The first three PRs don’t use the interface at all.

    I guess I was more curious what you think the bulk of the issue is, rather than just the bulk of the code. Do you think the lack of desire for review (compared to reviewing the capnproto protocol instead) comes from the encryption, the framing logic, the connman management, etc?

    I think you misunderstand the intended setup: the idea is to have a bitcoin-node application to which you connect via unix socket a bitcoin-stratum Template Provider application (https://github.com/Sjors/bitcoin/pull/68 or something based on SRI). That application then listens on the stratum port, uses noise encryption, etc.

    I understand that’s the goal, but why on earth would anyone actually implement a pool/miner that way? Bitcoin Core is defining a standard protocol to output work, forcing people to use a translation proxy to convert from that standard protocol to another standard protocol with basically equivalent messages is just latency, complexity, and possibility for failure. It’d be nice to have it listen on TCP, but that’s much better accomplished with netcat than some kind of protocol translation.

    One obvious DoS issue is that as clients receive multiple templates the node holds on to transaction references even as the mempool drops them. I still have to open a followup issue to discuss that. One suggested solution is to have the node refuse to make new templates are some time, or give the client a way to measure memory footprint so it can choose which older templates to drop. That doesn’t have to be fully resolved on the Bitcoin Core end: it sees bitcoin-stratum as a trusted process, it will have to deal with the problem.

    What’s wrong with just having a fixed timeout? As Bitcoin Core is now the thing responsible for deciding when a new template is created it is in full control of the memory used and can keep track of how much memory it is using. It could even use the flush bit to clear out old work if the mempool fee is climbing rapidly.

    More generally many historical bugs were in p2p code. Adding more p2p functionality adds more unknown unknowns.

    Yea, but that’s why this is an almost-push-only protocol, much fewer unknown unknowns :).

    Hopefully I clarified above that this doesn’t replace stratum v2, it’s a layer in between. I’m able to reuse the same code for both the integrated and via-IPC Template Provider, so there’s no loss in functionality. The only big difference is division of responsibility for maintaining the functionality. I think Stratum v2 should remain the standard for that.

    I strongly disagree, there’s a separate discussion on the Sv2 github (which I know you disagree with) on dropping the TP in favor of this new IPC TP, but I see zero reason why we should define a secondary Sv2-specific protocol for TP(ing) when there’s a perfectly fine one in Core. I’m not really a fan of the ecosystem standardizing a protocol around capnproto and everyone being forced to take a dependency on it (or maybe its encoding is simple enough that others can implement it by hand? not sure) and hope that this new protocol eventually gets exposed over TCP with cryptographic authentication, but even with those there’s no reason to have some Sv2-specific nonsense. Please let’s not have two nearly-identical protocols for the same thing, that’s insane.

  9. Sjors commented at 3:26 pm on October 21, 2024: member

    Do you think the lack of desire for review (compared to reviewing the capnproto protocol instead) comes from the encryption, the framing logic, the connman management, etc?

    The “instead” misses the mark: reviewing capnproto was already necessary to achieve all the other use cases of multiprocess. The mining interface just provided an easier opportunity to make progress there.

    Trying to redesign the interface in a way that makes it DoS-safe brings back the need for a lot of the review that not including https://github.com/Sjors/bitcoin/pull/68 is trying to avoid. Which in practice probably means the interface won’t get shipped for ages.

  10. plebhash commented at 7:09 am on November 26, 2024: none

    on the pre-IPC Sv2 Template Provider that @Sjors provides on his sv2 branch, we have two CLI parameters for bitcoind:

    • -sv2interval
    • -sv2feedelta

    these two parameters dictated how often TP would send templates to its clients (Pool or JDC), which could happen either on a regular interval (sv2interval), or ad-hoc when needed (sv2feedelta).

    it seems that the new IPC approach makes communication between bitcoin-stratum and bitcoin-node completely poll-based.

    in other words, it’s up to bitcoin-stratum to continuously poll bitcoin-node for new template changes.

    wouldn’t that potentially introduce inefficiencies? what if fee delta goes above the desired threshold? is bitcoin-stratum forced to wait until the next polling cycle before it get the update?

  11. Sjors commented at 8:25 am on November 26, 2024: member
    @plebhash I pushed an update to the bitcoin-mine (Template Provider) application yesterday in https://github.com/Sjors/bitcoin/pull/48, so you can test it. It has the same -sv2interval and -sv2feedelta arguments. This uses waitNext() from #31283 under the hood. There’s no polling involved!
  12. Sjors commented at 9:44 am on November 26, 2024: member

    In both versions -sv2interval specifies the minimum time before sending a fee update, and it currently can’t / shouldn’t be set lower than 1 second. This minimum is not related to polling between two processes.

    It’s there because determining the new fees requires the node to construct a whole new block template under the hood. This is currently inefficient so the node waits one second between each template generation. But once it finds a better template the node pushses it to the bitcoin-mine process.

    The minimum interval can hopefully be dropped after Cluster Mempool is done #30289, as that should make it much cheaper to figure out how much fees for the next block have increased. At that point you can just set -sv2interval=0 so it won’t require an interface change.

    The default value is 30 seconds which is probably a bit overkill conservative. It’s there to avoid spamming the pool. And because perhaps ASICs have some overhead when loading a new template. Happy to take recommendations for a different default.

  13. ryanofsky commented at 2:02 pm on December 12, 2024: contributor
    I added #31375 to the list in the description since I think it will also be needed for a multirprocess release
  14. Sjors commented at 6:45 am on December 16, 2024: member
    #31346 landed, marked as such in the description. Many to go, most of them are pretty simple and already have some review; just need more eyes to get them over the finish line.
  15. Sjors commented at 8:02 am on December 18, 2024: member

    #31197, #31318 and #31196 landed. Suggested review: #31325 or #31375.

    Update 2024-12-20: #31325 landed, suggested review: #31283 or #31375.

  16. Sjors commented at 2:52 am on December 20, 2024: member

    Updated description to hint at some new interface methods I plan to propose in the future: createFutureBlock(), waitNewPowBlock() and verifyBlock().

    However for now let’s focus on the last interface method that the Template Provider needs, namely waitNext(). And on all the multiprocess changes.


Sjors TheBlueMatt plebhash ryanofsky

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-26 03:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me