Stratum v2 Transport #30315

pull Sjors wants to merge 8 commits into bitcoin:master from Sjors:2024/06/sv2_transport changing 24 files +2660 −178
  1. Sjors commented at 3:59 pm on June 20, 2024: member

    Based on #29346. Followed by https://github.com/Sjors/bitcoin/pull/50. Parent PR #29432.

    Introduces Sv2Transport::Transport which is very similar to V2Transport.

    This shoehorns Sv2NetMsg into a CSerializedNetMsg in SetMessageToSend, and into a CNetMessage in GetReceivedMessage.

    See discussion in #30209.

  2. DrahtBot commented at 3:59 pm on June 20, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31045 (ci: Add missing -DWERROR=ON to test-each-commit by maflcko)
    • #30988 (Split CConnman by vasild)
    • #30595 (kernel: Introduce initial C header API by TheCharlatan)
    • #29415 (Broadcast own transactions only via short-lived Tor or I2P connections by vasild)
    • #28792 (Embed default ASMap as binary dump header file by fjahr)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. Sjors commented at 4:03 pm on June 20, 2024: member
    Would it make sense to add a libbitcoin_net library? Either at the same level as libbitcoin_common or on top of it. I could see how other node implementations could benefit from that, but it’s also a way to limit libbitcoin_common in size. Afaik only libbitcoin_node needs networking (and zmq?), and a future libbitcoin_sv2, and maybe libbitcoin_rpc.
  4. Sjors force-pushed on Jun 20, 2024
  5. DrahtBot added the label CI failed on Jun 20, 2024
  6. DrahtBot commented at 4:05 pm on June 20, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/26477378291

  7. Sjors force-pushed on Jun 20, 2024
  8. Sjors force-pushed on Jun 21, 2024
  9. Sjors commented at 11:18 am on June 21, 2024: member

    It compiles and the tests run, but it’s not pretty.

    The conversion between Sv2NetMsg and CSerializedNetMsg / CNetMessage is done in https://github.com/bitcoin/bitcoin/pull/30315/commits/7d937674de425d8e2525bac9e719ac93a775ea45, which is a bit of a hack at the moment.

    I also haven’t put much thought into Sv2NetMsg ever since I took over #27854, so perhaps this design can be improved as well.

  10. Sjors force-pushed on Jun 21, 2024
  11. Sjors force-pushed on Jun 21, 2024
  12. sipa commented at 1:34 pm on June 21, 2024: member
    I only had a very brief look, but my guess would be that it would be easier if Sv2NetMsg did not contain an Sv2NetHeader, and just stored type and message. The Sv2 Transport would then construct the header at submitting or sending time instead. This means Sv2NetMsg would be more of a dumb container for what higher-level code cares about, while the protocol details would be more abstracted away in the Transport.
  13. DrahtBot removed the label CI failed on Jun 21, 2024
  14. Sjors force-pushed on Jun 21, 2024
  15. Sjors commented at 3:44 pm on June 21, 2024: member
    @sipa done and managed to clean up sv2_messages.h a bit in the process.
  16. in src/common/sv2_messages.h:14 in e108385a6e outdated
     9+#include <span.h>
    10+#include <streams.h>
    11+#include <string>
    12+#include <util/check.h>
    13+
    14+namespace node {
    


    Sjors commented at 3:47 pm on June 21, 2024:
    639be6df22800d2e2a170c1af74d51951597834c: this namespace is outdated, since this isn’t part of the node anymore.
  17. Sjors force-pushed on Jun 21, 2024
  18. in src/test/sv2_transport_tests.cpp:248 in 2f673b5959 outdated
    242+
    243+        // Payload
    244+        Span<const std::byte> payload_plain = MakeByteSpan(msg.data());
    245+        // TODO: truncate very long messages, about 100 bytes at the start and end
    246+        //       is probably enough for most debugging.
    247+        LogPrintLevel(BCLog::SV2, BCLog::Level::Trace, "Payload: %s\n", HexStr(payload_plain));
    


    Sjors commented at 5:15 pm on June 21, 2024:

    sipa commented at 5:20 pm on June 21, 2024:
    Pretty sure you mean MakeByteSpan(msg), not MakeByteSpan(msg.data()).

    Sjors commented at 6:15 pm on June 21, 2024:
    I did! Did I? Should be fixed now.
  19. Sjors force-pushed on Jun 21, 2024
  20. DrahtBot commented at 6:14 pm on June 21, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/26526802432

  21. DrahtBot added the label CI failed on Jun 21, 2024
  22. Sjors force-pushed on Jun 24, 2024
  23. DrahtBot removed the label CI failed on Jun 24, 2024
  24. Sjors force-pushed on Jun 25, 2024
  25. Sjors force-pushed on Jun 25, 2024
  26. Sjors force-pushed on Jun 28, 2024
  27. Sjors force-pushed on Jun 28, 2024
  28. Sjors commented at 3:14 pm on June 28, 2024: member
    I dropped the refactoring commits e18887d3c45d4c0b2d414392ec33dbc36c0d8df7, cf67a72aea67b8ecf2c24200f69c68d1e7f4de4b and 912b800581ba6c21b0ed43832c6307f98e7a3af1 which move Transport and its prerequisites to libbitcoin_common. See https://github.com/Sjors/bitcoin/pull/47 for a more ambitious attempt to introduce a whole new libbitcoin_net, but that became real big and distracting real fast :-)
  29. Sjors force-pushed on Jul 1, 2024
  30. Sjors force-pushed on Jul 2, 2024
  31. DrahtBot added the label CI failed on Jul 7, 2024
  32. DrahtBot commented at 5:04 am on July 7, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/26955137717

  33. Sjors force-pushed on Jul 11, 2024
  34. DrahtBot removed the label CI failed on Jul 11, 2024
  35. Sjors force-pushed on Jul 17, 2024
  36. Sjors force-pushed on Jul 19, 2024
  37. Sjors force-pushed on Aug 8, 2024
  38. Sjors force-pushed on Aug 13, 2024
  39. hebasto added the label Needs CMake port on Aug 16, 2024
  40. Sjors force-pushed on Aug 29, 2024
  41. Sjors commented at 11:12 am on August 29, 2024: member
    Rebased for CMake (only) as well as #30571.
  42. hebasto removed the label Needs CMake port on Aug 29, 2024
  43. Sjors force-pushed on Aug 29, 2024
  44. DrahtBot added the label Needs rebase on Sep 3, 2024
  45. Sjors force-pushed on Sep 5, 2024
  46. DrahtBot removed the label Needs rebase on Sep 5, 2024
  47. Sjors force-pushed on Sep 10, 2024
  48. DrahtBot added the label CI failed on Sep 11, 2024
  49. Sjors force-pushed on Sep 13, 2024
  50. DrahtBot removed the label CI failed on Sep 13, 2024
  51. Sjors force-pushed on Sep 19, 2024
  52. Sjors force-pushed on Sep 19, 2024
  53. Sjors force-pushed on Sep 20, 2024
  54. Sjors commented at 9:13 am on September 20, 2024: member
    I added a commit (again) that moves CNetMessage, CSerializedNetMsg and Transport to common/transport.h. To keep the diff small it doesn’t move the implementation; that could be done in a separate PR. This avoids a circular dependency between bitcoin-node and bitcoin-sv2.
  55. Sjors force-pushed on Sep 20, 2024
  56. DrahtBot added the label CI failed on Sep 20, 2024
  57. DrahtBot commented at 9:25 am on September 20, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/30422811066

    Make sure to run all tests locally, according to the documentation.

    The failure may happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  58. Sjors commented at 3:56 pm on September 20, 2024: member
    Each of these CI failures seem spurious, some sort of timeout after many hours. However there’s real failures in the followup PR. Going to rebase this to avoid confusion.
  59. Sjors force-pushed on Sep 20, 2024
  60. DrahtBot removed the label CI failed on Sep 20, 2024
  61. DrahtBot added the label CI failed on Sep 26, 2024
  62. DrahtBot removed the label CI failed on Sep 30, 2024
  63. DrahtBot added the label Needs rebase on Sep 30, 2024
  64. Add sv2 log category for Stratum v2 b4a84abda9
  65. build: libbitcoin_sv2 scaffold d08a2ebfd0
  66. ci: always build WITH_SV2=ON 5f36962cfc
  67. Add sv2 noise protocol
    Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>
    b588ff85ae
  68. Add sv2 message CoinbaseOutputDataSize
    This commit adds the simplest stratum v2 message. The remaining messages are introduced in later commits.
    
    Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>
    1c8d46e167
  69. Move CNetMessage and Transport headers to common
    This avoids a circular dependency between bitcoin-sv2 and bitcoin-node.
    8352f32985
  70. Convert between Sv2NetMsg and CSerializedNetMsg
    This allows us to subclass Transport.
    a8cbde120a
  71. Introduce Sv2Transport
    Implemented starting from a copy of V2Transport and the V2TransportTester,
    modifying it to fit Stratum v2 and Noise Protocol requirements.
    
    Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>
    Co-Authored-By: Fi3
    62b255c99f
  72. Sjors force-pushed on Oct 4, 2024
  73. DrahtBot removed the label Needs rebase on Oct 4, 2024
  74. DrahtBot added the label Needs rebase on Oct 8, 2024
  75. DrahtBot commented at 3:35 pm on October 8, 2024: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

  76. Sjors commented at 8:57 am on October 16, 2024: member
    I moved this to https://github.com/Sjors/bitcoin/pull/67 so we can focus on building an interface for external applications to use, and getting multiprocess in a release.
  77. Sjors closed this on Oct 16, 2024


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-21 12:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me