refactor: P2P transport without serialize version and type #28892

pull maflcko wants to merge 4 commits into bitcoin:master from maflcko:2311-p2p-no-nVersion- changing 15 files +145 −197
  1. maflcko commented at 2:16 pm on November 16, 2023: member

    Now that the serialize framework ignores the serialize version and serialize type, everything related to it can be removed from the code.

    This is the first step, removing dead code from the P2P stack. A different pull will remove it from the wallet and other parts.

  2. DrahtBot commented at 2:16 pm on November 16, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK ajtowns
    Stale ACK sipa

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #28463 (p2p: Increase inbound capacity for block-relay only connections by mzumsande)
    • #27826 (validation: log which peer sent us a header by Sjors)
    • #26326 (net: don’t lock cs_main while reading blocks in net processing by andrewtoth)
    • #21515 (Erlay: bandwidth-efficient transaction relay protocol by naumenkogs)
    • #10102 (Multiprocess bitcoin by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot renamed this:
    refactor: P2P transport without serialize version and type
    refactor: P2P transport without serialize version and type
    on Nov 16, 2023
  4. DrahtBot added the label Refactoring on Nov 16, 2023
  5. in src/net_processing.cpp:1988 in fa2c1e88b5 outdated
    1984@@ -1985,7 +1985,7 @@ void PeerManagerImpl::BlockDisconnected(const std::shared_ptr<const CBlock> &blo
    1985 void PeerManagerImpl::NewPoWValidBlock(const CBlockIndex *pindex, const std::shared_ptr<const CBlock>& pblock)
    1986 {
    1987     auto pcmpctblock = std::make_shared<const CBlockHeaderAndShortTxIDs>(*pblock);
    1988-    const CNetMsgMaker msgMaker(PROTOCOL_VERSION);
    1989+    const NetMsgMaker msgMaker{};
    


    ajtowns commented at 2:50 am on November 17, 2023:
    Why not const NetMsgMaker msgMaker; with no curly brackets?

    maflcko commented at 8:17 am on November 17, 2023:

    I like {}, because it has many benefits:

    • No need for code readers to waste brain cycles on thinking whether an initialization was missed
    • A smaller --word-diff-regex=., when in the future a designated initializer or constructor argument is added
    • I looks nice
  6. in src/netmessagemaker.h:15 in fa1f283409 outdated
    11@@ -12,14 +12,14 @@
    12 class CNetMsgMaker
    13 {
    14 public:
    15-    explicit CNetMsgMaker(int nVersionIn) : nVersion(nVersionIn){}
    16+    explicit CNetMsgMaker(int /*unused*/) {}
    


    ajtowns commented at 2:52 am on November 17, 2023:
    There’s not really any need for a NetMsgMaker objects anymore after this change – NetMsgMaker::Make() could just be a global/static function.

    maflcko commented at 8:20 am on November 17, 2023:

    Correct, but it seems easier to make any such cleanups in your follow-up when you move the PushMsg method from connman to cnode/connection. You could then also add a second PushMsg template overload that takes the message parts and internally calls the NetMsgMaker::Make(), thus completely removing it in most call places.

    Happy to do that here, if you think this pull is too small to review, or would otherwise benefit from it.


    ajtowns commented at 2:23 pm on November 17, 2023:
    Fair enough

    maflcko commented at 2:44 pm on November 17, 2023:

    I haven’t tried it, but I presume that introducing the second template overload requires some header/module shuffling to avoid circular dependencies.

    Happy to take a look, I just didn’t want to step on your toes of the cnode/connection pull request (or anything else you are working on).


    ajtowns commented at 3:38 pm on November 17, 2023:
    Could move NetMsgMaker entirely into PushMessage at that point, perhaps. shrug

    maflcko commented at 4:07 pm on November 17, 2023:

    Could move NetMsgMaker entirely into PushMessage at that point, perhaps. shrug

    I don’t think so, because a message may be constructed once and then sent to several peers, so this wouldn’t be a refactor and potentially cause a performance penalty?

    Edit: See for example the code around const std::shared_future<CSerializedNetMsg> lazy_ser.


    maflcko commented at 4:26 pm on November 17, 2023:

    There’s not really any need for a NetMsgMaker objects anymore after this change – NetMsgMaker::Make() could just be a global/static function.

    Thanks, done in #28892 (comment)


    theuni commented at 4:28 pm on November 17, 2023:

    Could move NetMsgMaker entirely into PushMessage at that point, perhaps. shrug

    FWIW, NetMsgMaker was originally created exactly to prevent this for the sake of logical separation. The idea is/was that net shouldn’t have any idea how to serialize a (for example) CBlock, rather it should just receive the dumb bytes.


    ajtowns commented at 5:11 pm on November 17, 2023:
    @theuni I think my idea there was that template<typename T...> PushMessage(CNode& node, T... stuff) that accepts CBlock and so on would be in net_processing.cpp, and that would call CNode::PushMessage(msg) after serializing? That would fit with https://github.com/ajtowns/bitcoin/commit/6be38ef045cd1b974ae13d705260e83b769a48b5 at least. I’m just making this up as I go along though :man_shrugging:

    maflcko commented at 5:45 pm on November 17, 2023:
    Oh, I see you’d add the function as a method to PeerManagerImpl. That should be simple enough to push here as well. I’ll try to do that next week.

    maflcko commented at 2:14 pm on November 19, 2023:
    Done
  7. refactor: VectorWriter without nVersion
    The field is unused, so remove it.
    
    This is also required for future commits.
    fa0ed07941
  8. Remove unused Make() overload in netmessagemaker.h 66669da4a5
  9. maflcko force-pushed on Nov 17, 2023
  10. sipa commented at 2:38 pm on November 17, 2023: member

    utACK fade05b27348f18d1f9cf8c0b5cc26eaffe6b452

    I’m also ok with a commit in this PR to drop NetMsgMaker entirely.

  11. ajtowns commented at 3:36 pm on November 17, 2023: contributor
    ACK fade05b27348f18d1f9cf8c0b5cc26eaffe6b452
  12. fanquake requested review from theuni on Nov 17, 2023
  13. maflcko force-pushed on Nov 17, 2023
  14. maflcko force-pushed on Nov 17, 2023
  15. DrahtBot added the label CI failed on Nov 17, 2023
  16. maflcko commented at 4:24 pm on November 17, 2023: member
    Removed the class and replaced it with a template function in a namespace to address the feedback of both reviewers (Thanks!)
  17. in src/test/net_tests.cpp:860 in fa3e1ff64c outdated
    856@@ -858,13 +857,13 @@ BOOST_AUTO_TEST_CASE(initial_advertise_from_version_message)
    857     std::chrono::microseconds time_received_dummy{0};
    858 
    859     const auto msg_version =
    860-        msg_maker.Make(NetMsgType::VERSION, PROTOCOL_VERSION, services, time, services, CAddress::V1_NETWORK(peer_us));
    861+        NetMsg::Make(NetMsgType::VERSION, PROTOCOL_VERSION, services, time, services, CAddress::V1_NETWORK(peer_us));
    


    sipa commented at 4:27 pm on November 17, 2023:
    There is a stale PROTOCOL_VERSION here I think.

    maflcko commented at 4:32 pm on November 17, 2023:

    There is a stale PROTOCOL_VERSION here I think.

    Are you sure? According to the Bitcoin wiki the first 4 bytes in the version message are there to " Identifies protocol version being used by the node “, so changing this may change the test, so better done in a follow-up?

    The goal here is only to remove the serialize version and type.


    sipa commented at 4:45 pm on November 17, 2023:
    Oof, yes, you’re right.
  18. DrahtBot removed the label CI failed on Nov 17, 2023
  19. maflcko force-pushed on Nov 19, 2023
  20. in src/net_processing.cpp:1283 in fae1f4dc79 outdated
    1279@@ -1277,14 +1280,14 @@ void PeerManagerImpl::MaybeSetPeerAsAnnouncingHeaderAndIDs(NodeId nodeid)
    1280             // As per BIP152, we only get 3 of our peers to announce
    1281             // blocks using compact encodings.
    1282             m_connman.ForNode(lNodesAnnouncingHeaderAndIDs.front(), [this](CNode* pnodeStop){
    1283-                m_connman.PushMessage(pnodeStop, CNetMsgMaker(pnodeStop->GetCommonVersion()).Make(NetMsgType::SENDCMPCT, /*high_bandwidth=*/false, /*version=*/CMPCTBLOCKS_VERSION));
    1284+                PushMessage(*pnodeStop, NetMsg::Make(NetMsgType::SENDCMPCT, /*high_bandwidth=*/false, /*version=*/CMPCTBLOCKS_VERSION));
    


    ajtowns commented at 5:50 am on November 20, 2023:

    Calling NetMsg::Make within PushMessage seems unnecessary; just add an extra PeerManagerImpl helper:

    0    template <typename... Args>
    1    void PushMessage(CNode& node, std::string msg_type, Args&&... args) const
    2    {
    3        PushMessage(node, NetMsg::Make(std::move(msg_type), args...));
    4    }
    

    cf https://github.com/ajtowns/bitcoin/commit/0fe129b1b437d39221cef843f969f1adc31b543e


    maflcko commented at 7:19 am on November 20, 2023:
    Correct, but I wasn’t sure if reviewers preferred the slightly more verbose way to clarify that serialization is involved on every call.

    ajtowns commented at 11:40 am on November 20, 2023:
    Could give it a distinct name like void MakeAndPushMessage(node, msg_type, args) to make that clear, I guess?

    maflcko commented at 1:07 pm on November 20, 2023:

    MakeAndPushMessage https://github.com/ajtowns/bitcoin/commit/0fe129b1b437d39221cef843f969f1adc31b543e

    Thanks, done both. Looks like you forgot std::forward? Added it, and added you as co-author

  21. refactor: NetMsg::Make() without nVersion
    The nVersion field is unused, so remove it.
    
    This is also required for future commits.
    
    Also, add PushMessage aliases in PeerManagerImpl to make calling code
    less verbose.
    
    Co-Authored-By: Anthony Towns <aj@erisian.com.au>
    fa9b5f4fe3
  22. maflcko force-pushed on Nov 20, 2023
  23. in src/net.h:244 in fafd5e5bec outdated
    242     uint32_t m_message_size{0};          //!< size of the payload
    243     uint32_t m_raw_message_size{0};      //!< used wire size of the message (including header/checksum)
    244     std::string m_type;
    245 
    246-    CNetMessage(CDataStream&& recv_in) : m_recv(std::move(recv_in)) {}
    247+    CNetMessage(DataStream&& recv_in) : m_recv(std::move(recv_in)) {}
    


    ajtowns commented at 7:49 am on November 21, 2023:
    Maybe add explicit to this?

    maflcko commented at 7:26 am on November 22, 2023:
    Thanks, will do in one of the follow-up pulls, or here if I have to re-touch, whichever happens earlier.

    maflcko commented at 12:44 pm on November 23, 2023:
    According to https://corecheck.dev/bitcoin/bitcoin/pulls/28892 there were three instances in total, so I went ahead and fixed them all.
  24. ajtowns commented at 0:58 am on November 22, 2023: contributor
    ACK fafd5e5becf035426a45179ff592c11ccf2d5da5
  25. DrahtBot requested review from sipa on Nov 22, 2023
  26. refactor: P2P transport without serialize version and type fa79a881ce
  27. maflcko force-pushed on Nov 23, 2023
  28. ajtowns commented at 5:09 am on November 27, 2023: contributor
    reACK fa79a881ce0537e1d74da296a7513730438d2a02
  29. fanquake merged this on Nov 28, 2023
  30. fanquake closed this on Nov 28, 2023

  31. maflcko deleted the branch on Nov 28, 2023
  32. bitcoin locked this on Nov 27, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-22 00:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me