Split CConnman #30988

pull vasild wants to merge 16 commits into bitcoin:master from vasild:sockman changing 19 files +1361 −726
  1. vasild commented at 7:57 am on September 27, 2024: contributor

    Currently CConnman is a mixture of:

    • low level socket handling, e.g. send, recv, poll, bind, listen, connect, and
    • higher level logic that is specific to the Bitcoin P2P protocol, e.g. V1/V2 transport, choosing which address to connect to, if we manage to connect mark the address good in AddrMan, maintaining the number of inbound and outbound connections, banning of peers, interacting with PeerManager.

    This PR splits the socket handling into a new class which makes the code more modular and reusable. Having more modular and reusable code is a good thing on its own, even if the code is not reused. Stratum V2 and libevent-less RPC/HTTP server could benefit from this, but it makes sense on its own, even without those projects.


    The socket operations are driven by the new class SockMan which informs the higher level via provided methods when e.g. new data arrives on the socket or a new connection is accepted. For this, SockMan provides some non-virtual methods to start it rolling and then it calls pure virtual methods which are implemented by the higher level (e.g. CConnman) on certain events, for example “got this new data on this node’s socket”.

    The public interface of SockMan is:

      0/**
      1 * A socket manager class which handles socket operations.
      2 * To use this class, inherit from it and implement the pure virtual methods.
      3 * Handled operations:
      4 * - binding and listening on sockets
      5 * - starting of necessary threads to process socket operations
      6 * - accepting incoming connections
      7 * - making outbound connections
      8 * - closing connections
      9 * - waiting for IO readiness on sockets and doing send/recv accordingly
     10 */
     11class SockMan
     12{
     13public:
     14
     15    //
     16    // Non-virtual functions, to be reused by children classes.
     17    //
     18
     19    /**
     20     * Bind to a new address:port, start listening and add the listen socket to `m_listen`.
     21     * Should be called before `StartSocketsThreads()`.
     22     * [@param](/bitcoin-bitcoin/contributor/param/)[in] to Where to bind.
     23     * [@param](/bitcoin-bitcoin/contributor/param/)[out] errmsg Error string if an error occurs.
     24     * [@retval](/bitcoin-bitcoin/contributor/retval/) true Success.
     25     * [@retval](/bitcoin-bitcoin/contributor/retval/) false Failure, `strError` will be set.
     26     */
     27    bool BindAndStartListening(const CService& to, bilingual_str& errmsg);
     28
     29    /**
     30     * Start the necessary threads for sockets IO.
     31     */
     32    void StartSocketsThreads(const Options& options);
     33
     34    /**
     35     * Join (wait for) the threads started by `StartSocketsThreads()` to exit.
     36     */
     37    void JoinSocketsThreads();
     38
     39    /**
     40     * Make an outbound connection, save the socket internally and return a newly generated node id.
     41     * [@param](/bitcoin-bitcoin/contributor/param/)[in] to The address to connect to, either as CService or a host as string and port as
     42     * an integer, if the later is used, then `proxy` must be valid.
     43     * [@param](/bitcoin-bitcoin/contributor/param/)[in] is_important If true, then log failures with higher severity.
     44     * [@param](/bitcoin-bitcoin/contributor/param/)[in] proxy Proxy to connect through if `proxy.IsValid()` is true.
     45     * [@param](/bitcoin-bitcoin/contributor/param/)[out] proxy_failed If `proxy` is valid and the connection failed because of the
     46     * proxy, then it will be set to true.
     47     * [@param](/bitcoin-bitcoin/contributor/param/)[out] me If the connection was successful then this is set to the address on the
     48     * local side of the socket.
     49     * [@return](/bitcoin-bitcoin/contributor/return/) Newly generated node id, or std::nullopt if the operation fails.
     50     */
     51    std::optional<NodeId> ConnectAndMakeNodeId(const std::variant<CService, StringHostIntPort>& to,
     52                                               bool is_important,
     53                                               const Proxy& proxy,
     54                                               bool& proxy_failed,
     55                                               CService& me);
     56
     57    /**
     58     * Disconnect a given peer by closing its socket and release resources occupied by it.
     59     * [@return](/bitcoin-bitcoin/contributor/return/) Whether the peer existed and its socket was closed by this call.
     60     */
     61    bool CloseConnection(NodeId node_id);
     62
     63    /**
     64     * Try to send some data to the given node.
     65     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Identifier of the node to send to.
     66     * [@param](/bitcoin-bitcoin/contributor/param/)[in] data The data to send, it might happen that only a prefix of this is sent.
     67     * [@param](/bitcoin-bitcoin/contributor/param/)[in] will_send_more Used as an optimization if the caller knows that they will
     68     * be sending more data soon after this call.
     69     * [@param](/bitcoin-bitcoin/contributor/param/)[out] errmsg If <0 is returned then this will contain a human readable message
     70     * explaining the error.
     71     * [@retval](/bitcoin-bitcoin/contributor/retval/) >=0 The number of bytes actually sent.
     72     * [@retval](/bitcoin-bitcoin/contributor/retval/) <0 A permanent error has occurred.
     73     */
     74    ssize_t SendBytes(NodeId node_id,
     75                      Span<const unsigned char> data,
     76                      bool will_send_more,
     77                      std::string& errmsg) const;
     78
     79    /**
     80     * Close all sockets.
     81     */
     82    void CloseSockets();
     83
     84    //
     85    // Pure virtual functions must be implemented by children classes.
     86    //
     87
     88    /**
     89     * Be notified when a new connection has been accepted.
     90     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Id of the newly accepted connection.
     91     * [@param](/bitcoin-bitcoin/contributor/param/)[in] me The address and port at our side of the connection.
     92     * [@param](/bitcoin-bitcoin/contributor/param/)[in] them The address and port at the peer's side of the connection.
     93     * [@retval](/bitcoin-bitcoin/contributor/retval/) true The new connection was accepted at the higher level.
     94     * [@retval](/bitcoin-bitcoin/contributor/retval/) false The connection was refused at the higher level, so the
     95     * associated socket and node_id should be discarded by `SockMan`.
     96     */
     97    virtual bool EventNewConnectionAccepted(NodeId node_id,
     98                                            const CService& me,
     99                                            const CService& them) = 0;
    100
    101    /**
    102     * Called when the socket is ready to send data and `ShouldTryToSend()` has
    103     * returned true. This is where the higher level code serializes its messages
    104     * and calls `SockMan::SendBytes()`.
    105     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Id of the node whose socket is ready to send.
    106     * [@param](/bitcoin-bitcoin/contributor/param/)[out] cancel_recv Should always be set upon return and if it is true,
    107     * then the next attempt to receive data from that node will be omitted.
    108     */
    109    virtual void EventReadyToSend(NodeId node_id, bool& cancel_recv) = 0;
    110
    111    /**
    112     * Called when new data has been received.
    113     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Node for which the data arrived.
    114     * [@param](/bitcoin-bitcoin/contributor/param/)[in] data Data buffer.
    115     * [@param](/bitcoin-bitcoin/contributor/param/)[in] n Number of bytes in `data`.
    116     */
    117    virtual void EventGotData(NodeId node_id, const uint8_t* data, size_t n) = 0;
    118
    119    /**
    120     * Called when the remote peer has sent an EOF on the socket. This is a graceful
    121     * close of their writing side, we can still send and they will receive, if it
    122     * makes sense at the application level.
    123     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Node whose socket got EOF.
    124     */
    125    virtual void EventGotEOF(NodeId node_id) = 0;
    126
    127    /**
    128     * Called when we get an irrecoverable error trying to read from a socket.
    129     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Node whose socket got an error.
    130     * [@param](/bitcoin-bitcoin/contributor/param/)[in] errmsg Message describing the error.
    131     */
    132    virtual void EventGotPermanentReadError(NodeId node_id, const std::string& errmsg) = 0;
    133
    134    //
    135    // Non-pure virtual functions can be overridden by children classes or left
    136    // alone to use the default implementation from SockMan.
    137    //
    138
    139    /**
    140     * SockMan would only call EventReadyToSend() if this returns true.
    141     * Can be used to temporary pause sends for a node.
    142     * The implementation in SockMan always returns true.
    143     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Node for which to confirm or cancel a call to EventReadyToSend().
    144     */
    145    virtual bool ShouldTryToSend(NodeId node_id) const;
    146
    147    /**
    148     * SockMan would only call Recv() on a node's socket if this returns true.
    149     * Can be used to temporary pause receives for a node.
    150     * The implementation in SockMan always returns true.
    151     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Node for which to confirm or cancel a receive.
    152     */
    153    virtual bool ShouldTryToRecv(NodeId node_id) const;
    154
    155    /**
    156     * SockMan has completed the current send+recv iteration for a node.
    157     * It will do another send+recv for this node after processing all other nodes.
    158     * Can be used to execute periodic tasks for a given node.
    159     * The implementation in SockMan does nothing.
    160     * [@param](/bitcoin-bitcoin/contributor/param/)[in] node_id Node for which send+recv has been done.
    161     */
    162    virtual void EventIOLoopCompletedForNode(NodeId node_id);
    163
    164    /**
    165     * SockMan has completed send+recv for all nodes.
    166     * Can be used to execute periodic tasks for all nodes.
    167     * The implementation in SockMan does nothing.
    168     */
    169    virtual void EventIOLoopCompletedForAllPeers();
    170
    171    /**
    172     * Be notified of a change in the state of listening for incoming I2P connections.
    173     * The default behavior, implemented by `SockMan`, is to ignore this event.
    174     * [@param](/bitcoin-bitcoin/contributor/param/)[in] addr Our listening address.
    175     * [@param](/bitcoin-bitcoin/contributor/param/)[in] success If true then the listen succeeded and we are now
    176     * listening for incoming I2P connections at `addr`. If false then the
    177     * call failed and now we are not listening (even if this was invoked
    178     * before with `true`).
    179     */
    180    virtual void EventI2PListen(const CService& addr, bool success);
    181};
    

    Resolves: #30694


    Review hint: this PR moves some code around, so reviewers may find this helpful: git show --color-moved --color-moved-ws=allow-indentation-change.

  2. DrahtBot commented at 7:57 am on September 27, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30988.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK tdb3
    Stale ACK pinheadmz

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #31519 (refactor: Use std::span over Span by maflcko)
    • #31022 (test: Add mockable steady clock, tests for PCP and NATPMP implementations by laanwj)
    • #31014 (net: Use GetAdaptersAddresses to get local addresses on Windows by laanwj)
    • #30951 (net: option to disallow v1 connection on ipv4 and ipv6 peers by stratospher)
    • #30381 ([WIP] net: return result from addnode RPC by willcl-ark)
    • #29641 (scripted-diff: Use LogInfo over LogPrintf [WIP, NOMERGE, DRAFT] by maflcko)
    • #29415 (Broadcast own transactions only via short-lived Tor or I2P connections by vasild)
    • #28584 (Fuzz: extend CConnman tests by vasild)
    • #28521 (net, net_processing: additional and consistent disconnect logging by Sjors)
    • #28463 (p2p: Increase inbound capacity for block-relay only connections by mzumsande)
    • #25832 (tracing: network connection tracepoints by 0xB10C)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. in src/rpc/net.cpp:210 in 03f6cc2b4a outdated
    202@@ -203,6 +203,15 @@ static RPCHelpMan getpeerinfo()
    203     std::vector<CNodeStats> vstats;
    204     connman.GetNodeStats(vstats);
    205 
    206+    // An undocumented side effect of how CConnman previously stored nodes was
    207+    // that they were returned ordered by id. At least some functional tests
    208+    // rely on that, so keep it that way. An alternative is to remove this sort
    209+    // and fix the tests and take the risk of breaking other users of the
    210+    // "getpeerinfo" RPC.
    


    maflcko commented at 8:41 am on September 27, 2024:
    nit: Arrays are ordered data structures and I don’t think it is user-friendly to shuffle them by ID. I think this comment can just be removed.

    vasild commented at 3:05 pm on September 27, 2024:
    Removed, thanks!
  4. DrahtBot added the label CI failed on Sep 27, 2024
  5. Sjors commented at 11:41 am on September 27, 2024: member

    Nice! I’ll try to use this for Sv2Connman in https://github.com/Sjors/bitcoin/pull/50 and will let you know if anything is missing.

    Can you put sockman.h in libbitcoin_common instead of libbitcoin_node? For the Template Provider I’m trying to prevent a circular dependency on the node. This should do the trick: https://github.com/bitcoin/bitcoin/commit/4dd51b2924860bf10466da080ea4ff7bed1a3e3f

  6. vasild force-pushed on Sep 27, 2024
  7. vasild commented at 3:07 pm on September 27, 2024: contributor

    03f6cc2b4a...70c2f13f83: fix CI failure, and address suggestions

    Can you put sockman.h in libbitcoin_common

    Done.

  8. DrahtBot removed the label CI failed on Sep 27, 2024
  9. Sjors commented at 4:51 pm on September 27, 2024: member

    Here’s an initial sketch of making Sv2Connman a subclass of SockMan. The test gets through the handshake but fails later on, so I’ll need to study it a bit more closely.

    https://github.com/Sjors/bitcoin/pull/64

  10. DrahtBot added the label CI failed on Sep 29, 2024
  11. DrahtBot removed the label CI failed on Sep 29, 2024
  12. tdb3 commented at 10:05 pm on September 29, 2024: contributor
    Concept ACK
  13. pinheadmz commented at 1:38 pm on September 30, 2024: member

    Concept ACK

    This looks great and the API in the header looks easy, thanks.

    I’m in the process of cleaning up my HTTP branch for a pull request and then I can start reviewing this and rebasing on top.

    One element of libevent I’m not immediately seeing here is timed events. Really the only thing HTTP needs it for is walletpassphrase which calls RPCRunLater() which interacts with HTTPRPCTimerInterface(). I don’t think Conman has a specific mechanism for this because timed things are attached directly to nodes like m_last_getheaders_timestamp etc. The current HTTPRPCTimerInterface uses libevent event_new() and evtimer_add(), I accomplish this with a map of timestamps and callback functions in my event loop: https://github.com/pinheadmz/bitcoin/commit/42b7240378ffc4890ab4ba1453623e5986ea2a71

  14. maflcko commented at 2:07 pm on September 30, 2024: member

    I accomplish this with a map of timestamps and callback functions in my event loop

    I wonder why the existing scheduler can’t be used for re-locking the wallet? I know there is #18488 and #14289, but the thread is already filled with random stuff such as BerkeleyDatabase::PeriodicFlush(), and relocking the wallet seems(?) fast (I haven’t benchmarked), so should be fine to put in there as well, at least from that perspective?

  15. vasild commented at 10:07 am on October 3, 2024: contributor

    @pinheadmz, I think that the functionality of “execute this code after some time”, is not much related to the sockets handling and better be implemented at some higher level, not inside SockMan. Maybe the scheduler, like @maflcko suggested, or in the EventIOLoopCompletedForAllPeers() method which will be called periodically by SockMan:

    0    /**
    1     * SockMan has completed send+recv for all nodes.
    2     * Can be used to execute periodic tasks for all nodes.
    3     * The implementation in SockMan does nothing.
    4     */
    5    virtual void EventIOLoopCompletedForAllPeers();
    

    Edit: I guess TriggerEvents() from https://github.com/pinheadmz/bitcoin/commit/42b7240378ffc4890ab4ba1453623e5986ea2a71 can be called from EventIOLoopCompletedForAllPeers() or from the scheduler.

  16. Sjors commented at 1:54 pm on October 4, 2024: member
    @vasild if you rebase past #31011, tidy might point out that sockman.cpp.o depends on i2p.cpp. So you probably need to either move i2p.cpp to common as well, or remove the dependency.
  17. Jacksonearl2468 approved
  18. DrahtBot added the label CI failed on Oct 7, 2024
  19. in src/common/sockman.cpp:5 in 70c2f13f83 outdated
    0@@ -0,0 +1,528 @@
    1+// Copyright (c) 2024-present The Bitcoin Core developers
    2+// Distributed under the MIT software license, see the accompanying
    3+// file COPYING or https://opensource.org/license/mit/.
    4+
    5+#include <config/bitcoin-config.h> // IWYU pragma: keep
    


    Sjors commented at 11:15 am on October 16, 2024:
    Silent merge conflict with #30937
  20. in src/net.cpp:1715 in 1bfc1ca9b6 outdated
    1795@@ -1796,10 +1796,10 @@ void CConnman::CreateNodeFromAcceptedSocket(std::unique_ptr<Sock>&& sock,
    1796 
    1797     CNode* pnode = new CNode(id,
    1798                              std::move(sock),
    1799-                             addr,
    1800+                             CAddress{addr, NODE_NONE},
    1801                              CalculateKeyedNetGroup(addr),
    1802                              nonce,
    1803-                             addr_bind,
    1804+                             CAddress{addr_bind, NODE_NONE},
    


    pinheadmz commented at 7:27 pm on October 16, 2024:

    1bfc1ca9b6b68c6356ffc23ecf01e417152ade95

    Why do CNode .addr and .addrBind still need to be CAddress? maybe it makes some sense for .addr if that’s where we also store the service flags for easier gossiping, but why would the local bind address need anything besides an IP and port?


    vasild commented at 12:43 pm on December 3, 2024:

    CNode::addr needs to be CAddress because at least PeerManagerImpl::PushNodeVersion() is using addr.nServices.

    CNode::addrBind indeed does not need to be CAddress. I changed it to CService.

  21. in src/common/sockman.cpp:88 in 41c87ddb3d outdated
    52+            strError = strprintf(Untranslated("Error setting IPV6_PROTECTION_LEVEL on socket: %s, continuing anyway"), NetworkErrorString(WSAGetLastError()));
    53+            LogPrintf("%s\n", strError.original);
    54+        }
    55+#endif
    56+    }
    57+
    


    pinheadmz commented at 7:38 pm on October 16, 2024:

    41c87ddb3d7a18d2c0fa7eccfbde57e9d6e898c2

    Maybe these are expected to be set by callers in future commits, so I’m just leaving myself a note to look for SO_KEEPALIVE and TCP_NODELAY


    pinheadmz commented at 3:19 pm on October 17, 2024:

    Ok I see NODELAY set by ConnMan in EventNewConnectionAccepted in b94f9d338fd0d3b47a4ea6165ce9cde48f3b19bc

    I’m guessing we dont use KEEPALIVE for bitcoin p2p because we have out own ping/pong protocol


    pinheadmz commented at 6:04 pm on October 17, 2024:
    Moved to NewSockAccepted() in 9d1b352a4d050ec88b59cdf59209c0f086fea4e6
  22. in src/common/sockman.h:23 in 5d4920f630 outdated
     8@@ -9,9 +9,12 @@
     9 #include <util/sock.h>
    10 #include <util/translation.h>
    11 
    12+#include <atomic>
    13 #include <memory>
    14 #include <vector>
    15 
    16+typedef int64_t NodeId;
    


    pinheadmz commented at 2:53 pm on October 17, 2024:

    5d4920f630417ceeee1c79304faac6803795cd64

    I don’t really have any better suggestion for this but the term “node” is now a higher-level ConnMan context, for HTTP and StratumV2 I’d probably refer to them as “client”.


    vasild commented at 12:46 pm on December 3, 2024:
    “Peer” is a good description for this, but that is already used in net_processing.
  23. in src/common/sockman.cpp:308 in b94f9d338f outdated
    160+    };
    161+
    162+    while (!interruptNet) {
    163+
    164+        if (!m_i2p_sam_session->Listen(conn)) {
    165+            EventI2PListen(conn.me, /*success=*/false);
    


    pinheadmz commented at 3:11 pm on October 17, 2024:

    b94f9d338fd0d3b47a4ea6165ce9cde48f3b19bc

    Reminding myself to check for a test that covers a child class with i2p true but no Event listener

  24. in src/common/sockman.h:105 in b94f9d338f outdated
    100+     * Be notified when a new connection has been accepted.
    101+     * @param[in] sock Connected socket to communicate with the peer.
    102+     * @param[in] me The address and port at our side of the connection.
    103+     * @param[in] them The address and port at the peer's side of the connection.
    104+     */
    105+    virtual void EventNewConnectionAccepted(std::unique_ptr<Sock>&& sock,
    


    pinheadmz commented at 3:29 pm on October 17, 2024:

    b94f9d338fd0d3b47a4ea6165ce9cde48f3b19bc

    Seems weird to me to move the unique pointer to a connected socket out of sockman, implying that sockman itself doesn’t own the connections. Peeking ahead at future commits though I think this gets resolved.


    pinheadmz commented at 6:22 pm on October 17, 2024:
    resolved in b94f9d338fd0d3b47a4ea6165ce9cde48f3b19bc
  25. in src/net.h:1397 in b96beb27d2 outdated
    1410@@ -1410,7 +1411,7 @@ class CConnman : private SockMan
    1411     std::vector<AddedNodeParams> m_added_node_params GUARDED_BY(m_added_nodes_mutex);
    1412 
    1413     mutable Mutex m_added_nodes_mutex;
    1414-    std::vector<CNode*> m_nodes GUARDED_BY(m_nodes_mutex);
    1415+    std::unordered_map<NodeId, CNode*> m_nodes GUARDED_BY(m_nodes_mutex);
    


    pinheadmz commented at 4:52 pm on October 17, 2024:

    b96beb27d2b3d06a45644c537c114a08f0ccd285

    Love this, great optimization and makes a lot of sense.

  26. in src/test/fuzz/connman.cpp:49 in b96beb27d2 outdated
    44@@ -45,12 +45,13 @@ FUZZ_TARGET(connman, .init = initialize_connman)
    45     connman.Init(options);
    46 
    47     CNetAddr random_netaddr;
    48-    CNode random_node = ConsumeNode(fuzzed_data_provider);
    49+    NodeId node_id{0};
    50+    CNode random_node = ConsumeNode(fuzzed_data_provider, node_id++);
    


    pinheadmz commented at 4:55 pm on October 17, 2024:

    b96beb27d2b3d06a45644c537c114a08f0ccd285

    Why not use GetNewNodeId() here? (and below) Don’t you have a Connman with its own m_next_node_id?


    vasild commented at 1:10 pm on December 3, 2024:
    GetNewNodeId() is private. Not worth weakening the encapsulation for this test.
  27. in src/common/sockman.h:228 in bb5b91d430 outdated
    119+     */
    120+    virtual bool ShouldTryToSend(NodeId node_id) const;
    121+
    122+    /**
    123+     * SockMan would only call Recv() on a node's socket if this returns true.
    124+     * Can be used to temporary pause receives for a node.
    


    pinheadmz commented at 5:03 pm on October 17, 2024:

    bb5b91d430026c4826a2107c8c1a311518cc97ce

    nit s/temporary/temporarily

    and above


    vasild commented at 1:13 pm on December 3, 2024:
    Done
  28. in src/common/sockman.h:250 in 50cb52470e outdated
    139+    /**
    140+     * SockMan has completed send+recv for all nodes.
    141+     * Can be used to execute periodic tasks for all nodes.
    142+     * The implementation in SockMan does nothing.
    143+     */
    144+    virtual void EventIOLoopCompletedForAllPeers();
    


    pinheadmz commented at 5:12 pm on October 17, 2024:

    50cb52470ea6cc4de5a5e5260b8f308353942ec0

    This is great, can be used for timed actions (like locking the wallet) we used to depend on libevent for

  29. in src/net.cpp:2035 in 50cb52470e outdated
    2030+void CConnman::EventIOLoopCompletedForAllPeers()
    2031+{
    2032+    AssertLockNotHeld(m_nodes_mutex);
    2033+    AssertLockNotHeld(m_reconnections_mutex);
    2034+
    2035+    DisconnectNodes();
    


    pinheadmz commented at 5:19 pm on October 17, 2024:

    50cb52470ea6cc4de5a5e5260b8f308353942ec0

    Interesting that the parent class handles accepting connections but the child is expected to handle disconnections, and inside the utility event loop function. Of course only the derived class would know when disconnection is appropriate, I wonder if anything it makes sense just to add a comment in sockman.h that EventIOLoopCompletedForAllPeers() is expected to handle that, erasing items from m_nodes


    vasild commented at 1:34 pm on December 3, 2024:

    Good observation. I guess this is the nature of things: accepting new connections is driven (initiated) by low level socket event when a new connection arrives, while disconnecting can be low level driven or high level driven - e.g. misbehaving peer at the application layer.

    I extended the comment of EventIOLoopCompletedForAllPeers() a little bit:

    0    -+     * Can be used to execute periodic tasks for all nodes.                   
    1    ++     * Can be used to execute periodic tasks for all nodes, like disconnecting
    2    ++     * nodes due to higher level logic.           
    
  30. in src/common/sockman.cpp:24 in 3bb0145514 outdated
     9@@ -10,6 +10,19 @@
    10 #include <util/sock.h>
    11 #include <util/thread.h>
    12 
    13+CService GetBindAddress(const Sock& sock)
    14+{
    15+    CService addr_bind;
    16+    struct sockaddr_storage sockaddr_bind;
    17+    socklen_t sockaddr_bind_len = sizeof(sockaddr_bind);
    18+    if (!sock.GetSockName((struct sockaddr*)&sockaddr_bind, &sockaddr_bind_len)) {
    


    pinheadmz commented at 5:32 pm on October 17, 2024:

    3bb0145514978092ae966e30693cf619e4034837

    the else {error} looked backwards to me here at first, might be more clear to use == 0 instead of ! but i dunno what the official style is for this.


    vasild commented at 1:41 pm on December 3, 2024:

    I don’t think we have an official recommendation. My personal preference is to not treat integers as booleans: use if (x == 0) instead of if (!x) when x is an integer. But I am leaving this as it is because it is like that in master and this PR only moves it from net.cpp to sockman.cpp verbatim. I hope it is easier to reviewers to just check that a blob of code is moved without having to validate its correctness since that same code is already in master. This helps with such moves:

    ~/.gitconfig:

    0[diff]  
    1        colorMoved = dimmed-zebra
    2        colorMovedWS = allow-indentation-change
    
  31. in src/common/sockman.cpp:13 in 3bb0145514 outdated
     9@@ -10,6 +10,19 @@
    10 #include <util/sock.h>
    11 #include <util/thread.h>
    12 
    13+CService GetBindAddress(const Sock& sock)
    


    pinheadmz commented at 5:35 pm on October 17, 2024:

    3bb0145514978092ae966e30693cf619e4034837

    Would this make more sense as a direct method of Sock ? Especially now that it returns the simpler CService instead of CAddress


    vasild commented at 1:46 pm on December 3, 2024:
    Maybe yes, but sock.{h,cpp} does not currently include netaddress.h and I guess including it may cause circular dependency. It would also increase the scope of this PR. I am leaving it as it is.
  32. in src/common/sockman.h:103 in 3bb0145514 outdated
     99+     * local side of the socket.
    100+     * @param[out] sock Connected socket, if the operation is successful.
    101+     * @param[out] i2p_transient_session I2P session, if the operation is successful.
    102+     * @return Newly generated node id, or std::nullopt if the operation fails.
    103+     */
    104+    std::optional<NodeId> ConnectAndMakeNodeId(const std::variant<CService, StringHostIntPort>& to,
    


    pinheadmz commented at 5:44 pm on October 17, 2024:

    3bb0145514978092ae966e30693cf619e4034837

    This is great. Won’t be needed immediately for HTTP or StratumV2 but means we can more easily remove libevent from bitcoin-cli !

  33. in src/common/sockman.cpp:282 in c133a634d1 outdated
    334@@ -262,8 +335,16 @@ void SockMan::EventIOLoopCompletedForAllPeers() {}
    335 
    336 void SockMan::EventI2PListen(const CService&, bool) {}
    337 
    338+void SockMan::TestOnlyAddExistentNode(NodeId node_id, std::unique_ptr<Sock>&& sock)
    339+{
    340+    LOCK(m_connected_mutex);
    341+    m_connected.emplace(node_id, std::make_shared<NodeSockets>(std::move(sock)));
    


    pinheadmz commented at 2:57 pm on October 18, 2024:

    c133a634d10e172652ebbf05ca114037c8e551b9

    I know this is for testing only, but what happens if there is a NodeId collision (An existing node is inserted with a duplicate ID, replacing a connected node in sockman)?


    vasild commented at 1:53 pm on December 3, 2024:
    It would be a mess. I added an assert to make it clear that a test which inserts duplicate ids is broken.
  34. in src/common/sockman.cpp:340 in c133a634d1 outdated
    390+        // listening sockets in one call ("readiness" as in poll(2) or
    391+        // select(2)). If none are ready, wait for a short while and return
    392+        // empty sets.
    393+        auto io_readiness{GenerateWaitSockets()};
    394+        if (io_readiness.events_per_sock.empty() ||
    395+            !io_readiness.events_per_sock.begin()->first->WaitMany(SELECT_TIMEOUT,
    


    pinheadmz commented at 3:13 pm on October 18, 2024:

    c133a634d10e172652ebbf05ca114037c8e551b9

    You might want to add another comment here - when I was trying to handle socks myself I found this a bit weird and wrote:

    0    // WaitMany() mine as well be a static function, the context
    1    // of the first Sock in the vector is not relevant.
    

    vasild commented at 1:57 pm on December 3, 2024:
    Done
  35. pinheadmz approved
  36. pinheadmz commented at 3:14 pm on October 18, 2024: member

    ACK 70c2f13f83a5cc740330d0b4af9cbd74515be6b2

    Built and ran tests on macos/arm as well as debian/x86.

    Synced to signet with this build on a debian server, then synced from that node locally using the macos build.

    I left a lot of comments below but most are for myself to track during rebases, indicate to the maintainers the depth that I’m actually reviewing, and also some notes for myself – I’ll be rebaing my non-libevent http server on this banch and consuming the new SockMan.

    I have a working branch with my own sock handler for http, so I was able to compare that work to the code in this PR to follow along.

    There are two areas I am not entirely clear on that I will review more closely in the coming weeks which is I2P stuff, and reviewing all the mutex stuff.

     0-----BEGIN PGP SIGNED MESSAGE-----
     1Hash: SHA256
     2
     3ACK 70c2f13f83a5cc740330d0b4af9cbd74515be6b2
     4-----BEGIN PGP SIGNATURE-----
     5
     6iQIzBAEBCAAdFiEE5hdzzW4BBA4vG9eM5+KYS2KJyToFAmcSeTIACgkQ5+KYS2KJ
     7yTp/1A//ZzhksZzWlBbV6rcfIXnUVQS0SIgmY7sZCN8AN0RoiN4ECALWyrXz9Rdg
     8fONCWiQSTyxE5u2TnFo7os+o0Z+3ybxApufAx/OzqHaJwR8d5wnZPHU/uFTPYm/z
     9qArPT3SHQgKb1sftt8S5ve7j37RW7IbshiJ+6bq8+DNMY1Ep1DX4BBzZgwbzx2ek
    10qMcRFQkOkTbJg3/yOT1Vz361qCPLmbKjvbRyjYgYYG7oxtdwb5r/ezeagSCWKDJo
    11Yx/DJ2cwZ3uMWDvfnhNJOeql4MhOLZcU5hKyGGTRT3B5ER7ELNye5+Sz+pv0cJuP
    12R+zfoxAfYV3opqB/ewTD1sM+gtFZFUiLNHigPgKFmK5GerLguXUqY+vgQ/4VIEpg
    13XD2tHOCwwDULsekVLZ+hXJPSTBDmZAlHmJvcCBQgzLkjCMvrlmxJR8KpKg3lDgKM
    14GwFcYoIWubyFKgRCAQL6vc1qFnGc7PJ1TWJLoKxpjXeyE6igHquSK0KE3flhCJ97
    15CpWhZflrHwcaomSf9bDYsSaNvMN2MiFUzC4lbmLRWTzeOCS9oMFRMcIayQ+psUqC
    160gmbZzi+Qcf32gCNYvhxYwKqTeAlKih9x+vurTuK8831AZoUYwusetGaTsklwvOn
    17bDpgEGoqFPdPIB8yHAiFBwQPcGmn7tVmTbVhck59B3HvdS/wOr4=
    18=rnjt
    19-----END PGP SIGNATURE-----
    

    pinheadmz’s public key is on keybase

  37. DrahtBot requested review from tdb3 on Oct 18, 2024
  38. Sjors commented at 3:09 pm on October 21, 2024: member
    cc @theuni any takes?
  39. DrahtBot added the label Needs rebase on Oct 28, 2024
  40. Sjors commented at 4:47 pm on November 14, 2024: member

    This needs a rebase. Otherwise it’s non-trivial for me to keep https://github.com/Sjors/bitcoin/pull/67 rebased.

    Conflicts:

  41. vasild force-pushed on Nov 14, 2024
  42. vasild commented at 4:58 pm on November 14, 2024: contributor

    70c2f13f83...c4f51c7f61: rebase due to conflicts, next I will look at @pinheadmz feedback above and implement changes from coredev discussions:

    • Store the node id -> CNode map in SockMan (then the SockMan class becomes templated)
    • SockMan <-> Connman calls based on CNode, not node id
    • Make the virtual methods in SockMan private, since they are called only from SockMan
  43. DrahtBot removed the label CI failed on Nov 14, 2024
  44. DrahtBot removed the label Needs rebase on Nov 14, 2024
  45. vasild force-pushed on Nov 19, 2024
  46. vasild commented at 1:28 pm on November 19, 2024: contributor

    c4f51c7f61...2b0103705f: rebase and make the virtual methods of SockMan private since they are only used by SockMan.

    About storing the CNode in SockMan (which would be templated like SockMan<CNode>) and making the communication between SockMan and the higher class (e.g. CConnman) based on CNode instead of node id: this is an excellent idea that will make the code more straight-forward and the higher classes simpler. However it would make this PR larger. I will leave it off for now. There is some specific CNode ref-counting in CConnman and the code around deleting a node is a bit complicated, so it would require further non-trivial changes to move that to SockMan.

    I think in general, in the long term, independently of this PR, it would be good to get rid of the manual CNode ref-counting and tap std::shared_ptr to do that for us. In other words, to revive #28222.

  47. vasild force-pushed on Dec 3, 2024
  48. vasild commented at 1:59 pm on December 3, 2024: contributor
    2b0103705f...645f625e29: rebase and address suggestions
  49. DrahtBot added the label Needs rebase on Dec 6, 2024
  50. in src/common/sockman.h:128 in 645f625e29 outdated
    123+     * explaining the error.
    124+     * @retval >=0 The number of bytes actually sent.
    125+     * @retval <0 A permanent error has occurred.
    126+     */
    127+    ssize_t SendBytes(NodeId node_id,
    128+                      Span<const unsigned char> data,
    


    pinheadmz commented at 8:12 pm on December 10, 2024:
    Why not use std::span? I thought Span was only a temporary fill-in until C++20 something something? Also, I’m using std::span elsewhere… is Span the preference?

    Sjors commented at 7:10 am on December 16, 2024:
    Even in relatively new code folks often use Span, e.g. #30043. cc @sipa, @laanwj

    vasild commented at 5:47 am on December 19, 2024:

    Must have been just inertia. The caller of the newly added SendBytes() has an argument that is already Span from elsewhere in the code, coming from here:

    https://github.com/bitcoin/bitcoin/blob/c1252b14d714448295200a595086dd3e78b46c8f/src/net.h#L307-L311

    Now I changed the argument of the newly added SendBytes() to std::span, looks better and the Span variable is accepted. Thanks!

    wen s/Span/std::span/ all over the place?


    vasild commented at 6:36 pm on December 19, 2024:

    wen s/Span/std::span/ all over the place?

    now: https://github.com/bitcoin/bitcoin/pull/31519

  51. Sjors commented at 2:07 am on December 19, 2024: member
    Looks like the merge conflict is with #31072 as well as #31223.
  52. net: reduce CAddress usage to CService or CNetAddr
    * `CConnman::CalculateKeyedNetGroup()` needs `CNetAddr`, not `CAddress`,
      thus change its argument.
    
    * Both callers of `CConnman::CreateNodeFromAcceptedSocket()` create a
      dummy `CAddress` from `CService`, so use `CService` instead.
    
    * `GetBindAddress()` only needs to return `CAddress`.
    
    * `CNode::addrBind` does not need to be `CAddress`.
    de88060834
  53. net: split CConnman::BindListenPort() off CConnman
    Introduce a new low-level socket managing class `SockMan`
    and move the `CConnman::BindListenPort()` method to it.
    
    Also, separate the listening socket from the permissions -
    they were coupled in `struct ListenSocket`, but the socket
    is protocol agnostic, whereas the permissions are specific
    to the application of the Bitcoin P2P protocol.
    89ef5287dc
  54. style: modernize the style of SockMan::BindListenPort()
    It was copied verbatim from `CConnman::BindListenPort()` in the previous
    commit. Modernize its variables and style and log the error messages
    from the caller.
    a0a8a72218
  55. net: split CConnman::AcceptConnection() off CConnman
    Move the `CConnman::AcceptConnection()` method to `SockMan` and split
    parts of it:
    * the flip-to-CJDNS part: to just after the `AcceptConnection()` call
    * the permissions part: at the start of `CreateNodeFromAcceptedSocket()`
    919d275a50
  56. style: modernize the style of SockMan::AcceptConnection() 552a0f6382
  57. net: move the generation of ids for new nodes from CConnman to SockMan e0cb576c77
  58. net: move CConnman-specific parts away from ThreadI2PAcceptIncoming()
    CConnman-specific or in other words, Bitcoin P2P specific. Now
    the `ThreadI2PAcceptIncoming()` method is protocol agnostic and
    can be moved to `SockMan`.
    5eb0b66c9a
  59. net: move I2P-accept-incoming code from CConnman to SockMan 8e2ea1c344
  60. net: index nodes in CConnman by id
    Change `CConnman::m_nodes` from `std::vector<CNode*>` to
    `std::unordered_map<NodeId, CNode*>` because interaction
    between `CConnman` and `SockMan` is going to be based on
    `NodeId` and finding a node by its id would better be fast.
    
    As a nice side effect the existent search-by-id operations in
    `CConnman::AttemptToEvictConnection()`,
    `CConnman::DisconnectNode()` and
    `CConnman::ForNode()` now become `O(1)` (were `O(number of nodes)`),
    as well as the erase in `CConnman::DisconnectNodes()`.
    014ea3b378
  61. net: isolate P2P specifics from GenerateWaitSockets()
    Move the parts of `CConnman::GenerateWaitSockets()` that are specific to
    the Bitcoin-P2P protocol to dedicated methods:
    `ShouldTryToSend()` and `ShouldTryToRecv()`.
    
    This brings us one step closer to moving `GenerateWaitSockets()` to the
    protocol agnostic `SockMan` (which would call `ShouldTry...()` from
    `CConnman`).
    9fda56c033
  62. net: isolate P2P specifics from SocketHandlerConnected() and ThreadSocketHandler()
    Move some parts of `CConnman::SocketHandlerConnected()` and
    `CConnman::ThreadSocketHandler()` that are specific to the Bitcoin-P2P
    protocol to dedicated methods:
    `EventIOLoopCompletedForNode()` and `EventIOLoopCompletedForAllPeers()`.
    
    This brings us one step closer to moving `SocketHandlerConnected()` and
    `ThreadSocketHandler()` to the protocol agnostic `SockMan` (which would
    call `EventIOLoopCompleted...()` from `CConnman`).
    361673b595
  63. net: isolate all remaining P2P specifics from SocketHandlerConnected()
    Introduce 4 new methods for the interaction between `CConnman` and
    `SockMan`:
    
    * `EventReadyToSend()`:
      called when there is readiness to send and do the actual sending of data.
    
    * `EventGotData()`, `EventGotEOF()`, `EventGotPermanentReadError()`:
      called when the corresponing recv events occur.
    
    These methods contain logic that is specific to the Bitcoin-P2P protocol
    and move it away from `CConnman::SocketHandlerConnected()` which will
    become a protocol agnostic method of `SockMan`.
    
    Also, move the counting of sent bytes to `CConnman::SocketSendData()` -
    both callers of that method called `RecordBytesSent()` just after the
    call, so move it from the callers to inside
    `CConnman::SocketSendData()`.
    a30c7a6212
  64. net: split CConnman::ConnectNode()
    Move the protocol agnostic parts of `CConnman::ConnectNode()` into
    `SockMan::ConnectAndMakeNodeId()` and leave the Bitcoin-P2P specific
    stuff in `CConnman::ConnectNode()`.
    
    Move the protocol agnostic `CConnman::m_unused_i2p_sessions`, its mutex
    and `MAX_UNUSED_I2P_SESSIONS_SIZE` to `SockMan`.
    
    Move `GetBindAddress()` from `net.cpp` to `sockman.cpp`.
    35db6971ce
  65. net: tweak EventNewConnectionAccepted()
    Move `MaybeFlipIPv6toCJDNS()`, which is Bitcoin P2P specific from the
    callers of `CConnman::EventNewConnectionAccepted()` to inside that
    method.
    
    Move the IsSelectable check, the `TCP_NODELAY` option set and the
    generation of new node id out of `CConnman::EventNewConnectionAccepted()`
    because those are protocol agnostic. Move those to a new method
    `SockMan::NewSockAccepted()` which is called instead of
    `CConnman::EventNewConnectionAccepted()`.
    22c04d49a9
  66. net: move sockets from CNode to SockMan
    Move `CNode::m_sock` and `CNode::m_i2p_sam_session` to `SockMan::m_connected`.
    Also move all the code that handles sockets to `SockMan`.
    
    `CNode::CloseSocketDisconnect()` becomes
    `CConnman::MarkAsDisconnectAndCloseConnection()`.
    
    `CConnman::SocketSendData()` is renamed to
    `CConnman::SendMessagesAsBytes()` and its sockets-touching bits are moved to
    `SockMan::SendBytes()`.
    
    `CConnman::GenerateWaitSockets()` goes to
    `SockMan::GenerateWaitSockets()`.
    
    `CConnman::ThreadSocketHandler()` and
    `CConnman::SocketHandler()` are combined into
    `SockMan::ThreadSocketHandler()`.
    
    `CConnman::SocketHandlerConnected()` goes to
    `SockMan::SocketHandlerConnected()`.
    
    `CConnman::SocketHandlerListening()` goes to
    `SockMan::SocketHandlerListening()`.
    8d08948491
  67. net: move-only: improve encapsulation of SockMan
    `SockMan` members
    
    `AcceptConnection()`
    `NewSockAccepted()`
    `GetNewNodeId()`
    `m_i2p_sam_session`
    `m_listen private`
    
    are now used only by `SockMan`, thus make them private.
    b8b042626e
  68. vasild force-pushed on Dec 19, 2024
  69. vasild commented at 5:48 am on December 19, 2024: contributor
    645f625e29...b8b042626e: rebase due to conflicts and use std::span instead of Span in new code: #30988 (review)
  70. DrahtBot removed the label Needs rebase on Dec 19, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-21 09:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me