See individual commits for more description. In particular, 75ead75890b9b5ca5565a31d5fe9ad31dc9a96fa.
This (along with #8707) fixes the current maxupload test failure.
5037@@ -5038,7 +5038,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv,
5038
5039 // Be shy and don't send version until we hear
5040 if (pfrom->fInbound)
5041- pfrom->PushVersion();
5042+ connman.PushVersion(pfrom, GetAdjustedTime());
388@@ -389,6 +389,9 @@ CNode* CConnman::ConnectNode(CAddress addrConnect, const char *pszDest, bool fCo
389
390 // Add node
391 CNode* pnode = new CNode(GetNewNodeId(), nLocalServices, GetBestHeight(), hSocket, addrConnect, CalculateKeyedNetGroup(addrConnect), pszDest ? pszDest : "", false);
392+
393+ PushVersion(pnode, GetTime());
On Oct 6, 2016 1:06 PM, “Matt Corallo” notifications@github.com wrote:
@TheBlueMatt commented on this pull request.
Got halfway through a review and realized I just dont know how I feel about this…CNode moving towards a “I speak to the CConnman and serialize messages for the wire” seems like a sane design (and seems to be what you’re going for here), but it means there is just no good place to put serialization-version logic that isnt a complete layer violation.
Yea, I agree. I started creating a CSendMessage that would hold the serialized result, which could go directly to CConnman, but decided for the more straightforward change here. I’m happy to make that change, either as a part of this pr or after.
That has the benefit of testing the serialized result for p2p without actually requiring the send.
In src/main.cpp:
@@ -5038,7 +5038,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, // Be shy and don’t send version until we hear if (pfrom->fInbound) - pfrom->PushVersion(); + connman.PushVersion(pfrom, GetAdjustedTime());
Any idea why we use a different time for inbound vs outbound peers? If anything I’d think we’d announce our real time to inbound peers and our adjusted time to outbound ones, but we do the opposite.
Unsure of the reason there.
In src/net.cpp:
@@ -389,6 +389,9 @@ CNode* CConnman::ConnectNode(CAddress addrConnect, const char pszDest, bool fCo // Add node CNode pnode = new CNode(GetNewNodeId(), nLocalServices, GetBestHeight(), hSocket, addrConnect, CalculateKeyedNetGroup(addrConnect), pszDest ? pszDest : “”, false); + + PushVersion(pnode, GetTime());
Can we go ahead and move the PushVersion logic into InitializeNode? I think thats the right place since it will move into net_processing.cpp with the rest of the “node-message-processing” stuff.
I had planned on doing exactly that as a next step, but it looked kinda out of place by itself. With your changes, it makes perfect sense. Happy to change it to whatever makes it easier for you to move.
Glad to see we’re in sync :)
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
Yea, seems like we’re mostly on the same page here… Probably reasonable to give the serialization to net_processing, though I am concerned with how to merge type with P2P encryption, given that it also changes the message hashing, but only after messages are exchanged (I believe? Maybe @jonasschnelli wants to comment). Even more unsure of how to handle that.
As for this PR, I’m fine with it as it is now, noting that we want to tweak things further as we split code, though I’d prefer we go ahead and move the version message handling to Initialize so we don’t end up moving the same block three times.
On October 6, 2016 3:25:54 PM EDT, Cory Fields notifications@github.com wrote:
On Oct 6, 2016 1:06 PM, “Matt Corallo” notifications@github.com wrote:
@TheBlueMatt commented on this pull request.
Got halfway through a review and realized I just dont know how I feel about this…CNode moving towards a “I speak to the CConnman and serialize messages for the wire” seems like a sane design (and seems to be what you’re going for here), but it means there is just no good place to put serialization-version logic that isnt a complete layer violation.
Yea, I agree. I started creating a CSendMessage that would hold the serialized result, which could go directly to CConnman, but decided for the more straightforward change here. I’m happy to make that change, either as a part of this pr or after.
That has the benefit of testing the serialized result for p2p without actually requiring the send.
In src/main.cpp:
@@ -5038,7 +5038,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, // Be shy and don’t send version until we hear if (pfrom->fInbound) - pfrom->PushVersion(); + connman.PushVersion(pfrom, GetAdjustedTime());
Any idea why we use a different time for inbound vs outbound peers? If anything I’d think we’d announce our real time to inbound peers and our adjusted time to outbound ones, but we do the opposite.
Unsure of the reason there.
In src/net.cpp:
@@ -389,6 +389,9 @@ CNode* CConnman::ConnectNode(CAddress addrConnect, const char pszDest, bool fCo // Add node CNode pnode = new CNode(GetNewNodeId(), nLocalServices, GetBestHeight(), hSocket, addrConnect, CalculateKeyedNetGroup(addrConnect), pszDest ? pszDest : “”, false); + + PushVersion(pnode, GetTime());
Can we go ahead and move the PushVersion logic into InitializeNode? I think thats the right place since it will move into net_processing.cpp with the rest of the “node-message-processing” stuff.
I had planned on doing exactly that as a next step, but it looked kinda out of place by itself. With your changes, it makes perfect sense. Happy to change it to whatever makes it easier for you to move.
Glad to see we’re in sync :)
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #8708 (comment)
2567 {
2568- ENTER_CRITICAL_SECTION(cs_vSend);
2569- assert(ssSend.size() == 0);
2570- ssSend << CMessageHeader(Params().MessageStart(), pszCommand, 0);
2571- LogPrint("net", "sending: %s ", SanitizeString(pszCommand));
2572+ return {SER_NETWORK, nVersion ? nVersion | flags : pnode->GetSendVersion() | flags, CMessageHeader(Params().MessageStart(), sCommand.c_str(), 0) };
2581+ // Set the size
2582+ unsigned int nSize = strm.size() - CMessageHeader::HEADER_SIZE;
2583+ WriteLE32((uint8_t*)&strm[CMessageHeader::MESSAGE_SIZE_OFFSET], nSize);
2584+ // Set the checksum
2585+ uint256 hash = Hash(strm.begin() + CMessageHeader::HEADER_SIZE, strm.end());
2586+ assert(strm.size () >= CMessageHeader::CHECKSUM_OFFSET + CMessageHeader::CHECKSUM_SIZE);
2588
2589- LogPrint("net", "(aborted)\n");
2590 }
2591
2592-void CNode::EndMessage(const char* pszCommand) UNLOCK_FUNCTION(cs_vSend)
2593+void CConnman::PushMessage(CNode* pnode, CDataStream& strm, const std::string& sCommand)
Er, this isn’t intended to be used that way. This is internal and should be private.
The next set of changes will introduce a CSendMessage which is fully serialized. PushMessage will take that instead. That fixes the current layer violation here, and your issue at the same time I think?
413@@ -411,6 +414,24 @@ CNode* CConnman::ConnectNode(CAddress addrConnect, const char *pszDest, bool fCo
414 return NULL;
415 }
416
417+void CConnman::PushVersion(CNode* pnode, int64_t nTime)
369+ CAddress addr = pnode->addr;
370+
371+ CAddress addrYou = (addr.IsRoutable() && !IsProxy(addr) ? addr : CAddress(CService(), addr.nServices));
372+ CAddress addrMe = CAddress(CService(), nLocalNodeServices);
373+
374+ connman.PushMessageWithVersion(pnode, INIT_PROTO_VERSION, NetMsgType::VERSION, PROTOCOL_VERSION, (uint64_t)nLocalNodeServices, GetTime(), addrYou, addrMe,
361- state.name = pnode->addrName;
362- state.address = pnode->addr;
363+void PushNodeVersion(CNode *pnode, CConnman& connman)
364+{
365+ ServiceFlags nLocalNodeServices = pnode->GetLocalServices();
366+ uint64_t nonce = pnode->GetLocalNonce();
I think half the functions here need locks or to become atomics to be correct. (unless I’m missing a required lock to call this function, in which case please AssertLockHeld to make it obvious).
May be easiest to just make half the ints in CNode atomic.
758+ // only one version message is allowed per session. We can therefore
759+ // treat this value as const and even atomic as long as it's only used
760+ // once the handshake is complete. Any attempt to set this twice is an
761+ // error.
762+ assert(nSendVersion == 0);
763+ *const_cast<int*>(&nSendVersion) = nVersionIn;
Updated for @TheBlueMatt’s nits. Since the changes were very small, I went ahead and squashed.
PRs are coming up to address the const issues, as well as fixing the serialization layer violation.
2579-
2580- LEAVE_CRITICAL_SECTION(cs_vSend);
2581+ // Set the size
2582+ unsigned int nSize = strm.size() - CMessageHeader::HEADER_SIZE;
2583+ WriteLE32((uint8_t*)&strm[CMessageHeader::MESSAGE_SIZE_OFFSET], nSize);
2584+ assert(strm.size () >= CMessageHeader::CHECKSUM_OFFSET + CMessageHeader::CHECKSUM_SIZE);
2620- // Set the checksum
2621- uint256 hash = Hash(ssSend.begin() + CMessageHeader::HEADER_SIZE, ssSend.end());
2622- assert(ssSend.size () >= CMessageHeader::CHECKSUM_OFFSET + CMessageHeader::CHECKSUM_SIZE);
2623- memcpy((char*)&ssSend[CMessageHeader::CHECKSUM_OFFSET], hash.begin(), CMessageHeader::CHECKSUM_SIZE);
2624+ unsigned int nSize = strm.size() - CMessageHeader::HEADER_SIZE;
2625+ LogPrint("net", "%s (%d bytes) peer=%d\n", sCommand.c_str(), nSize, pnode->id);
No, they just ended up combined since the actual data push is all done in one place now.
Is there a reason to prefer having them split? How about adding back the “sending” for easy grepping, and the sanitizestring for extra paranoia, but leaving it as a combined message?
1136@@ -1156,10 +1137,6 @@ void CConnman::ThreadSocketHandler()
1137 {
1138 TRY_LOCK(pnode->cs_vSend, lockSend);
1139 if (lockSend) {
1140- if (pnode->nOptimisticBytesWritten) {
It was a hack. In fact, this PR was originally created to fix this.
If the node is in charge of pushing data, Connman has no way of knowing if optimistic writes have succeeded. The hackish solution was to cache the periodically check for optimistic written bytes (stored by CNode) and add them to the node’s total, but that fails miserably if the optimistic writes succeed most of the time.
Now that CConnman is in charge of the writes, it knows immediately how much was written, and doesn’t have to try to catch up with it later.
Oh totally, didn’t mean to imply the sending message should be two prints, just that you changed the text spuriously.
On November 1, 2016 5:08:13 PM EDT, Cory Fields notifications@github.com wrote:
theuni commented on this pull request.
- // Set the checksum
- uint256 hash = Hash(ssSend.begin() + CMessageHeader::HEADER_SIZE, ssSend.end());
- assert(ssSend.size () >= CMessageHeader::CHECKSUM_OFFSET + CMessageHeader::CHECKSUM_SIZE);
- memcpy((char*)&ssSend[CMessageHeader::CHECKSUM_OFFSET], hash.begin(), CMessageHeader::CHECKSUM_SIZE);
- unsigned int nSize = strm.size() - CMessageHeader::HEADER_SIZE;
- LogPrint(“net”, “%s (%d bytes) peer=%d\n”, sCommand.c_str(), nSize, pnode->id);
No, they just ended up combined since the actual data push is all done in one place now.
Is there a reason to prefer having them split? How about adding back the “sending” for easy grepping, and the sanitizestring for extra paranoia, but leaving it as a combined message?
You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #8708
839- void EndMessage(const char* pszCommand) UNLOCK_FUNCTION(cs_vSend);
840-
841- void PushVersion();
842-
843-
844- void PushMessage(const char* pszCommand)
2593+void CConnman::PushMessage(CNode* pnode, CDataStream& strm, const std::string& sCommand)
2594 {
2595- // The -*messagestest options are intentionally not documented in the help message,
2596- // since they are only used during development to debug the networking code and are
2597- // not intended for end-users.
2598- if (mapArgs.count("-dropmessagestest") && GetRand(GetArg("-dropmessagestest", 2)) == 0)
135@@ -136,6 +136,33 @@ class CConnman
136
137 bool ForNode(NodeId id, std::function<bool(CNode* pnode)> func);
138
139+ template <typename... Args>
PushMessageWithVersionAndFlag could be private I suppose. It’s only used by the others.
PushMessageWithFlag is the same as before.
PushMessageWithVersion is used explicitly before the handshake is complete, before we know what upgraded version to use.
Note that as next steps, an intermediary and fully serialized “CSendMessage” (or so) class will be added, which CConnman accepts rather than bare arguments, and forwards to the network. This makes CConnman mostly oblivious as to what it’s pushing (other than the type of message), which further improves encapsulation.
Also, The sendversion logic will move to CNodeState once the locking situation there is improved.
2686@@ -2686,6 +2687,52 @@ void CNode::EndMessage(const char* pszCommand) UNLOCK_FUNCTION(cs_vSend)
2687 LEAVE_CRITICAL_SECTION(cs_vSend);
2688 }
2689
2690+CDataStream CConnman::BeginMessage(CNode* pnode, int nVersion, int flags, const std::string& sCommand)
2691+{
2692+ return {SER_NETWORK, nVersion ? nVersion | flags : pnode->GetSendVersion() | flags, CMessageHeader(Params().MessageStart(), sCommand.c_str(), 0) };
(nVersion ? nVersion : pnode->GetSendVersion()) | flags
?
2717+ LOCK(pnode->cs_vSend);
2718+ if(pnode->hSocket == INVALID_SOCKET) {
2719+ return;
2720+ }
2721+ bool optimisticSend(pnode->vSendMsg.empty());
2722+ pnode->vSendMsg.emplace_back(strm.begin(), strm.end());
354@@ -355,10 +355,10 @@ void UpdatePreferredDownload(CNode* node, CNodeState* state)
355 }
356
357 void InitializeNode(NodeId nodeid, const CNode *pnode) {
358+ CAddress addr = pnode->addr;
Also add a variadic CDataStream ctor for ease-of-use.
The changes here are dense and subtle, but hopefully all is more explicit
than before.
- CConnman is now in charge of sending data rather than the nodes themselves.
This is necessary because many decisions need to be made with all nodes in
mind, and a model that requires the nodes calling up to their manager quickly
turns to spaghetti.
- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
send-version. Since the send version for serialization can only change once
per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
they are sent before the handshake. With this done, there's no need to lock
for access to nSendVersion.
Also, a new stream is used for each message, so there's no need to lock
during the serialization process.
- This takes care of accounting for optimistic sends, so the
nOptimisticBytesWritten hack can be removed.
- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
they haven't been used in years.
Drop all of the old stuff.
This is now handled properly in realtime.
Rebased after #9050. The net.cpp diff before/after the rebase:
0git diff 19d3daef936905743ef108aee1ccfe46fd77b283..c7b8effb96726ba5367a7b1c9c7374c4d2cc10e2 -- net.cpp
can be seen here: https://gist.github.com/theuni/cc4eb87f0196ddadb637d1a41c5c5b2e.
17@@ -18,6 +18,7 @@
18 #include "init.h"
19 #include "merkleblock.h"
20 #include "net.h"
21+#include "netbase.h"
theuni
laanwj
jtimon
TheBlueMatt
sipa
rebroad
MarcoFalke
Labels
P2P