Rather than ProcessMessage() being a static function in net_processing.cpp, make it a private member function of PeerLogicValidation. This is the start of moving static functions and global variables into PeerLogicValidation to make it better encapsulated.
Net processing: move ProcessMessage() to PeerLogicValidation #19704
pull jnewbery wants to merge 2 commits into bitcoin:master from jnewbery:2020-07-process-message-plv changing 6 files +115 −131-
jnewbery commented at 11:38 AM on August 12, 2020: member
- fanquake added the label P2P on Aug 12, 2020
-
jnewbery commented at 11:42 AM on August 12, 2020: member
This was requested by @MarcoFalke @sdaftuar and @theuni in #19607 (https://github.com/bitcoin/bitcoin/pull/19607#discussion_r462032894).
Changing
CConnman* connmantoCConnman& m_connmanis the continuation of work in #19174. See #19542 (comment) and #19174 (comment) for further discussion. Prior to this PR, connman was a pointer in PeerLogicValidation, but a reference everywhere else in net_processing. Changing it to a reference in PeerLogicValidation is required before moving the rest of the functions into PeerLogicValidation.EDIT: I've started by moving
ProcessMessage()into PLV because in order to move the newPeerstruct into PLV,Misbehaving()needs to be a member function of PLV, which means everything above it in this call tree https://doxygen.bitcoincore.org/net__processing_8cpp.html#ad2983e754c4d90bd68adf91b208664f5 needs to be a member (or hold a reference topeer_logic, which I don't think makes sense) -
Crypt-iQ commented at 12:23 PM on August 12, 2020: contributor
Fuzzers fail because
ProcessMessageis used in the harnesses. -
theStack commented at 12:57 PM on August 12, 2020: member
Concept ACK -- this is definitely a more appropriate location for
ProcessMessagethan the whole net_processing module scope. - jnewbery force-pushed on Aug 12, 2020
-
jnewbery commented at 1:20 PM on August 12, 2020: member
I've fixed the fuzz build (I think). Let's see what Travis thinks.
-
c556770b5e
[net_processing] Change PeerLogicValidation to hold a connman reference
Hold a reference to connman rather than a pointer because: - PeerLogicValidation can't run without a connman - The pointer never gets reseated The alternative is to always assert that the pointer is non-null before dereferencing. Change the name from connman to m_connman at the same time to conform with current style guidelines.
-
jnewbery commented at 1:30 PM on August 12, 2020: member
rebased
- jnewbery force-pushed on Aug 12, 2020
-
jonatack commented at 8:09 PM on August 13, 2020: member
Light approach ACK, if there is a follow-up code branch to see where this is heading I'd look at it.
-
JeremyRubin commented at 8:39 PM on August 13, 2020: contributor
Concept Ack and lite-cr ack (doesn't seem to change any logic, just refactor).
In terms of approach might be nice to encapsulate processmessage as a subclass that only has access to the relevant references out of the PeerLogicValidation state, but I don't think that's required (just thinking ahead that we might want all the individual message handlers in their own class).
edit: another reason for this is you can appropriately bind things in the interior class as const where relevant.
-
jnewbery commented at 9:44 AM on August 14, 2020: member
Lots of discussion on this at http://www.erisian.com.au/meetbot/bitcoin-core-dev/2020/bitcoin-core-dev.2020-08-13-19.00.log.html#l-104
-
promag commented at 2:03 PM on August 14, 2020: member
Code review ACK cbcb80abc8b3bddaba81e8ba2b22c7d957f02f37.
Am I missing something or this should have refactoring label?
First 2 commits could be merged on its own.
- jnewbery added the label Refactoring on Aug 14, 2020
-
DrahtBot commented at 8:37 PM on August 20, 2020: member
<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--174a7506f384e20aa4161008e828411d-->
Conflicts
Reviewers, this pull request conflicts with the following ones:
- #19763 (net: don't try to relay to the address' originator by vasild)
- #19753 (p2p: don't add AlreadyHave transactions to recentRejects by troygiorshev)
- #19724 ([net] Cleanup connection types- followups by amitiuttarwar)
- #19697 (Improvements on ADDR caching by naumenkogs)
- #19668 (Do not hide compile-time thread safety warnings by hebasto)
- #19610 (p2p: refactor AlreadyHave(), CInv::type, INV/TX processing by jonatack)
- #19498 (Tidy up ProcessOrphanTx by jnewbery)
- #19339 (validation: re-delegate absurd fee checking from mempool to clients by gzhao408)
- #19306 (refactor: Replace RecursiveMutex with Mutex in CTxMemPool by hebasto)
- #19107 (p2p: Move all header verification into the network layer, extend logging by troygiorshev)
- #19031 (Implement ADDRv2 support (part of BIP155) by vasild)
- #17785 (p2p: Unify Send and Receive protocol versions by hebasto)
- #17479 (Return BlockValidationState from ProcessNewBlock if CheckBlock/AcceptBlock fails by jnewbery)
- #15606 ([experimental] UTXO snapshots by jamesob)
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
-
in src/test/denialofservice_tests.cpp:155 in eb170099f3 outdated
149 | @@ -149,8 +150,9 @@ static void AddRandomOutboundPeer(std::vector<CNode *> &vNodes, PeerLogicValidat 150 | 151 | BOOST_AUTO_TEST_CASE(stale_tip_peer_management) 152 | { 153 | + const CChainParams& chainparams = Params(); 154 | auto connman = MakeUnique<CConnmanTest>(0x1337, 0x1337); 155 | - auto peerLogic = MakeUnique<PeerLogicValidation>(*connman, nullptr, *m_node.scheduler, *m_node.chainman, *m_node.mempool); 156 | + auto peerLogic = MakeUnique<PeerLogicValidation>(chainparams, *connman, nullptr, *m_node.scheduler, *m_node.chainman, *m_node.mempool);
MarcoFalke commented at 7:36 AM on August 21, 2020:in commit eb170099f3ea91555b56fdaaae89a64fe04b93be: It seems a bit confusing to pass in chainparams to be stored as a reference in a member variable, but then pass in the consensus params via a parameter of a function call, even though they can be "derived" from the chain params.
I'd suggest the following patch:
diff --git a/src/net_processing.cpp b/src/net_processing.cpp index 71b235f56b..4a5c7acc46 100644 --- a/src/net_processing.cpp +++ b/src/net_processing.cpp @@ -9,6 +9,7 @@ #include <banman.h> #include <blockencodings.h> #include <blockfilter.h> +#include <chainparams.h> #include <consensus/validation.h> #include <hash.h> #include <index/blockfilterindex.h> @@ -1219,13 +1220,12 @@ PeerLogicValidation::PeerLogicValidation(const CChainParams& chainparams, CConnm // same probability that we have in the reject filter). g_recent_confirmed_transactions.reset(new CRollingBloomFilter(48000, 0.000001)); - const Consensus::Params& consensusParams = Params().GetConsensus(); // Stale tip checking and peer eviction are on two different timers, but we // don't want them to get out of sync due to drift in the scheduler, so we // combine them in one function and schedule at the quicker (peer-eviction) // timer. static_assert(EXTRA_PEER_CHECK_INTERVAL < STALE_CHECK_INTERVAL, "peer eviction timer should be less than stale tip check timer"); - scheduler.scheduleEvery([this, consensusParams] { this->CheckForStaleTipAndEvictPeers(consensusParams); }, std::chrono::seconds{EXTRA_PEER_CHECK_INTERVAL}); + scheduler.scheduleEvery([this] { this->CheckForStaleTipAndEvictPeers(); }, std::chrono::seconds{EXTRA_PEER_CHECK_INTERVAL}); // schedule next run for 10-15 minutes in the future const std::chrono::milliseconds delta = std::chrono::minutes{10} + GetRandMillis(std::chrono::minutes{5}); @@ -4011,7 +4011,7 @@ void PeerLogicValidation::EvictExtraOutboundPeers(int64_t time_in_seconds) } } -void PeerLogicValidation::CheckForStaleTipAndEvictPeers(const Consensus::Params &consensusParams) +void PeerLogicValidation::CheckForStaleTipAndEvictPeers() { LOCK(cs_main); @@ -4022,7 +4022,7 @@ void PeerLogicValidation::CheckForStaleTipAndEvictPeers(const Consensus::Params if (time_in_seconds > m_stale_tip_check_time) { // Check whether our tip is stale, and if so, allow using an extra // outbound peer - if (!fImporting && !fReindex && m_connman.GetNetworkActive() && m_connman.GetUseAddrmanOutgoing() && TipMayBeStale(consensusParams)) { + if (!fImporting && !fReindex && m_connman.GetNetworkActive() && m_connman.GetUseAddrmanOutgoing() && TipMayBeStale(m_chainparams.GetConsensus())) { LogPrintf("Potential stale tip detected, will try using extra outbound peer (last tip update: %d seconds ago)\n", time_in_seconds - g_last_tip_update); m_connman.SetTryNewOutboundPeer(true); } else if (m_connman.GetTryNewOutboundPeer()) { diff --git a/src/net_processing.h b/src/net_processing.h index b7a680f8ff..819c2c3d9f 100644 --- a/src/net_processing.h +++ b/src/net_processing.h @@ -6,12 +6,11 @@ #ifndef BITCOIN_NET_PROCESSING_H #define BITCOIN_NET_PROCESSING_H -#include <chainparams.h> -#include <consensus/params.h> #include <net.h> #include <sync.h> #include <validationinterface.h> +class CChainParams; class CTxMemPool; class ChainstateManager; @@ -82,7 +81,7 @@ public: /** Consider evicting an outbound peer based on the amount of time they've been behind our tip */ void ConsiderEviction(CNode& pto, int64_t time_in_seconds) EXCLUSIVE_LOCKS_REQUIRED(cs_main); /** Evict extra outbound peers. If we think our tip may be stale, connect to an extra outbound */ - void CheckForStaleTipAndEvictPeers(const Consensus::Params &consensusParams); + void CheckForStaleTipAndEvictPeers(); /** If we have extra outbound peers, try to disconnect the one with the oldest block announcement */ void EvictExtraOutboundPeers(int64_t time_in_seconds) EXCLUSIVE_LOCKS_REQUIRED(cs_main); /** Retrieve unbroadcast transactions from the mempool and reattempt sending to peers */ diff --git a/src/test/denialofservice_tests.cpp b/src/test/denialofservice_tests.cpp index c94d7c2d00..024345326d 100644 --- a/src/test/denialofservice_tests.cpp +++ b/src/test/denialofservice_tests.cpp @@ -169,7 +169,7 @@ BOOST_AUTO_TEST_CASE(stale_tip_peer_management) AddRandomOutboundPeer(vNodes, *peerLogic, connman.get()); } - peerLogic->CheckForStaleTipAndEvictPeers(consensusParams); + peerLogic->CheckForStaleTipAndEvictPeers(); // No nodes should be marked for disconnection while we have no extra peers for (const CNode *node : vNodes) { @@ -180,7 +180,7 @@ BOOST_AUTO_TEST_CASE(stale_tip_peer_management) // Now tip should definitely be stale, and we should look for an extra // outbound peer - peerLogic->CheckForStaleTipAndEvictPeers(consensusParams); + peerLogic->CheckForStaleTipAndEvictPeers(); BOOST_CHECK(connman->GetTryNewOutboundPeer()); // Still no peers should be marked for disconnection @@ -193,7 +193,7 @@ BOOST_AUTO_TEST_CASE(stale_tip_peer_management) // required time connected check should be satisfied). AddRandomOutboundPeer(vNodes, *peerLogic, connman.get()); - peerLogic->CheckForStaleTipAndEvictPeers(consensusParams); + peerLogic->CheckForStaleTipAndEvictPeers(); for (int i=0; i<max_outbound_full_relay; ++i) { BOOST_CHECK(vNodes[i]->fDisconnect == false); } @@ -206,7 +206,7 @@ BOOST_AUTO_TEST_CASE(stale_tip_peer_management) // peer, and check that the next newest node gets evicted. UpdateLastBlockAnnounceTime(vNodes.back()->GetId(), GetTime()); - peerLogic->CheckForStaleTipAndEvictPeers(consensusParams); + peerLogic->CheckForStaleTipAndEvictPeers(); for (int i=0; i<max_outbound_full_relay-1; ++i) { BOOST_CHECK(vNodes[i]->fDisconnect == false); }
jnewbery commented at 10:06 AM on August 21, 2020:good idea. Done
in src/net_processing.h:9 in eb170099f3 outdated
5 | @@ -6,6 +6,7 @@ 6 | #ifndef BITCOIN_NET_PROCESSING_H 7 | #define BITCOIN_NET_PROCESSING_H 8 | 9 | +#include <chainparams.h>
MarcoFalke commented at 7:38 AM on August 21, 2020:in commit eb170099f3ea91555b56fdaaae89a64fe04b93be: Seems a bit odd to require the whole header when only the name is needed (the compiler doesn't need to know the struct memory layout to keep a pointer or reference as a member variable)
Note that txmemool.h isn't included either.
I suggest the patch from the other comment
jnewbery commented at 10:07 AM on August 21, 2020:yup. That's better. Done.
MarcoFalke approvedMarcoFalke commented at 7:42 AM on August 21, 2020: membercr ACK cbcb80abc8b3bddaba81e8ba2b22c7d957f02f37 🎤
<details><summary>Show signature and timestamp</summary>
Signature:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 cr ACK cbcb80abc8b3bddaba81e8ba2b22c7d957f02f37 🎤 -----BEGIN PGP SIGNATURE----- iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p pUgG4wwAjDXVdxnfaSOSedVWUkK0Xn7dJWYM57677OgNjxyljDUvsMvySG6Gg60O ffe/wB6mwYqXkW2+O+B+Tw2VQbC0AnaXlw9vNnuBDv3S8QDU3f1HE/a+48DWYqPc ipnSEOMZuDsWi31mQrtOOBHxD5hDulhF0Wx6I8SJ+iMhEk20Nd2T1ZQKZR6yJ2oS zOUeRjr/7n3/aeXdCJZ/AiTkVLCZskNSn9fbTSpFsEdefw/kVFzfm4YViCfRvLsn sb7Cs2Us07P64erk5JsJIzIh+C8QjWgWo57UW/sOb9zF05q7fCgpXjzf2iWxMlBe jctdKBoo9Y6SVKY5+Lc3XX+Y36/ae6tKtO5KobjNCDACwLLRnITaJJzltKLs+dTU AriR/ssfIsamz8cbhiMeJiDggdlNugaenPaW9U/EIDx6seOhkR10FkMl4aHJhTUx VBoBzUnGnCf0nW0WVKd9uujdC8qG33ub5Ppq1s21i1NC47I8uv6hS4XZ8tUxCPlD 80mbsjhw =WycG -----END PGP SIGNATURE-----Timestamp of file with hash
c58fbfd8242739033e1f88dae62117af90263272c9149109ad99aba5f1597e88 -</details>
jnewbery force-pushed on Aug 21, 2020jnewbery commented at 10:07 AM on August 21, 2020: memberGreat suggestions @MarcoFalke . I've taken them all.
jonatack commented at 11:08 AM on August 21, 2020: membertravis and cirrus:
MemorySanitizer: use-of-uninitialized-value src/net_processing.cpp:656:82 in (anonymous namespace)::TipMayBeStale(Consensus::Params const&)https://travis-ci.org/github/bitcoin/bitcoin/jobs/719886303
https://github.com/bitcoin/bitcoin/pull/19704/checks?check_run_id=1011864356
in src/net_processing.h:90 in 4d6c4d2058 outdated
87 | void EvictExtraOutboundPeers(int64_t time_in_seconds) EXCLUSIVE_LOCKS_REQUIRED(cs_main); 88 | /** Retrieve unbroadcast transactions from the mempool and reattempt sending to peers */ 89 | void ReattemptInitialBroadcast(CScheduler& scheduler) const; 90 | 91 | + /** Process a single message from a peer. Public for fuzz testing */ 92 | + void ProcessMessage(CNode& pfrom, const std::string& msg_type, CDataStream& vRecv,
jonatack commented at 11:28 AM on August 21, 2020:Does this need to be moved from the fuzz test into the codebase?
jnewbery commented at 12:14 PM on August 21, 2020:I don't understand the question. The method is public so it can be called by the fuzz test.
jonatack commented at 9:31 AM on August 24, 2020:Yes, nvm.
jonatack commented at 11:29 AM on August 21, 2020: memberRe-reviewing, I like these changes. One question below, in addition to the msan use-of-uninitialized-value issue.
[net_processing] Move ProcessMessage to PeerLogicValidation daed542a12jnewbery force-pushed on Aug 21, 2020jnewbery commented at 12:13 PM on August 21, 2020: membertravis and cirrus:
MemorySanitizer: use-of-uninitialized-value src/net_processing.cpp:656:82 in (anonymous namespace)::TipMayBeStale(Consensus::Params const&)
It looks like that happens because
SelectParams()is called twice in theTestChain100Setupconstructor, including after thePeerLogicValidationobject is constructed (see https://github.com/bitcoin/bitcoin/commit/666696b673c5d3b7ab467a890cafd13ac19be503).Rather than fight against the unit test framework, I've removed that commit. It's not essential for this PR.
theStack approvedtheStack commented at 10:10 PM on August 23, 2020: memberCode Review ACK daed542a12e0a6a4692aca12a61b84cd55accc33 It's nice to see that the interface of ProcessMessage() shrinks down from 10 to 6 parameters (or 7, if you also count the implicit this pointer).
MarcoFalke commented at 8:57 AM on August 24, 2020: memberre-ACK daed542a12, only change is removing second commit 🎴
<details><summary>Show signature and timestamp</summary>
Signature:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 re-ACK daed542a12, only change is removing second commit 🎴 -----BEGIN PGP SIGNATURE----- iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p pUi57gv/VAtt+n9kwA+RHgQQQsRU5ybObKW5pKc8cpR0kcgd6rwgJPvCdnYUZyIG eDpCMi4O4oHW7usvpn9koFWtHr8S3NPGqXbmF2PiIDb0aK8qkaUafqLWNgb/vI5T xcTs1XpOF1SW3IB6LnrVtwjk9j9iCeQ67Uu1JdEa9o6e8d84CnMbYBiealLA4f5V UunvG+QZ/vCXzPks4+Q2u9k9FBGj68ED723xJILw743+JLa3Czu4JXT+SPLsC33D 9THMkP38eaWpothKivyE8aNWzG9vSidi3R12apLHCisg54MuWRWNYygD3t5QEvpb Ca8ZBXZy665M7TOkfrfayNtgWi39zNI7TJabFgH7OAa/InPlGKGtdMepgd1a3+EF ZPhItdhgzwy221a80W2NtEyzxXnA3cWY0JBumvWA0P1HY6PIkAAe24rY5Zu25dyk Nj3dBfg9LV/6cSSEJqIdZyWJ0/dPSeMDzejzFWwRGgF/gX4bpyYAhotHXcf1CcBY CYtzu5vy =l6KP -----END PGP SIGNATURE-----Timestamp of file with hash
08eeaa12e1f73cc25bec71f7f0ba83ae3ad3bc6225f47932f8a0d313daf8ffd8 -</details>
jonatack commented at 9:30 AM on August 24, 2020: memberACK daed542a12e0a6a4692aca12a61b84cd55accc33 code review and debug tested
promag commented at 9:36 AM on August 24, 2020: memberCode review ACK daed542a12e0a6a4692aca12a61b84cd55accc33.
fanquake merged this on Aug 24, 2020fanquake closed this on Aug 24, 2020jnewbery deleted the branch on Aug 24, 2020sidhujag referenced this in commit 91657de432 on Aug 24, 2020fanquake referenced this in commit 92735e45ba on Aug 26, 2020fanquake referenced this in commit 6a2ba62685 on Aug 26, 2020sidhujag referenced this in commit 914f824016 on Aug 26, 2020sidhujag referenced this in commit 60a866c284 on Aug 26, 2020MarcoFalke referenced this in commit 147d50d63e on Sep 7, 2020Fabcien referenced this in commit fd797bb033 on Jan 6, 2021Fabcien referenced this in commit a5e788409e on Jan 6, 2021DrahtBot locked this on Feb 15, 2022
This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-05-02 12:14 UTC
More mirrored repositories can be found on mirror.b10c.me