p2p: protect onions in AttemptToEvictConnection(), add eviction protection test coverage #20197

pull jonatack wants to merge 8 commits into bitcoin:master from jonatack:AttemptToEvictConnection-identify-onions-with-m_inbound_onion changing 6 files +439 −168
  1. jonatack commented at 1:45 pm on October 20, 2020: member

    Now that #19991 and #20210 have been merged, we can determine inbound onion peers using CNode::m_inbound_onion and add it to the localhost peers protection in AttemptToEvictConnection, which was added in #19670 to address issue #19500.

    Update 28 February 2021: I’ve updated this to follow gmaxwell’s suggestion in #20197 (comment).

    This branch now protects up to 1/4 onion peers (connected via our tor control service), if any, sorted by longest uptime. If any (or all) onion slots remain after that operation, they are then allocated to protect localhost peers, or a minimum of 2 localhost peers in the case that no onion slots remain and 2 or more onion peers were protected, sorted as before by longest uptime.

    This patch also adds test coverage for the longest uptime, localhost, and onion peer eviction protection logic to build on the welcome initial unit testing of #20477.

    Suggest reviewing the commits that move code with colorMoved = dimmed-zebra and colorMovedWs = allow-indentation-change.

    Closes #11537.

  2. fanquake added the label P2P on Oct 20, 2020
  3. sdaftuar commented at 2:07 pm on October 20, 2020: member
    Concept ACK
  4. DrahtBot commented at 2:35 pm on October 20, 2020: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #21160 (net/net processing: Move tx inventory into net_processing by jnewbery)
    • #20196 (net: fix GetListenPort() to derive the proper port by vasild)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  5. practicalswift commented at 3:00 pm on October 20, 2020: contributor
    Concept ACK
  6. hebasto commented at 4:51 pm on October 20, 2020: member
    Concept ACK.
  7. jonatack commented at 10:15 am on October 21, 2020: member
    Added CNode::m_inbound_onion unit test coverage in #20210.
  8. gmaxwell commented at 8:43 pm on October 21, 2020: contributor
    Sounds good, although you might want to add a couple slots for actual localhost peers. :)
  9. sipa commented at 2:03 am on November 5, 2020: member

    Sounds good, although you might want to add a couple slots for actual localhost peers. :)

    Anyone with an old manually-configured hidden service not using the new bind style will have their inbound Tor connections not classified as such, so this may be degradation for them if there is no explicit affordance for localhost peers.

  10. jonatack commented at 7:58 am on November 5, 2020: member
    Thank you for the feedback @gmaxwell and @sipa, will update soon.
  11. DrahtBot added the label Needs rebase on Nov 5, 2020
  12. jonatack force-pushed on Dec 23, 2020
  13. DrahtBot removed the label Needs rebase on Dec 23, 2020
  14. practicalswift commented at 2:09 pm on December 23, 2020: contributor
    Now that we have unit testing of the eviction logic in master (see merged PR #20477): what about adding a small test for this change? :)
  15. jonatack commented at 2:41 pm on December 23, 2020: member

    Rebased, and updated to fall back to localhost if no onion peers are removed. Depending on feedback, this could go further and count onion peers removed to activate localhost if too few onion peers are removed.

    Now that we have unit testing of the eviction logic in master (see merged PR #20477): what about adding a small test for this change? :) @practicalswift I agree and spent time yesterday writing tests after extracting some of the untested parts of SelectNodeToEvict() to a separate function for deterministic testability of the current code. Edit: done.

  16. jonatack force-pushed on Dec 25, 2020
  17. jonatack commented at 11:51 pm on December 25, 2020: member
    Rebased and updated after merge of #19972 that created a silent merge conflict.
  18. in src/test/fuzz/node_eviction.cpp:24 in 0d167683b6 outdated
    31-            fuzzed_data_provider.ConsumeBool(),
    32-            fuzzed_data_provider.ConsumeIntegral<uint64_t>(),
    33-            fuzzed_data_provider.ConsumeBool(),
    34-            fuzzed_data_provider.ConsumeBool(),
    35+            /* id */ fuzzed_data_provider.ConsumeIntegral<NodeId>(),
    36+            /* nTimeConnected */ fuzzed_data_provider.ConsumeIntegral<int64_t>(),
    


    vasild commented at 1:16 pm on January 1, 2021:

    there is a nice syntax for this:

    0{.id = 123, .nTimeConnected = 567, ...}
    

    but last time I tried some old version of GCC did not support it. Maybe some day…


    jnewbery commented at 10:34 am on January 14, 2021:


    jnewbery commented at 12:47 pm on January 27, 2021:
    Interesting. Are the fuzzers compiled with –std=c++2a?

    vasild commented at 1:04 pm on January 27, 2021:
    No. My guess is that fuzz tests are not compiled with the old compiler that chokes on this syntax and that most compilers support it even in -std=c++17 mode.
  19. in src/net.h:1285 in 0d167683b6 outdated
    1246@@ -1247,6 +1247,7 @@ struct NodeEvictionCandidate
    1247     uint64_t nKeyedNetGroup;
    1248     bool prefer_evict;
    1249     bool m_is_local;
    1250+    bool m_is_onion;
    


    vasild commented at 1:25 pm on January 1, 2021:
    The commit message reads “p2p: add m_inbound_onion to AttemptToEvictConnection()”. Shouldn’t that be s/AttemptToEvictConnection()/NodeEvictionCandidate/?

    jonatack commented at 3:57 pm on February 11, 2021:
    Sure, I think the salient change in that commit is adding m_inbound_onion to AttemptToEvictConnection but happy to append “and add m_is_onion to NodeEvictionCandidate” to the commit message body.

    jonatack commented at 4:11 pm on February 11, 2021:
    Done.
  20. in src/net.cpp:944 in c16ed372bd outdated
    945+    std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareOnionTimeConnected);
    946+    const size_t local_erase_size = total_protect_size / 2;
    947+    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_onion; }), vEvictionCandidates.end());
    948+
    949+    // If no onion peers were removed, extend the same protection to localhost peers, as manually
    950+    // configured hidden services not using -bind will not be detected as inbound onion connections.
    


    vasild commented at 1:39 pm on January 1, 2021:
    0    // configured hidden services not using -bind=addr:port=onion will not be detected as inbound onion connections.
    

    jonatack commented at 4:14 pm on February 11, 2021:
    Thanks, replaced -bind with -bind=<addr>[:<port>]=onion like the -bind help.
  21. in src/net.cpp:941 in c16ed372bd outdated
    942-    size_t local_erase_size = total_protect_size / 2;
    943-    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_local; }), vEvictionCandidates.end());
    944+    // Pick out up to 1/4 peers that are connected via our tor onion service, sorted by longest uptime.
    945+    std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareOnionTimeConnected);
    946+    const size_t local_erase_size = total_protect_size / 2;
    947+    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_onion; }), vEvictionCandidates.end());
    


    vasild commented at 2:41 pm on January 1, 2021:

    What about extending EraseLastKElements() with one more argument “predicate” and remove elements only if it returns true on them (with a default that assumes true for any element)?

    Then this code would look like:

    0EraseLastKElements(
    1    vEvictionCandidates,
    2    CompareOnionTimeConnected,
    3    local_erase_size,
    4    [](NodeEvictionCandidate const &n) { return n.m_is_onion; });
    

    maybe out of the scope of this PR.


    vasild commented at 9:16 am on January 15, 2021:

    Something like this may work (untested, feel free to ignore):

      0diff --git i/src/net.cpp w/src/net.cpp
      1index ddf33c38a..64900b8b5 100644
      2--- i/src/net.cpp
      3+++ w/src/net.cpp
      4@@ -41,12 +41,13 @@
      5 // with Ubuntu 16.04 LTS and Debian 8 libminiupnpc-dev packages.
      6 static_assert(MINIUPNPC_API_VERSION >= 10, "miniUPnPc API version >= 10 assumed");
      7 #endif
      8 
      9 #include <algorithm>
     10 #include <cstdint>
     11+#include <functional> /* std::function */
     12 #include <unordered_map>
     13 
     14 #include <math.h>
     15 
     16 /** Maximum number of block-relay-only anchor connections */
     17 static constexpr size_t MAX_BLOCK_RELAY_ONLY_ANCHORS = 2;
     18@@ -893,38 +894,46 @@ static bool CompareNodeBlockRelayOnlyTime(const NodeEvictionCandidate &a, const
     19     if (a.fRelayTxes != b.fRelayTxes) return a.fRelayTxes;
     20     if (a.nLastBlockTime != b.nLastBlockTime) return a.nLastBlockTime < b.nLastBlockTime;
     21     if (a.fRelevantServices != b.fRelevantServices) return b.fRelevantServices;
     22     return a.nTimeConnected > b.nTimeConnected;
     23 }
     24 
     25-//! Sort an array by the specified comparator, then erase the last K elements.
     26-template<typename T, typename Comparator>
     27-static void EraseLastKElements(std::vector<T> &elements, Comparator comparator, size_t k)
     28+//! Sort an array by the specified comparator, then erase each one of the last K elements if
     29+//! `pred(element)` returns true.
     30+template <typename T, typename Comparator>
     31+static void EraseLastKElements(
     32+    std::vector<T>& elements,
     33+    Comparator comparator,
     34+    size_t k,
     35+    std::function<bool(const T&)> pred = [](const T&) { return true; })
     36 {
     37     std::sort(elements.begin(), elements.end(), comparator);
     38     size_t eraseSize = std::min(k, elements.size());
     39-    elements.erase(elements.end() - eraseSize, elements.end());
     40+    elements.erase(std::remove_if(elements.end() - eraseSize, elements.end(), pred),
     41+                   elements.end());
     42 }
     43 
     44 [[nodiscard]] Optional<NodeId> SelectNodeToEvict(std::vector<NodeEvictionCandidate>&& vEvictionCandidates)
     45 {
     46     // Protect connections with certain characteristics
     47 
     48+    std::function<bool(const NodeEvictionCandidate&)> pred;
     49+
     50     // Deterministically select 4 peers to protect by netgroup.
     51     // An attacker cannot predict which netgroups will be protected
     52     EraseLastKElements(vEvictionCandidates, CompareNetGroupKeyed, 4);
     53     // Protect the 8 nodes with the lowest minimum ping time.
     54     // An attacker cannot manipulate this metric without physically moving nodes closer to the target.
     55     EraseLastKElements(vEvictionCandidates, ReverseCompareNodeMinPingTime, 8);
     56     // Protect 4 nodes that most recently sent us novel transactions accepted into our mempool.
     57     // An attacker cannot manipulate this metric without performing useful work.
     58     EraseLastKElements(vEvictionCandidates, CompareNodeTXTime, 4);
     59     // Protect up to 8 non-tx-relay peers that have sent us novel blocks.
     60-    std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareNodeBlockRelayOnlyTime);
     61-    size_t erase_size = std::min(size_t(8), vEvictionCandidates.size());
     62-    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return !n.fRelayTxes && n.fRelevantServices; }), vEvictionCandidates.end());
     63+    const size_t erase_size = std::min(size_t(8), vEvictionCandidates.size());
     64+    pred = [](const NodeEvictionCandidate& n) { return !n.fRelayTxes && n.fRelevantServices; };
     65+    EraseLastKElements(vEvictionCandidates, CompareNodeBlockRelayOnlyTime, erase_size, pred);
     66 
     67     // Protect 4 nodes that most recently sent us novel blocks.
     68     // An attacker cannot manipulate this metric without performing useful work.
     69     EraseLastKElements(vEvictionCandidates, CompareNodeBlockTime, 4);
     70 
     71     // Protect the half of the remaining nodes which have been connected the longest.
     72@@ -933,22 +942,22 @@ static void EraseLastKElements(std::vector<T> &elements, Comparator comparator,
     73     // they're not longest-uptime overall. This helps protect tor peers, which
     74     // tend to be otherwise disadvantaged under our eviction criteria.
     75     const size_t initial_size = vEvictionCandidates.size();
     76     size_t total_protect_size = initial_size / 2;
     77 
     78     // Pick out up to 1/4 peers that are connected via our tor onion service, sorted by longest uptime.
     79-    std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareOnionTimeConnected);
     80     const size_t local_erase_size = total_protect_size / 2;
     81-    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_onion; }), vEvictionCandidates.end());
     82+    pred = [](const NodeEvictionCandidate& n) { return n.m_is_onion; };
     83+    EraseLastKElements(vEvictionCandidates, CompareOnionTimeConnected, local_erase_size, pred);
     84 
     85     // If no onion peers were removed, extend the same protection to localhost peers, as manually
     86     // configured hidden services not using -bind will not be detected as inbound onion connections.
     87     if (initial_size == vEvictionCandidates.size()) {
     88         // Pick out up to 1/4 peers that are localhost, sorted by longest uptime.
     89-        std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareLocalHostTimeConnected);
     90-        vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_local; }), vEvictionCandidates.end());
     91+        pred = [](const NodeEvictionCandidate& n) { return n.m_is_local; };
     92+        EraseLastKElements(vEvictionCandidates, CompareLocalHostTimeConnected, local_erase_size, pred);
     93     }
     94 
     95     // Calculate how many we removed, and update our total number of peers that
     96     // we want to protect based on uptime accordingly.
     97     total_protect_size -= initial_size - vEvictionCandidates.size();
     98     EraseLastKElements(vEvictionCandidates, ReverseCompareNodeTimeConnected, total_protect_size);
     99@@ -956,14 +965,15 @@ static void EraseLastKElements(std::vector<T> &elements, Comparator comparator,
    100     if (vEvictionCandidates.empty()) return nullopt;
    101 
    102     // If any remaining peers are preferred for eviction consider only them.
    103     // This happens after the other preferences since if a peer is really the best by other criteria (esp relaying blocks)
    104     //  then we probably don't want to evict it no matter what.
    105     if (std::any_of(vEvictionCandidates.begin(),vEvictionCandidates.end(),[](NodeEvictionCandidate const &n){return n.prefer_evict;})) {
    106-        vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.begin(),vEvictionCandidates.end(),
    107-                                  [](NodeEvictionCandidate const &n){return !n.prefer_evict;}),vEvictionCandidates.end());
    108+        pred = [](const NodeEvictionCandidate& n) { return !n.prefer_evict; };
    109+        EraseLastKElements(vEvictionCandidates, ReverseCompareNodeTimeConnected,
    110+                           vEvictionCandidates.size(), pred);
    111     }
    112 
    113     // Identify the network group with the most connections and youngest member.
    114     // (vEvictionCandidates is already sorted by reverse connect time)
    115     uint64_t naMostConnections;
    116     unsigned int nMostConnections = 0;
    

    jonatack commented at 9:33 am on January 15, 2021:

    Thanks! I started with the quick patch below on January 2, will compare with your ideas above.

     0-//! Sort an array by the specified comparator, then erase the last K elements.
     1+//! Sort an array by the specified comparator, then erase the last K elements where predicate is true.
     2 template<typename T, typename Comparator>
     3-static void EraseLastKElements(std::vector<T> &elements, Comparator comparator, size_t k)
     4+static void EraseLastKElements(std::vector<T> &elements, Comparator comparator, size_t k, std::function<bool(const NodeEvictionCandidate&)> predicate = [](NodeEvictionCandidate const &n) { return true ;})
     5 {
     6     std::sort(elements.begin(), elements.end(), comparator);
     7-    size_t eraseSize = std::min(k, elements.size());
     8-    elements.erase(elements.end() - eraseSize, elements.end());
     9+    size_t erase_size = std::min(k, elements.size());
    10+    elements.erase(std::remove_if(elements.end() - erase_size, elements.end(), predicate), elements.end());
    11 }
    12 
    13 [[nodiscard]] Optional<NodeId> SelectNodeToEvict(std::vector<NodeEvictionCandidate>&& vEvictionCandidates)
    14@@ -934,16 +934,15 @@ static void EraseLastKElements(std::vector<T> &elements, Comparator comparator,
    15     size_t total_protect_size = initial_size / 2;
    16 
    17     // Pick out up to 1/4 peers that are connected via our tor onion service, sorted by longest uptime.
    18-    std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareOnionTimeConnected);
    19     const size_t local_erase_size = total_protect_size / 2;
    20-    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_onion; }), vEvictionCandidates.end());
    21+    EraseLastKElements(vEvictionCandidates, CompareOnionTimeConnected, local_erase_size, [](NodeEvictionCandidate const &n) { return n.m_is_onion; });
    22+;
    23 
    24     // If no onion peers were removed, extend the same protection to localhost peers, as manually
    25-    // configured hidden services not using -bind will not be detected as inbound onion connections.
    26+    // configured hidden services not using -bind=addr:port=onion will not be detected as inbound onion connections.
    27     if (initial_size == vEvictionCandidates.size()) {
    28         // Pick out up to 1/4 peers that are localhost, sorted by longest uptime.
    29-        std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareLocalHostTimeConnected);
    30-        vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_local; }), vEvictionCandidates.end());
    31+        EraseLastKElements(vEvictionCandidates, CompareLocalHostTimeConnected, local_erase_size, [](NodeEvictionCandidate const &n) { return n.m_is_local; });
    32     }
    
  22. in src/net.cpp:945 in c16ed372bd outdated
    946+    const size_t local_erase_size = total_protect_size / 2;
    947+    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - local_erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return n.m_is_onion; }), vEvictionCandidates.end());
    948+
    949+    // If no onion peers were removed, extend the same protection to localhost peers, as manually
    950+    // configured hidden services not using -bind will not be detected as inbound onion connections.
    951+    if (initial_size == vEvictionCandidates.size()) {
    


    vasild commented at 2:54 pm on January 1, 2021:

    Just to note - this assumes that if there are recognizable onion connections (via -bind=addr:port=onion) then there are not non-recognizable ones (statically/manually configured tor daemon pointing to our standard p2p port, e.g. 8333).

    This assumption may not be true as there could be a node that is configured both via -bind=addr:port=onion (connections coming to 8334 are recognized as onion) and also a static tor config redirecting to 8333.

    I think the code is ok as it is because the above would be rare and maybe not worth the code complexity to tweak the eviction policy for it.

  23. vasild approved
  24. vasild commented at 2:59 pm on January 1, 2021: member

    ACK c16ed372bdc71795fbebc5dbf519a1d748a136dc

    The code looks ok, after some staring at it. Just some minor notes below.

  25. MarcoFalke referenced this in commit ae8f797135 on Jan 2, 2021
  26. sidhujag referenced this in commit 1a9cf218d6 on Jan 2, 2021
  27. MarcoFalke referenced this in commit 1f514332ed on Jan 31, 2021
  28. sidhujag referenced this in commit ab208da4d8 on Feb 2, 2021
  29. sdaftuar commented at 8:59 pm on February 17, 2021: member

    Sounds good, although you might want to add a couple slots for actual localhost peers. :)

    I interpreted this comment from @gmaxwell as suggesting that we should protect some actual localhost peers, even if we are also protecting some onion peers – looks like this patch right now would only try to protect some localhost peers if there are no onion peers that we’re protecting. Any reason not to protect some of each? I think that having more categories/metrics that we use to distinguish inbounds will make it harder for an adversary to take over all our inbound slots.

  30. jonatack force-pushed on Feb 21, 2021
  31. jonatack renamed this:
    p2p: improve onion detection in AttemptToEvictConnection()
    p2p: protect onions in AttemptToEvictConnection(), add eviction protection test coverage
    on Feb 21, 2021
  32. jonatack commented at 0:44 am on February 22, 2021: member
    Thank you @vasild and @sdaftuar. Updated to potentially protect up to 1/4 onion peers, independently of the existing 1/4 localhost protection, for an unchanged 1/2 of the remaining peers overall. The latest push also adds test coverage for the longest uptime, localhost, and onion peer eviction protection logic.
  33. jonatack force-pushed on Feb 22, 2021
  34. jonatack commented at 2:38 pm on February 22, 2021: member
    Improved the last test commit, adding combined onion + localhost tests.
  35. jonatack force-pushed on Feb 22, 2021
  36. jonatack force-pushed on Feb 22, 2021
  37. jonatack force-pushed on Feb 22, 2021
  38. in src/test/net_peer_eviction_tests.cpp:77 in f1c02b3ef9 outdated
    71+        }
    72+        if (std::find(unprotected_peer_ids.begin(), unprotected_peer_ids.end(), candidate.id) != unprotected_peer_ids.end()) {
    73+            // this peer remains in the eviction candidates, as expected
    74+            ++unprotected_as_expected;
    75+        }
    76+    }
    


    vasild commented at 12:49 pm on February 25, 2021:

    This loop is O(num_peers * (num_protected + num_unprotected)). It can easily be reduced to O(num_peers) by replacing the std::vector arguments with std::unordered_set (which can tell if an element is present in O(1) time). This will also simplify the lookups:

    0-if (std::find(protected_peer_ids.begin(), protected_peer_ids.end(), candidate.id) != protected_peer_ids.end()) {
    1+if (protected_peer_ids.count(candidate.id) > 0) {
    

    jonatack commented at 4:06 pm on February 28, 2021:
    Nice, thank you! And no need for the > 0. Done for both IsProtected and IsEvicted.
  39. in src/test/net_peer_eviction_tests.cpp:112 in f1c02b3ef9 outdated
    107+
    108+    // Expect 1/4 onion peers to be protected from eviction,
    109+    // independently of other characteristics.
    110+    BOOST_CHECK(IsProtected(
    111+        num_peers, [](NodeEvictionCandidate& c) {
    112+            c.m_is_onion = (c.id == 3 || c.id == 8 || c.id == 9) ? true : false;
    


    vasild commented at 1:01 pm on February 25, 2021:

    nit: here and below, expr ? true : false is equivalent to expr

    0            c.m_is_onion = (c.id == 3 || c.id == 8 || c.id == 9);
    

    jonatack commented at 4:06 pm on February 28, 2021:
    Done, thank you.
  40. in src/test/net_peer_eviction_tests.cpp:63 in f1c02b3ef9 outdated
    58+    Shuffle(candidates.begin(), candidates.end(), random_context);
    59+
    60+    const size_t expected{candidates.size() / 2}; // expect half the candidates will be protected
    61+    ProtectEvictionCandidates(candidates);
    62+    const size_t actual{candidates.size()};
    63+    BOOST_CHECK(actual == expected || actual == expected + 1);
    


    vasild commented at 1:12 pm on February 25, 2021:

    ProtectEvictionCandidates() will always remove size() / 2 elements (no matter whether the size is odd or even), so the new size will always be old_size - old_size / 2.

    0    const size_t expected{candidates.size() - candidates.size() / 2}; // expect half the candidates will be protected
    1    ProtectEvictionCandidates(candidates);
    2    const size_t actual{candidates.size()};
    3    BOOST_CHECK_EQUAL(actual, expected);
    

    jonatack commented at 4:06 pm on February 28, 2021:
    That’s better indeed. Done.
  41. in src/net.cpp:944 in f1c02b3ef9 outdated
    961-    if (std::any_of(vEvictionCandidates.begin(),vEvictionCandidates.end(),[](NodeEvictionCandidate const &n){return n.prefer_evict;})) {
    962-        vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.begin(),vEvictionCandidates.end(),
    963-                                  [](NodeEvictionCandidate const &n){return !n.prefer_evict;}),vEvictionCandidates.end());
    964+    if (std::any_of(vEvictionCandidates.begin(), vEvictionCandidates.end(), [](const NodeEvictionCandidate& n) { return n.prefer_evict; })) {
    965+        EraseLastKElements(vEvictionCandidates, ReverseCompareNodeTimeConnected, vEvictionCandidates.size(),
    966+                           [](const NodeEvictionCandidate& n) { return !n.prefer_evict; });
    


    vasild commented at 1:29 pm on February 25, 2021:
    Note to other reviewers and maybe worth adding a comment: the sorting function is irrelevant here because we process the entire array (last size() elements).

    jonatack commented at 4:06 pm on February 28, 2021:
    Good idea. Decided for now to not touch this line instead.
  42. vasild commented at 2:01 pm on February 25, 2021: member

    Approach ACK f1c02b3ef9fa4bbd82144eebbeaf8c9c9df238c5

    About how many to protect - I do not feel strongly about it, but I think we should protect 1/4 (localhost or tor), not 1/4 localhost and 1/4 tor. Because localhost likely means a tor peer that somehow is not recognized as such. So, a tor peer is (surely recognized as tor OR unrecognized one, coming from localhost (likely)) and the original code aims to protect 1/4 tor peers and my understanding is that this PR’s purpose is not to possibly extend that to 1/2 (1/4 + 1/4).

    Just to be explicit what I mean:

     0diff --git i/src/net.cpp w/src/net.cpp
     1index e0e0ba9d8..96a1259e7 100644
     2--- i/src/net.cpp
     3+++ w/src/net.cpp
     4@@ -827,23 +827,24 @@ static bool ReverseCompareNodeMinPingTime(const NodeEvictionCandidate &a, const
     5 
     6 static bool ReverseCompareNodeTimeConnected(const NodeEvictionCandidate &a, const NodeEvictionCandidate &b)
     7 {
     8     return a.nTimeConnected > b.nTimeConnected;
     9 }
    10 
    11-static bool CompareLocalHostTimeConnected(const NodeEvictionCandidate &a, const NodeEvictionCandidate &b)
    12-{
    13-    if (a.m_is_local != b.m_is_local) return b.m_is_local;
    14-    return a.nTimeConnected > b.nTimeConnected;
    15-}
    16-
    17-static bool CompareOnionTimeConnected(const NodeEvictionCandidate& a, const NodeEvictionCandidate& b)
    18-{
    19-    if (a.m_is_onion != b.m_is_onion) return b.m_is_onion;
    20-    return a.nTimeConnected > b.nTimeConnected;
    21-}
    22+/**
    23+ * Compare eviction candidates in such a way, that (localhost or onion) ones are at
    24+ * the end. Within them put the ones that have been connected longer (have smaller
    25+ * `nTimeConnected`) at the end.
    26+ */
    27+static bool CompareThisIsHard(const NodeEvictionCandidate& a, const NodeEvictionCandidate& b)
    28+{
    29+    const bool a_is_special = a.m_is_local || a.m_is_onion;
    30+    const bool b_is_special = b.m_is_local || b.m_is_onion;
    31+    if (a_is_special != b_is_special) return b_is_special;
    32+    return a.nTimeConnected > b.nTimeConnected;
    33+}
    34 
    35 static bool CompareNetGroupKeyed(const NodeEvictionCandidate &a, const NodeEvictionCandidate &b) {
    36     return a.nKeyedNetGroup < b.nKeyedNetGroup;
    37 }
    38 
    39 static bool CompareNodeBlockTime(const NodeEvictionCandidate &a, const NodeEvictionCandidate &b)
    40@@ -891,21 +892,18 @@ void ProtectEvictionCandidates(std::vector<NodeEvictionCandidate>& vEvictionCand
    41     // they're not longest-uptime overall. This helps protect tor peers, which
    42     // tend to be otherwise disadvantaged under our eviction criteria.
    43     const size_t initial_size = vEvictionCandidates.size();
    44     size_t total_protect_size = initial_size / 2;
    45     const size_t local_erase_size = total_protect_size / 2;
    46 
    47-    // Pick out up to 1/4 peers connected via our onion service, sorted by longest uptime.
    48-    EraseLastKElements(vEvictionCandidates, CompareOnionTimeConnected, local_erase_size,
    49-                       [](const NodeEvictionCandidate& n) { return n.m_is_onion; });
    50-
    51-    // Pick out up to 1/4 peers that are localhost, sorted by longest uptime,
    52-    // as manually configured hidden services not using `-bind=addr[:port]=onion`
    53-    // will not be detected as inbound onion connections.
    54-    EraseLastKElements(vEvictionCandidates, CompareLocalHostTimeConnected, local_erase_size,
    55-                       [](const NodeEvictionCandidate& n) { return n.m_is_local; });
    56+    // Pick out up to 1/4 localhost peers (likely they are tor ones that do not benefit from
    57+    // `-bind=addr[:port]=onion`) or peers connected via our onion service, sorted by longest uptime.
    58+    EraseLastKElements(vEvictionCandidates,
    59+                       CompareThisIsHard,
    60+                       local_erase_size,
    61+                       [](const NodeEvictionCandidate& n) { return n.m_is_local || n.m_is_onion; });
    62 
    63     // Calculate how many we removed, and update our total number of peers that
    64     // we want to protect based on uptime accordingly.
    65     total_protect_size -= initial_size - vEvictionCandidates.size();
    66     EraseLastKElements(vEvictionCandidates, ReverseCompareNodeTimeConnected, total_protect_size);
    67 }
    

    Again, I do not have a strong opinion on this. Curious to see what other reviewers think.

  43. jonatack force-pushed on Feb 28, 2021
  44. jonatack commented at 4:45 pm on February 28, 2021: member

    Thank you @vasild for your excellent review! I’ve taken your inline code feedback.

    WRT the question of “how many to protect and how to do it,” I think it may be best to continue picking out the onion peers and the localhost peers separately in order to strive for more peer diversity. The idea is that if we do somehow have both onion and localhost peers, it seems worthwhile to ensure we protect some of each, even if one has worse latency than the other.

    Thus, at the cost of a wee bit more complexity, I’ve updated this to follow @gmaxwell’s suggestion more closely.

    This branch now protects up to 1/4 onion peers (connected via our tor control service), if any, sorted by longest uptime. If any (or all) onion slots remain after that operation, they are then allocated to protect localhost peers, or a minimum of 2 localhost peers in the case that no onion slots remain and 2 or more onion peers were protected, sorted as before by longest uptime.

  45. jonatack force-pushed on Feb 28, 2021
  46. jonatack commented at 6:02 pm on February 28, 2021: member
    Improved the unit tests.
  47. DrahtBot added the label Needs rebase on Mar 4, 2021
  48. jonatack force-pushed on Mar 4, 2021
  49. jonatack commented at 4:01 pm on March 4, 2021: member
    Rebased. @naumenkogs, @ariard, @jnewbery, @dongcarl, as reviewers of #19670 you might be interested in this. @practicalswift, this updates your unit tests and adds missing ones as you had requested. @wtogami, this might address the issues you discussed with me recently.
  50. in src/net.h:1285 in ea9f19ee66 outdated
    1287@@ -1287,8 +1288,11 @@ struct NodeEvictionCandidate
    1288     uint64_t nKeyedNetGroup;
    1289     bool prefer_evict;
    1290     bool m_is_local;
    1291+    bool m_is_onion;
    


    ariard commented at 4:14 pm on March 4, 2021:
    Maybe called it m_is_onion_service to dissociate from the manually configured hidden services you mentioned above?

    jonatack commented at 2:29 pm on March 17, 2021:
    Thanks @ariard. The latter ones would be m_is_local. I think I prefer to go with m_is_onion for CNode::m_inbound_onion and following the naming style of m_is_local.
  51. DrahtBot removed the label Needs rebase on Mar 4, 2021
  52. ariard commented at 4:20 pm on March 4, 2021: member

    Concept ACK.

    What type of localhost peers coming from automated-connections-but-manually-configured do we have right now ? I2P? Otherwise if you want to protect localhost peers from manual connections maybe better just to redirect them on a whitebind=noban@addr ?

  53. laanwj added this to the "Blockers" column in a project

  54. vasild commented at 4:39 pm on March 9, 2021: member
    #11537 is related to this PR.
  55. vasild approved
  56. vasild commented at 5:31 pm on March 9, 2021: member
    ACK ea9f19ee66ff66e1f5c384c19def994ec3037e13
  57. vasild commented at 5:36 pm on March 9, 2021: member

    @ariard, I2P peers are not localhost ones. They can easily be distinguished because their address’s CNetAddr::IsI2P() is true. The complication with Tor is because incoming tor connections come from an IP address (not from a Tor address), usually 127.0.0.1.

    I am not sure what you mean by “automated-connections-but-manually-configured” or “protect localhost peers from manual connections”.

  58. DrahtBot added the label Needs rebase on Mar 17, 2021
  59. jonatack force-pushed on Mar 17, 2021
  60. jonatack commented at 2:24 pm on March 17, 2021: member
    Rebased due to conflicts from the merge of #21415 git range-diff a9d1b40 ea9f19e c18fd59
  61. DrahtBot removed the label Needs rebase on Mar 17, 2021
  62. vasild approved
  63. vasild commented at 5:08 pm on March 17, 2021: member
    ACK c18fd59329a855c12ba370474a3ec39032611f2b
  64. in src/net.cpp:882 in fd3cbfda2b outdated
    878@@ -879,6 +879,26 @@ static void EraseLastKElements(std::vector<T> &elements, Comparator comparator,
    879     elements.erase(elements.end() - eraseSize, elements.end());
    880 }
    881 
    882+void ProtectEvictionCandidates(std::vector<NodeEvictionCandidate>& vEvictionCandidates)
    


    sipa commented at 2:29 am on March 19, 2021:

    In commit “Extract ProtectEvictionCandidates() from SelectNodeToEvict()”

    Can you add a comment explaining exactly which protection rules are implemented by this function, or give it a bit more specific name?

    It seems it’s just a subset of those currently in SelectNodeToEvict. If new eviction protections are added, how to decide if they should go into this function or into SelectNodeToEvict?


    jonatack commented at 7:26 pm on March 19, 2021:
    Good idea. Done, renamed the function to ProtectEvictionCandidatesByRatio() (open to suggestions) and added Doxygen documentation.

    jonatack commented at 7:33 pm on March 19, 2021:
    Also updated the commit names/descriptions.
  65. sipa commented at 3:25 am on March 19, 2021: member
    utACK c18fd59329a855c12ba370474a3ec39032611f2b. I didn’t review the last (unit test) commit in detail.
  66. Extract ProtectEvictionCandidatesByRatio from SelectNodeToEvict
    to allow deterministic unit testing of the ratio-based peer eviction protection
    logic, which protects peers having longer connection times and those connected
    via higher-latency networks.
    
    Add documentation.
    f126cbd6de
  67. Move peer eviction tests to a separate test file
    out of net_tests, because the eviction tests:
    
    - are a different domain of test coverage, with different dependencies
    
    - run more slowly than the net tests
    
    - will be growing in size, in this PR branch and in the future, as eviction
      test coverage is improved
    41f84d5ecc
  68. Use std::unordered_set instead of std::vector in IsEvicted()
    An unordered set can tell if an element is present in ~O(1) time (constant on
    average, worst case linear to the size of the container), which speeds up and
    simplifies the lookup in IsEvicted().
    
    Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
    ca63b53ecd
  69. Add unit tests for ProtectEvictionCandidatesByRatio()
    Thank you to Vasil Dimov (vasild) for the suggestion to use std::unordered_set
    rather than std::vector for the IsProtected() peer id arguments.
    72e30e8e03
  70. Add m_inbound_onion to AttemptToEvictConnection()
    and an `m_is_onion` struct member to NodeEvictionCandidate and tests.
    
    We'll use these in the peer eviction logic to protect inbound onion peers
    in addition to the existing protection of localhost peers.
    8b1e156143
  71. Use EraseLastKElements() throughout SelectNodeToEvict()
    Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
    8f1a53eb02
  72. Protect onion+localhost peers in ProtectEvictionCandidatesByRatio()
    Now that we have a reliable way to detect inbound onion peers, this commit
    updates our existing eviction protection of 1/4 localhost peers to instead
    protect up to 1/4 onion peers (connected via our tor control service), sorted by
    longest uptime. Any remaining slots of the 1/4 are then allocated to protect
    localhost peers, or 2 localhost peers if no slots remain and 2 or more onion
    peers are protected, sorted by longest uptime.
    
    The goal is to avoid penalizing onion peers, due to their higher min ping times
    relative to IPv4 and IPv6 peers, and improve our diversity of peer connections.
    
    Thank you to Gregory Maxwell, Suhas Daftuar, Vasil Dimov and Pieter Wuille
    for valuable review feedback that shaped the direction.
    caa21f586f
  73. Add unit test coverage for our onion peer eviction protection 0cca08a8ee
  74. jonatack force-pushed on Mar 19, 2021
  75. jonatack commented at 7:30 pm on March 19, 2021: member

    Added documentation and renamed a function per #20197 (review).

     0diff --git a/src/net.cpp b/src/net.cpp
     1index be93426377..66d81b115f 100644
     2--- a/src/net.cpp
     3+++ b/src/net.cpp
     4@@ -887,7 +887,7 @@ static void EraseLastKElements(
     5     elements.erase(std::remove_if(elements.end() - eraseSize, elements.end(), predicate), elements.end());
     6 }
     7 
     8-void ProtectEvictionCandidates(std::vector<NodeEvictionCandidate>& vEvictionCandidates)
     9+void ProtectEvictionCandidatesByRatio(std::vector<NodeEvictionCandidate>& vEvictionCandidates)
    10 {
    11     // Protect the half of the remaining nodes which have been connected the longest.
    12     // This replicates the non-eviction implicit behavior, and precludes attacks that start later.
    13@@ -945,7 +945,9 @@ void ProtectEvictionCandidates(std::vector<NodeEvictionCandidate>& vEvictionCand
    14     // An attacker cannot manipulate this metric without performing useful work.
    15     EraseLastKElements(vEvictionCandidates, CompareNodeBlockTime, 4);
    16 
    17-    ProtectEvictionCandidates(vEvictionCandidates);
    18+    // Protect some of the remaining eviction candidates by ratios of desirable
    19+    // or disadvantaged characteristics.
    20+    ProtectEvictionCandidatesByRatio(vEvictionCandidates);
    21 
    22     if (vEvictionCandidates.empty()) return std::nullopt;
    23 
    24diff --git a/src/net.h b/src/net.h
    25index 449e0c54b7..eb7fa079ab 100644
    26--- a/src/net.h
    27+++ b/src/net.h
    28@@ -1285,8 +1285,36 @@ struct NodeEvictionCandidate
    29     bool m_is_onion;
    30 };
    31 
    32-void ProtectEvictionCandidates(std::vector<NodeEvictionCandidate>& vEvictionCandidates);
    33-
    34+/**
    35+ * Select an inbound peer to evict after filtering out (protecting) peers having
    36+ * distinct, difficult-to-forge characteristics. The protection logic picks out
    37+ * fixed numbers of desirable peers per various criteria, followed by (mostly)
    38+ * ratios of desirable or disadvantaged peers. If any eviction candidates
    39+ * remain, the selection logic chooses a peer to evict.
    40+ */
    41 [[nodiscard]] std::optional<NodeId> SelectNodeToEvict(std::vector<NodeEvictionCandidate>&& vEvictionCandidates);
    42 
    43+/** Protect desirable or disadvantaged inbound peers from eviction by ratio.
    44+ *
    45+ * This function protects half of the peers which have been connected the
    46+ * longest, to replicate the non-eviction implicit behavior and preclude attacks
    47+ * that start later.
    48+ *
    49+ * Half of these protected spots (1/4 of the total) are reserved for onion peers
    50+ * connected via our tor control service, if any, sorted by longest uptime, even
    51+ * if they're not longest uptime overall. Any remaining slots of the 1/4 are
    52+ * then allocated to protect localhost peers, if any (or up to 2 localhost peers
    53+ * if no slots remain and 2 or more onion peers were protected), sorted by
    54+ * longest uptime, as manually configured hidden services not using
    55+ * `-bind=addr[:port]=onion` will not be detected as inbound onion connections.
    56+ *
    57+ * This helps protect onion peers, which tend to be otherwise disadvantaged
    58+ * under our eviction criteria for their higher min ping times relative to IPv4
    59+ * and IPv6 peers, and favorise the diversity of peer connections.
    60+ *
    61+ * This function was extracted from SelectNodeToEvict() to be able to test the
    62+ * ratio-based protection logic deterministically.
    63+ */
    64+void ProtectEvictionCandidatesByRatio(std::vector<NodeEvictionCandidate>& vEvictionCandidates);
    65+
    66 #endif // BITCOIN_NET_H
    67diff --git a/src/test/net_peer_eviction_tests.cpp b/src/test/net_peer_eviction_tests.cpp
    68index 7b88a3e9b6..31d391bf7d 100644
    69--- a/src/test/net_peer_eviction_tests.cpp
    70+++ b/src/test/net_peer_eviction_tests.cpp
    71@@ -43,9 +43,9 @@ std::vector<NodeEvictionCandidate> GetRandomNodeEvictionCandidates(const int n_c
    72 }
    73 
    74 // Create `num_peers` random nodes, apply setup function `candidate_setup_fn`,
    75-// apply protection logic, and then return true if all of `protected_peer_ids`
    76-// and none of `unprotected_peer_ids` are protected from eviction, i.e. removed
    77-// from the eviction candidates.
    78+// call ProtectEvictionCandidatesByRatio() to apply protection logic, and then
    79+// return true if all of `protected_peer_ids` and none of `unprotected_peer_ids`
    80+// are protected from eviction, i.e. removed from the eviction candidates.
    81 bool IsProtected(int num_peers,
    82                  std::function<void(NodeEvictionCandidate&)> candidate_setup_fn,
    83                  const std::unordered_set<NodeId>& protected_peer_ids,
    84@@ -60,7 +60,7 @@ bool IsProtected(int num_peers,
    85 
    86     const size_t size{candidates.size()};
    87     const size_t expected{size - size / 2}; // Expect half the candidates will be protected.
    88-    ProtectEvictionCandidates(candidates);
    89+    ProtectEvictionCandidatesByRatio(candidates);
    90     BOOST_CHECK_EQUAL(candidates.size(), expected);
    
  76. vasild approved
  77. vasild commented at 8:15 am on March 22, 2021: member
    ACK 0cca08a8ee33b4e05ff586ae4fd914f5ea860cea
  78. ghost commented at 8:46 pm on March 27, 2021: none
    Interesting PR. Concept ACK. Compiled successfully on Ubuntu. Tests passed.
  79. laanwj commented at 2:20 pm on March 30, 2021: member
    Code review ACK 0cca08a8ee33b4e05ff586ae4fd914f5ea860cea I did not do anything in terms of manual testing, but it looks like the change is well-covered by tests.
  80. laanwj merged this on Mar 30, 2021
  81. laanwj closed this on Mar 30, 2021

  82. jonatack deleted the branch on Mar 30, 2021
  83. in src/net.cpp:940 in 0cca08a8ee
    936@@ -893,30 +937,17 @@ static void EraseLastKElements(std::vector<T> &elements, Comparator comparator,
    937     // An attacker cannot manipulate this metric without performing useful work.
    938     EraseLastKElements(vEvictionCandidates, CompareNodeTXTime, 4);
    939     // Protect up to 8 non-tx-relay peers that have sent us novel blocks.
    940-    std::sort(vEvictionCandidates.begin(), vEvictionCandidates.end(), CompareNodeBlockRelayOnlyTime);
    941-    size_t erase_size = std::min(size_t(8), vEvictionCandidates.size());
    942-    vEvictionCandidates.erase(std::remove_if(vEvictionCandidates.end() - erase_size, vEvictionCandidates.end(), [](NodeEvictionCandidate const &n) { return !n.fRelayTxes && n.fRelevantServices; }), vEvictionCandidates.end());
    943+    const size_t erase_size = std::min(size_t(8), vEvictionCandidates.size());
    


    mzumsande commented at 4:03 pm on March 30, 2021:
    Since EraseLastKElements determines the minimum again, this line seems unnecessary (could just pass size_t(8)).

    jonatack commented at 6:27 pm on March 30, 2021:
    Good eye! Will update in #21261.

    jonatack commented at 2:37 pm on April 20, 2021:
    Thanks again @mzumsande. Addressed in #21261.
  84. mzumsande commented at 4:10 pm on March 30, 2021: member

    Post-Merge Comment (reviewed but didn’t finish in time yesterday evening):

    I understand the motivation, but just noting that the fixed localhost_min_protect_size in the ratio-based ProtectEvictionCandidatesByRatio makes things unintuitive at a lower initial_size:

    Both cases are nicely covered by the unit tests, but it doesn’t make a lot of sense to me that at 6 total peers, we’d protect 1  onion, 0 localhost and 2 by generic uptime, whereas at 8 peers, we’d protect 2 onion, 2 localhost and 0 by generic uptime (raising the percentage of potentially protected tor peers to 50% instead of the targeted 25%, so that no slots are left for protecting generic uptime peers).

  85. jonatack commented at 4:33 pm on March 30, 2021: member
    Thanks for having a look! Here we have an initial improvement and unit testing in place, and in the next step in #21261 we’ll re-evaluate the algorithm and ratios between onion, localhost and I2P peers (considering that other privacy networks may be added soon as well). Please do weigh in there with the ratios you would suggest.
  86. sidhujag referenced this in commit 6161b6736f on Mar 30, 2021
  87. fanquake removed this from the "Blockers" column in a project

  88. jonatack commented at 2:40 pm on April 20, 2021: member
    @mzumsande I’ve updated the inbound eviction protection proposal in #21261 to be generalizable to multiple networks (with I2P and CJDNS on the way) and fully ratio-based, which should address your feedback. Will be bringing it out of draft shortly.
  89. laanwj referenced this in commit 5c4f0c4d46 on Jun 14, 2021
  90. sidhujag referenced this in commit b9e1e110ed on Jun 14, 2021
  91. Fabcien referenced this in commit 8314163358 on Feb 4, 2022
  92. Fabcien referenced this in commit e17e1b2061 on Feb 4, 2022
  93. Fabcien referenced this in commit 7128106e3b on Feb 4, 2022
  94. Fabcien referenced this in commit e1af69cbce on Feb 4, 2022
  95. Fabcien referenced this in commit 443b5d94ae on Feb 4, 2022
  96. Fabcien referenced this in commit 0cc0457d68 on Feb 4, 2022
  97. laanwj referenced this in commit 8b6cd42c62 on Mar 2, 2022
  98. PastaPastaPasta referenced this in commit a83e04a6b0 on Jul 17, 2022
  99. DrahtBot locked this on Aug 16, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me