rpc, p2p: add addpermissionflags RPC and allow whitelisting outbound #26441

pull brunoerg wants to merge 7 commits into bitcoin:master from brunoerg:2022-10-whitelist-rpc changing 12 files +168 −38
  1. brunoerg commented at 6:26 pm on November 1, 2022: contributor

    Built this PR on top of #17167 (that’s been closed due to inactivity but had some Concept ACK). So, it allows whitelisting outbound peers.

    This PR adds a new RPC addpermissionflags to be able to set up permission flags -whitelist thru RPC, so we don’t need to restart our node if we want to add new flags.

    E.g.

    0$ ./src/bitcoin-cli addpermissionflags ["noban", "mempool", "in", "out"] "127.0.0.1"
    
  2. brunoerg renamed this:
    rpc, p2p: allow whitelisting outgoing connections (17167) + add `addwhitelist` RPC
    rpc, p2p: add `addwhitelist` RPC
    on Nov 1, 2022
  3. brunoerg force-pushed on Nov 1, 2022
  4. brunoerg force-pushed on Nov 1, 2022
  5. DrahtBot commented at 10:39 pm on November 1, 2022: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK stickies-v, hernanmarino
    Approach ACK pablomartin4btc

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #27114 (p2p: Allow whitelisting outgoing connections by brunoerg)
    • #26988 (rpc: Add test-only RPC getaddrmaninfo for new/tried table address count by stratospher)
    • #26837 (I2P network optimizations by vasild)
    • #25515 (net, test: Virtualise CConnman and add initial PeerManager unit tests by dergoegge)
    • #24748 (test/BIP324: functional tests for v2 P2P encryption by stratospher)
    • #24545 (BIP324: Enable v2 P2P encrypted transport by dhruv)
    • #17783 (util: Fix -norpcwhitelist, -norpcallowip, and similar corner case behavior by ryanofsky)
    • #17581 (refactor: Remove settings merge reverse precedence code by ryanofsky)
    • #17580 (refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags by ryanofsky)
    • #17493 (util: Forbid ambiguous multiple assignments in config file by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  6. brunoerg force-pushed on Nov 2, 2022
  7. brunoerg force-pushed on Nov 5, 2022
  8. brunoerg marked this as ready for review on Nov 5, 2022
  9. luke-jr changes_requested
  10. luke-jr commented at 8:03 pm on November 5, 2022: member
    I think it would be better to specify as [String "IP" or Number nodeid, [list of permissions], [list of permissions to remove]] or similar.
  11. in src/rpc/net.cpp:277 in cf50266b14 outdated
    271@@ -272,6 +272,49 @@ static RPCHelpMan getpeerinfo()
    272     };
    273 }
    274 
    275+static RPCHelpMan addwhitelist()
    276+{
    277+    return RPCHelpMan{"addwhitelist",
    


    maflcko commented at 8:57 am on November 7, 2022:
    Wouldn’t it be better to call this RPC addpermissionflags, to better convey that this is setting flags and not a single black/white bool?

    ajtowns commented at 3:25 am on November 8, 2022:
    +1 – we also have an rpc whitelist, so making it clear this changes net permission flags, not the rpc whitelist itself seems good

    brunoerg commented at 2:16 pm on November 8, 2022:
    I agree. It would be better.
  12. in src/net.cpp:604 in cf50266b14 outdated
    601+    if (NetPermissions::HasFlag(flags, NetPermissionFlags::BloomFilter)) {
    602+        service_flags = static_cast<ServiceFlags>(service_flags | NODE_BLOOM);
    603+    }
    604+}
    605+
    606+std::string ConnectionTypeAsString(ConnectionType conn_type)
    


    maflcko commented at 8:59 am on November 7, 2022:
    Is this used for anything?

    brunoerg commented at 1:05 pm on November 10, 2022:
    Seems not, gonna remove it. It seems a copy/paste from CNode
  13. in src/rpc/net.cpp:301 in cf50266b14 outdated
    296+    CConnman& connman = EnsureConnman(node);
    297+
    298+    const auto permissions{request.params[0].get_array()};
    299+
    300+    for (size_t i = 0; i < permissions.size(); i++) {
    301+        NetWhitelistPermissions subnet;
    


    stickies-v commented at 4:13 pm on November 7, 2022:

    nit: m_subnet is a member of NetWhitelistPermissions, isn’t permissions a more accurate name?

    0        NetWhitelistPermissions permissions;
    

    brunoerg commented at 6:48 pm on November 11, 2022:
    Agreed.
  14. in src/rpc/net.cpp:304 in cf50266b14 outdated
    299+
    300+    for (size_t i = 0; i < permissions.size(); i++) {
    301+        NetWhitelistPermissions subnet;
    302+        ConnectionDirection connection_direction;
    303+        bilingual_str error;
    304+        if (!NetWhitelistPermissions::TryParse(permissions[i].get_str(), subnet, connection_direction, error)) throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid permissions");
    


    stickies-v commented at 4:14 pm on November 7, 2022:

    nit: line length

    0        if (!NetWhitelistPermissions::TryParse(permissions[i].get_str(), subnet, connection_direction, error)) {
    1            throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid permissions");
    2        }
    
  15. in src/rpc/net.cpp:300 in cf50266b14 outdated
    295+    NodeContext& node = EnsureAnyNodeContext(request.context);
    296+    CConnman& connman = EnsureConnman(node);
    297+
    298+    const auto permissions{request.params[0].get_array()};
    299+
    300+    for (size_t i = 0; i < permissions.size(); i++) {
    


    stickies-v commented at 4:15 pm on November 7, 2022:
    0    for (const auto& permission : permissions) {
    
  16. in src/rpc/net.cpp:275 in cf50266b14 outdated
    271@@ -272,6 +272,49 @@ static RPCHelpMan getpeerinfo()
    272     };
    273 }
    274 
    275+static RPCHelpMan addwhitelist()
    


    stickies-v commented at 4:22 pm on November 7, 2022:
    A question rather than a suggestion: should the docs highlight that these permissions do not persist beyond a node reboot? Intuitively, I’d have expected this kind of effect to be persistent, so maybe highlighting that is useful if I’m not the only one?

    brunoerg commented at 2:03 pm on November 14, 2022:
    It’s intuitive for me these permissions aren’t persistent since there is no way to remove them (thinking about a follow-up adding an option for removal in this RPC), but sounds goods for me to add it in the docs to clarify.
  17. in src/rpc/net.cpp:280 in cf50266b14 outdated
    271@@ -272,6 +272,49 @@ static RPCHelpMan getpeerinfo()
    272     };
    273 }
    274 
    275+static RPCHelpMan addwhitelist()
    276+{
    277+    return RPCHelpMan{"addwhitelist",
    278+                "\nAdd permission flags to the peers using the given IP address (e.g. 1.2.3.4) or\n"
    279+                "CIDR-notated network (e.g. 1.2.3.0/24). Uses the same permissions as -whitebind.\n"
    280+                "Additional flags in and out control whether permissions apply to incoming connections and/or outgoing (default: incoming only).\n",
    


    stickies-v commented at 4:26 pm on November 7, 2022:

    Perhaps also useful to include this in the example?

    0                "Additional flags \"in\" and \"out\" control whether permissions apply to incoming and/or outgoing connections (default: incoming only).\n",
    
  18. in src/net_permissions.cpp:120 in cf50266b14 outdated
    116@@ -104,11 +117,11 @@ bool NetWhitebindPermissions::TryParse(const std::string& str, NetWhitebindPermi
    117     return true;
    118 }
    119 
    120-bool NetWhitelistPermissions::TryParse(const std::string& str, NetWhitelistPermissions& output, bilingual_str& error)
    121+bool NetWhitelistPermissions::TryParse(const std::string& str, NetWhitelistPermissions& output, ConnectionDirection& output_connection_direction, bilingual_str& error)
    


    stickies-v commented at 4:35 pm on November 7, 2022:
    Not a big fan of adding extra out-parameters, I think util::Result would be more appropriate here (but also a bit of an overhaul so wouldn’t be opposed to doing it in follow-up)
  19. stickies-v commented at 4:42 pm on November 7, 2022: contributor

    Concept ACK-ish, in that whitelisting shouldn’t require a node restart.

    Where I’m not 100% sure (as commented), is that this conceptually feels much more like a persistent setting instead of the ephemeral nature (i.e. effect gone after node restart) that these RPC commands have. Could be a source of confusion?

  20. brunoerg marked this as a draft on Nov 8, 2022
  21. brunoerg commented at 2:19 pm on November 8, 2022: contributor
    I changed it to draft to work on some stuff some reviewers suggested here. Gonna advertise when it ready for review.
  22. brunoerg force-pushed on Nov 11, 2022
  23. net: Move PF_ISIMPLICIT interpretation from AcceptConnection to new InitializePermissionFlags f359fff7ab
  24. net: Move extra service flag into InitializePermissionFlags 229b76e4f1
  25. Accept "in" and "out" flags to -whitelist to allow whitelisting outgoing connections 7dc3ac9cd5
  26. net: make `vWhitelistedRange` and `vWhitelistedRangeOutgoing` public d2977e818b
  27. brunoerg force-pushed on Nov 11, 2022
  28. brunoerg renamed this:
    rpc, p2p: add `addwhitelist` RPC
    rpc, p2p: add `addpermissionflags` RPC
    on Nov 11, 2022
  29. p2p, rpc: add `addpermissionflags` 6e82ca53db
  30. test: add coverage for `addpermissionflags` RPC 7b442d6709
  31. doc: add release notes for `addpermissionflags` 153e86ea84
  32. brunoerg force-pushed on Nov 12, 2022
  33. brunoerg commented at 1:58 pm on November 14, 2022: contributor

    Force-pushed changing the approach and addressing reviews!

    • Instead of using the same format from -whitelist= (e.g. permissions@ip address or network), now it has 2 params - an array of flags and an ip address or network. E.g.
      0./src/bitcoin-cli addpermissionflags ["noban", "mempool", "in", "out"] "127.0.0.1"
      
    • Addressed #26441 (review) by changing this RPC to addpermissionflags
    • Addressed other minor changes
  34. brunoerg marked this as ready for review on Nov 14, 2022
  35. glozow added the label RPC/REST/ZMQ on Jan 27, 2023
  36. in src/rpc/net.cpp:278 in 153e86ea84
    271@@ -272,6 +272,61 @@ static RPCHelpMan getpeerinfo()
    272     };
    273 }
    274 
    275+static RPCHelpMan addpermissionflags()
    276+{
    277+    return RPCHelpMan{"addpermissionflags",
    278+                "\nAdd permission flags to the peers using the given IP address (e.g. 1.2.3.4) or\n"
    


    codo1 commented at 7:58 am on February 10, 2023:
    It took a while before I realized the added permission flags are only applied to new peers (connecting after the rpc command). Not sure if that should have been obvious to me. Maybe change “peers” to “future peers” or something similar?
  37. brunoerg renamed this:
    rpc, p2p: add `addpermissionflags` RPC
    rpc, p2p: add `addpermissionflags` RPC and allow whitelisting outbound
    on Feb 13, 2023
  38. in src/net.h:1001 in d2977e818b outdated
     998@@ -993,12 +999,6 @@ class CConnman
     999     // P2P timeout in seconds
    1000     std::chrono::seconds m_peer_connect_timeout;
    1001 
    1002-    // Whitelisted ranges. Any node connecting from these is automatically
    1003-    // whitelisted (as well as those connecting to whitelisted binds).
    1004-    std::vector<NetWhitelistPermissions> vWhitelistedRange;
    1005-    // Whitelisted ranges for outgoing connections.
    1006-    std::vector<NetWhitelistPermissions> vWhitelistedRangeOutgoing;
    1007-
    


    furszy commented at 1:55 pm on February 13, 2023:
    I don’t think that making the members public is good, it breaks the class encapsulation and allows multiple threads to access/modify the fields without any contention (read/write operations should be guarded).
  39. pablomartin4btc commented at 10:06 pm on February 13, 2023: member

    Approach ACK, run automated tests (unit, functional & fuzz), pending of manual testing.

    Regarding whitelisting outbound connections found some comments from @gmaxwell & @laanwj here.

    Please add “Review club” label (*).

  40. in src/net.cpp:962 in f359fff7ab outdated
    965-        if (gArgs.GetBoolArg("-whitelistforcerelay", DEFAULT_WHITELISTFORCERELAY)) NetPermissions::AddFlag(permission_flags, NetPermissionFlags::ForceRelay);
    966-        if (gArgs.GetBoolArg("-whitelistrelay", DEFAULT_WHITELISTRELAY)) NetPermissions::AddFlag(permission_flags, NetPermissionFlags::Relay);
    967-        NetPermissions::AddFlag(permission_flags, NetPermissionFlags::Mempool);
    968-        NetPermissions::AddFlag(permission_flags, NetPermissionFlags::NoBan);
    969-    }
    970+    AddWhitelistPermissionFlags(permission_flags, addr, vWhitelistedRange);
    


    LarryRuane commented at 5:03 am on February 15, 2023:
    The first commit (“net: Move PF_ISIMPLICIT interpretation from AcceptConnection to …”) doesn’t compile; this call to AddWhitelistPermissionFlags() should not have the vWhitelistedRange until the third commit, “Accept “in” and “out” flags to -whitelist…”.

    brunoerg commented at 8:32 pm on February 16, 2023:
    Done in #27114
  41. in src/net.cpp:564 in 7dc3ac9cd5 outdated
    559@@ -555,7 +560,8 @@ CNode* CConnman::ConnectNode(CAddress addrConnect, const char *pszDest, bool fCo
    560                              pszDest ? pszDest : "",
    561                              conn_type,
    562                              /*inbound_onion=*/false,
    563-                             CNodeOptions{ .i2p_sam_session = std::move(i2p_transient_session) });
    564+                             CNodeOptions{ .i2p_sam_session = std::move(i2p_transient_session),
    565+                                           .permission_flags = permission_flags });
    


    LarryRuane commented at 5:32 am on February 15, 2023:
    I got a compile error with the third commit, “Accept “in” and “out” flags to -whitelist…” because these are out of order, which you fixed later, but each commit should compile.

    brunoerg commented at 8:32 pm on February 16, 2023:
    Done in #27114
  42. LarryRuane changes_requested
  43. LarryRuane commented at 5:43 am on February 15, 2023: contributor
    I’ll review more soon, but here are a couple of instances where changes are not in the correct commits. Each commit should compile (and all tests should pass, but I didn’t check that).
  44. hernanmarino commented at 6:27 pm on February 15, 2023: contributor
    tested ACK. This might help with a couple of functional tests.
  45. brunoerg marked this as a draft on Feb 16, 2023
  46. brunoerg commented at 8:32 pm on February 16, 2023: contributor
    Converted it to draft to address suggestions and re-build it on top of #27114.
  47. in src/net_permissions.cpp:24 in 153e86ea84
    20@@ -21,9 +21,10 @@ const std::vector<std::string> NET_PERMISSIONS_DOC{
    21 namespace {
    22 
    23 // Parse the following format: "perm1,perm2@xxxxxx"
    24-bool TryParsePermissionFlags(const std::string& str, NetPermissionFlags& output, size_t& readen, bilingual_str& error)
    25+bool TryParsePermissionFlags(const std::string& str, NetPermissionFlags& output, ConnectionDirection* output_connection_direction, size_t& readen, bilingual_str& error)
    


    Riahiamirreza commented at 9:59 am on February 18, 2023:
    nit: while the other parameter is output_connection_direction, isn’t it good idea to change output to output_permission_flags or output_permissions?
  48. in src/net_permissions.cpp:101 in 153e86ea84
     97@@ -85,7 +98,7 @@ bool NetWhitebindPermissions::TryParse(const std::string& str, NetWhitebindPermi
     98 {
     99     NetPermissionFlags flags;
    100     size_t offset;
    101-    if (!TryParsePermissionFlags(str, flags, offset, error)) return false;
    102+    if (!TryParsePermissionFlags(str, flags, nullptr, offset, error)) return false;
    


    Riahiamirreza commented at 10:04 am on February 18, 2023:
    nit: Why not using ConnectionDirection::None rather than nullptr to make it more clear?
  49. Riahiamirreza approved
  50. DrahtBot added the label Needs rebase on Feb 22, 2023
  51. DrahtBot commented at 6:46 pm on February 22, 2023: contributor

    🐙 This pull request conflicts with the target branch and needs rebase.

  52. DrahtBot commented at 1:16 am on May 23, 2023: contributor

    There hasn’t been much activity lately and the patch still needs rebase. What is the status here?

    • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
    • Is it no longer relevant? ➡️ Please close.
    • Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
  53. DrahtBot commented at 0:49 am on August 21, 2023: contributor

    There hasn’t been much activity lately and the patch still needs rebase. What is the status here?

    • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
    • Is it no longer relevant? ➡️ Please close.
    • Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
  54. DrahtBot commented at 1:17 am on November 19, 2023: contributor

    There hasn’t been much activity lately and the patch still needs rebase. What is the status here?

    • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
    • Is it no longer relevant? ➡️ Please close.
    • Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
  55. brunoerg commented at 6:09 pm on January 12, 2024: contributor
    Closing it to focus on other approach.
  56. brunoerg closed this on Jan 12, 2024


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-05 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me