net, refactor: pass uint16 CService::port as uint16 #21328

pull jonatack wants to merge 4 commits into bitcoin:master from jonatack:pass-uint16_t-CService-port-as-uint16_t changing 18 files +82 −56
  1. jonatack commented at 10:24 pm on March 1, 2021: member
    As noticed during review today in #20685 (review) of the upcoming I2P network support, CService::port is uint16_t but is passed around the codebase and into the ctors as int, which causes uneeded conversions and casts. We can avoid these (including in the incoming I2P code without further changes to it) by using ports with the correct type. The remaining conversions are pushed out to the user input boundaries where they can be range-checked and raise with user feedback in the next patch.
  2. DrahtBot added the label P2P on Mar 1, 2021
  3. DrahtBot added the label RPC/REST/ZMQ on Mar 1, 2021
  4. DrahtBot added the label Utils/log/libs on Mar 1, 2021
  5. DrahtBot added the label Validation on Mar 1, 2021
  6. jonatack force-pushed on Mar 2, 2021
  7. jonatack force-pushed on Mar 2, 2021
  8. jonatack commented at 2:04 am on March 2, 2021: member
    Just discovered #8394, will check if it contains anything I’ve overlooked.
  9. in src/test/netbase_tests.cpp:89 in f8b1c2140e outdated
    82@@ -83,31 +83,31 @@ BOOST_AUTO_TEST_CASE(netbase_properties)
    83 
    84 }
    85 
    86-bool static TestSplitHost(std::string test, std::string host, int port)
    87+bool static TestSplitHost(const std::string& test, const std::string& host, uint16_t port)
    88 {
    89     std::string hostOut;
    90-    int portOut = -1;
    91+    uint16_t portOut = 1;
    


    vasild commented at 5:44 am on March 2, 2021:

    Usually 0 is used as “dummy/invalid” port. Port 1 is reserved for (from /etc/services):

    0tcpmux            1/tcp    #TCP Port Service Multiplexer
    1tcpmux            1/udp    #TCP Port Service Multiplexer
    

    jonatack commented at 9:29 pm on March 6, 2021:
    Thanks, done.
  10. in src/util/strencodings.cpp:111 in f8b1c2140e outdated
    106@@ -107,7 +107,8 @@ std::vector<unsigned char> ParseHex(const std::string& str)
    107     return ParseHex(str.c_str());
    108 }
    109 
    110-void SplitHostPort(std::string in, int &portOut, std::string &hostOut) {
    111+void SplitHostPort(std::string in, uint16_t& portOut, std::string& hostOut)
    112+{
    


    vasild commented at 5:52 am on March 2, 2021:

    (github does not let me to comment on line 118, so commenting on line 111 instead):

    0118        int32_t n;
    1119        if (ParseInt32(in.substr(colon + 1), &n) && n > 0 && n < 0x10000) {
    2120            in = in.substr(0, colon);
    3121            portOut = n;
    4122        }
    
    • Line 121 does not cause a warning?
    • Maybe change 0x10000 to std::numeric_limits<uint16_t>::max().
    • If this pattern if (ParseUint32() && n is in [0,64k]) is used in other places, then maybe worth adding ParseUint16(). We have ParseUint{8,32,64}().

    jonatack commented at 9:30 pm on March 6, 2021:
    Good ideas! (no warning seen). Added a ParseUInt16() method and updated SplitHostPort() to use it.

    jonatack commented at 10:05 pm on March 6, 2021:
    We have unit tests for ParseUInt32() and ParseUInt64() but not for ParseUInt8() or ParseUInt16(). Can add in a follow-up or here depending on reviewer preference.
  11. in src/httpserver.cpp:293 in f8b1c2140e outdated
    289@@ -290,7 +290,7 @@ static bool ThreadHTTP(struct event_base* base)
    290 /** Bind HTTP server to specified addresses */
    291 static bool HTTPBindAddresses(struct evhttp* http)
    292 {
    293-    int http_port = gArgs.GetArg("-rpcport", BaseParams().RPCPort());
    294+    uint16_t http_port = gArgs.GetArg("-rpcport", BaseParams().RPCPort());
    


    vasild commented at 5:59 am on March 2, 2021:

    GetArg() returns uint64_t and here we would silently narrow it down to uint16_t. It is ok since before this PR we would narrow it down to int and later narrow the int down to uint16_t.

    If the user makes a typo and specifies -rpcport=88332 we will bind on 22796 instead (with and without this PR). Best to print an error and refuse to start, not necessary in this PR.


    jonatack commented at 9:30 pm on March 6, 2021:
    Decided to use named casts at each of these user input entry points. I agree with better user input parsing and errors….this PR has become a bit large but happy to look at it in a follow-up.
  12. vasild approved
  13. vasild commented at 6:11 am on March 2, 2021: member

    ACK f8b1c2140eea958ab0e6f42ef2daab7000375d82

    I think this PR is good and safe as it is. Some non-blocker comments below. It would be even better if it adds some checks that ports given by the user are in range, like in https://github.com/bitcoin/bitcoin/pull/8394/files#diff-134ccc4e5308f84fb9c03a9a1e1599209560370e49e30c7f545fa7b9ba7e07deR153

  14. DrahtBot commented at 10:13 am on March 2, 2021: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #20457 (util: Make Parse{Int,UInt}{32,64} use locale independent std::from_chars(…) (C++17) instead of locale dependent strto{l,ll,ul,ull} by practicalswift)
    • #20452 (util: Replace use of locale dependent atoi(…) with locale-independent std::from_chars(…) (C++17) by practicalswift)
    • #20196 (net: fix GetListenPort() to derive the proper port by vasild)
    • #18061 (util: Pass size to ParseHex to assist preallocation by elichai)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  15. practicalswift commented at 12:40 pm on March 2, 2021: contributor

    Strong Concept ACK

    Thanks for removing this implicit conversion gotcha :)

    uint16_t should be used consistently for port numbers.

    Somewhat related: #19415 (comment)

  16. laanwj commented at 11:34 am on March 3, 2021: member

    I like the idea of using specific types. But if we’re going to change this, I think I would feel slightly better defining a typedef Port for this. What if we want to support a protocol in the future with 32-bit ports? I do not see this as likely… my intuition is that things are moving to one-endpoint-per-address conceptually instead of ports… and yes the port as a concept is typical to current Internet protocols, where they’re all 16 bits. But I guess it’s clearer too to have a named type.

    Anyhow concept ACK.

  17. DrahtBot added the label Needs rebase on Mar 3, 2021
  18. jonatack force-pushed on Mar 6, 2021
  19. jonatack commented at 9:36 pm on March 6, 2021: member
    Thank you @vasild, @practicalswift and @laanwj. Updated per all of your suggestions.
  20. jonatack force-pushed on Mar 6, 2021
  21. jonatack force-pushed on Mar 6, 2021
  22. DrahtBot removed the label Needs rebase on Mar 6, 2021
  23. practicalswift commented at 6:22 pm on March 7, 2021: contributor

    Not super excited by the introduction of Port TBH.

    The problem is probably best illustrated by this UB quiz:

    Q. Which of the two cases rely on undefined behaviour?

    0// Snippet 1
    1Port p;
    2if (p != other_p) {
    3}
    
    0// Snippet 2
    1uint16_t p;
    2if (p != other_p) {
    3}
    

    Both snippets rely on undefined behaviour (they are technically equivalent thanks to the typedef).

    But that’s not the point.

    The point is that while the UB in snippet 2 is immediately clear (read of uninitialized uint16_t) the UB in snippet 1 requires looking up information that is not available in the code snippet (more specifically the typedef).

    Trying to make it as easy as possible to spot unsafe code is an important part of safe coding practice.

    Personally I find that the introduction of Port makes it somewhat harder to spot unsafe code as illustrated above.

  24. in src/httpserver.cpp:318 in 6be96959ea outdated
    316 
    317     // Bind addresses
    318-    for (std::vector<std::pair<std::string, uint16_t> >::iterator i = endpoints.begin(); i != endpoints.end(); ++i) {
    319-        LogPrint(BCLog::HTTP, "Binding RPC on address %s port %i\n", i->first, i->second);
    320+    for (std::vector<std::pair<std::string, Port> >::iterator i = endpoints.begin(); i != endpoints.end(); ++i) {
    321+        LogPrint(BCLog::HTTP, "Binding RPC on address %s port %u\n", i->first, i->second);
    


    vasild commented at 10:53 am on March 9, 2021:
    These changes %i -> %u are not necessary because LogPrint() is type safe. The commit p2p: update port logging to unsigned integers can be dropped.

    jonatack commented at 10:08 pm on March 10, 2021:
    done
  25. vasild commented at 11:13 am on March 9, 2021: member

    ACK 9b7055f9a9bd7345fc7dfb51ed0bace8257e07cd

    Notice that this is not the last commit - I tend to agree with @practicalswift wrt uint16_t vs Port so I ACK the commit before those changes. Two more things to consider about Port:

    • The name Port may be too generic. I can imagine that “port” in some other context means something else, e.g. “usb port for connecting to a hardware wallet”. Maybe IPPort would be a better name.
    • At one place we do Port n; ParseUInt16(..., &n); which kind of defeats the purpose of the Port abstraction. I don’t see IP ports ever changing to something other than 16 bit unsigned integers.
  26. jonatack commented at 12:59 pm on March 9, 2021: member

    Thanks for the feedback!

    I agree that Port is less greppable than IPPort (and we do already have a ToStringIPPort() method).

    No strong opinion, but here are a few reasons why I tend to agree with @laanwj and took the time to do it:

    • ISTM not having a type alias here is why we’re in this situation in the first place (and since a long time)
    • I’m happy that we have an alias for CAmount, for example; it makes clear to contributors which type should be used for amounts and the same would be true for port id numbers
    • This was a somewhat tedious change to make and I don’t plan to do it again, take advantage of it ;)
    • I think we’ll be happier down the road if this is done than if we don’t do it

    I’m not sure whether to wait for more feedback or separate the type alias to another pull. Keep the feedback coming :pray:

  27. vasild commented at 2:41 pm on March 9, 2021: member
    Yeah, those are also valid points :scroll: :thinking: :thought_balloon:
  28. MarcoFalke commented at 3:07 pm on March 9, 2021: member
    If a new name is introduce, wouldn’t it be better to make it fully type safe?
  29. laanwj commented at 6:06 pm on March 9, 2021: member

    Q. Which of the two cases rely on undefined behaviour?

    Don’t disagree with the point but whyy does C have to turn everything into UB, I hate this, I really do, wish we could use a sane language

  30. practicalswift commented at 10:27 pm on March 9, 2021: contributor

    I agree with MarcoFalke: if we are to introduce Port consider making it fully type safe in order to avoid the pitfall I described above.

    Portability is important, but safety is more important (in general) :)

    I suggest moving the Port commit to a separate PR since that is the only part of this PR that is not ready for merge.

    Q. Which of the two cases rely on undefined behaviour?

    Don’t disagree with the point but whyy does C have to turn everything into UB, I hate this, I really do, wish we could use a sane language

    Heh, just don’t shoot the messenger! :)

  31. jonatack force-pushed on Mar 10, 2021
  32. vasild approved
  33. vasild commented at 7:37 am on March 11, 2021: member

    ACK cf9952da0528b22af77c169822777587bb2a87ee

    If in doubt - KISS.

  34. jonatack commented at 8:58 am on March 11, 2021: member
    Thanks @vasild! Yes, dropped three commits: the logging one, and the two for the type alias that could be a follow-up. No other changes.
  35. DrahtBot added the label Needs rebase on Mar 15, 2021
  36. jonatack force-pushed on Mar 15, 2021
  37. jonatack commented at 3:37 pm on March 15, 2021: member
    Rebased following the merge of #19415git range-diff eceb3f77 cf9952d ae6eb4c
  38. vasild approved
  39. vasild commented at 4:21 pm on March 15, 2021: member
    ACK ae6eb4c8d549526232bcf6197a31de660b74b8af
  40. practicalswift commented at 4:39 pm on March 15, 2021: contributor
    cr ACK ae6eb4c8d549526232bcf6197a31de660b74b8af: patch looks correct
  41. DrahtBot removed the label Needs rebase on Mar 15, 2021
  42. laanwj referenced this in commit cb0aafbd21 on Mar 16, 2021
  43. DrahtBot added the label Needs rebase on Mar 16, 2021
  44. jonatack force-pushed on Mar 16, 2021
  45. jonatack commented at 2:44 pm on March 16, 2021: member
    Rebased following merge of #21444git range-diff 01bb3afb ae6eb4c a05736
  46. vasild approved
  47. vasild commented at 3:13 pm on March 16, 2021: member
    ACK a0573615cb31670743b3915135e3a03418fb9629
  48. DrahtBot removed the label Needs rebase on Mar 16, 2021
  49. jonatack commented at 6:39 pm on March 16, 2021: member
    Ah, the fuzz CI is signalling a needed change since the merge of #19415: “UndefinedBehaviorSanitizer: implicit-signed-integer-truncation test/fuzz/netbase_dns_lookup.cpp:48:45”. Will update.
  50. p2p, refactor: pass and use uint16_t CService::port as uint16_t 6423c8175f
  51. util: add ParseUInt16(), use it in SplitHostPort() 2875a764f7
  52. util: add missing braces and apply clang format to SplitHostPort() 6f09c0f6b5
  53. test: add missing netaddress include headers 52dd40a9fe
  54. jonatack force-pushed on Mar 16, 2021
  55. jonatack commented at 6:57 pm on March 16, 2021: member

    Thanks @vasild! Updated one line per git diff a057361 52dd40a to make a change needed after the merge of #19415.

    0+++ b/src/test/fuzz/netbase_dns_lookup.cpp
    1@@ -18,7 +18,7 @@ FUZZ_TARGET(netbase_dns_lookup)
    2     const std::string name = fuzzed_data_provider.ConsumeRandomLengthString(512);
    3     const unsigned int max_results = fuzzed_data_provider.ConsumeIntegral<unsigned int>();
    4     const bool allow_lookup = fuzzed_data_provider.ConsumeBool();
    5-    const int default_port = fuzzed_data_provider.ConsumeIntegral<int>();
    6+    const uint16_t default_port = fuzzed_data_provider.ConsumeIntegral<uint16_t>();
    
  56. practicalswift commented at 8:57 pm on March 16, 2021: contributor
    cr ACK 52dd40a9febec1f4e70d777821b6764830bdec61: patch looks correct
  57. vasild approved
  58. vasild commented at 11:18 am on March 17, 2021: member
    ACK 52dd40a9febec1f4e70d777821b6764830bdec61
  59. MarcoFalke removed the label RPC/REST/ZMQ on Mar 17, 2021
  60. MarcoFalke removed the label Utils/log/libs on Mar 17, 2021
  61. MarcoFalke removed the label Validation on Mar 17, 2021
  62. MarcoFalke added the label Refactoring on Mar 17, 2021
  63. vasild commented at 5:18 pm on March 19, 2021: member

    Looks like master (at 05757aa86) has a problem that would be fixed by this PR - my “private” CI got this error:

    0netbase.cpp:212:37: runtime error: implicit conversion from type 'int' of value -2147483632 (32-bit, signed) to type 'uint16_t' (aka 'unsigned short') changed the value to 16 (16-bit, unsigned)
    

    which seems unrelated to the changes I am testing there.

    Edit: also in another PR: https://cirrus-ci.com/task/6488952220680192?command=ci#L3334

  64. sipa commented at 5:23 pm on March 19, 2021: member
    @vasild Conversions to unsigned types are actually well-defined in the lamguage. Our sanitizer runs in sort of an over-eager mode and triggers on that because such conversions are often a sign of bugs. But here I suspect it’s intentional, and valid.
  65. vasild commented at 5:28 pm on March 19, 2021: member
    @sipa, I agree. However, a failing CI is a problem :) it would be fixed either by this PR or an explicit cast like CService(vIP[i], static_cast<uint16_t>(port)).
  66. MarcoFalke commented at 7:46 pm on March 19, 2021: member
    cr ACK 52dd40a9febec1f4e70d777821b6764830bdec61
  67. MarcoFalke merged this on Mar 19, 2021
  68. MarcoFalke closed this on Mar 19, 2021

  69. jonatack deleted the branch on Mar 19, 2021
  70. sidhujag referenced this in commit d165b55c0f on Mar 20, 2021
  71. MarcoFalke referenced this in commit d2a78ee928 on Mar 21, 2021
  72. sidhujag referenced this in commit fc65f9795d on Mar 21, 2021
  73. Fabcien referenced this in commit 67221c860e on Feb 21, 2022
  74. Fabcien referenced this in commit c38178957a on Feb 21, 2022
  75. Fabcien referenced this in commit d349817bc5 on Feb 21, 2022
  76. DrahtBot locked this on Aug 16, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-18 18:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me