Logging cleanup #29798

pull vasild wants to merge 4 commits into bitcoin:master from vasild:logging_cleanup changing 4 files +14 −20
  1. vasild commented at 11:53 am on April 3, 2024: contributor
    • Move special cases from LOG_CATEGORIES_BY_STR to GetLogCategory() (suggested here).

    • Remove "none" and "0" from RPC logging help because that help text was wrong. "none" resulted in an error and "0" was ignored itself (contrary to what the help text suggested).

    • Remove unused LOG_CATEGORIES_BY_STR[""] (suggested here).

    This is a followup to #29419, addressing leftover suggestions + more.

  2. DrahtBot commented at 11:53 am on April 3, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK LarryRuane, ryanofsky
    Concept ACK kevkevinpal
    Stale ACK ajtowns

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #29415 (Broadcast own transactions only via short-lived Tor or I2P connections by vasild)
    • #26697 (logging: use bitset for categories by LarryRuane)
    • #17783 (util: Fix -norpcwhitelist, -norpcallowip, and similar corner case behavior by ryanofsky)
    • #17581 (refactor: Remove settings merge reverse precedence code by ryanofsky)
    • #17580 (refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags by ryanofsky)
    • #17493 (util: Forbid ambiguous multiple assignments in config file by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. in src/logging.cpp:346 in d98ca056a8 outdated
    315@@ -317,8 +316,6 @@ namespace BCLog {
    316 
    317 std::string BCLog::Logger::GetLogPrefix(BCLog::LogFlags category, BCLog::Level level) const
    318 {
    319-    if (category == LogFlags::NONE) category = LogFlags::ALL;
    


    maflcko commented at 12:35 pm on April 3, 2024:
    not sure. Just because it currently isn’t called with that value, I fail to see what prevents that in the future.

    vasild commented at 3:39 pm on April 3, 2024:
    Calling this with NONE does not make sense logically and no caller does it currently. Thus I removed this. But you are right that a (nonsensical?) caller could be added in the future. Would you prefer an assert like assert(category != LogFlags::NONE); here or to restore this line as it is (was)?

    ryanofsky commented at 2:09 pm on April 5, 2024:

    re: #29798 (review) In commit “logging: remove unused code” (d98ca056a82264884bc1aa2da3af540a73ee8f26)

    Would you prefer an assert

    I’d suggest just dropping this commit entirely. If you adopt my suggestion from the first commit the {"", BCLog::NONE} map entry already disappears so this does not provide an additional simplification to the map, and I don’t think there’s really another reason to change GetLogPrefix behavior here, at least not as part of this PR.


    vasild commented at 5:30 pm on May 5, 2024:
    Dropped the commit entirely as suggested.
  4. jonatack commented at 12:42 pm on April 3, 2024: member
    See #27231.
  5. vasild commented at 3:44 pm on April 3, 2024: contributor

    See #27231.

    There you mention:

    I’m going to open an alternate pull that does what you suggest…

    Is the current PR doing what you meant? E.g. drop that help text and return an error for “0” (“none” already returns an error in master)?

  6. DrahtBot added the label CI failed on Apr 3, 2024
  7. DrahtBot removed the label CI failed on Apr 5, 2024
  8. in src/logging.cpp:198 in 9fb321ad2b outdated
    199-    if (str.empty()) {
    200+    if (str.empty() || str == "1") {
    201         flag = BCLog::ALL;
    202         return true;
    203     }
    204+    if (str == "0") {
    


    ryanofsky commented at 2:01 pm on April 5, 2024:

    In commit “logging: move special cases to GetLogCategory()” (9fb321ad2b0176955718317b796715250eac01f4)

    Would be good if commit title had a “refactor:” prefix so it is clear this is not intended to change behavior.

    Also, I think this commit should remove ALL and NONE cases from the map, so special cases are handled completely within the GetLogCategory() and LogCategoryToStr() functions, and there are no more special cases in the map and LogCategoriesList(). Would suggest:

     0--- a/src/logging.cpp
     1+++ b/src/logging.cpp
     2@@ -142,7 +142,6 @@ bool BCLog::Logger::DefaultShrinkDebugFile() const
     3 }
     4 
     5 static const std::map<std::string, BCLog::LogFlags> LOG_CATEGORIES_BY_STR{
     6-    {"", BCLog::NONE},
     7     {"net", BCLog::NET},
     8     {"tor", BCLog::TOR},
     9     {"mempool", BCLog::MEMPOOL},
    10@@ -174,7 +173,6 @@ static const std::map<std::string, BCLog::LogFlags> LOG_CATEGORIES_BY_STR{
    11     {"txreconciliation", BCLog::TXRECONCILIATION},
    12     {"scan", BCLog::SCAN},
    13     {"txpackages", BCLog::TXPACKAGES},
    14-    {"all", BCLog::ALL},
    15 };
    16 
    17 static const std::unordered_map<BCLog::LogFlags, std::string> LOG_CATEGORIES_BY_FLAG{
    18@@ -191,11 +189,10 @@ static const std::unordered_map<BCLog::LogFlags, std::string> LOG_CATEGORIES_BY_
    19 
    20 bool GetLogCategory(BCLog::LogFlags& flag, const std::string& str)
    21 {
    22-    if (str.empty() || str == "1") {
    23+    if (str.empty() || str == "1" || str == "all") {
    24         flag = BCLog::ALL;
    25         return true;
    26-    }
    27-    if (str == "0") {
    28+    } else if (str == "0") {
    29         flag = BCLog::NONE;
    30         return true;
    31     }
    32@@ -226,6 +223,11 @@ std::string BCLog::Logger::LogLevelToStr(BCLog::Level level)
    33 
    34 std::string LogCategoryToStr(BCLog::LogFlags category)
    35 {
    36+    if (category == BCLog::NONE) {
    37+        return "";
    38+    } else if (category == BCLog::ALL) {
    39+        return "all";
    40+    }
    41     auto it = LOG_CATEGORIES_BY_FLAG.find(category);
    42     assert(it != LOG_CATEGORIES_BY_FLAG.end());
    43     return it->second;
    44@@ -252,9 +254,7 @@ std::vector<LogCategory> BCLog::Logger::LogCategoriesList() const
    45 {
    46     std::vector<LogCategory> ret;
    47     for (const auto& [category, flag] : LOG_CATEGORIES_BY_STR) {
    48-        if (flag != BCLog::NONE && flag != BCLog::ALL) {
    49-            ret.push_back(LogCategory{.category = category, .active = WillLogCategory(flag)});
    50-        }
    51+        ret.push_back(LogCategory{.category = category, .active = WillLogCategory(flag)});
    52     }
    53     return ret;
    54 }
    

    And would suggest a commit message like:

    0logging, refactor: make category special cases explicit
    1
    2Make special cases explicit in GetLogCategory() and LogCategoryToStr()
    3functions. Simplify the LOG_CATEGORIES_BY_STR and LOG_CATEGORIES_BY_FLAG
    4mappings and LogCategoriesList() function.
    

    vasild commented at 5:31 pm on May 5, 2024:
    Done as suggested.
  9. in src/logging.h:122 in 1fae741495 outdated
    103@@ -104,13 +104,15 @@ namespace BCLog {
    104         std::atomic<Level> m_log_level{DEFAULT_LOG_LEVEL};
    105 
    106         /** Log categories bitfield. */
    107-        std::atomic<uint32_t> m_categories{0};
    108+        std::atomic<uint32_t> m_categories{BCLog::NONE};
    


    ryanofsky commented at 2:03 pm on April 5, 2024:

    In commit “logging: minor encapsulation improvement and use BCLog::NONE instead of 0” (1fae74149507e5914a7bb7986e4906053b8dbbbb)

    Would be good to add “refactor:” prefix to commit message so it is obvious this is not supposed to change behavior.


    vasild commented at 5:31 pm on May 5, 2024:
    Done.
  10. in src/logging.cpp:198 in e78866a109 outdated
    194@@ -195,10 +195,6 @@ bool GetLogCategory(BCLog::LogFlags& flag, const std::string& str)
    195         flag = BCLog::ALL;
    196         return true;
    197     }
    198-    if (str == "0") {
    


    ryanofsky commented at 2:07 pm on April 5, 2024:

    In commit “rpc: remove misleading text from logging help” (e78866a10973c64a59e8a71eed737328b6bc0ca3)

    Right now this returns NONE/true when the empty string "" is passed. Should return false in that case too and never return NONE?

    Also, this is a nice change, but I found description confusing because the whole thing sounds it this like a help text change, but then the last sentence mentions it also adds a new error. Would suggest a description more like:

    0rpc: make logging method reject "0" category
    1
    2Current logging RPC method documentation claims to accept "0" and "none"
    3categories, but the "none" argument is actually rejected and the "0" argument is
    4ignored. Update the implementation to refuse both categories, and remove the
    5help text claiming to support them.
    

    kevkevinpal commented at 11:49 pm on April 9, 2024:
    I noticed in 9fb321ad2b0176955718317b796715250eac01f4 that we introduce this code, I am not sure if it makes sense to remove this 0 value as the first commit so we’re not adding code and removing within two commits

    vasild commented at 5:40 pm on May 5, 2024:

    Reworded the commit message as suggested, but I didn’t get this:

    Right now this returns NONE/true when the empty string "" is passed.

    Hmm, no? It returns ALL/true when "" is passed, in master and in this PR - there is an if (str.empty() ... a few lines above.

    Should return false in that case too

    You mean to return false when empty string is passed? That would be some further change that maybe makes sense, but is not included in this PR.

    and never return NONE?

    It never returns NONE (in this PR).


    vasild commented at 5:45 pm on May 5, 2024:
    Yeah, good observation! I tried to reorder the commits but then if the rpc commit is the first one I would have to remove "0" from LOG_CATEGORIES_BY_STR which is used outside of GetLogCategory() and then it becomes difficult to asses the change. So I left it as it is.

    ryanofsky commented at 5:05 pm on May 6, 2024:

    Right now this returns NONE/true when the empty string "" is passed.

    Hmm, no? It returns ALL/true when "" is passed, in master and in this PR - there is an if (str.empty() ... a few lines above.

    Right, I think I was confused when I wrote that comment. Makes sense to keep "" behavior unchanged, too.

  11. ryanofsky approved
  12. ryanofsky commented at 2:12 pm on April 5, 2024: contributor

    Code review ACK d98ca056a82264884bc1aa2da3af540a73ee8f26

    Nice cleanups! Main feedback I have is to more clearly indicate which commits are refactoring, and which have behavior changes. Would suggest putting “logging, refactor:” in refactoring commit titles and “logging:” in commits that change behavior. Also I would consider dropping any behavior changes that aren’t obvious improvments and doing them in separate PRs so they have more visibility and are not lost in “logging cleanup.”

  13. kevkevinpal commented at 1:30 am on April 10, 2024: contributor

    Concept ACK

    just a question is there any reason to keep BCLog::NONE in logging.h and not completely delete it I did grep --exclude-dir=".deps" -nri "BCLog::NONE" ./src/ --binary-files=without-match and grep --exclude-dir=".deps" -nri "LogFlags::NONE" ./src/ --binary-files=without-match and only see it being used in ./src/logging.{h,cpp} and ./src//qt/transactiondesc.cpp

  14. vasild force-pushed on May 5, 2024
  15. vasild commented at 5:29 pm on May 5, 2024: contributor

    d98ca056a8...bee22409ea: rebase and address suggestions

    Edit: @kevkevinpal, I am not sure about dropping BCLog::NONE on the technical side, but what is sure is that it would further increase the scope of this PR, so I am leaving it as it is :)

  16. vasild force-pushed on May 5, 2024
  17. DrahtBot added the label CI failed on May 5, 2024
  18. DrahtBot commented at 5:53 pm on May 5, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/24605970251

  19. vasild commented at 5:55 pm on May 5, 2024: contributor

    bee22409ea...df2422c5f9: fix CI

    this compiles locally but not on the CI:

    0ret.emplace_back(category, WillLogCategory(flag));
    

    so instead use:

    0ret.push_back(LogCategory{.category = category, .active = WillLogCategory(flag)});
    
  20. DrahtBot removed the label CI failed on May 5, 2024
  21. ryanofsky approved
  22. ryanofsky commented at 5:12 pm on May 6, 2024: contributor
    Code review ACK df2422c5f91e5b1b42d6b718cc2527dc413712fc. Looks good, this gets rid of some unnecessary complexity and strange behavior in logging RPC.
  23. DrahtBot requested review from kevkevinpal on May 6, 2024
  24. DrahtBot added the label CI failed on May 22, 2024
  25. DrahtBot removed the label CI failed on May 22, 2024
  26. ajtowns commented at 5:41 pm on July 3, 2024: contributor

    Consider also updating the help text for -debug and -debugexclude:

     0--- a/src/init/common.cpp
     1+++ b/src/init/common.cpp
     2@@ -27,9 +27,9 @@ void AddLoggingArgs(ArgsManager& argsman)
     3 {
     4     argsman.AddArg("-debuglogfile=<file>", strprintf("Specify location of debug log file (default: %s). Relative paths will be prefixed by a net-specific datadir location. Pass -nodebuglogfile to disable writing the log to a file.", DEFAULT_DEBUGLOGFILE), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     5     argsman.AddArg("-debug=<category>", "Output debug and trace logging (default: -nodebug, supplying <category> is optional). "
     6-        "If <category> is not supplied or if <category> = 1, output all debug and trace logging. <category> can be: " + LogInstance().LogCategoriesString() + ". This option can be specified multiple times to output multiple categories.",
     7+        "If <category> is not supplied or if <category> is 1 or \"all\", output all debug logging. If <category> is 0 or \"none\", any other categories are ignored. Other valid values for <category> are: " + LogInstance().LogCategoriesString() + ". This option can be specified multiple times to output multiple categories.",
     8         ArgsManager::ALLOW_ANY, OptionsCategory::DEBUG_TEST);
     9-    argsman.AddArg("-debugexclude=<category>", "Exclude debug and trace logging for a category. Can be used in conjunction with -debug=1 to output debug and trace logging for all categories except the specified category. This option can be specified multiple times to exclude multiple categories.", ArgsManager::ALLOW_ANY, OptionsCategory::DEBUG_TEST);
    10+    argsman.AddArg("-debugexclude=<category>", "Exclude debug and trace logging for a category. Can be used in conjunction with -debug=1 to output debug and trace logging for all categories except the specified category. This option can be specified multiple times to exclude multiple categories. This takes priority over \"-debug\"", ArgsManager::ALLOW_ANY, OptionsCategory::DEBUG_TEST);
    
  27. ajtowns commented at 5:53 pm on July 3, 2024: contributor
    ACK df2422c5f91e5b1b42d6b718cc2527dc413712fc ; looks like it behaves correctly to me
  28. DrahtBot added the label Needs rebase on Jul 26, 2024
  29. in src/logging.cpp:251 in df2422c5f9 outdated
    217@@ -225,8 +218,14 @@ std::string BCLog::Logger::LogLevelToStr(BCLog::Level level)
    218     assert(false);
    219 }
    220 
    221-std::string LogCategoryToStr(BCLog::LogFlags category)
    222+static std::string LogCategoryToStr(BCLog::LogFlags category)
    223 {
    224+    if (category == BCLog::NONE) {
    225+        return "";
    226+    }
    


    LarryRuane commented at 0:54 am on July 27, 2024:

    nit, in commit “logging, refactor: make category special cases explicit” (eacf21f7cb3d4122fd196a2e49b2d7857a611bcb) I don’t think it’s possible for the argument to be NONE, so perhaps:

    0    assert(category != BCLog::NONE)'
    

    (or maybe add a comment that this should never happen)


    vasild commented at 12:51 pm on August 2, 2024:
    I am fine either way. The only caller of LogCategoryToStr() has checked explicitly that it is not NONE. This was suggested here: #29798 (review), @ryanofsky, what do you think?

    LarryRuane commented at 3:49 pm on August 2, 2024:
    I would still prefer the assertion, but if we’re going to return a string, “none” is probably better than an empty string. But it’s a minor point either way.

    ryanofsky commented at 5:10 pm on August 2, 2024:

    re: #29798 (review)

    Good catch. Since this case can’t happen and the function is already written to assert on other invalid categories, agree it’s better to remove this unused special case. I think it might be overkill to add an assert for this since there is an assert right below that will catch it, but no real opinion on that.


    vasild commented at 4:45 am on August 4, 2024:
    Alright, even better! I just removed that if, no need for two asserts.
  30. in src/logging.cpp:251 in df2422c5f9 outdated
    224+    if (category == BCLog::NONE) {
    225+        return "";
    226+    }
    227+    if (category == BCLog::ALL) {
    228+        return "all";
    229+    }
    


    LarryRuane commented at 0:56 am on July 27, 2024:

    very nit

    0    if (category == BCLog::ALL) return "all";
    

    vasild commented at 1:03 pm on August 2, 2024:
    I prefer to always use { as it makes debugging easier (set a breakpoint on the line with the return statement) and also if a line happens to be added in the body of the if, then the diff is smaller.
  31. in src/logging.cpp:207 in df2422c5f9 outdated
    178 };
    179 
    180 static const std::unordered_map<BCLog::LogFlags, std::string> LOG_CATEGORIES_BY_FLAG{
    181     // Swap keys and values from LOG_CATEGORIES_BY_STR.
    182     [](const std::map<std::string, BCLog::LogFlags>& in) {
    183         std::unordered_map<BCLog::LogFlags, std::string> out;
    


    LarryRuane commented at 1:03 am on July 27, 2024:

    In commit “logging, refactor: make category special cases explicit” (eacf21f7cb3d4122fd196a2e49b2d7857a611bcb) Although you’re not touching this line, perhaps as long as you’re in the vicinity, add:

    0        out.reserve(in.size());
    

    vasild commented at 1:05 pm on August 2, 2024:
    I try to avoid doing unrelated changes to surrounding lines because that makes the diff harder to review.

    LarryRuane commented at 3:36 pm on August 2, 2024:
    Good point, and now that I look at it again, this is insanely unimportant for performance, because this happens once during startup!
  32. LarryRuane commented at 1:13 am on July 27, 2024: contributor
    ACK (but needs rebase, will re-ACK), this will make #26697 simpler (so I think the current PR should merge first). Comments are minor, feel free to ignore.
  33. vasild force-pushed on Aug 2, 2024
  34. vasild commented at 12:43 pm on August 2, 2024: contributor
    df2422c5f9...397211b573: rebase due to conflicts
  35. DrahtBot removed the label Needs rebase on Aug 2, 2024
  36. ryanofsky approved
  37. ryanofsky commented at 5:14 pm on August 2, 2024: contributor
    Code review ACK 397211b573921d2e9d34906ca7720ca2092b8bd7, just since rebase since last review. @vasild AJ’s documentation suggestion also seems like it would be an improvement #29798 (comment) (though I didn’t look into the details and verify it is accurate).
  38. DrahtBot requested review from LarryRuane on Aug 2, 2024
  39. DrahtBot requested review from ajtowns on Aug 2, 2024
  40. LarryRuane approved
  41. LarryRuane commented at 5:33 pm on August 2, 2024: contributor
    ACK 397211b573921d2e9d34906ca7720ca2092b8bd7 I will re-ACK if you make that one small suggested change.
  42. logging, refactor: make category special cases explicit
    Make special cases explicit in GetLogCategory() and LogCategoryToStr()
    functions. Simplify the LOG_CATEGORIES_BY_STR and LOG_CATEGORIES_BY_FLAG
    mappings and LogCategoriesList() function.
    
    This makes the maps `LOG_CATEGORIES_BY_STR` and `LOG_CATEGORIES_BY_FLAG`
    consistent (one is exactly the opposite of the other).
    160706aa38
  43. logging, refactor: minor encapsulation improvement and use BCLog::NONE instead of 0
    * Make the standalone function `LogCategoryToStr()` private inside
      `logging.cpp` (aka `static`) - it is only used in that file.
    
    * Make the method `Logger::GetLogPrefix()` `private` - it is only
      used within the class.
    
    * Use `BCLog::NONE` to initialize `m_categories` instead of `0`.
      We later check whether it is `BCLog::NONE` (in
      `Logger::DefaultShrinkDebugFile()`).
    8c6f3bf163
  44. rpc: make logging method reject "0" category and correct the help text
    Current logging RPC method documentation claims to accept "0" and "none"
    categories, but the "none" argument is actually rejected and the "0"
    argument is ignored. Update the implementation to refuse both
    categories, and remove the help text claiming to support them.
    74dd33cb0a
  45. logging: clarify -debug and -debugexclude descriptions a7432dd6ed
  46. vasild force-pushed on Aug 4, 2024
  47. vasild commented at 4:47 am on August 4, 2024: contributor
    397211b573...a7432dd6ed: pick #29798 (comment) and #29798 (review)
  48. LarryRuane commented at 3:23 pm on August 4, 2024: contributor
    ACK a7432dd6ed3e13a272d62ecde535e6d562cc932c
  49. DrahtBot requested review from ryanofsky on Aug 4, 2024
  50. ryanofsky approved
  51. ryanofsky commented at 0:52 am on August 5, 2024: contributor
    Code review ACK a7432dd6ed3e13a272d62ecde535e6d562cc932c. Only changes since last review are removing dead if statement and adding AJ’s suggested -debug and -debugexclude help improvements, which look accurate and much more clear.
  52. ryanofsky merged this on Aug 5, 2024
  53. ryanofsky closed this on Aug 5, 2024

  54. ryanofsky commented at 1:08 am on August 5, 2024: contributor
    Merged with 2 current acks and a stale ack since changes after the stale ack were pretty minor (rebase, dropping dead code, updating help as suggested).
  55. vasild deleted the branch on Aug 12, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-22 12:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me