logging: add LogPrintfCategory to log unconditionally with category #25306

pull jonatack wants to merge 2 commits into bitcoin:master from jonatack:2022-06-logprintfcategory changing 8 files +21 −11
  1. jonatack commented at 11:46 am on June 8, 2022: contributor

    These are the next two commits from #25203.

    • Add LogPrintfCategory to log unconditionally while prefixing the output with the passed category name. Add documentation and a unit test, and update the lint-logs.py and lint-format-strings.py scripts.

    • Replace the log messages that manually print a category, with LogPrintfCategory. In upcoming commits, it will likely be used in many other cases, such as to replace LogPrintf where it makes sense.

  2. jonatack force-pushed on Jun 8, 2022
  3. in src/test/logging_tests.cpp:119 in 90787ffa92 outdated
    114@@ -114,7 +115,8 @@ BOOST_FIXTURE_TEST_CASE(logging_LogPrintMacros, LogSetup)
    115         "[net:debug] foo7: bar7",
    116         "[net:info] foo8: bar8",
    117         "[net:warning] foo9: bar9",
    118-        "[net:error] foo10: bar10"};
    119+        "[net:error] foo10: bar10",
    120+        "[validation] foo11: bar11"};
    


    MarcoFalke commented at 11:55 am on June 8, 2022:
    0        "[validation] foo11: bar11",
    1        };
    

    nit, to make future changes less verbose


    jonatack commented at 12:05 pm on June 8, 2022:
    thanks, done (we may need to update our clang-format)
  4. MarcoFalke approved
  5. MarcoFalke commented at 11:55 am on June 8, 2022: member
    lgtm
  6. logging: add LogPrintfCategory to log unconditionally with category
    prefixing the output with the passed category name.
    
    - add documentation
    - add a unit test
    - update lint-logs.py
    - update lint-format-strings.py
    eb8aab759f
  7. logging: use LogPrintfCategory rather than a manual category
    Here we update only the log messages that manually print a category.
    
    In upcoming commits, LogPrintCategory will likely be used in many
    other cases, such as to replace `LogPrintf` where it makes sense.
    ecff20db28
  8. jonatack force-pushed on Jun 8, 2022
  9. DrahtBot added the label P2P on Jun 8, 2022
  10. DrahtBot added the label RPC/REST/ZMQ on Jun 8, 2022
  11. MarcoFalke removed the label RPC/REST/ZMQ on Jun 8, 2022
  12. MarcoFalke removed the label P2P on Jun 8, 2022
  13. MarcoFalke added the label Utils/log/libs on Jun 8, 2022
  14. DrahtBot commented at 3:07 pm on June 8, 2022: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #25308 (refactor: Reduce number of LoadChainstate parameters and return values by ryanofsky)
    • #25203 (logging: update to severity-based logging by jonatack)
    • #21603 (log: Mitigate disk filling attacks by rate limiting LogPrintf by dergoegge)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  15. klementtan commented at 4:55 pm on June 8, 2022: contributor
    Code Review ACK ecff20db286e2f5d3afe32cfaae72de69d34d23c
  16. laanwj commented at 4:57 pm on June 8, 2022: member

    Concept ACK

    I do think this is an intermediate state. I imagine at some point we’ll want to have a category and log level for every message, so the need to log messages with just a category goes away again.

  17. brunoerg approved
  18. brunoerg commented at 5:14 pm on June 8, 2022: contributor

    ACK ecff20db286e2f5d3afe32cfaae72de69d34d23c

    Just reviewed the code and it’s good. I built and ran it (signet) and checked debug.log, lgtm.

    Ex:

    02022-06-08T17:03:23Z Script verification uses 7 additional threads
    12022-06-08T17:03:23Z scheduler thread start
    22022-06-08T17:03:23Z [http] creating work queue of depth 16
    32022-06-08T17:03:23Z Using random cookie authentication.
    42022-06-08T17:03:23Z Generated RPC authentication cookie /Users/brunogarcia/Library/Application Support/Bitcoin/signet/.cookie
    52022-06-08T17:03:23Z [http] starting 4 worker threads
    62022-06-08T17:03:23Z Using wallet directory /Users/brunogarcia/Library/Application Support/Bitcoin/signet/wallets
    
  19. jonatack commented at 5:20 pm on June 8, 2022: contributor
    @laanwj No strong opinion but I thought the same, tried a version of it, and initially concluded that (a) [category:all] for unconditional logging might be unnecessarily verbose, and (b) all doesn’t seem useful as a severity level that a user can set in our logging (it would do the same as setting debug currently), though I could be overlooking a use case.
  20. MarcoFalke commented at 5:39 pm on June 8, 2022: member
    I also was under the assumption that this is permanent. If it wasn’t, we probably should not do it, but then I am also wondering if there is a better alternative.
  21. laanwj commented at 5:48 pm on June 8, 2022: member

    Right, my understanding was that the None category and level would eventually go away, and it’d be more like logging systems in other applications. That it’s an intermediate solution until the existing messages are “sorted out”. And after threshold configuration (like #25287) was added. But I could be wrong.

    In any case, permanent or not, I’m ok with this as intermediate solution.

  22. jonatack commented at 5:57 pm on June 8, 2022: contributor
    I could imagine a scenario where LogPrintf and LogPrintfCategory might go away someday (along with None) if they are more or less converted to, say, Info level, and users setting the level to Warning or Error just wouldn’t care to see those messages. But that seems a little radical for now (or not, IDK, Info will be the default and I would use that or a more detailed level myself).
  23. MarcoFalke commented at 5:58 pm on June 8, 2022: member

    I guess the statements under discussion are mostly:

    • Single logs (per process lifetime), for example init logs
    • Heartbeat logs (when a block came in)

    I can see how they are useful unconditionally to make later debugging easier, which would be my reason to keep them unconditional. However, I can also see how there might be a user only interested in Warning/Error logs, not in Info logs. So I guess one alternative is to change None to Info and make it the default severity. (That is probably also the way python does it)

  24. jonatack commented at 6:05 pm on June 8, 2022: contributor
    For an idea, open https://github.com/bitcoin/bitcoin/pull/25203/files, click to load the files with a larger diff, and search for LogPrintfCategory. They probably could all be changed to Error, Warning, or Info. The default debug log would be shockingly quiet, though.
  25. laanwj commented at 6:11 pm on June 8, 2022: member

    I can see how they are useful unconditionally to make later debugging easier, which would be my reason to keep them unconditional.

    Yes, I don’t disagree. It does feel somewhat strange to me to have log messages that figure higher than Error in severity, but aren’t errors. But if used sparingly for this purpose it makes sense.

    So I guess one alternative is to change None to Info and make it the default severity. (That is probably also the way python does it)

    The default debug log would be shockingly quiet, though.

    I think the default logging should remain the same. So if we use Info as default log level for what is now unconditional, for most categories we’d want to have the standard threshold at Info. For some (leveldb, qt) it could deviate, becuase they are spammy.

  26. jonatack commented at 6:22 pm on June 8, 2022: contributor

    Users would currently need to turn on debug logging though, to see Info logging.

    Edit: right, change Info to be unconditional. Hmm. That may need some shifting, though in the latest pass I moved most non-essential logging into debug anyway.

  27. laanwj commented at 6:26 pm on June 8, 2022: member

    Users would currently need to turn on debug logging though, to see Info logging.

    Yes. The current hardcoded LogAcceptCategory thresholding implementation is silly I temporarily added it for leveldb category, basically, until there’s more fine-grained configuration possible. The default threshold for non-spammy cagtegories should probably be Info and higher.

    I think the current LogPrint messages should be of level Debug, and current LogPrintf should be Info or higher. I’m fine with having an unconditional None level for marker messages as @MarcoFalke says.

  28. jonatack commented at 6:29 pm on June 8, 2022: contributor
    In the parent PR I propose also a trace level, for the really verbose low-level messages.
  29. laanwj commented at 6:30 pm on June 8, 2022: member
    Right, makes sense. Though we shouldn’t go over the top with number of severity levels either. But trace sounds OK for some cases like low-level networking logging.
  30. laanwj commented at 9:45 am on June 14, 2022: member
    Code review ACK ecff20db286e2f5d3afe32cfaae72de69d34d23c
  31. laanwj merged this on Jun 14, 2022
  32. laanwj closed this on Jun 14, 2022

  33. jonatack deleted the branch on Jun 14, 2022
  34. sidhujag referenced this in commit 78150ec7b6 on Jun 15, 2022
  35. bitcoin locked this on Jul 19, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 19:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me