kernel, logging: Pass Logger instances to kernel objects #30342

pull ryanofsky wants to merge 10 commits into bitcoin:master from ryanofsky:pr/gklog changing 48 files +653 −302
  1. ryanofsky commented at 5:52 pm on June 26, 2024: contributor

    Pass Logger instances to BlockManager, CCoinsViewDB, CDBWrapper, Chainstate, ChainstateManager, CoinsViews, and CTxMemPool classes so libbitcoinkernel applications and tests have the option to control where log output goes instead of having all output sent to the global logger.

    This PR is an alternative to #30338. It is more limited because it only changes kernel code while leaving other parts of the codebase alone. It also takes the opportunity to migrate legacy LogPrint / LogPrintf calls to the new log macros introduced in #28318.


    This is based on #29256. The non-base commits are:

  2. DrahtBot commented at 5:52 pm on June 26, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30342.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK TheCharlatan

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #33680 (validation: do not wipe utxo cache for stats/scans/snapshots by l0rinc)
    • #33669 (Add libbitcoinkernel example files by w0xlt)
    • #33646 (log: check fclose() results and report safely in logging.cpp by cedwies)
    • #33629 (Cluster mempool by sdaftuar)
    • #33591 (Cluster mempool followups by sdaftuar)
    • #33553 (validation: Improve warnings in case of chain corruption by mzumsande)
    • #33512 (coins: use number of dirty cache entries in flush warnings/logs by l0rinc)
    • #33443 (log: reduce excessive “rolling back/forward” messages during block replay by l0rinc)
    • #33324 (blocks: add -reobfuscate-blocks arg to xor existing blk/rev on startup by l0rinc)
    • #33042 (refactor: inline constant return values from dbwrapper write methods by l0rinc)
    • #32950 (validation: remove BLOCK_FAILED_CHILD by stratospher)
    • #32543 (kernel: Remove dependency on clientversion by TheCharlatan)
    • #32427 ((RFC) kernel: Replace leveldb-based BlockTreeDB with flat-file based store by TheCharlatan)
    • #32380 (Modernize use of UTF-8 in Windows code by hebasto)
    • #32317 (kernel: Separate UTXO set access from validation functions by TheCharlatan)
    • #32297 (bitcoin-cli: Add -ipcconnect option by ryanofsky)
    • #31860 (init: Take lock on blocks directory in BlockManager ctor by TheCharlatan)
    • #31644 (leveldb: show non-default options during init by l0rinc)
    • #31533 (fuzz: Add fuzz target for block index tree and related validation events by mzumsande)
    • #31382 (kernel: Flush in ChainstateManager destructor by TheCharlatan)
    • #31260 (scripted-diff: Type-safe settings retrieval by ryanofsky)
    • #31132 (validation: fetch block inputs on parallel threads >10% faster IBD by andrewtoth)
    • #30595 (kernel: Introduce C header API by TheCharlatan)
    • #30214 (refactor: Improve assumeutxo state representation by ryanofsky)
    • #30155 (validation: Make ReplayBlocks interruptible by mzumsande)
    • #30059 (Add option dbfilesize to control LevelDB target (“max”) file size by luke-jr)
    • #29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)
    • #29641 (scripted-diff: Use LogInfo over LogPrintf [WIP, NOMERGE, DRAFT] by maflcko)
    • #26022 (Add util::ResultPtr class by ryanofsky)
    • #25722 (refactor: Use util::Result class for wallet loading by ryanofsky)
    • #25665 (refactor: Add util::Result failure values, multiple error and warning messages by ryanofsky)
    • #19461 (multiprocess: Add bitcoin-gui -ipcconnect option by ryanofsky)
    • #19460 (multiprocess: Add bitcoin-wallet -ipcconnect option by ryanofsky)
    • #10102 (Multiprocess bitcoin by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • NOTE: the logger instances is leaked on exit. -> instance [subject-verb agreement: “instances is” should be “instance is”]
    • not wanting to allow users filter by category -> not wanting to allow users to filter by category [missing “to” before verb “filter”]

    drahtbot_id_5_m

  3. TheCharlatan commented at 8:21 pm on June 26, 2024: contributor
    Concept ACK.
  4. ryanofsky force-pushed on Jun 26, 2024
  5. DrahtBot added the label CI failed on Jun 26, 2024
  6. DrahtBot commented at 10:35 pm on June 26, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/26722091796

  7. ryanofsky commented at 10:36 pm on June 26, 2024: contributor
    Updated 45423355735770f01d2bac738cc8b0b41415e921 -> db1b9f7696af80036de739408cd9eae5d06481ec (pr/gklog.1 -> pr/gklog.2, compare) moving code to an earlier commit to fix a “test-each-commit” test failure https://github.com/bitcoin/bitcoin/actions/runs/9684398780/job/26722073921?pr=30342, and cleaning up code to fix a clang-tidy warning https://cirrus-ci.com/task/5893151045451776
  8. DrahtBot removed the label CI failed on Jun 27, 2024
  9. DrahtBot added the label Needs rebase on Jul 2, 2024
  10. ryanofsky referenced this in commit ecadc8f6e5 on Jul 3, 2024
  11. ryanofsky force-pushed on Jul 9, 2024
  12. ryanofsky commented at 10:23 pm on July 9, 2024: contributor

    Rebased db1b9f7696af80036de739408cd9eae5d06481ec -> 42d0e06d1a4343c2bf3257f26cf18b79072c6b0d (pr/gklog.2 -> pr/gklog.3, compare) to fix conflict with #30141. Also added a commit to let kernel applications direct low-level util output to custom log instances. Updated 42d0e06d1a4343c2bf3257f26cf18b79072c6b0d -> fcaed3faf0f12b3b322e65df1e81d455bbee7db5 (pr/gklog.3 -> pr/gklog.4, compare) to fix CI errors Updated fcaed3faf0f12b3b322e65df1e81d455bbee7db5 -> f005677f762754ae09417c2a418e27d28ef3c7e1 (pr/gklog.4 -> pr/gklog.5, compare) to fix more CI errors Rebased f005677f762754ae09417c2a418e27d28ef3c7e1 -> dcd055ecf9c20b84930c2ce45d922427cf0112ea (pr/gklog.5 -> pr/gklog.6, compare) due to conflicts with #30425 and #30407 Rebased dcd055ecf9c20b84930c2ce45d922427cf0112ea -> 53d7933727ebe0670dceb88ec9f1020bd7b0b280 (pr/gklog.6 -> pr/gklog.7, compare) due to conflict with #28052 Rebased 53d7933727ebe0670dceb88ec9f1020bd7b0b280 -> 10c275342dca365a6678da5fc016ca9d46d9dfca (pr/gklog.7 -> pr/gklog.8, compare) due to conflict with #30485 Rebased 10c275342dca365a6678da5fc016ca9d46d9dfca -> 493dd26b75d0e35422b3230be3c72a0f96eb5e8a (pr/gklog.8 -> pr/gklog.9, compare) due to various conflicts Rebased 493dd26b75d0e35422b3230be3c72a0f96eb5e8a -> e971028fa5e570acf585073494acd65f4831ab1c (pr/gklog.9 -> pr/gklog.10, compare) due to conflicts with #31393, #31490, #30965

    Rebased e971028fa5e570acf585073494acd65f4831ab1c -> 282db88a9b1789f0d78928c87c7204402ac11ac3 (pr/gklog.10 -> pr/gklog.11, compare)

    Rebased 282db88a9b1789f0d78928c87c7204402ac11ac3 -> ec6d771d3b34ec979e373f025c4f064ae44df860 (pr/gklog.11 -> pr/gklog.12, compare)

    Rebased ec6d771d3b34ec979e373f025c4f064ae44df860 -> b1b72969c9764636721a7518a2f4453d0eb3845d (pr/gklog.12 -> pr/gklog.13, compare)

    Rebased b1b72969c9764636721a7518a2f4453d0eb3845d -> d0e2944bb18e44c599312b980900c324ae698464 (pr/gklog.13 -> pr/gklog.14, compare) on updated base PR to fix CI errors (https://github.com/bitcoin/bitcoin/actions/runs/18514131135?pr=30342)

  13. DrahtBot removed the label Needs rebase on Jul 10, 2024
  14. ryanofsky force-pushed on Jul 10, 2024
  15. ryanofsky force-pushed on Jul 10, 2024
  16. DrahtBot added the label CI failed on Jul 10, 2024
  17. DrahtBot commented at 6:22 pm on July 10, 2024: contributor

    🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

    Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    Leave a comment here, if you need help tracking down a confusing failure.

    Debug: https://github.com/bitcoin/bitcoin/runs/27284943547

  18. DrahtBot removed the label CI failed on Jul 10, 2024
  19. DrahtBot added the label Needs rebase on Jul 16, 2024
  20. ryanofsky force-pushed on Jul 18, 2024
  21. DrahtBot removed the label Needs rebase on Jul 18, 2024
  22. DrahtBot added the label CI failed on Jul 22, 2024
  23. DrahtBot commented at 11:07 am on July 22, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/27590643116

    Make sure to run all tests locally, according to the documentation.

    The failure may happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  24. DrahtBot removed the label CI failed on Jul 22, 2024
  25. DrahtBot added the label Needs rebase on Jul 26, 2024
  26. ryanofsky force-pushed on Aug 7, 2024
  27. ryanofsky force-pushed on Aug 7, 2024
  28. DrahtBot removed the label Needs rebase on Aug 7, 2024
  29. DrahtBot added the label CI failed on Aug 13, 2024
  30. DrahtBot commented at 7:50 pm on August 13, 2024: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/28438476134

    Make sure to run all tests locally, according to the documentation.

    The failure may happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  31. DrahtBot added the label Needs rebase on Aug 21, 2024
  32. ryanofsky force-pushed on Dec 9, 2024
  33. DrahtBot removed the label Needs rebase on Dec 9, 2024
  34. DrahtBot removed the label CI failed on Dec 9, 2024
  35. DrahtBot added the label CI failed on Dec 11, 2024
  36. DrahtBot removed the label CI failed on Dec 17, 2024
  37. DrahtBot added the label Needs rebase on Dec 18, 2024
  38. ryanofsky force-pushed on Mar 12, 2025
  39. DrahtBot removed the label Needs rebase on Mar 12, 2025
  40. DrahtBot added the label Needs rebase on Mar 14, 2025
  41. ryanofsky referenced this in commit 6d0745d22b on Apr 3, 2025
  42. ryanofsky force-pushed on Apr 3, 2025
  43. DrahtBot removed the label Needs rebase on Apr 3, 2025
  44. ryanofsky referenced this in commit 2be41ef85f on Apr 3, 2025
  45. ryanofsky referenced this in commit eaac991552 on Apr 4, 2025
  46. ryanofsky force-pushed on Apr 4, 2025
  47. DrahtBot added the label CI failed on Apr 4, 2025
  48. DrahtBot commented at 3:37 am on April 4, 2025: contributor

    🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/39937617956

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  49. DrahtBot removed the label CI failed on Apr 4, 2025
  50. DrahtBot added the label Needs rebase on Apr 8, 2025
  51. log, test: Add test for currently accepted logging arguments
    Test will be extended in next commit to cover support for context objects.
    260170d014
  52. log, refactor: Allow log macros to accept context arguments
    Allow LogDebug(), LogTrace(), LogInfo(), LogWarning(), and LogError() macros to
    accept context arguments to provide more information in log messages and more
    control over logging to callers.
    
    This functionality is used in followup PRs:
    
    - https://github.com/bitcoin/bitcoin/pull/30342 - To let libbitcoinkernel send
      output to specfic `BCLog::Logger` instances instead of a global instance, so
      output can be disambiguated and applications can have more control over
      logging.
    
    - https://github.com/bitcoin/bitcoin/pull/30343 - To replace custom
      `WalletLogPrintf` calls with standard logging calls that automatically include
      wallet names and don't log everything at info level.
    
    This commit does not change behavior of current log prints or require them to
    be updated. It includes tests and documentation covering the new functionality.
    
    Co-Authored-By: Hodlinator <172445034+hodlinator@users.noreply.github.com>
    e4ca362858
  53. logging: Add support for custom log contexts
    Custom log contexts allow overridding log formatting and adding metadata such
    as request ids or wallet names to log messages while still using standard
    macros for logging. This is used to replace WalletLogPrintf() functions with
    LogInfo() calls in followup PR #30343.
    98e2cef781
  54. log, refactor: Add preprocessor workaround for MSVC
    Needed to fix errors like:
    
    const Context &GetContext(const Context &)': expects 1 arguments - 3 provided
    
    due to a compiler bug:
    
    https://stackoverflow.com/questions/5134523/msvc-doesnt-expand-va-args-correctly/5134656#5134656
    
    Example CI failure:
    
    https://github.com/bitcoin/bitcoin/actions/runs/8072891468/job/22055528830?pr=29256
    b8b97370ce
  55. log, refactor: Disallow category args to logging macros
    Disallow passing BCLog category constants to LogInfo(), LogWarning(), and
    LogError() macros, and disallow omitting BCLog categories when calling
    LogDebug() and LogTrace() macros.
    
    Additionally, drop category information from log output at higher levels as
    suggested by Hodlinator
    https://github.com/bitcoin/bitcoin/pull/29256#discussion_r2040671181.
    
    None of these restrictions are technically necessary, and not everybody
    believes they are good. However, they have existed since the
    Log{Trace,Debug,Info,Warning,Error} macros were introduced in #28318, and
    removing these restrictions is orthogonal to the goals of this PR which are:
    (1) to allow the Log macros to work without accessing global state and (2) to
    support local logging customization with additional metadata.
    
    This change just more clearly documents the current logging restrictions and
    provides better error messages to developers when enforcing them.
    
    Co-Authored-By: Hodlinator <172445034+hodlinator@users.noreply.github.com>
    8428899fb9
  56. Merge branch 'pr/bclog' into pr/gklog 19b90ba7c7
  57. refactor: Pass Logger instances to kernel objects
    Pass Logger instances to BlockManager, CCoinsViewDB, CDBWrapper,
    ChainstateManager, and CoinsViews instances so libbitcoinkernel applications
    and test code have the option to control where log output goes instead of
    having all output sent to the global logger.
    
    This commit just passes the logger objects without using them. The next commit
    updates log print statements to use the new objects.
    8656985439
  58. refactor: Log kernel output to local log instances
    This is a mechanical change updating kernel code that currently uses the global
    log instance to log to local instances instead.
    bcfecce29b
  59. refactor: Pass Logger instance to CTxMemPool
    This allows libbitcoinkernel applications and test code to have the option to
    control where log output goes instead of having all output sent to the global
    logger.
    7e569a5bd4
  60. kernel: Drop global Logger instance
    Change LogInstance() function to no longer allocate (and leak) a BCLog::Logger
    instance. Instead allow kernel applications to initialize their own logging
    instances that can be returned by LogInstance().
    
    The LogInstance() function is not actually used for the majority of logging in
    kernel code. Most kernel log output goes directly to BCLog::Logger instances
    specified when kernel objects like ChainstateManager and CTxMemPool are
    constructed, which allows applications to create multiple Logger instances and
    control which log output is sent to them.
    
    The only kernel code that does rely on LogInstance() is certain low level code in
    the util library, like the RNG seeder, that is not passed a specific instance
    and needs to rely on the global LogInstance() function.
    
    Other code outside the kernel library uses LogInstance() heavily, and may
    continue to do so. bitcoind and test code now just create a log instance before
    the first LogInstance() call, which it returns, so all behavior is the same as
    before.
    d0e2944bb1
  61. ryanofsky referenced this in commit 8e3ee9d7e7 on Oct 15, 2025
  62. ryanofsky referenced this in commit 9e7dabc35d on Oct 15, 2025
  63. ryanofsky force-pushed on Oct 15, 2025
  64. DrahtBot removed the label Needs rebase on Oct 15, 2025
  65. ryanofsky referenced this in commit a5700c7911 on Oct 16, 2025
  66. ryanofsky force-pushed on Oct 16, 2025
  67. DrahtBot added the label Needs rebase on Oct 29, 2025
  68. DrahtBot commented at 0:03 am on October 29, 2025: contributor
    🐙 This pull request conflicts with the target branch and needs rebase.

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-10-31 12:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me