kernel, logging: Pass Logger instances to kernel objects

ryanofsky commented at 5:52 pm on June 26, 2024: contributor

Pass Logger instances to BlockManager, CCoinsViewDB, CDBWrapper, Chainstate, ChainstateManager, CoinsViews, and CTxMemPool classes so libbitcoinkernel applications and tests have the option to control where log output goes instead of having all output sent to the global logger.

This PR is an alternative to #30338. It is more limited because it only changes kernel code while leaving other parts of the codebase alone. It also takes the opportunity to migrate legacy LogPrint / LogPrintf calls to the new log macros introduced in #28318.

This is based on #29256. The non-base commits are:

DrahtBot commented at 5:52 pm on June 26, 2024: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	TheCharlatan

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#30673 (coins: remove logic for spent-and-FRESH cache entries and writing non-DIRTY entries by andrewtoth)
#30644 (fuzz: Faster utxo_snapshot fuzz target by maflcko)
#30610 (validation: do not wipe utxo cache for stats/scans/snapshots by sipa)
#30571 (test: [refactor] Use m_rng directly by maflcko)
#30509 (multiprocess: Add -ipcbind option to bitcoin-node by ryanofsky)
#30155 (validation: Make ReplayBlocks interruptible by mzumsande)
#30059 (Add option dbfilesize to control LevelDB target (“max”) file size by luke-jr)
#30039 (dbwrapper: Bump LevelDB max file size to 128 MiB to avoid system slowdown from high disk cache flush rate by maciejsszmigiero)
#29641 (scripted-diff: Use LogInfo/LogDebug over LogPrintf/LogPrint by maflcko)
#29640 (Fix tiebreak when loading blocks from disk (and add tests for comparing chain ties) by sr-gi)
#27826 (validation: log which peer sent us a header by Sjors)
#26697 (logging: use bitset for categories by LarryRuane)
#26022 (Add util::ResultPtr class by ryanofsky)
#25665 (refactor: Add util::Result failure values, multiple error and warning messages by ryanofsky)
#19461 (multiprocess: Add bitcoin-gui -ipcconnect option by ryanofsky)
#19460 (multiprocess: Add bitcoin-wallet -ipcconnect option by ryanofsky)
#10102 (Multiprocess bitcoin by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

TheCharlatan commented at 8:21 pm on June 26, 2024: contributor

Concept ACK.

ryanofsky force-pushed on Jun 26, 2024

DrahtBot added the label CI failed on Jun 26, 2024

DrahtBot commented at 10:35 pm on June 26, 2024: contributor

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/26722091796

ryanofsky commented at 10:36 pm on June 26, 2024: contributor

Updated 45423355735770f01d2bac738cc8b0b41415e921 -> db1b9f7696af80036de739408cd9eae5d06481ec (pr/gklog.1 -> pr/gklog.2, compare) moving code to an earlier commit to fix a “test-each-commit” test failure https://github.com/bitcoin/bitcoin/actions/runs/9684398780/job/26722073921?pr=30342, and cleaning up code to fix a clang-tidy warning https://cirrus-ci.com/task/5893151045451776

DrahtBot removed the label CI failed on Jun 27, 2024

DrahtBot added the label Needs rebase on Jul 2, 2024

ryanofsky referenced this in commit ecadc8f6e5 on Jul 3, 2024

ryanofsky force-pushed on Jul 9, 2024

ryanofsky commented at 10:23 pm on July 9, 2024: contributor

Rebased db1b9f7696af80036de739408cd9eae5d06481ec -> 42d0e06d1a4343c2bf3257f26cf18b79072c6b0d (pr/gklog.2 -> pr/gklog.3, compare) to fix conflict with #30141. Also added a commit to let kernel applications direct low-level util output to custom log instances. Updated 42d0e06d1a4343c2bf3257f26cf18b79072c6b0d -> fcaed3faf0f12b3b322e65df1e81d455bbee7db5 (pr/gklog.3 -> pr/gklog.4, compare) to fix CI errors Updated fcaed3faf0f12b3b322e65df1e81d455bbee7db5 -> f005677f762754ae09417c2a418e27d28ef3c7e1 (pr/gklog.4 -> pr/gklog.5, compare) to fix more CI errors Rebased f005677f762754ae09417c2a418e27d28ef3c7e1 -> dcd055ecf9c20b84930c2ce45d922427cf0112ea (pr/gklog.5 -> pr/gklog.6, compare) due to conflicts with #30425 and #30407

DrahtBot removed the label Needs rebase on Jul 10, 2024

ryanofsky force-pushed on Jul 10, 2024

DrahtBot added the label CI failed on Jul 10, 2024

DrahtBot commented at 6:22 pm on July 10, 2024: contributor

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/27284943547

DrahtBot removed the label CI failed on Jul 10, 2024

DrahtBot added the label Needs rebase on Jul 16, 2024

ryanofsky force-pushed on Jul 18, 2024

DrahtBot removed the label Needs rebase on Jul 18, 2024

DrahtBot added the label CI failed on Jul 22, 2024

DrahtBot commented at 11:07 am on July 22, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/27590643116

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

DrahtBot removed the label CI failed on Jul 22, 2024

DrahtBot added the label Needs rebase on Jul 26, 2024

logging: Improve new LogDebug/Trace/Info/Warning/Error Macros

Improve new LogDebug(), LogTrace(), LogInfo(),  LogWarning(), LogError() macros
introduced in #28318:

- Make them always accept log categories to make it possible to only log
  messages from a particular component.
- Make them not rely on a global LogInstance, to provide better control of
  logging in controlled environments, such as unit tests that want to
  selectively capture log output, or libbitcoinkernel applications that want to
  have multiple instances of validation objects without mixing their log output.
- Make them consistent, now all accepting the same parameters.
- Make them less verbose by not requiring BCLog category constants to be
  specified in individual log prints

9e598591dc

logging: Add preprocessor workaround for MSVC

Needed to fix errors like:

const Source &_LogSource(const Source &)': expects 1 arguments - 3 provided
const Source &_LogSource(const Source &)': expects 1 arguments - 3 provided

due to a compiler bug:

https://stackoverflow.com/questions/5134523/msvc-doesnt-expand-va-args-correctly/5134656#5134656

Example CI failure:

https://github.com/bitcoin/bitcoin/actions/runs/8072891468/job/22055528830?pr=29256

3560100bfe

test: Clean up LogPrintf_ test to check LogPrintStr directly

Change was suggested by hodlinator in https://github.com/bitcoin/bitcoin/pull/29256#discussion_r1655562082

46f5e27e86

Merge remote-tracking branch 'origin/pull/29256/head' 08e90c8a6f

refactor: Pass Logger instances to kernel objects

Pass Logger instances to BlockManager, CCoinsViewDB, CDBWrapper,
ChainstateManager, and CoinsViews instances so libbitcoinkernel applications
and test code have the option to control where log output goes instead of
having all output sent to the global logger.

This commit just passes the logger objects without using them. The next commit
updates log print statements to use the new objects.

90e941d3db

refactor: Log kernel output to local log instances

This is a mechanical change updating kernel code that currently uses the global
log instance to log to local instances instead.

aecba82de7

refactor: Pass Logger instance to CTxMemPool

This allows libbitcoinkernel applications and test code to have the option to
control where log output goes instead of having all output sent to the global
logger.

4b6e43275b

kernel: Drop global Logger instance

Change LogInstance() function to no longer allocate (and leak) a BCLog::Logger
instance. Instead allow kernel applications to initialize their own logging
instances that can be returned by LogInstance().

The LogInstance() function is not actually used for the majority of logging in
kernel code. Most kernel log output goes directly to BCLog::Logger instances
specified when kernel objects like ChainstateManager and CTxMemPool are
constructed, which allows applications to create multiple Logger instances and
control which log output is sent to them.

The only kernel code that does rely on LogInstance() is certain low level code in
the util library, like the RNG seeder, that is not passed a specific instance
and needs to rely on the global LogInstance() function.

Other code outside the kernel library uses LogInstance() heavily, and may
continue to do so. bitcoind and test code now just create a log instance before
the first LogInstance() call, which it returns, so all behavior is the same as
before.

10c275342d

ryanofsky force-pushed on Aug 7, 2024

DrahtBot removed the label Needs rebase on Aug 7, 2024

DrahtBot added the label CI failed on Aug 13, 2024

DrahtBot commented at 7:50 pm on August 13, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/28438476134

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

DrahtBot added the label Needs rebase on Aug 21, 2024

DrahtBot commented at 4:34 am on August 21, 2024: contributor

🐙 This pull request conflicts with the target branch and needs rebase.

DrahtBot commented at 0:08 am on November 18, 2024: contributor

⌛ There hasn’t been much activity lately and the patch still needs rebase. What is the status here?

Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.

kernel, logging: Pass Logger instances to kernel objects #30342

Code Coverage

Reviews

Conflicts