kernel: improve BlockChecked ownership semantics #33078

pull stickies-v wants to merge 2 commits into bitcoin:master from stickies-v:2025-07/validation-interface-ownership changing 9 files +42 −33
  1. stickies-v commented at 1:38 pm on July 28, 2025: contributor

    Subscribers to the BlockChecked validation interface event may need access to the block outside of the callback scope. Currently, this is only possible by copying the block, which makes exposing this validation interface event publicly either cumbersome or with significant copy overhead.

    By using shared_ptr, we make the shared ownership explicit and allow users to safely use the block outside of the callback scope. By using a const-ref shared_ptr, no atomic reference count cost is incurred if a subscriber does not require block ownership.

    For example: in #30595, this would allow us to drop the kernel_BlockPointer handle entirely, and generalize everything into kernel_Block. This PoC is implemented in https://github.com/stickies-v/bitcoin/commits/kernel/remove-blockpointer/.


    Performance

    I have added a benchmark in a separate branch, to ensure this change does not lead to a problematic performance regression. Since most of the overhead comes from the subscribers, I have added scenarios for One, Two, and Ten subscribers. From these results, it appears there is no meaningful performance difference on my machine.

    When BlockChecked() takes a const CBlock& reference (master):

    ns/op op/s err% total benchmark
    170.09 5,879,308.26 0.3% 0.01 BlockCheckedOne
    1,603.95 623,460.10 0.5% 0.01 BlockCheckedTen
    336.00 2,976,173.37 1.1% 0.01 BlockCheckedTwo

    When BlockChecked() takes a const std::shared_ptr<const CBlock>& (this PR):

    ns/op op/s err% total benchmark
    172.20 5,807,155.33 0.1% 0.01 BlockCheckedOne
    1,596.79 626,254.52 0.0% 0.01 BlockCheckedTen
    333.38 2,999,603.17 0.3% 0.01 BlockCheckedTwo
  2. kernel: refactor: ConnectTip to pass block pointer by value
    By passing by value, we can remove the need to allocate a new pointer if
    the callsite uses move semantics. In the process, simplify naming.
    9ba1fff29e
  3. DrahtBot added the label Validation on Jul 28, 2025
  4. DrahtBot commented at 1:38 pm on July 28, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33078.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK w0xlt, ryanofsky, TheCharlatan, yuvicc, achow101
    Concept ACK stringintech, purpleKarrot

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #32317 (kernel: Separate UTXO set access from validation functions by TheCharlatan)
    • #30595 (kernel: Introduce initial C header API by TheCharlatan)
    • #29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  5. stickies-v force-pushed on Jul 28, 2025
  6. DrahtBot added the label CI failed on Jul 28, 2025
  7. DrahtBot commented at 1:58 pm on July 28, 2025: contributor

    🚧 At least one of the CI tasks failed. Task no wallet, libbitcoinkernel: https://github.com/bitcoin/bitcoin/runs/46861571669 LLM reason (✨ experimental): The CI failure is caused by a compilation error due to a method override mismatch in bitcoin-chainstate.cpp.

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  8. DrahtBot removed the label CI failed on Jul 28, 2025
  9. in src/validationinterface.h:153 in 742f2b5435 outdated
    149@@ -150,7 +150,7 @@ class CValidationInterface {
    150      * is guaranteed to be the current best block at the time the
    151      * callback was generated (not necessarily now).
    152      */
    153-    virtual void BlockChecked(const CBlock&, const BlockValidationState&) {}
    154+    virtual void BlockChecked(std::shared_ptr<const CBlock>, const BlockValidationState&) {}
    


    w0xlt commented at 7:08 pm on July 31, 2025:
    0    virtual void BlockChecked(const std::shared_ptr<const CBlock>, const BlockValidationState&) {}
    

    stickies-v commented at 7:52 pm on July 31, 2025:
    What’s the benefit of passing a const std::shared_ptr?

    w0xlt commented at 8:35 pm on July 31, 2025:
    Mostly for clarity and intent (with this change, cannot reassign or mutate the local shared_ptr copy). Other than that, no real effect. It is a nit, not blocking.

    maflcko commented at 11:17 am on August 1, 2025:
    I haven’t compiled or benchmarked this, but using const std::shared_ptr<const CBlock> & could avoid the copy and then subscribers are free to decide if they want read-only access, or if they want to create a copy themselves. That would also be in symmetry with the other block signals?

    stickies-v commented at 1:22 pm on August 1, 2025:
    Done, thanks. My initial idea was to use value semantics to simplify ownership, and to let the dispatcher implement move semantics where relevant, but that doesn’t work when there is one dispatcher and multiple subscribers as is the case with our validation interface. Letting the subscriber explicitly claim ownership by copying the reference makes sense and allows us to pay for what we use.
  10. in src/validation.cpp:3162 in 9ba1fff29e outdated
    3160     // Read block from disk.
    3161     const auto time_1{SteadyClock::now()};
    3162-    std::shared_ptr<const CBlock> pthisBlock;
    3163-    if (!pblock) {
    3164+    if (!block_to_connect) {
    3165         std::shared_ptr<CBlock> pblockNew = std::make_shared<CBlock>();
    


    maflcko commented at 6:58 am on August 1, 2025:
    would be easier to do this in-place? block_to_connect=std::make_shared<CBlock>();, as you mutate the copy anyway

    stickies-v commented at 8:29 am on August 1, 2025:
    I don’t think that’s possible, because block_to_connect points to a const CBlock, so we can’t pass that to ReadBlock().
  11. stickies-v commented at 9:43 am on August 1, 2025: contributor
    Note: I have updated the PR description with benchmark results for this change, indicating a modest (~10%) performance decrease which I think is acceptable? edit: no more performance penalty since force-push from 742f2b5 to 1d9f1cb
  12. kernel: improve BlockChecked ownership semantics
    Subscribers to the BlockChecked validation interface event may need
    access to the block outside of the callback scope. Currently, this
    is only possible by copying the block, which makes exposing this
    validation interface event publicly either cumbersome or with significant
    copy overhead.
    
    By using shared_ptr, we make the shared ownership explicit and allow
    users to safely use the block outside of the callback scope.
    1d9f1cb4bd
  13. stickies-v force-pushed on Aug 1, 2025
  14. stickies-v commented at 1:24 pm on August 1, 2025: contributor
    Force-pushed to call BlockChecked with const std::shared_ptr<const CBlock>&, addressing review feedback to avoid atomic reference counting overhead when the subscriber doesn’t need ownership. Benchmarks indicate this PR now no longer has a significant performance impact, on my machine.
  15. ryanofsky approved
  16. ryanofsky commented at 6:56 pm on August 8, 2025: contributor
    Code review ACK 1d9f1cb4bd6b119e1d56cbdd7f6ce4d4521fffa3. These all seem like simple changes that make sense
  17. TheCharlatan approved
  18. TheCharlatan commented at 10:12 pm on August 8, 2025: contributor
    ACK 1d9f1cb4bd6b119e1d56cbdd7f6ce4d4521fffa3
  19. purpleKarrot commented at 4:24 pm on August 14, 2025: contributor

    According to https://youtu.be/ZbVCGCy3mGQ?feature=shared&t=2672 (please watch the talk from the beginning), std::shared<T const> const& should be used for “optional ownership transfer”.

    If the function with such an argument does not perform a copy of that argument inside an if-block, the argument should be changed. If it does a copy unconditionally, the argument should be changed to std::shared<T const> and the copy should be turned into a move. If no copy is happening at all inside the function and all the function does is dereference the shared pointer, the argument should be changed to T const& and the dereferencing should be done at the call site.

    I think there are a lot of functions where violations of that guideline are introduced here, so that is a NACK from my side.

  20. stickies-v commented at 4:47 pm on August 14, 2025: contributor

    Thanks @purpleKarrot. I’m not sure I understand your NACK. Do you mean Concept or Approach? If approach, do you have a preferred alternative?

    std::shared<T const> const& should be used for “optional ownership transfer”.

    Optional ownership transfer is indeed the intended use case here with BlockChecked. We know that some callers don’t require ownership, e.g. they just inspect the hash of the block. We also want to allow callers to take ownership, especially as we expose this functionality in the bitcoinkernel API.

    So:

    • std::shared<T const> would incur unnecessary reference counting in some cases (we can’t use std::move because the event can be subsribed to multiple times)
    • T const& doesn’t allow taking ownership without copying

    I think that leaves us with std::shared<T const> const&?

  21. ryanofsky commented at 5:06 pm on August 14, 2025: contributor

    I think there are a lot of functions where violations of that guideline are introduced here, so that is a NACK from my side.

    I’m not sure if the “use T const& if not sharing, shared_ptr<T const> if sharing” should apply to virtual methods, because the base interface can’t know what concrete overrides will do. Different implementations may or may not take ownership, so const shared_ptr<...>& in the interface makes sense. It avoids unnecessary copies while still allowing any override to share the block if it wants to.

  22. purpleKarrot commented at 5:11 pm on August 14, 2025: contributor

    When I see a function that takes std::shared<T const> const&, and that function does not perform an optional ownership transfer, it makes me suspicious. We can apply the interface segregation principle by changing that to T const& and perform the derenferencing at the call site.

    Here, we have a special case because the function is virtual. If only one implementation requires optional ownership transfer, that requirement is imposed on all others, which is generally a problem with virtual functions.

    In that case, std::shared<T const> const& actually might be the correct approach, but to someone who has interned the “how to argue”-guidelines, it still looks wrong and we should prefer that correct code that looks right.

    The alternative would be to raise the level of abstraction and use std::shared_ptr<T const> as an implementation detail. I understand that this may conflict with your approach of giving access to both the pointer and the pointee through the public API. I would prefer to delay this PR until we reached consensus on the ownership approach for the API.

  23. ryanofsky commented at 5:40 pm on August 14, 2025: contributor

    We can apply the interface segregation principle

    Yeah I think I understand your concern that this interface is overexposing functionality client’s don’t necessarily need. But it seems like it was already doing that before this PR and this PR doesn’t change the situation very much. If you want to explore alternative approaches, that seems reasonable. I just think these type changes do make sense given the interface we have.

  24. purpleKarrot commented at 9:24 pm on August 14, 2025: contributor

    I just think these type changes do make sense given the interface we have.

    Maybe. I think we are pretty close to reaching consensus regarding the ownership model. Let’s just delay the change until we are 100% there.

  25. in src/validation.cpp:3151 in 1d9f1cb4bd
    3144@@ -3145,26 +3145,28 @@ class ConnectTrace {
    3145  *
    3146  * The block is added to connectTrace if connection succeeds.
    3147  */
    3148-bool Chainstate::ConnectTip(BlockValidationState& state, CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool)
    3149+bool Chainstate::ConnectTip(
    3150+    BlockValidationState& state,
    3151+    CBlockIndex* pindexNew,
    3152+    std::shared_ptr<const CBlock> block_to_connect,
    


    stringintech commented at 9:24 pm on August 15, 2025:
    pblock -> block_to_connect in ConnectTip() doc.

    stickies-v commented at 7:38 am on August 22, 2025:
  26. stringintech commented at 9:26 pm on August 15, 2025: contributor
    Concept ACK.
  27. yuvicc commented at 6:58 pm on August 19, 2025: contributor

    Code Review ACK 1d9f1cb4bd6b119e1d56cbdd7f6ce4d4521fffa3

    • the benchmark looks encouraging
    • avoids ref-counts and no atomic overhead.
  28. DrahtBot requested review from stringintech on Aug 19, 2025
  29. purpleKarrot commented at 3:40 pm on August 20, 2025: contributor

    The alternative would be to raise the level of abstraction and use std::shared_ptr<T const> as an implementation detail. I understand that this may conflict with your approach of giving access to both the pointer and the pointee through the public API. I would prefer to delay this PR until we reached consensus on the ownership approach for the API.

    It seams like we reached consensus that we will not provide access to the pointee through the public api. That means we can replace std::shared_ptr<CBlock const> with a value type in the long run. Passing a value type as const& to a function is perfectly fine. ACK

  30. achow101 commented at 5:28 pm on August 20, 2025: member
    ACK 1d9f1cb4bd6b119e1d56cbdd7f6ce4d4521fffa3
  31. achow101 merged this on Aug 20, 2025
  32. achow101 closed this on Aug 20, 2025

  33. alexanderwiederin referenced this in commit 67b164b083 on Aug 21, 2025

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-09-03 00:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me