kernel: Use spans instead of vectors for passing block headers to validation functions #30742

pull TheCharlatan wants to merge 3 commits into bitcoin:master from TheCharlatan:headersSpan changing 9 files +19 −16
  1. TheCharlatan commented at 4:13 pm on August 28, 2024: contributor

    Makes it friendlier for potential future users of the kernel library if they do not store the headers in a std::vector, but can guarantee contiguous memory.

    Take this opportunity to also change the argument of ImportBlocks previously taking a std::vector to a std::span.

  2. DrahtBot commented at 4:13 pm on August 28, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK stickies-v, maflcko, danielabrozzoni, achow101
    Concept ACK theuni
    Stale ACK jonatack

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

  3. DrahtBot added the label Validation on Aug 28, 2024
  4. in src/validation.cpp:4388 in 035a71fcbe outdated
    4384@@ -4384,7 +4385,7 @@ bool ChainstateManager::AcceptBlockHeader(const CBlockHeader& block, BlockValida
    4385 }
    4386 
    4387 // Exposed wrapper for AcceptBlockHeader
    4388-bool ChainstateManager::ProcessNewBlockHeaders(const std::vector<CBlockHeader>& headers, bool min_pow_checked, BlockValidationState& state, const CBlockIndex** ppindex)
    4389+bool ChainstateManager::ProcessNewBlockHeaders(const std::span<CBlockHeader> headers, bool min_pow_checked, BlockValidationState& state, const CBlockIndex** ppindex)
    


    theuni commented at 5:24 pm on August 28, 2024:
    I think you want std::span<const CBlockHeader> for these. In addition to preserving constness, it allows elements to be passed in as {{foo}}, rather than specifying spans everywhere.
  5. theuni commented at 5:25 pm on August 28, 2024: member
    Concept ACK.
  6. theuni commented at 5:28 pm on August 28, 2024: member
    Actually, I read this assuming we’d already migrated away from Span, which isn’t the case. So probably best not to start introducing std::spans until that’s happened?
  7. theuni commented at 5:29 pm on August 28, 2024: member
    Relevant: #29119
  8. TheCharlatan force-pushed on Aug 28, 2024
  9. TheCharlatan commented at 5:51 pm on August 28, 2024: contributor

    Thanks for the review @theuni!

    Updated 65e440433e7223511a4effb568cdb187fa0aa22c -> c584c1f23b8107dade11d81ada2baf741acbac62 (headersSpan_0 -> headersSpan_1, compare)

    • Addressed @theuni’s comment, made the span take a const CBlockHeader, which allows dropping the annoying std::span invocations again.

    Actually, I read this assuming we’d already migrated away from Span, which isn’t the case. So probably best not to start introducing std::spans until that’s happened?

    There already are users of std::span, so I did not have the impression that I was introducing something novel here. Now that #29071 is merged I would also expect #29119 to move forward soon. But can always revert to Span if that is what reviewers prefer.

  10. in src/validation.cpp:4388 in b7710426dd outdated
    4384@@ -4384,7 +4385,7 @@ bool ChainstateManager::AcceptBlockHeader(const CBlockHeader& block, BlockValida
    4385 }
    4386 
    4387 // Exposed wrapper for AcceptBlockHeader
    4388-bool ChainstateManager::ProcessNewBlockHeaders(const std::vector<CBlockHeader>& headers, bool min_pow_checked, BlockValidationState& state, const CBlockIndex** ppindex)
    4389+bool ChainstateManager::ProcessNewBlockHeaders(const std::span<const CBlockHeader> headers, bool min_pow_checked, BlockValidationState& state, const CBlockIndex** ppindex)
    


    theuni commented at 6:04 pm on August 28, 2024:
    No need to pass as const. foo(std::span<const CBlockHeader>) is functionally equivalent to foo(const std::vector<CBlockHeader>&).

    TheCharlatan commented at 6:17 pm on August 28, 2024:
    Whoops, missed that, thanks!
  11. theuni commented at 6:06 pm on August 28, 2024: member

    Now that #29071 is merged I would also expect #29119 to move forward soon. But can always revert to Span if that is what reviewers prefer. @maflcko ?

  12. TheCharlatan force-pushed on Aug 28, 2024
  13. TheCharlatan commented at 6:17 pm on August 28, 2024: contributor

    Updated c584c1f23b8107dade11d81ada2baf741acbac62 -> 45a90050290b046a8c0f417eab679f36032da297 (headersSpan_1 -> headersSpan_2, compare)

  14. in src/net_processing.cpp:5072 in 45a9005029 outdated
    5067@@ -5068,7 +5068,8 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
    5068             mapBlockSource.emplace(hash, std::make_pair(pfrom.GetId(), true));
    5069 
    5070             // Check claimed work on this block against our anti-dos thresholds.
    5071-            if (prev_block && prev_block->nChainWork + CalculateClaimedHeadersWork({pblock->GetBlockHeader()}) >= GetAntiDoSWorkThreshold()) {
    5072+            auto header{pblock->GetBlockHeader()};
    5073+            if (prev_block && prev_block->nChainWork + CalculateClaimedHeadersWork({{header}}) >= GetAntiDoSWorkThreshold()) {
    


    jonatack commented at 6:28 pm on August 28, 2024:

    nit, as this is only used inside the conditional, may as well keep the scope somaller

    0-            auto header{pblock->GetBlockHeader()};
    1-            if (prev_block && prev_block->nChainWork + CalculateClaimedHeadersWork({{header}}) >= GetAntiDoSWorkThreshold()) {
    2+            if (prev_block && prev_block->nChainWork + CalculateClaimedHeadersWork({{pblock->GetBlockHeader()}}) >= GetAntiDoSWorkThreshold()) {
    
  15. jonatack commented at 6:29 pm on August 28, 2024: member
    ACK 45a90050290b046a8c0f417eab679f36032da297
  16. DrahtBot requested review from theuni on Aug 28, 2024
  17. TheCharlatan force-pushed on Aug 28, 2024
  18. TheCharlatan commented at 7:15 pm on August 28, 2024: contributor

    Thanks for the review @jonatack.

    Updated 45a90050290b046a8c0f417eab679f36032da297 -> 797eede7609ca0d4d788e62f35e70e85103a19dc (headersSpan_2 -> headersSpan_3, compare)

  19. jonatack commented at 7:59 pm on August 28, 2024: member

    ACK 797eede7609ca0d4d788e62f35e70e85103a19dc

    Doesn’t conflict with the changes in #29119 and LGTM provided we’re migrating to std::span and modulo the tidy CI iwyu checks on the updated files (the task hasn’t run yet at the time of writing this).

  20. stickies-v approved
  21. stickies-v commented at 10:41 am on August 29, 2024: contributor
    ACK 797eede7609ca0d4d788e62f35e70e85103a19dc
  22. maflcko commented at 7:22 am on August 30, 2024: member

    Avoids the need for copying by having to wrap the header in a std::vector when just a single block header is passed.

    Can you explain this a bit more? In theory there is C++26 with https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p2447r6.html , but I don’t think any compiler implements it and the code here is compiled in C++20, so I haven’t checked if that fixes your problem.

    Currently in the code, you are even creating explicit copies via the GetBlockHeader method.

    Maybe I am missing something obvious?

  23. TheCharlatan commented at 8:01 am on August 30, 2024: contributor

    but I don’t think any compiler implements it and the code here is compiled in C++20, so I haven’t checked if that fixes your problem.

    Thanks, you are right, my initial patch avoided copies where possible by explicitly creating the span, but the brace initializers that theuni suggested instead create temporary arrays that copy the headers. I should update the description, or I can revert to what I was doing before. Do you have a preference? I don’t think the copies matter all that much either way. The main motiviation for me is that it makes life a bit easier for potential external callers if you use spans over vector references in the interfaces.

  24. maflcko commented at 8:10 am on August 30, 2024: member

    I think your initial commit also had some calls to GetBlockHeader to create a copy (explicitly), but I agree that it doesn’t matter much either way.

    Maybe remove it from the motivation, since it doesn’t seem to be a goal, and if it was a goal, it would be hard to achieve?

  25. validation: Use span for ProcessNewBlockHeaders
    Makes it friendlier for potential future users of the kernel library if
    they do not store the headers in a std::vector, but can guarantee
    contiguous memory.
    52575e96e7
  26. validation: Use span for CalculateClaimedHeadersWork
    Makes it friendlier for potential future users of the kernel library if
    they do not store the headers in a std::vector, but can guarantee
    contiguous memory.
    20515ea3f5
  27. TheCharlatan force-pushed on Aug 30, 2024
  28. TheCharlatan commented at 8:46 am on August 30, 2024: contributor

    797eede7609ca0d4d788e62f35e70e85103a19dc -> b31483e32b7d111eeef5fea8ebdc927a32c22bf4 (headersSpan_3 -> headersSpan_4, compare)

    • Corrected commit message and pull request description, dropping mentions of copies as per @maflcko’s suggestion.
    • Added a commit for also moving to std::span in ImportBlocks.
  29. maflcko commented at 9:05 am on August 30, 2024: member

    This changes vector to span, and where copies were done previously, they are still done.

    ACK b31483e32b7d111eeef5fea8ebdc927a32c22bf4 🖍

    Signature:

    0untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
    1RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
    2trusted comment: ACK b31483e32b7d111eeef5fea8ebdc927a32c22bf4 🖍
    3KluDd3R0iPVwSdNpepkr2LEtfCpZ259JdLgMHassQji86noEGgOSkRAX6C0unxwVFzvv9ROsLYzZd5fV2XJGDw==
    
  30. DrahtBot requested review from jonatack on Aug 30, 2024
  31. DrahtBot requested review from stickies-v on Aug 30, 2024
  32. in src/node/blockstorage.h:433 in b31483e32b outdated
    429@@ -429,7 +430,7 @@ class BlockManager
    430     void CleanupBlockRevFiles() const;
    431 };
    432 
    433-void ImportBlocks(ChainstateManager& chainman, std::vector<fs::path> vImportFiles);
    434+void ImportBlocks(ChainstateManager& chainman, std::span<const fs::path> vImportFiles);
    


    stickies-v commented at 10:18 am on August 30, 2024:

    nit: would be nice to modernize the name to e.g. import_paths since it’s no longer a vector.

    0void ImportBlocks(ChainstateManager& chainman, std::span<const fs::path> import_paths);
    

    TheCharlatan commented at 10:28 am on August 30, 2024:
    Oh right, finally!
  33. stickies-v approved
  34. stickies-v commented at 10:19 am on August 30, 2024: contributor
    ACK b31483e32b7d111eeef5fea8ebdc927a32c22bf4
  35. TheCharlatan force-pushed on Aug 30, 2024
  36. TheCharlatan commented at 10:34 am on August 30, 2024: contributor

    b31483e32b7d111eeef5fea8ebdc927a32c22bf4 -> a2955f09792b6232f3a45aa44a498b466279a8b7 (headersSpan_4 -> headersSpan_5, compare)

  37. validation: Use span for ImportBlocks paths
    Makes it friendlier for potential future users of the kernel library if
    they do not store the headers in a std::vector, but can guarantee
    contiguous memory.
    a2955f0979
  38. TheCharlatan force-pushed on Aug 30, 2024
  39. stickies-v commented at 10:46 am on August 30, 2024: contributor
    re-ACK a2955f09792b6232f3a45aa44a498b466279a8b7 - no changes except further walking the ~file~ path of modernizing variable names.
  40. DrahtBot requested review from maflcko on Aug 30, 2024
  41. maflcko commented at 11:21 am on August 30, 2024: member

    ACK a2955f09792b6232f3a45aa44a498b466279a8b7 🕑

    Signature:

    0untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
    1RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
    2trusted comment: ACK a2955f09792b6232f3a45aa44a498b466279a8b7 🕑
    3lTCtNlp6GgzlMVUtGi0UeE37JlHEPqVxzu84w9qah6aDOvdY6VVedlJf9Dorj/Kjkm8MX2kvBnBkv+eI9aifAQ==
    
  42. DrahtBot added the label CI failed on Sep 3, 2024
  43. danielabrozzoni approved
  44. danielabrozzoni commented at 2:12 pm on September 3, 2024: contributor
    ACK a2955f09792b6232f3a45aa44a498b466279a8b7
  45. maflcko commented at 2:31 pm on September 3, 2024: member
    CI failure looks unrelated and can be ignored: https://github.com/bitcoin/bitcoin/issues/30797
  46. achow101 commented at 7:32 pm on September 3, 2024: member
    ACK a2955f09792b6232f3a45aa44a498b466279a8b7
  47. achow101 merged this on Sep 3, 2024
  48. achow101 closed this on Sep 3, 2024

  49. beirut2 approved

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-29 01:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me