kernel: Remove key module from kernel library #29252

pull TheCharlatan wants to merge 5 commits into bitcoin:master from TheCharlatan:kernelRmKey changing 33 files +79 −71
  1. TheCharlatan commented at 5:44 pm on January 15, 2024: contributor

    The key module’s functionality is not used by the kernel library, but currently kernel users are still required to initialize the key module’s secp256k1_context_sign global as part of the kernel::Context through ECC_Start. So move the ECC_Start call to the NodeContext ctor instead to completely remove the key module from the kernel library.

    The gui tests currently keep multiple NodeContext objects in memory, so call ECC_Stop manually to avoid triggering an assertion on ECC_Start.


    This PR is part of the libbitcoinkernel project. It removes a module from the kernel library.

  2. DrahtBot commented at 5:44 pm on January 15, 2024: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage

    For detailed information about the code coverage, see the test coverage report.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK ryanofsky, theuni, achow101
    Stale ACK maflcko, hebasto

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    No conflicts as of last run.

  3. DrahtBot added the label Validation on Jan 15, 2024
  4. maflcko commented at 9:10 am on January 16, 2024: member

    The gui tests currently keep multiple NodeContext objects in memory, so relax the secp256k1_context_sign check in ECC_Start to instead return early if it has already been initialized.

    An alternative would be to call ECC_Stop in the gui tests instead of modifying key code?

  5. TheCharlatan force-pushed on Jan 16, 2024
  6. TheCharlatan commented at 3:42 pm on January 16, 2024: contributor

    Updated 67058d4a82e222f770ede96dc948f758b0a8386c -> 2cb38d93a8fea16bf84b072a90ac023ef345134d (kernelRmKey_0 -> kernelRmKey_1, compare)

    • Implemented @maflcko’s suggestion, calling ECC_Stop instead of relaxing the ECC_Start check.
  7. in src/kernel/context.h:22 in 2cb38d93a8 outdated
    18@@ -19,7 +19,6 @@ namespace kernel {
    19 //! should be stored to std::unique_ptr members pointing to opaque types.
    20 struct Context {
    21     Context();
    22-    ~Context();
    


    maflcko commented at 3:52 pm on January 16, 2024:
    Removing this will lead to compile errors when a unique_ptr member is added with a fwd decl, no?

    TheCharlatan commented at 3:59 pm on January 16, 2024:
    Yes.

    maflcko commented at 4:01 pm on January 16, 2024:
    Seems better to leave as-is, or remove the comment, if it is no longer accurate?

    TheCharlatan commented at 4:04 pm on January 16, 2024:
    Will leave as is then. Seems likely that something will end up here.

    TheCharlatan commented at 4:45 pm on January 16, 2024:

    clang-tidy doesn’t like it being left there. Its suggestion of defaulting it in the header seems counter to what we want to achieve here.

    0/home/drgrid/bitcoin/src/./kernel/context.h:22:5: error: class 'Context' can be made trivially destructible by defaulting the destructor on its first declaration [performance-trivially-destructible,-warnings-as-errors]
    1    ~Context();
    2    ^
    3               = default
    4kernel/context.cpp:23:10: note: destructor definition is here
    5Context::~Context() = default;
    6         ^
    

    maflcko commented at 7:10 pm on January 16, 2024:
    It is possible to suppress a clang tidy check on a specific line of code
  8. in src/kernel/checks.cpp:1 in 2cb38d93a8


    maflcko commented at 3:53 pm on January 16, 2024:
    need to remove include as well, no?
  9. maflcko commented at 4:02 pm on January 16, 2024: member

    lgtm ACK 2cb38d93a8fea16bf84b072a90ac023ef345134d 🍂

    Signature:

    0untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
    1RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
    2trusted comment: lgtm ACK 2cb38d93a8fea16bf84b072a90ac023ef345134d 🍂
    33Z6w9mvDbuK4T+f2+g2HGbUkY+ETKts/CGAUhdhsu/MjBFRcdyVX4GOLme1SMwENKvYIENkhNym7luWlGcHRAw==
    
  10. TheCharlatan force-pushed on Jan 16, 2024
  11. TheCharlatan commented at 10:09 pm on January 16, 2024: contributor

    Updated 2cb38d93a8fea16bf84b072a90ac023ef345134d -> a885a166cec6d84d08600f12b25d912bd28af80e (kernelRmKey_1 -> kernelRmKey_2, compare)

  12. maflcko commented at 9:01 am on January 17, 2024: member

    lgtm ACK a885a166cec6d84d08600f12b25d912bd28af80e 🔀

    Signature:

    0untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
    1RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
    2trusted comment: lgtm ACK a885a166cec6d84d08600f12b25d912bd28af80e 🔀
    3MgtL4Gf030oUpUZQHa33bgM/KAIA6X7ogn4k7avoYVc87vxEv1qrxN9wxdvRTYiOZ1Hd8faqpxGbrRFpJIA9DQ==
    
  13. fanquake requested review from theuni on Jan 17, 2024
  14. fanquake added this to the milestone 28.0 on Feb 27, 2024
  15. achow101 requested review from darosior on Apr 9, 2024
  16. hebasto commented at 7:38 am on May 6, 2024: member
    Concept ACK.
  17. in src/qt/test/rpcnestedtests.cpp:50 in a885a166ce outdated
    45@@ -46,6 +46,9 @@ void RPCNestedTests::rpcNestedTests()
    46         tableRPC.appendCommand(c.name, &c);
    47     }
    48 
    49+    // Call ECC_Stop before instantiating another NodeContext in TestingSetup
    50+    ECC_Stop();
    


    hebasto commented at 8:32 am on May 6, 2024:
    nit: Add #include <key.h>?
  18. hebasto approved
  19. hebasto commented at 8:35 am on May 6, 2024: member
    ACK a885a166cec6d84d08600f12b25d912bd28af80e.
  20. in src/qt/test/rpcnestedtests.cpp:49 in a885a166ce outdated
    45@@ -46,6 +46,9 @@ void RPCNestedTests::rpcNestedTests()
    46         tableRPC.appendCommand(c.name, &c);
    47     }
    48 
    49+    // Call ECC_Stop before instantiating another NodeContext in TestingSetup
    


    theuni commented at 9:11 pm on May 6, 2024:
    This is really unfortunate, but I can’t think of anything less hack-ish. And at least it’s relegated to the test code.

    ryanofsky commented at 2:07 pm on May 7, 2024:

    re: #29252 (review)

    This is really unfortunate, but I can’t think of anything less hack-ish. And at least it’s relegated to the test code.

    I debugged this, and the reason this is needed is that node.kernel.reset() calls during shutdown (in Shutdown() and BasicTestingSetup::~BasicTestingSetup) are no longer calling ECC_Stop. If those places are updated to still shut down ECC like before, this could go away.

  21. theuni approved
  22. theuni commented at 9:19 pm on May 6, 2024: member
    Very nice change. ACK a885a166cec6d84d08600f12b25d912bd28af80e
  23. in src/kernel/context.h:22 in a885a166ce outdated
    18@@ -19,7 +19,7 @@ namespace kernel {
    19 //! should be stored to std::unique_ptr members pointing to opaque types.
    20 struct Context {
    21     Context();
    22-    ~Context();
    23+    ~Context(); // NOLINT(performance-trivially-destructible)
    


    ryanofsky commented at 12:00 pm on May 7, 2024:

    In commit “kernel: Remove key module from kernel library” (a885a166cec6d84d08600f12b25d912bd28af80e)

    Is there a reason to suppress the lint warning instead of just deleting the destructor, since the destructor does not do anything? If the destructor is needed for some reason, it might be good to clarify in a comment.


    maflcko commented at 3:00 pm on May 7, 2024:

    it might be good to clarify in a comment.

    There is already a comment in line 19, which explains why this is done (“should be stored to std::unique_ptr members pointing to opaque types.”)

    While there currently is no unique_ptr, it is possible that one will be re-added in the future?


    ryanofsky commented at 3:22 pm on May 7, 2024:

    re: #29252 (review)

    There is already a comment in line 19, which explains why this is done (“should be stored to std::unique_ptr members pointing to opaque types.”)

    While there currently is no unique_ptr, it is possible that one will be re-added in the future?

    Oh interesting. I was wondering why NOLINT was needed here but not in NodeContext, but I guess it is because NodeContext does contain unique_ptr members, and this doesn’t have any.

    I think probably the simplest thing to do would be to wait until a destructor actually needs to be declared before adding one, instead of keeping a destructor that doesn’t do anything anything anymore and needing to use NOLINT. It is possible the kernel context struct could have unique ptr members added in the future, but at this point that is looking less likely to happen (see #29252 (review))

  24. TheCharlatan marked this as a draft on May 7, 2024
  25. in src/kernel/context.h:16 in a885a166ce outdated
    18@@ -19,7 +19,7 @@ namespace kernel {
    19 //! should be stored to std::unique_ptr members pointing to opaque types.
    20 struct Context {
    


    ryanofsky commented at 12:15 pm on May 7, 2024:

    In commit “kernel: Remove key module from kernel library” (a885a166cec6d84d08600f12b25d912bd28af80e)

    Not directly related to this PR, but I think the code comment above is basically out of date. Initial idea for kernel::Context was to have it hold a collection of variables that kernel code needed to run, so kernel code could use the context struct instead of global variables, and application code would still be convenient to write.

    But this was before Options structs were introduced, and currently code is only passing around options structs, not the Context struct as anticipated. So I think at this point it would be good describe the kernel::Context more accurately and probably make it a class instead of struct. Maybe:

     0//! RAII initializer for global kernel state, running initialization code that
     1//! should execute before other kernel code on construction, and running cleanup
     2//! meant to execute after other kernel code on destruction.
     3//!
     4//! This class initializes global state used by kernel code, including RNG and
     5//! SHA256 state. In the future, this could initialize more global state if more
     6//! globals are added, or become a no-op if globals are removed.
     7//!
     8//! Note: The `kernel::Context` class has a different role from `NodeContext`
     9//! and `WalletContext` structs, despite being similarly named. The kernel
    10//! context class is used to initialize global state, while the node and wallet
    11//! context structs are meant hold state themselves and remove the need for
    12//! global state.
    13class Context
    

    TheCharlatan commented at 8:11 pm on May 7, 2024:
    Some of the option fields do currently hold references and it might be a good idea to move ownership of those referred values to the kernel Context eventually. I agree though that the comment should reflect the current code, not some future hypothetical.

    ryanofsky commented at 2:04 pm on May 8, 2024:

    re: #29252 (review)

    Some of the option fields do currently hold references and it might be a good idea to move ownership of those referred values to the kernel Context eventually.

    Fair enough, new members could be added and kernel::Context could become something more like what was originally described, and the new comment I suggested would need to get updated in that case. To be sure, I don’t think any changes need to be made as part of this PR. I was more making an observation on the current state of things. It seems worth noting out that kernel context object plays a pretty different role than node and wallet context objects because its job is initialize external state, while node and wallet context objects are meant to be passive containers.

  26. in src/node/context.cpp:24 in a885a166ce outdated
    18@@ -18,6 +19,14 @@
    19 #include <validation.h>
    20 
    21 namespace node {
    22-NodeContext::NodeContext() = default;
    23-NodeContext::~NodeContext() = default;
    24+NodeContext::NodeContext()
    25+{
    26+    ECC_Start();
    


    ryanofsky commented at 2:28 pm on May 7, 2024:

    In commit “kernel: Remove key module from kernel library” (a885a166cec6d84d08600f12b25d912bd28af80e)

    I don’t think it’s a great idea to add these ECC_Start / ECC_Stop calls here, because this prevents writing lighter-weight tests that may need to pass a NodeContext argument to RPC or Init functions, but don’t need ECC. Also calling ECC_Start in the constructor makes it not possible for different NodeContext instances to be constructed at the same time. So for example you couldn’t write a test calling RPC or Init functions with different context objects. It also clashes a little with the NodeContext documentation which says it “isn’t intended to have any member functions”, just to be “a collection of references.”

    I think using RAII to initialize ECC state it good, but it should happen in separate class, rather than NodeContext.


    TheCharlatan commented at 8:12 pm on May 7, 2024:
    A RAII wrapper for the ECC state sounds like a nice improvement.

    ryanofsky commented at 2:04 pm on May 8, 2024:

    re: #29252 (review)

    A RAII wrapper for the ECC state sounds like a nice improvement.

    You probably saw this, but just for anyone else reading, this is implemented in the branch posted #29252#pullrequestreview-2042988944

  27. ryanofsky commented at 2:44 pm on May 7, 2024: contributor

    Code review a885a166cec6d84d08600f12b25d912bd28af80e. This is a good change, but I think it has a few issues pointed out in comments below. At a high level I think it makes sense to decouple ECC initialization from the kernel context struct, but I don’t think it makes sense to couple it so tightly with the node context struct.

    Would suggest an approach more like this branch: pr/ecc with commits:

    • c6b4b7c050393b4442f155d40c2a7f586eb19195 common: Add ECC_Context RAII wrapper for ECC_Start/ECC_Stop
    • 7184d36807a00f613b4f7dc0459b69bba314cfd6 test: Use ECC_Context helper in bench and fuzz tests
    • 668e2f8eeb03308702071a74a77c8278229d2df8 tools: Use ECC_Context helper in bitcoin-tx and bitcoin-wallet tools
    • 77190010b178ce253350ce2c4ad5b051e7a02ca2 kernel: Remove key module from kernel library

    Feel free to take any of this or do something different.

  28. TheCharlatan marked this as ready for review on May 7, 2024
  29. theuni commented at 6:17 pm on May 8, 2024: member
    @TheCharlatan Mind adding a commit to fixup the includes as per this conversation?
  30. TheCharlatan force-pushed on May 9, 2024
  31. TheCharlatan commented at 12:51 pm on May 9, 2024: contributor

    Thank you for the review @ryanofsky!

    a885a166cec6d84d08600f12b25d912bd28af80e -> 7dc7938731233f5f3299618e1e07ff92782a7cfd (kernelRmKey_2 -> kernelRmKey_3, compare)

    • Addressed @ryanofsky’s comment, cherry-picked the mentioned commits. Also added another commit for removing the now unused ECC_Start and ECC_Stop from the header.
    • Addressed @theuni’s comment, cleaning up some of the includes.
  32. TheCharlatan force-pushed on May 9, 2024
  33. TheCharlatan commented at 1:35 pm on May 9, 2024: contributor
    Rebased 7dc7938731233f5f3299618e1e07ff92782a7cfd -> 65a205166b3c7d082a7528ea5036dff551d7eb6e (kernelRmKey_3 -> kernelRmKey_4, compare)
  34. in src/key.h:240 in 65a205166b outdated
    235@@ -236,12 +236,6 @@ struct CExtKey {
    236     void SetSeed(Span<const std::byte> seed);
    237 };
    238 
    239-/** Initialize the elliptic curve support. May not be called twice without calling ECC_Stop first. */
    240-void ECC_Start();
    


    ryanofsky commented at 1:42 pm on May 9, 2024:

    In commit “Refactor: Remove ECC_Start and ECC_Stop from key header” (65a205166b3c7d082a7528ea5036dff551d7eb6e)

    It’s probably good to remove these declarations, but I think it would be good to move the documentation to the cpp file so it is not lost. Notes like “May not be called twice” “No-op if ECC_Start wasn’t called” seem useful to describe how these are intended to work.

    Also references to these function in the ECC_Context comment below should be removed if they are private. Maybe replace “RAII wrapper for ECC_Start() and ECC_End(), initializing global ECC state” with “RAII class initializing and deinitializing global state for elliptic curve support.”

  35. ryanofsky approved
  36. ryanofsky commented at 1:42 pm on May 9, 2024: contributor
    Code review ACK 65a205166b3c7d082a7528ea5036dff551d7eb6e
  37. DrahtBot requested review from maflcko on May 9, 2024
  38. DrahtBot requested review from theuni on May 9, 2024
  39. DrahtBot requested review from hebasto on May 9, 2024
  40. common: Add ECC_Context RAII wrapper for ECC_Start/ECC_Stop 538fedde1d
  41. test: Use ECC_Context helper in bench and fuzz tests 28905c1a64
  42. tools: Use ECC_Context helper in bitcoin-tx and bitcoin-wallet tools a08d2b3cb9
  43. kernel: Remove key module from kernel library
    The key module's functionality is not used by the kernel library, but
    currently kernel users are still required to initialize the key module's
    `secp256k1_context_sign` global as part of the `kernel::Context` through
    `ECC_Start`.
    41eba5bd71
  44. Refactor: Remove ECC_Start and ECC_Stop from key header
    They are unused outside of the key module now.
    96378fe734
  45. TheCharlatan force-pushed on May 9, 2024
  46. TheCharlatan commented at 2:01 pm on May 9, 2024: contributor

    Updated 65a205166b3c7d082a7528ea5036dff551d7eb6e -> 96378fe734e5fb6167eb20036d7170572a647edb (kernelRmKey_4 -> kernelRmKey_5, compare)

    • Addressed @ryanofsky’s comment, moving comments and improving description of ECC_Context.
  47. ryanofsky approved
  48. ryanofsky commented at 3:10 pm on May 9, 2024: contributor
    Code review ACK 96378fe734e5fb6167eb20036d7170572a647edb. Just suggested comment changes since last review.
  49. theuni approved
  50. theuni commented at 8:15 pm on May 9, 2024: member

    Very nice improvements since my last review. The RAII context is an obvious improvement, thanks @ryanofsky.

    I looked for a way to add a virtual lifetime class with on_startup/on_shutdown to replace the .init in fuzz.h, but didn’t have any luck because of a (afacs?) missing teardown callback. It seems the static context approach used here is actually what’s recommended.

    utACK 96378fe734e5fb6167eb20036d7170572a647edb

  51. achow101 commented at 5:07 pm on May 10, 2024: member
    ACK 96378fe734e5fb6167eb20036d7170572a647edb
  52. achow101 merged this on May 10, 2024
  53. achow101 closed this on May 10, 2024

  54. hebasto added the label Needs CMake port on May 20, 2024
  55. hebasto commented at 11:03 pm on May 20, 2024: member
    Ported to the CMake-based build system in https://github.com/hebasto/bitcoin/pull/202.
  56. hebasto removed the label Needs CMake port on May 20, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me