Introduce MempoolObserver interface to break “policy/fees -> txmempool -> policy/fees” circular dependency #13949

pull Empact wants to merge 1 commits into bitcoin:master from Empact:mempool-observer changing 8 files +70 −16
  1. Empact commented at 1:09 am on August 13, 2018: member

    A simple interface which make the implementations independent of one another.

    Note I also removed the inBlock argument to the public CBlockPolicyEstimator::removeTx, as all block-related removal calls are internal to the class. This simplifies the extracted interface and reduces CBlockPolicyEstimator surface area.

  2. fanquake added the label Refactoring on Aug 13, 2018
  3. fanquake added the label Mempool on Aug 13, 2018
  4. Empact force-pushed on Aug 13, 2018
  5. Empact force-pushed on Aug 13, 2018
  6. DrahtBot commented at 2:55 am on August 13, 2018: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #16066 (mempool: Skip estimator if block is older than X by promag)
    • #15946 (Allow maintaining the blockfilterindex when using prune by jonasschnelli)
    • #14193 (validation: Add missing mempool locks by MarcoFalke)
    • #10443 (Add fee_est tool for debugging fee estimation code by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  7. practicalswift commented at 8:28 am on August 13, 2018: contributor

    Concept ACK

    Thanks for working on these circular dependencies!

  8. in src/policy/fees.h:205 in 2a44b30ce5 outdated
    198@@ -198,8 +199,10 @@ class CBlockPolicyEstimator
    199     /** Process a transaction accepted to the mempool*/
    200     void processTransaction(const CTxMemPoolEntry& entry, bool validFeeEstimate);
    201 
    202-    /** Remove a transaction from the mempool tracking stats*/
    203-    bool removeTx(uint256 hash, bool inBlock);
    204+    /** Remove a transaction from the mempool tracking stats
    205+     * Removal is not inBlock as in-block removals are handled internally.
    206+     */
    207+    bool removeTx(uint256 hash);
    


    l2a5b1 commented at 9:37 am on August 14, 2018:

    You can mark the declarations of interfaces::MempoolObserver methods in CBlockPolicyEstimator with override:

    0    void processBlock(unsigned int nBlockHeight,
    1                      std::vector<const CTxMemPoolEntry*>& entries) override;
    2
    3    void processTransaction(const CTxMemPoolEntry& entry, bool validFeeEstimate) override;
    4
    5    bool removeTx(uint256 hash) override;
    
  9. in src/policy/fees.h:202 in 2a44b30ce5 outdated
    198@@ -198,8 +199,10 @@ class CBlockPolicyEstimator
    199     /** Process a transaction accepted to the mempool*/
    200     void processTransaction(const CTxMemPoolEntry& entry, bool validFeeEstimate);
    201 
    202-    /** Remove a transaction from the mempool tracking stats*/
    203-    bool removeTx(uint256 hash, bool inBlock);
    204+    /** Remove a transaction from the mempool tracking stats
    


    l2a5b1 commented at 9:59 am on August 14, 2018:

    To prevent redundant documentation, maybe you can declare the interfaces::MempoolObserver methods in CBlockPolicyEstimator without copying the documentation from the method declarations in interfaces::MempoolObserver.

    0    void processBlock(unsigned int nBlockHeight,
    1                      std::vector<const CTxMemPoolEntry*>& entries);
    2
    3    void processTransaction(const CTxMemPoolEntry& entry, bool validFeeEstimate);
    4
    5    /**
    6     * Removal is not inBlock as in-block removals are handled internally.
    7     */
    8    bool removeTx(uint256 hash);
    
  10. l2a5b1 commented at 10:27 am on August 14, 2018: contributor
    Concept ACK. Thanks @Empact!
  11. Empact force-pushed on Aug 14, 2018
  12. Empact commented at 10:08 pm on August 14, 2018: member
    Marked methods override and made the comments more appropriate
  13. Empact force-pushed on Aug 15, 2018
  14. Empact commented at 5:42 am on August 15, 2018: member
    Made the removeTx return type void and added interface arg comments.
  15. l2a5b1 commented at 12:07 pm on August 15, 2018: contributor

    utACK 56ec018 when squashed.

    One nit: I was wondering about the location of mempool_observer.h. If the file belongs in src/interfaces, where it currently resides, you may want to add an entry for the MempoolObserver interface to https://github.com/bitcoin/bitcoin/blob/master/src/interfaces/README.md, which contains a list of headers.

  16. Empact force-pushed on Aug 15, 2018
  17. Empact force-pushed on Aug 15, 2018
  18. Empact commented at 5:14 pm on August 15, 2018: member
    Thanks for calling out the readme. Fixed, rebased, and squashed.
  19. Empact force-pushed on Aug 15, 2018
  20. l2a5b1 commented at 6:21 pm on August 15, 2018: contributor
    yw, nice refactor! utACK f58e17e
  21. in src/policy/fees.cpp:512 in f58e17e386 outdated
    508@@ -509,6 +509,12 @@ void TxConfirmStats::removeTx(unsigned int entryHeight, unsigned int nBestSeenHe
    509 // tracked. Txs that were part of a block have already been removed in
    510 // processBlockTx to ensure they are never double tracked, but it is
    511 // of no harm to try to remove them again.
    512+void CBlockPolicyEstimator::removeTx(uint256 hash)
    


    practicalswift commented at 7:12 am on September 5, 2018:
    hash should be passed by const reference?
  22. Empact force-pushed on Sep 11, 2018
  23. Empact commented at 3:58 am on September 11, 2018: member
    Changed removeTx and processBlock to accept const hash references.
  24. MarcoFalke commented at 6:26 pm on March 25, 2019: member

    copy-paste from #15638 (review) by @ryanofsky:

    While I think in some cases fixing lint-circular-dependencies errors can make code organization better, it can just as easily make it worse.

    An example would be the circular dependency between txmempool and policy/fees. The fee estimation code needs access to the mempool entry struct, and the mempool code needs to call simple fee estimator add/remove functions when mempool entries get added and removed.

    The simple and obvious way to write this code is to declare the mempool entry struct in txmempool.h and the add/remove functions in fees.h and have a circular dependency between the two object files. As long as both object files are part of the same static library, this is perfectly fine and causes no problems. But if you want to appease the lint checker, you only have bad and mediocre options:

    • Combining files that shouldn’t be combined: merging fees and mempool files.
    • Splitting and scattering code around unnecessarily: declaring mempool entry struct in a separate file from the mempool struct.
    • Introducing unnecessary complexity: Registering fee estimator add/remove functions with the mempool at runtime, affecting runtime performance, and potentially introducing initialization bugs.

    It’s possible changes like these would be good for other reasons, but in general I think cleaning up spew from lint-circular-dependencies is not a good reason to make a code change if you can’t point to another actual reason for the change.

  25. ryanofsky commented at 7:08 pm on March 25, 2019: member

    re: #13949 (comment)

    Wow, I didn’t know about (or maybe forgot about?) this PR. Looking over this change, it may not be a bad idea if there are justifications for it other than fixing circular dependencies. In the pasted comment, I was actually complaining circular dependency errors in the context of #10443, not this PR.

  26. MarcoFalke commented at 7:16 pm on March 25, 2019: member
    This refactoring change is +80-17. Generally I’d prefer if refactoring simplified things, not made them harder
  27. Empact force-pushed on Apr 3, 2019
  28. Empact force-pushed on Apr 3, 2019
  29. Empact commented at 4:33 am on April 3, 2019: member

    Now +70 −16, with +43 coming from the interface header itself.

    I would argue this does make things simpler, by making CTxMemPool independent of CBlockPolicyEstimator. Both objects are now simpler because they both relate only to an abstract interface, which means there are no demands placed on either class apart from the capabilities and limitations of the interface.

    A practical benefit is that this makes CTxMemPool more flexible, for example it introduces the possibility of testing it indirectly via an observer object.

  30. l2a5b1 commented at 11:31 am on April 3, 2019: contributor
    utACK 8501bdc. The introduction of the MempoolObserver interface seems an elegant way to break the circular dependency.
  31. DrahtBot added the label Needs rebase on Apr 10, 2019
  32. Empact force-pushed on Jun 5, 2019
  33. DrahtBot removed the label Needs rebase on Jun 5, 2019
  34. DrahtBot added the label Needs rebase on Jun 6, 2019
  35. Extract MempoolObserver interface
    This is an abstract interface which presents removes the dependency between
    CBlockPolicyEstimator and CTxMemPool
    e3e91140fc
  36. Empact force-pushed on Jun 7, 2019
  37. DrahtBot removed the label Needs rebase on Jun 7, 2019
  38. DrahtBot added the label Needs rebase on Jul 3, 2019
  39. DrahtBot commented at 2:26 pm on July 3, 2019: member
  40. Empact commented at 7:41 pm on February 20, 2020: member
    Closing in favor of #17786
  41. Empact closed this on Feb 20, 2020

  42. Empact deleted the branch on Sep 7, 2020
  43. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me