cluster mempool: exploit SFL properties in txgraph #34085

pull sipa wants to merge 5 commits into bitcoin:master from sipa:202512_txgraph_sfl changing 4 files +93 −179
  1. sipa commented at 9:21 pm on December 16, 2025: member

    Part of #30289, follow-up to #32545.

    This gets rid of FixLinearization() by integrating the functionality into Linearize(), and makes txgraph exploit that (by delaying fixing of clusters until their first re-linearization). It also reduces (but does not eliminate) the number of calls to PostLinearize, as the SFL linearization effectively performs something very similar to postlinearization when loading in an existing linearization already.

  2. DrahtBot commented at 9:21 pm on December 16, 2025: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/34085.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Stale ACK instagibbs

    If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #34138 (Cluster mempool: more accurate cost model for SFL by sipa)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • // Finally set the cluster to NEEDS_FIX, so its linearization is fixed the next time its is // attempted to be made ACCEPTABLE.
      • its is -> it is [double “its” is incorrect; should read “it is attempted” for correct grammar and clarity]

    2025-12-29

  3. sipa force-pushed on Dec 19, 2025
  4. sipa marked this as ready for review on Dec 19, 2025
  5. fanquake requested review from instagibbs on Dec 19, 2025
  6. fanquake requested review from marcofleon on Dec 19, 2025
  7. sipa commented at 3:21 pm on December 19, 2025: member
    Rebased after merge of #32545, and marked as ready for review.
  8. in src/cluster_linearize.h:1579 in a5067dbc7f outdated
    1578@@ -1577,32 +1579,10 @@ void PostLinearize(const DepGraph<SetType>& depgraph, std::span<DepGraphIndex> l
    1579 template<typename SetType>
    


    instagibbs commented at 4:57 pm on December 19, 2025:

    a5067dbc7f5189848e02574f8f889c5a26fd3ceb

    retaining its ordering where possible

    micro nit: I know it’s going to get nuked later, but seems misleading at this commit IIUC


    sipa commented at 10:09 pm on December 29, 2025:
    Fixed.
  9. in src/txgraph.cpp:1189 in 1b2c53ae0c outdated
    1193-            quality = QualityLevel::NEEDS_SPLIT_ACCEPTABLE;
    1194-        } else {
    1195-            quality = QualityLevel::NEEDS_SPLIT;
    1196-        }
    1197-    } else {
    1198+    if (todo.Any()) {
    


    instagibbs commented at 5:24 pm on December 19, 2025:

    1b2c53ae0c9020791d39c85b032f7f907b468c99

    Could we go another step further and delete the

    // First remove all removals at the end of the linearization

    Stuff, drop this specific conditional and just filter or is this still a performance win


    sipa commented at 10:10 pm on December 29, 2025:
    Yeah, I was wondering about this too. I opted not to change it to minimize the diff, but you’re right, it’s better to simplify. Done.
  10. in src/txgraph.cpp:3269 in 28ff1be7b5 outdated
    3272@@ -3253,6 +3273,7 @@ std::vector<TxGraph::Ref*> TxGraphImpl::Trim() noexcept
    3273                                 .subspan(group_data.m_cluster_offset, group_data.m_cluster_count);
    3274         uint64_t size{0};
    3275         for (Cluster* cluster : cluster_span) {
    3276+            MakeAcceptable(*cluster, cluster->GetLevel(*this));
    


    instagibbs commented at 6:41 pm on December 19, 2025:

    28ff1be7b5cae0f0674e0d0045da0eee5e9adc6a

    Let’s me be more confident the cluster wasnt in a needs split state or something.

    0            MakeAcceptable(*cluster, cluster->GetLevel(*this));
    1            Assume(cluster->IsAcceptable());
    

    sipa commented at 10:10 pm on December 29, 2025:
    I’ve added this inside GenericClusterImpl::AppendTrimData(), as I think it may otherwise result in a non-optimizable dynamic call. Ok?
  11. in src/txgraph.cpp:1487 in ad895c9fd7 outdated
    1485@@ -1494,7 +1486,6 @@ void GenericClusterImpl::ApplyDependencies(TxGraphImpl& graph, int level, std::s
    1486     // Finally fix the linearization, as the new dependencies may have invalidated the
    1487     // linearization, and post-linearize it to fix up the worst problems with it.
    


    instagibbs commented at 7:13 pm on December 19, 2025:

    ad895c9fd7f70e435aa7e478a8054c31f7b2d2e3

    Stale comment about “post-linearize”


    instagibbs commented at 7:14 pm on December 19, 2025:

    28ff1be7b5cae0f0674e0d0045da0eee5e9adc6a

    stale comment about fixing + post-linearization


    sipa commented at 10:11 pm on December 29, 2025:
    Thanks, fixed.
  12. instagibbs approved
  13. instagibbs commented at 4:02 pm on December 29, 2025: member

    ACK 159bd23bcc8dca4d5e7029ee25c312c4ef78b0eb

    Only a few suggestions

  14. fanquake added this to the milestone 31.0 on Dec 29, 2025
  15. clusterlin: support fixing linearizations (feature)
    This also updates FixLinearization to just be a thin wrapper around Linearize.
    In a future commit, FixLinearization will be removed entirely.
    123f2e22fb
  16. txgraph: drop NEEDS_SPLIT_ACCEPTABLE (simplification)
    With the SFL algorithm, we will practically be capable of keeping
    most if not all clusters optimal. With that, it seems less valuable
    to avoid doing work after splitting an acceptable cluster, because by
    doing some work we may get it to OPTIMAL.
    
    This reduces the complexity of the code a bit as well.
    701ee7adc8
  17. txgraph: use PostLinearize less prior to linearizing
    With the new SFL algorithm, the process of loading an existing linearization into the
    SFL state is very similar to what PostLinearize does. This means there is little benefit
    to performing an explicit PostLinearize step before linearizing inside txgraph. Instead,
    it seems better to use our allotted CPU time to perform more SFL optimization steps.
    e5f455e730
  18. txgraph: permit non-topological clusters to defer fixing (optimization) 6f0e4ac334
  19. clusterlin: remove unused FixLinearization (cleanup) d9fd076f02
  20. sipa force-pushed on Dec 29, 2025


sipa DrahtBot instagibbs


marcofleon

Milestone
31.0


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-12-30 12:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me