bench: Expand mempool eviction benchmarking #27019

pull instagibbs wants to merge 1 commits into bitcoin:master from instagibbs:bench_eviction changing 1 files +79 −101
  1. instagibbs commented at 8:42 pm on February 1, 2023: member

    Took the existing benchmark and made is something more exhaustive, targeting eviction of 2500 transactions of different shapes. of dependencies.

    Magic number comes from: 100 children being RBF’d, each bumping 24 parents = 2500 transactions

    Turns out the benchmark ComplexMemPool already does something similar, so if we don’t want to expand it, maybe just remove it in favor of the existing better benchmark?

  2. DrahtBot commented at 8:42 pm on February 1, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process. A summary of reviews will appear here.

  3. maflcko renamed this:
    Expand eviction benchmarking
    bench: Expand eviction benchmarking
    on Feb 2, 2023
  4. DrahtBot added the label Tests on Feb 2, 2023
  5. in src/bench/mempool_eviction.cpp:43 in 1fed1e3b0a outdated
    80-    tx4.vout[0].nValue = 10 * COIN;
    81-    tx4.vout[1].scriptPubKey = CScript() << OP_4 << OP_EQUAL;
    82-    tx4.vout[1].nValue = 10 * COIN;
    83+            if (put_index < txns.size()) {
    84+                // We spend put_index's txn's in the next available slot for each previous transaction
    85+                tx.vin[put_index].prevout = COutPoint(txns[put_index]->GetHash(), put_index + txn_index - 1);
    


    maflcko commented at 8:27 am on February 2, 2023:
    0bench/mempool_eviction.cpp:42:105: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned long'
    

    instagibbs commented at 2:47 pm on February 2, 2023:
    found the issue, thanks
  6. bitcoin deleted a comment on Feb 2, 2023
  7. instagibbs force-pushed on Feb 2, 2023
  8. fanquake requested review from glozow on Mar 16, 2023
  9. in src/bench/mempool_eviction.cpp:71 in a3a2e82a6e outdated
    158+            AddTx(txns[i], (txns.size() - i)*1000LL, pool);
    159+        }
    160+        assert(pool.size() == txns.size());
    161+        pool.TrimToSize(pool.DynamicMemoryUsage() - 1);
    162+        assert(pool.size() == txns.size() - 1);
    163+        pool.TrimToSize(GetVirtualTransactionSize(*txns[0]));
    


    glozow commented at 6:19 pm on March 21, 2023:
    Why this size exactly - the first tx isn’t guaranteed to be the smallest or anything? Why not just 0?

    instagibbs commented at 7:48 pm on March 21, 2023:
    hmmm, I don’t recall. I can just set to 0
  10. in src/bench/mempool_eviction.cpp:27 in a3a2e82a6e outdated
    26-// unique transactions for a more meaningful performance measurement.
    27-static void MempoolEviction(benchmark::Bench& bench)
    28+void add_parents_child(std::vector<CTransactionRef>& txns, size_t package_size, size_t set_num)
    29+{
    30+    // N+1 N-input-N-output txns, in a parents-and-child package
    31+    // Where the final tx takes an input from all ancestors
    


    glozow commented at 6:29 pm on March 21, 2023:

    I could be reading this code wrong, but isn’t each tx taking inputs from each tx before it? It’s still parents-and-child technically but it’s also a chain.

    0    // Where each tx takes an input from all prior txns
    
  11. in src/bench/mempool_eviction.cpp:85 in a3a2e82a6e outdated
    173+
    174+    for (size_t i=0; i<num_packages; i++) {
    175+        add_parents_child(txns, package_size, i);
    176+    }
    177+
    178+    /* Now we're ready */
    


    glozow commented at 6:31 pm on March 21, 2023:
    Er, what does this comment mean?

    instagibbs commented at 7:54 pm on March 21, 2023:
    will remove

    glozow commented at 9:18 am on March 22, 2023:
    Didn’t remove?
  12. in src/bench/mempool_eviction.cpp:91 in a3a2e82a6e outdated
    187-        AddTx(tx6_r, 1100LL, pool);
    188-        AddTx(tx7_r, 9000LL, pool);
    189-        pool.TrimToSize(pool.DynamicMemoryUsage() * 3 / 4);
    190-        pool.TrimToSize(GetVirtualTransactionSize(*tx1_r));
    191+        for (size_t i=0; i<txns.size(); i++) {
    192+            // Monotonically decreasing fees
    


    glozow commented at 6:39 pm on March 21, 2023:
    Why? I don’t think it has any impact on the construction performance?

    instagibbs commented at 8:04 pm on March 21, 2023:
    looks like something that doesn’t matter, based on me getting rid of that factor. Removed.
  13. in src/bench/mempool_eviction.cpp:64 in a3a2e82a6e outdated
    151-    const CTransactionRef tx4_r{MakeTransactionRef(tx4)};
    152-    const CTransactionRef tx5_r{MakeTransactionRef(tx5)};
    153-    const CTransactionRef tx6_r{MakeTransactionRef(tx6)};
    154-    const CTransactionRef tx7_r{MakeTransactionRef(tx7)};
    155+    bench.run([&]() NO_THREAD_SAFETY_ANALYSIS {
    156+        for (size_t i=0; i<txns.size(); i++) {
    


    glozow commented at 6:42 pm on March 21, 2023:

    nit: I think we tend to prefer this

    0        for (size_t i{0}; i < txns.size(); ++i) {
    
  14. glozow commented at 6:45 pm on March 21, 2023: member
    Thanks for beefing up this bench! Basically lgtm, a few nits. Would like a bit clearer docs so future people can easily understand what the test is doing.
  15. jonatack commented at 6:59 pm on March 21, 2023: contributor
    Sorry for the drive-by suggestion: from the title, I thought this PR might be about the peer eviction benchmarks. Maybe update the title to s/eviction/mempool eviction/.
  16. instagibbs renamed this:
    bench: Expand eviction benchmarking
    bench: Expand mempool eviction benchmarking
    on Mar 21, 2023
  17. instagibbs commented at 7:01 pm on March 21, 2023: member
    @jonatack good point, updated
  18. Expand eviction benchmarking a6fc02baae
  19. instagibbs force-pushed on Mar 21, 2023
  20. instagibbs commented at 8:04 pm on March 21, 2023: member
    fixups included, squashed
  21. in src/bench/mempool_eviction.cpp:34 in a6fc02baae
    33+    const size_t num_txns = package_size;
    34+    for (size_t txn_index=0; txn_index<num_txns; txn_index++) {
    35+        CMutableTransaction tx = CMutableTransaction();
    36+        tx.vin.resize(num_puts);
    37+        tx.vout.resize(num_puts);
    38+        for (size_t put_index=0; put_index<num_puts; put_index++) {
    


    glozow commented at 9:18 am on March 22, 2023:

    Can you apply the changes to the other loops too? i.e.

    0        for (size_t put_index{0}; put_index < num_puts; ++put_index) {
    
  22. in src/bench/mempool_eviction.cpp:43 in a6fc02baae
    42+            tx.vout[put_index].nValue = txn_index;
    43+
    44+            if (put_index < txn_index) {
    45+                // We spend put_index's txn's in the next available slot for each previous transaction
    46+                assert(put_index + txn_index >= 1);
    47+                tx.vin[put_index].prevout = COutPoint(txns[put_index]->GetHash(), put_index + txn_index - 1);
    


    glozow commented at 9:26 am on March 22, 2023:

    Wait, if you’re using txns[put_index] and you’re using the same txns reference each time you call add_parents_child, aren’t you having all subsequent packages spend outputs from txns[0:24] too? That means all these packages conflict with each other.

    You could fix this by adding set_num * package_size or something to the index. But what would probably make a better interface is if add_parents_child returns the list of transactions it creates, and you append them to your larger list of packages.


    instagibbs commented at 2:07 pm on March 22, 2023:

    Hmmm, traced out a package size of 4, looks like I’m referencing non-existent outputs, not re-using them(maybe both :) ).

    Clearly this is confusing and broken either way.

    To take a step back, after doing this draft I realized https://github.com/bitcoin/bitcoin/blob/master/src/bench/mempool_stress.cpp exists. I’m fine if this doesn’t get merged if it doesn’t add anything new? Helped me learn about the benchmarks if nothing else. Thoughts?


    glozow commented at 2:12 pm on March 22, 2023:
    Ah true, I suppose ComplexMemPool is already benching eviction of packages. Fine with closing. The effort is appreciated :pray:

    instagibbs commented at 2:15 pm on March 22, 2023:
    I could change the PR to remove this silly bench.. :P
  23. instagibbs closed this on Mar 22, 2023

  24. bitcoin locked this on Mar 21, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 10:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me