bench: Add OrphanTxPool benchmark #19377

pull hebasto wants to merge 1 commits into bitcoin:master from hebasto:200624-m-bench changing 2 files +43 −0
  1. hebasto commented at 6:25 am on June 25, 2020: member

    This PR adds a new benchmark for adding/evicting orphan transactions to/from the mapOrphanTransactions.

    This benchmark could be useful while considering relevant code changes (e.g., #19374) or changing the DEFAULT_MAX_ORPHAN_TRANSACTIONS parameter.

    The output example:

    0$ ./src/bench/bench_bitcoin -filter="OrphanTxPool"
    1# Benchmark, evals, iterations, total, min, max, median
    2OrphanTxPool, 5, 10000, 2.3938, 4.76101e-05, 4.83722e-05, 4.77407e-0
    
  2. DrahtBot added the label Build system on Jun 25, 2020
  3. DrahtBot added the label P2P on Jun 25, 2020
  4. DrahtBot added the label Tests on Jun 25, 2020
  5. naumenkogs commented at 8:28 am on June 25, 2020: member
    utACK 059c8e6b31d55cad2ccf1dbb1cec0163cd58108d
  6. jnewbery commented at 3:16 pm on June 25, 2020: member
    @hebasto can you provide motivation for this change? In practice, only one orphan transactions is ever evicted at a time (in LimitOrphanTxSize() which is only called after the 101st orphan has been added. Picking a random element from a 101-entry container is always going to be fast, whatever the underlying data structure, so I don’t think there’s any benefit to having a microbenchmark for this. On the other hand, there is a cost to adding it - it means that there’s more work involved in changing or reviewing changes to the interface.
  7. hebasto commented at 3:30 pm on June 25, 2020: member

    @jnewbery

    @hebasto can you provide motivation for this change?

    It was motivated by discussion with @sipa in #19374.

  8. jnewbery commented at 4:19 pm on June 25, 2020: member
    I’m a -0.5 on this. I don’t think it’s worth adding a microbench for a different usage pattern than is used in the product, and for something that is always going to be quick.
  9. MarcoFalke commented at 4:42 pm on June 25, 2020: member

    microbenchmarks are never going to match the exact usage pattern in the end product. This is true for all existing microbenchmarks (like the mempool eviction one, etc …). Absent the details, I think having a benchmark is slightly preferable to not having a benchmark because it might catch accidental performance regressions.

    And since we don’t have any user-facing compatibility concerns for tests in general, adding and removing tests can be done much more easily than (let’s say) an RPC method. In general I think that the cost of a pull request that adds a test that passes can only be negative or null (when it is testing nothing). And it should always be an uncontroversial option to remove a not-too-useful test that is in the way of other changes/refactoring/features.

    I haven’t looked at the specifics of this pull request, but I think in general we need more benchmarks (or fuzzers) for p2p tx relay, as that is the primary DoS attack vector on the live network.

  10. DrahtBot commented at 5:14 pm on June 25, 2020: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  11. jnewbery commented at 5:57 pm on June 25, 2020: member

    In general I think that the cost of a pull request that adds a test that passes can only be negative or null (when it is testing nothing). And it should always be an uncontroversial option to remove a not-too-useful test that is in the way of other changes/refactoring/features.

    I’m sorry - this is wrong. All code added to the repo has a cost - in maintenance, in review, in discussion as to whether to remove it later. Tests/benches should only be added if they’re useful.

  12. in src/net_processing.cpp:1027 in 059c8e6b31 outdated
    1018@@ -1018,6 +1019,13 @@ unsigned int LimitOrphanTxSize(unsigned int nMaxOrphans)
    1019     return nEvicted;
    1020 }
    1021 
    1022+unsigned int LimitOrphanTxSize(unsigned int nMaxOrphans)
    1023+{
    1024+    LOCK(g_cs_orphans);
    1025+    SweepOutExpiredOrphans();
    1026+    return EvictOrphanTxs(nMaxOrphans);
    1027+}
    


    MarcoFalke commented at 6:13 pm on June 25, 2020:
    Why is net_processing changed in a pull that adds a benchmark? Seems unnecessary to me

    hebasto commented at 6:21 pm on June 25, 2020:

    To benchmark EvictOrphanTxs() without sweeping out functionality of the current LimitOrphanTxSize().

    What could you suggest to change?


    MarcoFalke commented at 8:44 pm on June 25, 2020:
    If you explicitly want to disable the time based eviction, it could be done with mocktime in the benchmark, no?

    hebasto commented at 7:03 am on June 26, 2020:
  13. in src/bench/orphan_tx_pool.cpp:17 in 059c8e6b31 outdated
    12+#include <script/script.h>
    13+#include <sync.h>
    14+
    15+#include <vector>
    16+
    17+static constexpr CAmount CENT{1000000};
    


    MarcoFalke commented at 6:14 pm on June 25, 2020:
    Why is this needed? values should not matter for performance

    hebasto commented at 7:03 am on June 26, 2020:
  14. in src/bench/orphan_tx_pool.cpp:19 in 059c8e6b31 outdated
    14+
    15+#include <vector>
    16+
    17+static constexpr CAmount CENT{1000000};
    18+extern bool AddOrphanTx(const CTransactionRef& tx, NodeId peer);
    19+extern unsigned int EvictOrphanTxs(unsigned int nMaxOrphans);
    


    MarcoFalke commented at 6:16 pm on June 25, 2020:
    Why is extern needed here?

    hebasto commented at 7:03 am on June 26, 2020:
  15. MarcoFalke commented at 6:18 pm on June 25, 2020: member
    Concept ACK. I think adding a benchmark for orphan handling is useful and needed
  16. hebasto force-pushed on Jun 26, 2020
  17. hebasto commented at 7:02 am on June 26, 2020: member

    Updated 059c8e6b31d55cad2ccf1dbb1cec0163cd58108d -> 2ee8c0b4be9e524f51572aba9ce4aafccbc0efef (pr19377.01 -> pr19377.02, diff):

  18. DrahtBot added the label Needs rebase on Jul 30, 2020
  19. bench: Add OrphanTxPool benchmark b0a5761286
  20. hebasto force-pushed on Aug 2, 2020
  21. hebasto commented at 8:25 pm on August 2, 2020: member
    Rebased 2ee8c0b4be9e524f51572aba9ce4aafccbc0efef -> b0a576128669c7ebe61a53a3c3a9d1e21c4c844e (pr19377.02 -> pr19377.03) due to the conflict with #18011.
  22. DrahtBot removed the label Needs rebase on Aug 3, 2020
  23. fanquake removed the label Build system on Apr 12, 2021
  24. in src/bench/orphan_tx_pool.cpp:37 in b0a5761286
    32+        tx.vout[0].scriptPubKey << OP_1;
    33+        txs.emplace_back(MakeTransactionRef(tx));
    34+    }
    35+
    36+    bench.batch(txs.size()).unit("byte").run([&] {
    37+        WITH_LOCK(g_cs_orphans, for (const auto& tx : txs) { AddOrphanTx(tx, 0); });
    


    jonatack commented at 5:52 pm on June 20, 2021:

    Concept ACK. Had a look as this “conflicts” with #22284, rebased it to current master and began building; it needs updating here and line 17 above:

    0bench/orphan_tx_pool.cpp:17:83: error: use of undeclared identifier 'g_cs_orphans'
    1bool AddOrphanTx(const CTransactionRef& tx, NodeId peer) EXCLUSIVE_LOCKS_REQUIRED(g_cs_orphans);
    2                                                                                  ^
    3bench/orphan_tx_pool.cpp:37:19: error: use of undeclared identifier 'g_cs_orphans'
    4        WITH_LOCK(g_cs_orphans, for (const auto& tx : txs) { AddOrphanTx(tx, 0); });
    5                  ^
    6bench/orphan_tx_pool.cpp:37:19: error: use of undeclared identifier 'g_cs_orphans'
    73 errors generated.
    
  25. jnewbery commented at 7:07 pm on June 22, 2021: member

    Had a look as this “conflicts” with #22284, rebased it to current master and began building; it needs updating here and line 17 above:

    This also conflicts with #21527, which is blocking quite a lot of important work in net_processing.

  26. hebasto marked this as a draft on Jun 23, 2021
  27. hebasto commented at 0:20 am on June 23, 2021: member
    Marked this as a “draft” for now.
  28. DrahtBot added the label Needs rebase on Dec 13, 2021
  29. DrahtBot commented at 11:19 pm on December 13, 2021: member

    🐙 This pull request conflicts with the target branch and needs rebase.

    Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a “draft”.

  30. DrahtBot commented at 1:07 pm on March 21, 2022: member
    • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
    • Is it no longer relevant? ➡️ Please close.
    • Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
  31. hebasto closed this on Apr 21, 2022

  32. DrahtBot locked this on Apr 21, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-08 22:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me