test: Fix bug in transaction generation in ComplexMempool benchmark #22856

pull shoryak wants to merge 1 commits into bitcoin:master from shoryak:complexmempool changing 1 files +10 −8
  1. shoryak commented at 5:53 pm on September 1, 2021: contributor

    This fixes issues with ComplexMempool benchmark introduced in #17292 , this stress test benchmarks performance of ancestor and descendant tracking of mempool graph algorithms on a complex Mempool.

    This Benchmark first creates 100 base transactions and stores them in available_coins vector. available_coins is used for selecting ancestor transactions while creating 800 new transactions. For this a random transaction is picked from available_coins and some of its outputs are mapped to the inputs of the new transaction being created.

    Now in case we exhaust all the outputs of an entry in available_coins then we need to remove it from available_coins before the next iteration of choosing a potential ancestor , it is now implemented with this patch.

    As the index of the entry is randomly chosen from available_coins , In order to remove it from the vector , if index of the selected entry is not at the end of available_coins vector , it is swapped with the entry at the back of the vector , then the entry at the end of available_coins is popped out.

    Earlier the code responsible for constructing outputs of the newly created transaction was inside the loop used for assigning ancestors to the transaction , which does some unnecessary work as it creates outputs of the transaction again and again , now it is moved out of the loop so outputs of the transaction are created just once before adding it to the final list of the transactions created. This one is a minor change to save some computation.

    These changes have changed the ComplexMempool benchmark results on bitcoin:master as follows :

    Before

    ns/op op/s err% total benchmark
    232,881,625.00 4.29 0.7% 2.55 ComplexMemPool

    After

    ns/op op/s err% total benchmark
    497,275,135.00 2.01 0.5% 5.49 ComplexMemPool
  2. shoryak force-pushed on Sep 1, 2021
  3. DrahtBot added the label Tests on Sep 1, 2021
  4. fanquake requested review from JeremyRubin on Sep 2, 2021
  5. fanquake commented at 2:20 am on September 2, 2021: member
    Please modify the commit message to actually explain the changes.
  6. in src/bench/mempool_stress.cpp:71 in 8ae1334314 outdated
    65@@ -66,15 +66,17 @@ static void ComplexMemPool(benchmark::Bench& bench)
    66                 tx.vin.back().scriptSig = CScript() << coin.tx_count;
    67                 tx.vin.back().scriptWitness.stack.push_back(CScriptNum(coin.tx_count).getvch());
    68             }
    69-            if (coin.vin_left == coin.ref->vin.size()) {
    70-                coin = available_coins.back();
    71+            if (coin.vin_left == coin.ref->vout.size()) {
    72+                if(available_coins.size()-1!=idx){ // if idx is not the last index swap it with the end index
    73+                   std::swap(available_coins[idx], available_coins[available_coins.size()-1]);
    


    JeremyRubin commented at 5:16 pm on September 2, 2021:
    can use available_coins.back() here too.
  7. in src/bench/mempool_stress.cpp:66 in 8ae1334314 outdated
    65@@ -66,15 +66,17 @@ static void ComplexMemPool(benchmark::Bench& bench)
    66                 tx.vin.back().scriptSig = CScript() << coin.tx_count;
    


    JeremyRubin commented at 5:21 pm on September 2, 2021:
    on line 59 above, should Available also be made a reference? Otherwise the modifications to the coin would not be recorded since it’s a copy?

    shoryak commented at 2:47 am on September 3, 2021:

    Oops My bad , yes Available needs to be a reference , fixing it changes the benchmark results substantially .

    ns/op op/s err% total benchmark
    1,103,652,315.00 0.91 2.6% 13.34 ComplexMemPool
  8. shoryak force-pushed on Sep 3, 2021
  9. shoryak force-pushed on Sep 4, 2021
  10. shoryak force-pushed on Sep 4, 2021
  11. shoryak force-pushed on Sep 4, 2021
  12. Fixes Bug in Transaction generation in ComplexMempool benchmark
    Available in line 59 is made a reference , so contents of the coin can be modified
    
    While generating transactions we select ancestors from available_coins ,in case we exhaust all the outputs of an entry in available_coins
    then we need to remove it from available_coins before the next iteration of choosing a potential ancestor , it is now implemented with
    this patch by ,As the index of the entry is randomly chosen from available_coins , In order to remove it from the vector if index of the
    selected entry is not at the end of available_coins vector, it is swapped with the entry at the back of the vector , then the entry at the
    end of available_coins is popped out.
    
    Code generating outputs for the transaction is moved out of the loop, as it needs to be done only once before adding the transaction to ordered_coins
    29e983386b
  13. shoryak force-pushed on Sep 4, 2021
  14. laanwj renamed this:
    Modifications in ComplexMempool benchmark
    test: Fix bug in transaction generation in ComplexMempool benchmark
    on Sep 16, 2021
  15. theStack commented at 10:03 pm on November 10, 2021: member

    Concept ACK

    If I understand correctly, this PR actually contains two bug fixes (remove correct element from available_coins, make modification to available_coins[idx] effective by declaring coin as reference) and one performance improvement?

    Would maybe be worth splitting up into two commits. Also the commit body looks a bit messy currently (too long lines, confusing interpunction, grammar). In the course of doing that, a rebase on master couldn’t hurt – I am actually surprised that there is no merge conflict, since #23157 generalized the CreateOrderedCoins helper function.

  16. JeremyRubin commented at 10:13 pm on November 10, 2021: contributor

    i’m not sure if @shoryak is still watching bitcoin core stuff since the semester is quite busy and this was a SoB project…

    i think it’d be fine to merge as is?

  17. laanwj commented at 4:26 pm on November 26, 2021: member

    That’s a valid reason to no longer work on it, but “author doesn’t have time” is not a valid reason for something to be merged while ignoring reviewer comments. Normally we’d close and label “up for grabs”.

    But in this particular case, yes, it’s probably fine like this. @theStack do you agree?

  18. theStack commented at 7:09 pm on November 26, 2021: member
    @laanwj: My concerns were mainly considering aesthetics (commit message formatting etc.), but the code change itself looks correct to me, so yes, I agree 👍
  19. shoryak commented at 8:37 am on December 2, 2021: contributor
    @theStack I would be happy to make any changes in the formatting of the commit message, though I prefer it to be just a single commit rather than two.
  20. MarcoFalke merged this on Dec 7, 2021
  21. MarcoFalke closed this on Dec 7, 2021

  22. MarcoFalke commented at 12:19 pm on December 7, 2021: member
    • /usr/bin/time --format='%M' ./src/bench/bench_bitcoin --filter=ComplexMemPool Before: 57016 After: 56532

    • /usr/bin/time --format='%M' valgrind ./src/bench/bench_bitcoin --filter=ComplexMemPool Before: 340496 After: 340420

  23. in src/bench/mempool_stress.cpp:62 in 29e983386b
    55@@ -56,7 +56,7 @@ static void ComplexMemPool(benchmark::Bench& bench)
    56         size_t n_ancestors = det_rand.randrange(10)+1;
    57         for (size_t ancestor = 0; ancestor < n_ancestors && !available_coins.empty(); ++ancestor){
    58             size_t idx = det_rand.randrange(available_coins.size());
    59-            Available coin = available_coins[idx];
    60+            Available& coin = available_coins[idx];
    61             uint256 hash = coin.ref->GetHash();
    62             // biased towards taking just one ancestor, but maybe more
    63             size_t n_to_take = det_rand.randrange(2) == 0 ? 1 : 1+det_rand.randrange(coin.ref->vout.size() - coin.vin_left);
    


    MarcoFalke commented at 2:01 pm on December 7, 2021:
    bench/mempool_stress.cpp:59:89: runtime error: unsigned integer overflow: 9 - 11 cannot be represented in type ‘unsigned long’

    MarcoFalke commented at 2:04 pm on December 7, 2021:
    I’ve reverted the merge in commit 95fe477fd189ae30e76050b95280086d913e78c2 again due to this

    JeremyRubin commented at 4:35 pm on December 7, 2021:
    interesting – why was it passing here but not on master?

    MarcoFalke commented at 5:02 pm on December 7, 2021:
    we changed from clang-12 to clang-13, maybe?
  24. sidhujag referenced this in commit a01792c0f2 on Dec 7, 2021
  25. RandyMcMillan referenced this in commit 7283e2c744 on Dec 23, 2021
  26. PastaPastaPasta referenced this in commit 6d6f644c78 on Apr 7, 2022
  27. PastaPastaPasta referenced this in commit 31fe43ecdd on Apr 7, 2022
  28. PastaPastaPasta referenced this in commit 3ed1cf0bf9 on Apr 7, 2022
  29. PastaPastaPasta referenced this in commit 0d1c8f914c on Apr 11, 2022
  30. DrahtBot locked this on Dec 7, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-30 00:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me