Basic CCheckQueue Benchmarks

JeremyRubin commented at 7:06 pm on January 9, 2017: contributor

Some basic benchmarks for the CCheckQueue, for future work to compare to.

Add Basic CheckQueue Benchmark 9f03110f32

JeremyRubin force-pushed on Jan 9, 2017

fanquake added the label Tests on Jan 9, 2017

in src/bench/checkqueue.cpp: in 9f03110f32 outdated

14+
15+// This Benchmark tests the CheckQueue with the lightest
16+// weight Checks, so it should make any lock contention
17+// particularly visible
18+static void CCheckQueueSpeed(benchmark::State& state)
19+{

ryanofsky commented at 9:00 pm on January 11, 2017:

Maybe declare some constants here with the various magic numbers & things that could be tuned in the benchmark, like size of the queue, number of threads, number of jobs to add, number of times to call add, etc.

in src/bench/checkqueue.cpp: in 9f03110f32 outdated

30+       tg.create_thread([&]{queue.Thread();});
31+    }
32+    while (state.KeepRunning()) {
33+        CCheckQueueControl<FakeJobNoWork> control(&queue);
34+        // We can make vChecks out of the loop because calling Add doesn't
35+        // change the size of the vector.

ryanofsky commented at 9:05 pm on January 11, 2017:

Maybe add an assert to the end of the benchmark to check this? It’s be easy to imagine the vector being moved from in the future, which would invalidate the benchmark.

ryanofsky commented at 9:11 pm on January 11, 2017:

Also, if this is true, any reason not to move vChecks declaration out of the while loop too?

in src/bench/checkqueue.cpp: in 9f03110f32 outdated

32+    while (state.KeepRunning()) {
33+        CCheckQueueControl<FakeJobNoWork> control(&queue);
34+        // We can make vChecks out of the loop because calling Add doesn't
35+        // change the size of the vector.
36+        std::vector<FakeJobNoWork> vChecks;
37+        vChecks.resize(30);

ryanofsky commented at 9:13 pm on January 11, 2017:

Could call the vector constructor taking a size to eliminate the resize line.

in src/bench/checkqueue.cpp: in 9f03110f32 outdated

72+    for (auto x = 0; x < std::max(2, GetNumCores()); ++x) {
73+       tg.create_thread([&]{queue.Thread();});
74+    }
75+    while (state.KeepRunning()) {
76+        // Make insecure_rand here so that each iteration is identical.
77+        FastRandomContext insecure_rand(true);

ryanofsky commented at 9:18 pm on January 11, 2017:

Maybe move this outside while loop, if whatever time this takes isn’t meaningful to the benchmark.

JeremyRubin commented at 9:38 pm on January 11, 2017:

It is meaningful for each iteration to be identical.

ryanofsky commented at 11:19 pm on January 11, 2017:

Oops, should have read the comment above. Though maybe I would reword it to something like “Start with a fresh insecure_rand here” rather than “Make insecure_rand here.”

in src/bench/checkqueue.cpp: in 9f03110f32 outdated

57+    struct PrevectorJob {
58+        prevector<28, uint8_t> p;
59+        PrevectorJob(){
60+        }
61+        PrevectorJob(FastRandomContext& insecure_rand){
62+            p.resize(insecure_rand.rand32() % 56);

ryanofsky commented at 9:24 pm on January 11, 2017:

Why do this in the constructor instead of the call operator? Wouldn’t it be more realistic to have the work done in the worker threads instead of the control thread? Maybe add a comment if this is the intended behavior.

JeremyRubin commented at 9:40 pm on January 11, 2017:

FastRandomContext is not thread safe.

Also we do want the main thread to be doing some work while the worker threads process.

ryanofsky approved

ryanofsky commented at 9:35 pm on January 11, 2017: member

utACK 9f03110f3256e81173c65c2ed1810efaa74fbc5b. Looks fine, just minor comments.

Address ryanofsky feedback on CCheckQueue benchmarks. Eliminated magic numbers, fixed scoping of vectors (and memory movement component of benchmark). aad4cb5059

ryanofsky approved

ryanofsky commented at 11:20 pm on January 11, 2017: member

utACK aad4cb50599b15d6fc439e465a616635e96466d1

fanquake commented at 1:59 am on January 12, 2017: member

0CCheckQueueSpeed,576,0.001812756061554,0.001984249800444,0.001944298545520,6149498,6731089,6595898
1CCheckQueueSpeed,576,0.001704573631287,0.001963026821613,0.001897984494766,5782270,6659114,6438475
2CCheckQueueSpeed,576,0.001838877797127,0.001967173069715,0.001926153070397,6237919,6675994,6534347
3CCheckQueueSpeed,576,0.001697123050690,0.001930065453053,0.001841966890626,5757193,6547343,6248762
4CCheckQueueSpeed,576,0.001837179064751,0.001974612474442,0.001906548523241,6232267,6698444,6467841

0CCheckQueueSpeedPrevectorJob,448,0.002283558249474,0.002333313226700,0.002298991062811,7746465,7915155,7799211
1CCheckQueueSpeedPrevectorJob,448,0.002310715615749,0.002362936735153,0.002330580460174,7838558,8015741,7906361
2CCheckQueueSpeedPrevectorJob,448,0.002285875380039,0.002489343285561,0.002338334918022,7754281,8444561,7932261
3CCheckQueueSpeedPrevectorJob,448,0.002281628549099,0.002457186579704,0.002311725169420,7739965,8335454,7842407
4CCheckQueueSpeedPrevectorJob,448,0.002286218106747,0.002332687377930,0.002305187284946,7755492,7913158,7820222

laanwj commented at 2:29 pm on January 19, 2017: member

utack aad4cb5

laanwj merged this on Jan 19, 2017

laanwj closed this on Jan 19, 2017

laanwj referenced this in commit 054d664215 on Jan 19, 2017

codablock referenced this in commit 9e17b0dfc1 on Jan 19, 2018

codablock referenced this in commit 2ef2986b8c on Jan 20, 2018

codablock referenced this in commit a84f8da136 on Jan 21, 2018

andvgal referenced this in commit cd5359b883 on Jan 6, 2019

CryptoCentric referenced this in commit 14864c57f3 on Feb 27, 2019

zkbot referenced this in commit aa225ebb0b on Jan 24, 2020

zkbot referenced this in commit 74ff73abab on Jan 24, 2020

furszy referenced this in commit 4ed15cc69d on Jun 8, 2020

MarcoFalke locked this on Sep 8, 2021

Basic CCheckQueue Benchmarks #9498