bench: use std::chrono rather than gettimeofday #11562

pull theuni wants to merge 2 commits into bitcoin:master from theuni:bench-clock-chrono changing 3 files +38 −29
  1. theuni commented at 9:09 PM on October 25, 2017: member

    gettimeofday has portability issues, see for example #11558.

    Regardless of large-scale clock refactors in the future, I think it's fine for bench to just use std::chrono itself.

    Note that this may slightly improve bench accuracy and changes the display from tiny floats to nanosecond counts instead.

  2. practicalswift commented at 9:19 PM on October 25, 2017: contributor

    Concept ACK

  3. fanquake added the label Tests on Oct 25, 2017
  4. MarcoFalke commented at 10:33 PM on October 25, 2017: member

    Concept ACK. This is what the google bench does.

  5. fanquake commented at 12:23 AM on October 26, 2017: member

    master https://github.com/bitcoin/bitcoin/commit/57ee73990f1ce29916adfd99f93eae1ccea1a43b

    src/bench/bench_bitcoin
#Benchmark,count,min,max,average,min_cycles,max_cycles,average_cycles
Base58CheckEncode,294912,0.000003700712114,0.000003791210474,0.000003750498258,12553,12860,12723
Base58Decode,1048576,0.000000986561645,0.000001043212251,0.000001009610287,3346,3542,3425
Base58Encode,425984,0.000002401517122,0.000002516913810,0.000002463796521,8146,8538,8357
BenchLockedPool,640,0.001663804054260,0.001768421381712,0.001725013926625,5644197,5998992,5852070
CCheckQueueSpeed,512,0.001896254718304,0.002109408378601,0.001992908306420,6432527,7155741,6760920
CCheckQueueSpeedPrevectorJob,448,0.002330623567104,0.002476744353771,0.002359484455415,7906058,8401791,8004012
CCoinsCaching,122880,0.000008048693417,0.000008630100638,0.000008307983323,27303,29275,28184
CoinSelection,576,0.001785077154636,0.001916944980621,0.001805677182145,6055513,6502807,6125742
DeserializeAndCheckBlockTest,112,0.008977383375168,0.009968042373657,0.009116992354393,30453379,33814404,30927281
DeserializeBlockTest,144,0.007569313049316,0.007755517959595,0.007618673973613,25677256,26308849,25846183
FastRandom_1bit,640,0.001670766621828,0.001709744334221,0.001680362224579,5671193,5799963,5700593
FastRandom_32bit,112,0.009404510259628,0.009604752063751,0.009498813322612,31902396,32581996,32222551
MempoolEviction,24576,0.000041037099436,0.000047371606342,0.000042666749020,139209,160698,144746
PrevectorClear,5120,0.000209093093872,0.000216422602534,0.000211418559775,709305,734155,717233
PrevectorDestructor,5120,0.000199727714062,0.000203179195523,0.000201328517869,677510,689255,682960
RIPEMD160,416,0.002455778419971,0.002532064914703,0.002477367910055,8330696,8589435,8404447
RollingBloom-refresh,1,0.000129,0.000129,0.000129
RollingBloom-refresh,1,0.000111,0.000111,0.000111
RollingBloom-refresh,1,0.000129,0.000129,0.000129
RollingBloom-refresh,1,0.000112,0.000112,0.000112
RollingBloom-refresh,1,0.000113,0.000113,0.000113
RollingBloom-refresh,1,0.000112,0.000112,0.000112
RollingBloom-refresh,1,0.000111,0.000111,0.000111
RollingBloom-refresh,1,0.000113,0.000113,0.000113
RollingBloom-refresh,1,0.000111,0.000111,0.000111
RollingBloom-refresh,1,0.000123,0.000123,0.000123
RollingBloom-refresh,1,0.000108,0.000108,0.000108
RollingBloom-refresh,1,0.000127,0.000127,0.000127
RollingBloom-refresh,1,0.000112,0.000112,0.000112
RollingBloom-refresh,1,0.000112,0.000112,0.000112
RollingBloom-refresh,1,0.000113,0.000113,0.000113
RollingBloom-refresh,1,0.000111,0.000111,0.000111
RollingBloom-refresh,1,0.000117,0.000117,0.000117
RollingBloom-refresh,1,0.000112,0.000112,0.000112
RollingBloom-refresh,1,0.000109,0.000109,0.000109
RollingBloom-refresh,1,0.00011,0.00011,0.00011
RollingBloom-refresh,1,0.000108,0.000108,0.000108
RollingBloom-refresh,1,0.000112,0.000112,0.000112
RollingBloom-refresh,1,0.000112,0.000112,0.000112
RollingBloom-refresh,1,0.00011,0.00011,0.00011
RollingBloom-refresh,1,0.000109,0.000109,0.000109
RollingBloom-refresh,1,0.000113,0.000113,0.000113
RollingBloom,1572864,0.000000646410626,0.000000685669875,0.000000676037568,2192,2327,2293
SHA1,576,0.001821309328079,0.001895938068628,0.001866892394092,6178301,6431563,6333002
SHA256,352,0.002996556460857,0.003085255622864,0.003031289712949,10165221,10466029,10283598
SHA256_32b,6,0.215636968612671,0.217754483222961,0.216444651285807,731612744,738682787,734277071
SHA512,384,0.002636194229126,0.002715311944485,0.002677739908298,8942669,9211179,9083624
SipHash_32b,28,0.036945462226868,0.039195537567139,0.037574112415314,125329483,132961911,127469598
Sleep100ms,10,0.102285981178284,0.104399442672729,0.102977490425110,347096871,354150647,349350507
Trig,67108864,0.000000015336752,0.000000016048432,0.000000015816315,52,54,53
VerifyScriptBench,6144,0.000164830125868,0.000173246022314,0.000169373194998,559145,587697,574596

    This PR https://github.com/bitcoin/bitcoin/pull/11562/commits/e2dc48c3e5f70c75b762be656e0c9419b2b0208f

    src/bench/bench_bitcoin
#Benchmark,count,min,max,average,min_cycles,max_cycles,average_cycles. Clock precision: nanosecond
Base58CheckEncode,262144,3873ns,4485ns,4025ns,13138,15215,13655
Base58Decode,983040,1022ns,1089ns,1042ns,3467,3694,3536
Base58Encode,393216,2509ns,2694ns,2568ns,8511,9140,8712
BenchLockedPool,576,1711427ns,1831078ns,1761460ns,5805626,6211513,5975351
CCheckQueueSpeed,576,1894807ns,1987571ns,1943943ns,6427702,6742379,6594382
CCheckQueueSpeedPrevectorJob,416,2375311ns,2435702ns,2410998ns,8057696,8262562,8178757
CCoinsCaching,122880,8041ns,8954ns,8476ns,27277,30374,28754
CoinSelection,576,1883259ns,1920119ns,1896561ns,6388520,6513567,6433648
DeserializeAndCheckBlockTest,112,9069196ns,10666107ns,9416574ns,30765154,36182297,31943563
DeserializeBlockTest,128,7655800ns,8043978ns,7823370ns,25970543,27287306,26538987
FastRandom_1bit,640,1677213ns,1759200ns,1725739ns,5689570,5967687,5854173
FastRandom_32bit,104,9361699ns,10403821ns,9620188ns,31757436,35292568,32634282
MempoolEviction,24576,41403ns,43570ns,42427ns,140452,147802,143927
PrevectorClear,5120,208818ns,234129ns,215746ns,708367,794231,731869
PrevectorDestructor,4608,217063ns,228702ns,221567ns,736337,775821,751618
RIPEMD160,416,2491869ns,2678229ns,2566195ns,8453095,9085277,8705227
RollingBloom-refresh,1,112950,112950,112950
RollingBloom-refresh,1,109470,109470,109470
RollingBloom-refresh,1,119208,119208,119208
RollingBloom-refresh,1,108423,108423,108423
RollingBloom-refresh,1,110924,110924,110924
RollingBloom-refresh,1,107718,107718,107718
RollingBloom-refresh,1,104799,104799,104799
RollingBloom-refresh,1,107850,107850,107850
RollingBloom-refresh,1,236540,236540,236540
RollingBloom-refresh,1,110421,110421,110421
RollingBloom-refresh,1,107586,107586,107586
RollingBloom-refresh,1,105012,105012,105012
RollingBloom-refresh,1,135634,135634,135634
RollingBloom-refresh,1,121365,121365,121365
RollingBloom-refresh,1,105733,105733,105733
RollingBloom-refresh,1,107545,107545,107545
RollingBloom-refresh,1,107570,107570,107570
RollingBloom-refresh,1,113778,113778,113778
RollingBloom-refresh,1,112316,112316,112316
RollingBloom-refresh,1,107881,107881,107881
RollingBloom-refresh,1,108146,108146,108146
RollingBloom-refresh,1,107903,107903,107903
RollingBloom-refresh,1,110589,110589,110589
RollingBloom-refresh,1,107555,107555,107555
RollingBloom,1441792,679ns,742ns,711ns,2305,2518,2414
SHA1,576,1845302ns,1918492ns,1887662ns,6259766,6508049,6403460
SHA256,352,3038257ns,3252807ns,3120301ns,10306589,11034401,10584908
SHA256_32b,6,220640728ns,226463297ns,222662923ns,748472847,768224917,755332837
SHA512,384,2663500ns,3062115ns,2776882ns,9035312,10387520,9419936
SipHash_32b,28,36830074ns,39666791ns,38085206ns,124937632,134560472,129195322
Sleep100ms,10,101757668ns,103232076ns,102764599ns,345189224,350191554,348605289
Trig,67108864,15ns,16ns,15ns,52,57,53
VerifyScriptBench,6144,166119ns,174340ns,169195ns,563523,591412,573955
  6. laanwj commented at 2:52 PM on October 26, 2017: member

    Does this also use a monotonic clock? That's another issue with using plain gettimeofday for benchmarks.

  7. theuni commented at 4:38 PM on October 26, 2017: member

    @laanwj Yes, if the resolution is no worse than the high precision clock. See the last commit: https://github.com/bitcoin/bitcoin/pull/11562/commits/e2dc48c3e5f70c75b762be656e0c9419b2b0208f

  8. laanwj commented at 7:00 PM on October 26, 2017: member

    @laanwj Yes, if the resolution is no worse than the high precision clock. See the last commit: e2dc48c

    Nice! concept ACK

  9. martinus commented at 7:28 AM on October 27, 2017: contributor

    I've also switched to std::chrono in https://github.com/bitcoin/bitcoin/pull/11517

  10. in src/bench/bench.cpp:38 in e2dc48c3e5 outdated 
      42 | +benchmark::BenchRunner::RunAll(benchmark::duration elapsedTimeForOne)
  43 |  {
  44 |      perf_init();
  45 |      std::cout << "#Benchmark" << "," << "count" << "," << "min" << "," << "max" << "," << "average" << ","
  46 | -              << "min_cycles" << "," << "max_cycles" << "," << "average_cycles" << "\n";
  47 | +              << "min_cycles" << "," << "max_cycles" << "," << "average_cycles. " << "Clock precision: "
    laanwj commented at 10:21 AM on November 6, 2017:

    Please use a ,, to separate here, instead of a dot, otherwise programs will get confused re: CSV parsing.

  11. in src/bench/bench.cpp:109 in e2dc48c3e5 outdated 
     106 | +    int64_t min_elapsed = std::chrono::duration_cast<std::chrono::nanoseconds>(minTime).count();
 107 | +    int64_t max_elapsed = std::chrono::duration_cast<std::chrono::nanoseconds>(maxTime).count();
 108 | +    int64_t avg_elapsed = std::chrono::duration_cast<std::chrono::nanoseconds>((now-beginTime)/count).count();
 109 |      int64_t averageCycles = (nowCycles-beginCycles)/count;
 110 | -    std::cout << std::fixed << std::setprecision(15) << name << "," << count << "," << minTime << "," << maxTime << "," << average << ","
 111 | +    std::cout << std::fixed << std::setprecision(15) << name << "," << count << "," << min_elapsed << "ns," << max_elapsed << "ns," << avg_elapsed << "ns,"
    laanwj commented at 10:23 AM on November 6, 2017:

    Please put the unit id in the row header, not in the data. Adding it to every record complicates parsing (for example, spreadsheets won't see it as number anymore, and will not be able to compute with it or plot it).

  12. laanwj commented at 10:23 AM on November 6, 2017: member

    utACK apart from output format nits.

  13. bench: switch to std::chrono for time measurements
    std::chrono removes portability issues.

Rather than storing doubles, store the untouched time_points. Then
convert to nanoseconds for display. This allows for maximum precision, while
keeping results comparable between differing hardware/operating systems.

Also, display full nanosecond counts rather than sub-second floats.
    c515d266ec
  14. bench: prefer a steady clock if the resolution is no worse 24a0bddf4a
  15. theuni force-pushed on Nov 7, 2017
  16. theuni commented at 10:20 PM on November 7, 2017: member

    Fixed up formatting as @laanwj suggested. I just dropped the commit which added the clock precision display, as there's no obvious (to me) way to show it without breaking csv.

  17. laanwj merged this on Nov 8, 2017
  18. laanwj closed this on Nov 8, 2017
  19. laanwj referenced this in commit 5776582b7f on Nov 8, 2017
  20. MarcoFalke commented at 3:53 PM on November 9, 2017: member

    post merge utACK 24a0bddf4ae13d8f1fa0436a547de67dcf6d4f2a

    Looks like a clear improvement.

  21. TheBlueMatt commented at 4:50 PM on November 9, 2017: member

    Postumous utACK 24a0bddf4ae13d8f1fa0436a547de67dcf6d4f2a modulo changes in #11646.

  22. laanwj referenced this in commit 68e021e3a3 on Dec 13, 2017
  23. MarcoFalke referenced this in commit afa9600020 on Aug 13, 2018
  24. PastaPastaPasta referenced this in commit 6d02d0b8e3 on Jul 18, 2019
  25. PastaPastaPasta referenced this in commit 6a18c3645e on Dec 22, 2019
  26. PastaPastaPasta referenced this in commit e6c46f3f02 on Dec 22, 2019
  27. PastaPastaPasta referenced this in commit 4607b9eb3c on Jan 2, 2020
  28. PastaPastaPasta referenced this in commit 4aea4b3f0d on Jan 2, 2020
  29. PastaPastaPasta referenced this in commit 55e7cfd604 on Jan 4, 2020
  30. PastaPastaPasta referenced this in commit 79723ea4da on Jan 4, 2020
  31. PastaPastaPasta referenced this in commit 66f8045fe7 on Jan 12, 2020
  32. PastaPastaPasta referenced this in commit 7610b8699f on Jan 12, 2020
  33. PastaPastaPasta referenced this in commit b8243e7644 on Jan 12, 2020
  34. PastaPastaPasta referenced this in commit 2afd200c33 on Jan 12, 2020
  35. PastaPastaPasta referenced this in commit 896e5a9717 on Jan 12, 2020
  36. PastaPastaPasta referenced this in commit b78253b2f6 on Jan 12, 2020
  37. PastaPastaPasta referenced this in commit d7c8ef3653 on Jan 12, 2020
  38. PastaPastaPasta referenced this in commit e80235ac83 on Jan 12, 2020
  39. PastaPastaPasta referenced this in commit efe3b9e1ca on Jan 12, 2020
  40. PastaPastaPasta referenced this in commit 106aab0c19 on Jan 12, 2020
  41. PastaPastaPasta referenced this in commit 97df283547 on Jan 12, 2020
  42. PastaPastaPasta referenced this in commit 01af073a40 on Jan 12, 2020
  43. PastaPastaPasta referenced this in commit 6d749f8d2a on Jan 16, 2020
  44. PastaPastaPasta referenced this in commit 6c9e49cd61 on Jan 16, 2020
  45. zkbot referenced this in commit aa225ebb0b on Jan 24, 2020
  46. zkbot referenced this in commit 74ff73abab on Jan 24, 2020
  47. PastaPastaPasta referenced this in commit 410113b7e3 on Jan 31, 2020
  48. PastaPastaPasta referenced this in commit 577f597e99 on Jan 31, 2020
  49. PastaPastaPasta referenced this in commit b12538634b on Feb 4, 2020
  50. PastaPastaPasta referenced this in commit 304f012632 on Feb 9, 2020
  51. PastaPastaPasta referenced this in commit 3b439789c6 on Feb 13, 2020
  52. PastaPastaPasta referenced this in commit 9d9d1499b0 on Feb 27, 2020
  53. PastaPastaPasta referenced this in commit 39a8e20de6 on Feb 27, 2020
  54. furszy referenced this in commit 4ed15cc69d on Jun 8, 2020
  55. ckti referenced this in commit 4a4ccdfe65 on Mar 28, 2021
  56. ckti referenced this in commit 26a91e1270 on Mar 28, 2021
  57. PastaPastaPasta referenced this in commit ffb6b7c210 on May 26, 2021
  58. gades referenced this in commit e391a0081c on Jun 26, 2021
  59. PastaPastaPasta referenced this in commit 0687c9a9f5 on Jun 27, 2021
  60. PastaPastaPasta referenced this in commit af67ca3268 on Jun 28, 2021
  61. PastaPastaPasta referenced this in commit f4d7177988 on Jun 29, 2021
  62. PastaPastaPasta referenced this in commit 2618b52d3c on Jun 29, 2021
  63. PastaPastaPasta referenced this in commit fd2b028aba on Jun 29, 2021
  64. PastaPastaPasta referenced this in commit 07e1a914af on Jun 29, 2021
  65. PastaPastaPasta referenced this in commit a9bf129d18 on Jun 29, 2021
  66. gades referenced this in commit ad3bab0111 on Jun 30, 2021
  67. linuxsh2 referenced this in commit 78e9c711a6 on Jul 29, 2021
  68. DrahtBot locked this on Sep 8, 2021
Contributors
theunipracticalswiftMarcoFalkefanquakelaanwjmartinusTheBlueMatt
Labels
Tests
Linked (view graph)
#11526 Visual Studio build configuration for Bitcoin Core#11558 Minimal code changes to allow msvc compilation
github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-13 21:15 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me