bench: Add SHA256 implementation specific benchmarks #27598

pull hebasto wants to merge 2 commits into bitcoin:master from hebasto:230508-bench changing 3 files +163 −23
  1. hebasto commented at 3:17 pm on May 8, 2023: member

    On the master branch, only the best available SHA256 implementation is being benchmarked. This PR makes bench_bitcoin benchmark all SHA256 implementations that are available on the system.

    For example:

    • on Linux:
     0$ ./src/bench/bench_bitcoin -filter=SHA.*
 1Using the 'x86_shani(1way,2way)' SHA256 implementation
 2
 3|             ns/byte |              byte/s |    err% |     total | benchmark
 4|--------------------:|--------------------:|--------:|----------:|:----------
 5|                1.00 |    1,002,545,462.93 |    0.4% |      0.01 | `SHA1`
 6|                2.91 |      344,117,991.18 |    0.1% |      0.03 | `SHA256 using the 'standard' SHA256 implementation`
 7|                2.21 |      453,081,794.40 |    0.1% |      0.02 | `SHA256 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
 8|                2.21 |      453,396,506.58 |    0.1% |      0.02 | `SHA256 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
 9|                0.53 |    1,870,520,687.49 |    0.1% |      0.01 | `SHA256 using the 'x86_shani(1way,2way)' SHA256 implementation`
10|                7.90 |      126,627,134.33 |    0.0% |      0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation`
11|                3.94 |      253,850,206.07 |    0.0% |      0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
12|                1.40 |      716,247,553.38 |    0.4% |      0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
13|                1.26 |      792,706,270.13 |    0.9% |      0.01 | `SHA256D64_1024 using the 'x86_shani(1way,2way)' SHA256 implementation`
14|                6.75 |      148,172,097.64 |    0.2% |      0.01 | `SHA256_32b using the 'standard' SHA256 implementation`
15|                4.90 |      204,156,289.96 |    0.1% |      0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way)' SHA256 implementation`
16|                4.90 |      204,101,274.22 |    0.1% |      0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
17|                1.70 |      589,052,595.35 |    0.4% |      0.01 | `SHA256_32b using the 'x86_shani(1way,2way)' SHA256 implementation`
18|                2.21 |      453,441,736.14 |    1.0% |      0.02 | `SHA3_256_1M`
19|                1.92 |      521,807,101.48 |    1.0% |      0.02 | `SHA512`
    • on macOS (M1):
     0% ./src/bench/bench_bitcoin -filter=SHA.\*
 1Using the 'arm_shani(1way,2way)' SHA256 implementation
 2
 3|             ns/byte |              byte/s |    err% |     total | benchmark
 4|--------------------:|--------------------:|--------:|----------:|:----------
 5|                1.36 |      737,644,274.00 |    0.6% |      0.02 | `SHA1`
 6|                3.08 |      324,556,777.15 |    0.2% |      0.03 | `SHA256 using the 'standard' SHA256 implementation`
 7|                0.45 |    2,198,104,135.18 |    0.3% |      0.01 | `SHA256 using the 'arm_shani(1way,2way)' SHA256 implementation`
 8|                8.84 |      113,131,299.18 |    0.0% |      0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation`
 9|                0.94 |    1,059,406,239.36 |    0.0% |      0.01 | `SHA256D64_1024 using the 'arm_shani(1way,2way)' SHA256 implementation`
10|                6.17 |      162,050,659.51 |    0.2% |      0.01 | `SHA256_32b using the 'standard' SHA256 implementation`
11|                1.15 |      866,637,155.98 |    0.0% |      0.01 | `SHA256_32b using the 'arm_shani(1way,2way)' SHA256 implementation`
12|                1.69 |      592,636,491.59 |    0.2% |      0.02 | `SHA3_256_1M`
13|                1.89 |      528,785,775.66 |    0.0% |      0.02 | `SHA512`

    Found it useful, while working on #24773.

  2. DrahtBot commented at 3:17 pm on May 8, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK sipa, martinus, MarcoFalke
    Concept ACK luke-jr

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    No conflicts as of last run.

  3. DrahtBot added the label Tests on May 8, 2023
  4. maflcko commented at 3:20 pm on May 8, 2023: member
    What about benchmarking always all that are available on the system?
  5. hebasto commented at 3:22 pm on May 8, 2023: member

    What about benchmarking all that are available on the system?

    You mean, as separated benchmarks? Without -sha-implementation option?

  6. hebasto force-pushed on May 9, 2023
  7. hebasto renamed this:
    bench: Add `-sha-implementation` command-line option
    bench: Benchmark all `SHA256` implementations that are available on the system
    on May 9, 2023
  8. hebasto force-pushed on May 9, 2023
  9. hebasto commented at 4:22 pm on May 9, 2023: member

    Reworked according to @MarcoFalke’s suggestion.

    Friendly ping @martinus :)

  10. in src/bench/bench.cpp:133 in e5ab4a84ce outdated 
    146-            func(bench);
147-        } else {
148-            for (auto n : args.asymptote) {
149-                bench.complexityN(n);
150-                func(bench);
151+        if (properties.iterate_sha256_implementations) {
    martinus commented at 12:49 pm on May 11, 2023:
    Hm, I don’t really like that all of that SHA256 specific code is becoming part of the benchmark framework. Wouldn’t it be sufficient to just adapt the SHA benchmark in crypto_hash.cpp so that there is a separate test for each of the implementations? There could be a separate benchmark for each value of the enum UseImplementation.
    fanquake commented at 12:06 pm on May 16, 2023:

    Hm, I don’t really like that all of that SHA256 specific code is becoming part of the benchmark framework.

    Yea. Concept NACK on the current approach.

    hebasto commented at 4:28 pm on May 16, 2023:

    @martinus

    Thanks! Reworked.

  11. in src/bench/bench_bitcoin.cpp:123 in e5ab4a84ce outdated 
    118@@ -118,6 +119,9 @@ int main(int argc, char** argv)
119         return EXIT_SUCCESS;
120     }
121 
122+    std::string sha256_algo = SHA256AutoDetect();
123+    tfm::format(std::cout, "Using the '%s' SHA256 implementation\n", sha256_algo);
    martinus commented at 12:50 pm on May 11, 2023:
    This always prints a SHA log even when no SHA related benchmark is run
    hebasto commented at 4:29 pm on May 16, 2023:
    Thanks! That change has been dropped.
  12. hebasto force-pushed on May 16, 2023
  13. hebasto commented at 4:28 pm on May 16, 2023: member
    Reworked according to @martinus’s comments.
  14. hebasto renamed this:
    bench: Benchmark all `SHA256` implementations that are available on the system
    bench: Add SHA256 implementation specific benchmarks
    on May 16, 2023
  15. hebasto force-pushed on May 18, 2023
  16. hebasto commented at 11:06 am on May 18, 2023: member
    Rebased on top of the merged #27615 as it was relevant.
  17. luke-jr commented at 4:50 pm on June 23, 2023: member
    Concept ACK
  18. achow101 requested review from sipa on Sep 20, 2023
  19. achow101 requested review from josibake on Sep 20, 2023
  20. Add ability to specify SHA256 implementation for benchmark purposes 5f72417176
  21. bench: Add SHA256 implementation specific benchmarks ce6df7df9b
  22. hebasto force-pushed on Sep 20, 2023
  23. hebasto commented at 8:26 pm on September 20, 2023: member
    Rebased for fresh CI simultaneously with #24773 that is based on this PR.
  24. DrahtBot added the label CI failed on Sep 21, 2023
  25. DrahtBot removed the label CI failed on Sep 21, 2023
  26. sipa commented at 3:12 pm on September 29, 2023: member

    ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476

    My output (Ryzen 5950X):

    ns/byte byte/s err% total benchmark
    1.29 776,698,179.43 0.3% 0.01 SHA256D64_1024_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
    0.67 1,492,914,034.43 0.1% 0.01 SHA256D64_1024_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation
    3.45 290,161,790.35 0.3% 0.01 SHA256D64_1024_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation
    6.96 143,610,645.45 0.2% 0.01 SHA256D64_1024_STANDARD using the 'standard' SHA256 implementation
    4.51 221,653,283.35 0.1% 0.01 SHA256_32b_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
    1.25 799,826,326.07 0.2% 0.01 SHA256_32b_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation
    4.54 220,365,747.83 0.1% 0.01 SHA256_32b_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation
    5.77 173,257,910.52 0.1% 0.01 SHA256_32b_STANDARD using the 'standard' SHA256 implementation
    2.01 498,088,336.96 0.2% 0.02 SHA256_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
    0.44 2,288,837,568.06 0.4% 0.01 SHA256_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation
    1.99 501,633,066.45 0.5% 0.02 SHA256_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation
    2.50 400,467,265.21 0.2% 0.03 SHA256_STANDARD using the 'standard' SHA256 implementation
  27. hebasto commented at 11:23 am on October 2, 2023: member

    @martinus

    Mind taking another look at this PR?

  28. martinus commented at 5:52 pm on October 2, 2023: contributor

    ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476. I would have created a helper function in the test to avoid the code duplication for each test, but that’s just me nitpicking. Here are results from my Ryzen 7950X, with ./src/bench/bench_bitcoin -filter="SHA256.*" -min-time=1000:

    ns/byte byte/s err% ins/byte bra/byte miss% total benchmark
    1.08 922,442,557.83 0.0% 18.39 0.01 0.3% 1.07 SHA256D64_1024_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
    0.59 1,696,233,239.51 0.1% 6.60 0.03 0.1% 1.10 SHA256D64_1024_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation
    2.94 339,763,959.72 0.0% 51.98 0.02 0.1% 1.10 SHA256D64_1024_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation
    5.79 172,580,960.50 0.1% 132.83 0.06 0.1% 1.10 SHA256D64_1024_STANDARD using the 'standard' SHA256 implementation
    3.82 261,777,995.08 0.1% 82.72 1.69 0.0% 1.10 SHA256_32b_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
    1.10 912,755,631.26 0.0% 14.09 1.53 0.0% 1.10 SHA256_32b_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation
    3.82 261,543,369.48 0.1% 82.72 1.69 0.0% 1.10 SHA256_32b_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation
    4.10 244,096,616.72 0.2% 112.94 1.53 0.0% 1.10 SHA256_32b_STANDARD using the 'standard' SHA256 implementation
    1.75 572,561,916.85 0.1% 36.80 0.09 0.0% 1.10 SHA256_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
    0.38 2,639,518,254.68 0.0% 2.56 0.02 0.0% 1.10 SHA256_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation
    1.75 572,393,692.61 0.2% 36.80 0.09 0.0% 1.10 SHA256_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation
    1.82 548,938,688.20 0.1% 51.91 0.02 0.0% 1.10 SHA256_STANDARD using the 'standard' SHA256 implementation
  29. martinus approved
  30. hebasto requested review from maflcko on Oct 4, 2023
  31. maflcko commented at 1:53 pm on October 4, 2023: member

    review ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476 🏵

    Signature:

    0untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
1RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
2trusted comment: review ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476 🏵
3GsNRGSniZroT9qJHZFN7xnASprLA9scBcdwWOood4TqNZX7pGDvXRFuAI0buRTCEQMVV9OT5ZVnJMymXcWDPCw==
  32. DrahtBot removed review request from maflcko on Oct 4, 2023
  33. fanquake merged this on Oct 4, 2023
  34. fanquake closed this on Oct 4, 2023
  35. hebasto deleted the branch on Oct 4, 2023
  36. Frank-GER referenced this in commit 004c46e90e on Oct 13, 2023
  37. bitcoin locked this on Oct 3, 2024


hebasto DrahtBot maflcko martinus fanquake luke-jr sipa


josibake

Labels
Tests

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-01-10 15:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me