ARMv8 sha2 support

Sjors commented at 8:13 pm on June 5, 2018: member

I assume #13191 make this less hard, although benefits may be small. Brief chat on IRC:

Me

While trying to get bitcoind to run on one the many *-pi’s out there, I wondered: has anyone ever tried to design a system on chip that’s optimal for this? @laanwj: provoostenator: you mean secp256k1 specific instructions? people have been thingking about it, could be done on a FPGA, but I don’t think it’s ever been done

Me:

echeveria seems to believe sha256 is the bottleneck (see #bitcoin), but also that anything outside the CPU would be too slow I/O to be worh it.

echeveria:

I looked at the Zynq combination FPGA / ARM devices a long time ago and came to the conclusion that the copy time even on the shared memory bus between the two chips would make it non viable. I’d enjoy being proved wrong though.

laanwj:

provoostenator: well sha256 extension instructions exist for ARM (supported on newer SoCs), I intend to add support for them at some point. But I would be surprised if that is the biggest bottleneck in validation.

echeveria:

yes, if there is high-bandwidth communication between two chpis that tends to dominate. I was > thinking of, say, RiscV extensions for secp256k1 validation so it’s in-core. for ARM it’s somewhat unlikely at this time

I have (at least) three devices to test this on, which all have 4 to 8 ARM Cortex-A53 cores, and 1- 4 GB RAM: an Android Xiaomi A1 (ABCore syncs the whole chain in less than a month), a NanoPi Neo Plus and a Khadas VIM2 Max.

Maybe this c++ code is useful: https://github.com/randombit/botan/issues/841

laanwj commented at 9:29 am on June 6, 2018: member

Thanks for creating an issue to track this. I’ll try to add some information in case someone wants to pick this up, if someone doesn’t want to wait for me to do so.

To check programmatically if SHA256 instructions are supported on an ARM platform in Linux, one can do:

0#include <arm_acle.h>
1#include <sys/auxv.h>
2#include <asm/hwcap.h>
3...
4#elif defined(__aarch64__)
5  return (getauxval(AT_HWCAP) & HWCAP_SHA2) != 0;
6#elif defined(__arm__)
7  return (getauxval(AT_HWCAP2) & HWCAP2_SHA2) != 0;
8#else...

Sjors commented at 6:17 pm on June 11, 2018: member

Also this: https://github.com/noloader/SHA-Intrinsics

fanquake added the label Build system on Sep 6, 2018

rollmeister commented at 7:23 pm on November 9, 2018: contributor

I have attempted to apply SHA256 ArmV8 acceleration for secp256k1 however due to this library using ANSI 89 standard code the intrinsic’s refuse to compile: I have however added it (using some dirty hacks) and CRC32 support for the rest of Bitcoin Core.in a fork I published. Block processing performance should improve further if it could be extended to secp256k1. Basically would be little more than a copy and paste of the code in sha256.cpp and changing the compiling scripts, at my guess anyway. I lack the skill to do so anytime soon. Benchmarking stats are presented in my fork repo readme.md

sipa commented at 7:59 pm on November 9, 2018: member

@rollmeister The SHA256 implementation inside libsecp256k1 isn’t performance critical (specifically, it’s only used for signing and not used during block validation). We should just focus on the SHA256 in the Bitcoin Core crypto/ directory.

rollmeister commented at 8:08 pm on November 9, 2018: contributor

@sipa Yes I did some more testing and its invoked there infrequently. My code is in the fork in either case and it is up to the Bitcoin developers (if they are interested) to merge the implementation into the main branch using maintainable separate source files and updated build scripts. I do not intend to do so myself, as I have satisfied my own curiosity and personal needs with the work I have done so far.

rollmeister commented at 1:24 pm on November 19, 2018: contributor

I need help in completing a port of Transform64 for ArmV8. I cannot work out transform 3. I have referred to the Intel Shani implementation. I can offer a small financial reward via Paypal using debit card for anyone able to solve it. The work is hard coded in my Bitcoin Core fork at https://github.com/rollmeister/bitcoin-armv8/blob/master/src/crypto/sha256.cpp . When benchmarked as is and the self test is disabled, it is 600% performant and makes nodes operating on cheap arm boards more viable. If successfully completed, I will make it (and the crc32 support also) into a maintainable separate source file and edit the configure.ac and do a pull request. *UPDATE I have made progress and figured out how to route the regular sha256 armv8 transform via the Transform64Wrapper. It is about 20% slower than a fully optimised implementation sha256d computations are still 500% faster than before and will settle for that. It is reindexing the blockchain much faster now.

denravonska commented at 4:04 pm on May 2, 2019: none

@rollmeister Has there been any update on your ARM optimizations? A lot of altcoins would be very happy to see it implemented.

laanwj added the label Android on Sep 30, 2019

laanwj commented at 8:37 am on January 3, 2021: member

Looks like I had a competing issue about the same open (#17414) 😄

prusnak commented at 2:14 pm on January 8, 2022: contributor

Since there are lot of people that are using Raspberry Pi for running a full node - this might be also worth investigating:

https://github.com/obijywk/qpusha - Hardware-accelerated SHA-256 hashing on the Raspberry Pi (utilizing VideoCore IV Quad Processors)

prusnak commented at 11:49 pm on January 20, 2022: contributor

PR implementing this in https://github.com/bitcoin/bitcoin/pull/24115

laanwj closed this on Feb 14, 2022

PastaPastaPasta referenced this in commit a7a3a93489 on Mar 29, 2022

PastaPastaPasta referenced this in commit 8278356354 on Mar 29, 2022

DrahtBot locked this on Feb 14, 2023

ARMv8 sha2 support #13401