depends: Add RISC-V support #13543

pull laanwj wants to merge 5 commits into bitcoin:master from laanwj:2018_06_riscv_depends changing 7 files +775 −754
  1. laanwj commented at 1:59 pm on June 27, 2018: member

    This adds support for riscv32 and riscv64 builds to the depends system.

    The change consists of documentation and build system changes. The most significant change is an update of config.sub and config.guess inside zeromq patch, as the current version does not recognize the riscv* host tuples (there’s no new version of ZeroMQ yet with newer ones).

    Good thing: RISC-V 64-bit toolchain packages can be installed out of the box on Ubuntu 18.04+.

    I would also like to add RISC-V 64-bit executables to gitian, but this will not be possible until #12511 .

  2. depends: Add RISC-V support 359e2e3525
  3. fanquake added the label Build system on Jun 27, 2018
  4. fanquake commented at 4:18 pm on June 27, 2018: member

    @laanwj Have you been building with NO_QT=1?

    I’ve just tested a make HOST=riscv64-linux-gnu depends build on Ubuntu 18.04, after installing the RISC-V toolchain and am seeing:

    0Configuring qrencode...
1checking build system type... x86_64-unknown-linux-gnu
2checking host system type... Invalid configuration `riscv64-linux-gnu': machine `riscv64' not recognized
3configure: error: /bin/bash use/config.sub riscv64-linux-gnu failed

    Building with NO_QT=1 completes successfully.

    I wonder if there’s a less verbose patch we could et away with to fix the zeromq build?

  5. laanwj commented at 4:32 pm on June 27, 2018: member

    Thanks for testing. Good point: yes, I have built without Qt, same as we do for ARM. Should probably mention that in the documentation, though I don’t think there’s any RISC-V boards at this time with a capable enough GPU and HDMI/VGA out to run even a desktop GUI (though maybe it’s possible to connect one through the PCI-e slot? I don’t know).

    I wonder if there’s a less verbose patch we could et away with to fix the zeromq build?

    I’m sure it could be cut down to less lines if that’s really worthwhile, I might personally prefer the dumb solution of simply copying the new autoconf files though.

  6. tmagik commented at 5:22 pm on June 27, 2018: none

    I can test this on a HiFiveU board next week. The problem with running a full bitcoin main net instance on a physical riscv board is going to be I/O.

    In the meantime, try running in Qemu as it will probably have better I/O

  7. laanwj commented at 5:25 pm on June 27, 2018: member

    I can test this on a HiFiveU board next week. The problem with running a full bitcoin main net instance on a physical riscv board is going to be I/O.

    Running test_bitcoin would already be helpful, even if you don’t want to sync a chain.

    In the meantime, try running in Qemu as it will probably have better I/O

    Good idea. I did this for the big-endian support, so it should certainly work for little-endian RISC-V.

  8. tmagik commented at 6:01 pm on June 27, 2018: none

    Thanks for testing. Good point: yes, I have built without Qt, same as we do for ARM. Should probably mention that in the documentation, though I don’t think there’s any RISC-V boards at this time with a capable enough GPU and HDMI/VGA out to run even a desktop GUI (though maybe it’s possible to connect one through the PCI-e slot? I don’t know).

    https://www.crowdsupply.com/microsemi/hifive-unleashed-expansion-board

    It runs X with a radeon graphics card and has more cores/memory/performance than the first machine I compiled bitcoin on. Bitcoin ATM vendors looking at high volume, low-cost embedded systems should probably be spending money on some evaluation systems.

  9. laanwj commented at 6:36 pm on June 27, 2018: member

    This is interesting:

     0user@bionic:~/bitcoin/src$ QEMU_LD_PREFIX=/usr/riscv64-linux-gnu qemu-riscv64 ./test/test_bitcoin
 1Running 291 test cases...
 2test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
 3test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
 4test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
 5test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
 6test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
 7test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
 8test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
 9test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
10test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
11test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
12test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
13test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
14test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
15test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
16test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
17test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
18test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
19test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
20test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
21test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
22test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
23test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
24test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
25test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
26test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
27test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
28test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
29test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
30test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
31test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
32test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
33test/crypto_tests.cpp(561): error: in "crypto_tests/sha256d64": check memcmp(out1, out2, 32 * i) == 0 has failed
34
35*** 32 failures are detected in the test module "Bitcoin Test Suite"

    It runs X with a radeon graphics card and has more cores/memory/performance than the first machine I compiled bitcoin on. Bitcoin ATM vendors looking at high volume, low-cost embedded systems should probably be spending money on some evaluation systems.

    Nice. I should get that one too.

  10. sipa commented at 6:46 pm on June 27, 2018: member
    @laanwj That’s very strange. It should be using the pure C++ SHA256 implementation. What SHA256 implementation does debug.log report (if you get that far)?
  11. laanwj commented at 7:21 pm on June 27, 2018: member

    @laanwj That’s very strange. It should be using the pure C++ SHA256 implementation. What SHA256 implementation does debug.log report (if you get that far)?

    It chooses the correct one:

    02018-06-27T19:16:24Z Using the 'standard' SHA256 implementation

    I’ll dive into what happens here.

  12. laanwj commented at 10:14 am on June 28, 2018: member

    It looks like the beginning of input in[] to SHA256D64 becomes corrupted by

    0        for (int j = 0; j < i; ++j) {
1            CHash256().Write(in + 64 * j, 64).Finalize(out1 + 32 * j);
2        }

    The function itself works correctly.

    I’m suspecting a miscompilation issue at the moment. It happens with -O1 and higher but not -O0. I’m not able to narrow it down to a specific optimization option, however it looks like it has something to do with computing stack offsets for inlining.

    I’d expect a newer toolchain will solve this, but if so it means the Ubuntu stock one (Ubuntu 7.3.0-16ubuntu3) is effectively broken that’s a disappointment.

  13. laanwj commented at 3:34 pm on June 29, 2018: member
    My sync in qemu is at block 270000, and has encountered no issues. I haven’t figured out yet why the crypto_tests/sha256d64 specifically has this issue. Strangely enough, moving the in array to the outer scope makes the test pass. I intend to try compilng with a more recent version of gcc to see if the test problem is resolved.
  14. tmagik commented at 11:00 pm on June 30, 2018: none

    I attempted to make a test case, and simplified the following down:

    0static inline uint64_t InsecureRandBits(int bits) { return insecure_seed++; }

    however this does not fail, so I’m now highly suspicious of inlining

  15. tmagik referenced this in commit bcc905ac67 on Jul 1, 2018
  16. tmagik commented at 0:25 am on July 1, 2018: none

    I can confirm the failure with gcc-7.3.1 on fedora on a HiFive Unleashed as well as gcc-7.3.0 on debian in qemu-user-mode emulation.

    Testcase at https://github.com/tmagik/riscv-sha256d-inline-fail/

  17. tmagik commented at 1:04 pm on July 1, 2018: none

    @laanwj Debian’s gcc-8 appears to fix the problem

    0riscv64-linux-gnu-gcc-8 --version
1riscv64-linux-gnu-gcc-8 (Debian 8.1.0-8) 8.1.0
  18. laanwj commented at 1:42 pm on July 4, 2018: member
    @tmagic Thanks for reproducing. Happy to hear it’s solved with newer gcc. It might be better to remove the note about using ubuntu’s cross-toolchain packages, then, until they have gcc 8 like debian.
  19. laanwj requested review from theuni on Jul 5, 2018
  20. DrahtBot commented at 7:31 am on July 6, 2018: member
    • #13578 ([depends, zmq, doc] upgrade zeromq to 4.2.5 and avoid deprecated zeromq api functions by mruddy)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  21. in depends/packages/zeromq.mk:6 in bab9f4e83d outdated 
    2@@ -3,7 +3,7 @@ $(package)_version=4.2.3
3 $(package)_download_path=https://github.com/zeromq/libzmq/releases/download/v$($(package)_version)/
4 $(package)_file_name=$(package)-$($(package)_version).tar.gz
5 $(package)_sha256_hash=8f1e2b2aade4dbfde98d82366d61baef2f62e812530160d2e6d0a5bb24e40bc0
6-$(package)_patches=0001-fix-build-with-older-mingw64.patch 0002-disable-pthread_set_name_np.patch
7+$(package)_patches=0001-fix-build-with-older-mingw64.patch 0002-disable-pthread_set_name_np.patch 0003-new_configure.patch
    theuni commented at 4:57 pm on July 6, 2018:
    Could we copy our files instead, since we have to keep ours up to date anyway?
    fanquake commented at 3:53 pm on July 8, 2018:
    I’ve added some commits to a branch that attempt to do that: https://github.com/fanquake/bitcoin/tree/riscv-depends-no-zmq-patching. It also updates the depends config.guess/sub to latest upstream. @laanwj You can cherry pick from there if you’d like. Note that branch doesn’t include re-running ./autogen.sh like below. I’ll comment about that shortly.
  22. in depends/packages/zeromq.mk:18 in bab9f4e83d outdated 
    12@@ -13,7 +13,9 @@ endef
13 
14 define $(package)_preprocess_cmds
15    patch -p1 < $($(package)_patch_dir)/0001-fix-build-with-older-mingw64.patch && \
16-   patch -p1 < $($(package)_patch_dir)/0002-disable-pthread_set_name_np.patch
17+   patch -p1 < $($(package)_patch_dir)/0002-disable-pthread_set_name_np.patch && \
18+   patch -p1 < $($(package)_patch_dir)/0003-new_configure.patch && \
19+   ./autogen.sh
    theuni commented at 5:02 pm on July 6, 2018:
    Is it definitely necessary to re-run autogen.sh? Afaik configure just calls out to config.guess/config.sub. Unless there’s some weird globbing involved, I should think that replacing those files would be enough.
  23. theuni approved
  24. theuni commented at 5:04 pm on July 6, 2018: member
    utACK, suggested changes would be a bonus if they work.
  25. depends: latest config.guess d7005e9988
  26. depends: latest config.sub 409481c465
  27. depends: update zmq config.guess/config.sub for riscv support 0d1f38c45f
  28. fanquake commented at 4:07 pm on July 8, 2018: member

    I’ve done some more testing of this using a branch that doesn’t patch zmq’s config.guess/sub, and instead just copies over our local versions, which I’ve updated from upstream.

    re running ./autogen.sh after copying. With or without './autogen.sh I see this warning from the zmq build:

    0Staging zeromq...
15.	make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.	5.	make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.

    The other difference (at least in output) is: With ./autogen.sh:

    0libtool: warning: remember to run 'libtool --finish /home/ubuntu/bitcoin/depends/riscv64-linux-gnu/lib'

    Without ./autogen.sh:

    0libtool: install: warning: remember to run `libtool --finish /home/ubuntu/bitcoin/depends/riscv64-linux-gnu/lib'

    Also noticed that when I pass --enable-werror to configure I see:

     0./configure --prefix=`pwd`/depends/riscv64-linux-gnu --enable-werror
 1<snip>
 2
 3  debug enabled = no
 4  gprof enabled = no
 5  werror        = yes
 6
 7  target os     = linux
 8  build os      = 
 9
10  CC            = riscv64-linux-gnu-gcc
11  CFLAGS        = -pipe -O2 
12  CPPFLAGS      =   -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -I/home/ubuntu/bitcoin/depends/riscv64-linux-gnu/share/../include/  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS
13  CXX           = riscv64-linux-gnu-g++ -std=c++11
14  CXXFLAGS      =   -Wstack-protector -fstack-protector-all    -Werror=vla  -pipe -O2 
15  LDFLAGS       = -pthread  -Wl,-z,relro -Wl,-z,now -pie  -L/home/ubuntu/bitcoin/depends/riscv64-linux-gnu/share/../lib 
16  ARFLAGS       = cr

    0  CXXLD    bench/bench_bitcoin
1  CXXLD    test/test_bitcoin
2/home/ubuntu/bitcoin/depends/riscv64-linux-gnu/share/../lib/libboost_unit_test_framework-mt.a(execution_monitor.o): In function `.L0 ':
3execution_monitor.cpp:(.text+0x9a4): warning: fedisableexcept is not implemented and will always fail
4execution_monitor.cpp:(.text+0x96a): warning: feenableexcept is not implemented and will always fail
5make[2]: Leaving directory '/home/ubuntu/bitcoin/src'

    All my testing has been done on Ubuntu 18.04.

  29. laanwj commented at 2:07 pm on July 9, 2018: member
    @theuni thanks for review @fanquake thanks for making the changes - will take over your branch
  30. depends: Mention RISC-V known compilation issue with gcc-7.3.x 974f0bf8e6
  31. laanwj force-pushed on Jul 9, 2018
  32. laanwj commented at 2:31 pm on July 9, 2018: member

    Updated branch:

    • Took over @fanquake’s branch that copies the autoconf artifacts instead of the big ugly patch
    • Added a comment about the RISC-V compilation issue
  33. theuni commented at 7:54 pm on July 9, 2018: member
    utACK 974f0bf8e684696be7796dbf3d48ff0a41f4ac26.
  34. laanwj merged this on Jul 10, 2018
  35. laanwj closed this on Jul 10, 2018
  36. laanwj referenced this in commit 6c6a3001e5 on Jul 10, 2018
  37. sipa added the label Needs release notes on Aug 14, 2018
  38. fanquake removed the label Needs release note on Mar 20, 2019
  39. deadalnix referenced this in commit 3ed52fce4f on Apr 3, 2020
  40. 10xcryptodev referenced this in commit 0c8f3447e4 on May 16, 2020
  41. 10xcryptodev referenced this in commit ce99b584ea on May 16, 2020
  42. 10xcryptodev referenced this in commit 4a7c3c22a7 on May 17, 2020
  43. ftrader referenced this in commit 3220b7ace3 on Aug 17, 2020
  44. gades referenced this in commit 96fb7f3b6b on Jun 24, 2021
  45. CryptoCentric referenced this in commit 7c4387a1a6 on Jul 2, 2021
  46. DrahtBot locked this on Dec 16, 2021


laanwj fanquake tmagik sipa DrahtBot theuni

Labels
Build system

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-05 16:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me