RFC: Riscv bare metal CI job

sedited commented at 8:30 am on December 5, 2024: contributor

This adds a CI job for building the static consensus library and linking it to an executable. It uses newlib-cygwin as a C library for the final linking step. This ensure compatibility with this target going forward and can serve as a starting point for enabling bare metal builds for the entire kernel library. This would have also caught the error fixed in #31365.

DrahtBot commented at 8:30 am on December 5, 2024: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31425.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	laanwj, hebasto, fanquake

If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#34495 (Replace boost signals with minimal compatible implementation by theuni)
#34491 (ci: add FreeBSD Clang cross job by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

sedited force-pushed on Dec 5, 2024

DrahtBot commented at 9:12 am on December 5, 2024: contributor

🚧 At least one of the CI tasks failed. Debug: https://github.com/bitcoin/bitcoin/runs/33960779294

Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

DrahtBot added the label CI failed on Dec 5, 2024

maflcko commented at 9:16 am on December 5, 2024: member

can serve as a starting point for enabling bare metal builds for the entire kernel library.

Interesting. Do you think this is possible at all, given that the kernel library links leveldb, which I’d presume is not bare-metal ready?

So I guess this mostly serves as a check that users can ship their own-brewed libbitcoinconsensus (or so, or subset of it)?

sedited force-pushed on Dec 5, 2024

sedited commented at 9:26 am on December 5, 2024: contributor

Interesting. Do you think this is possible at all, given that the kernel library links leveldb, which I’d presume is not bare-metal ready?

Yes, definitely not ready. I think the main hurdle is the background compaction, which I am not sure how to tackle. Maybe we’ll find a solution for it eventually though, either by patching it, or allowing the user to bring their own utxos.

So I guess this mostly serves as a check that users can ship their own-brewed libbitcoinconsensus (or so, or subset of it)?

Yes, that is the goal for now.

sedited marked this as ready for review on Dec 5, 2024

sedited force-pushed on Dec 5, 2024

in ci/test/03_test_script.sh:168 in 4910ae5d0d outdated

163+        -Wl,--no-whole-archive \
164+        src/crypto/libbitcoin_crypto.a \
165+        src/secp256k1/lib/libsecp256k1.a \
166+        /opt/riscv-ilp32/riscv32-unknown-elf/lib/libstdc++.a \
167+        /riscv/newlib/build/riscv32-unknown-elf/newlib/libc.a \
168+        /riscv/newlib/build/riscv32-unknown-elf/newlib/libm.a \

laanwj commented at 11:20 am on December 5, 2024:

Any specific reason to use the intermediate build and not the installed libraries from /opt/newlib, here?

in ci/test/01_base_install.sh:98 in 4910ae5d0d outdated

93+
94+    ${CI_RETRY_EXE} git clone --depth=1 https://sourceware.org/git/newlib-cygwin.git -b topic/3.6 /riscv/newlib
95+    cd /riscv/newlib
96+    mkdir build && cd build
97+    ../configure \
98+        --target=riscv32-unknown-elf --disable-newlib-io-float --enable-newlib-io-long-long --enable-newlib-io-long-double --with-arch=rv32gc --with-abi=ilp32 --disable-shared --disable-multilib\

laanwj commented at 11:22 am on December 5, 2024:

Are you sure enabling i/o for long-double is needed? i don’t believe we use this type anywhere.

sedited commented at 2:25 pm on December 6, 2024:

Sorry, all these flags were a mess. I was experimenting with linking in a range of other functionality, as well as running it on linux directly, and didn’t prune stuff out nicely. Removed most of them again.

laanwj commented at 11:35 am on December 5, 2024: member

It uses newlib-cygwin as a C library for the final linking step.

Mentioning this because i had to look it up to be sure: newlib-cygwin has nothing to do with Windows whatsoever. It’s simply a minimalist libc.

I think the main hurdle is the background compaction, which I am not sure how to tackle.

Could be a periodic foreground task, if threads aren’t available? But yes, this would imply patching leveldb, there is no such API right now.

laanwj added the label Build system on Dec 5, 2024

laanwj added the label Tests on Dec 5, 2024

DrahtBot removed the label CI failed on Dec 5, 2024

in ci/test/01_base_install.sh:89 in 4910ae5d0d outdated

83@@ -84,6 +84,29 @@ if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
84   rm -rf /msan/llvm-project
85 fi
86 
87+if [[ ${BARE_METAL_RISCV} == "true" ]]; then
88+    ${CI_RETRY_EXE} git clone --depth=1 https://github.com/riscv-collab/riscv-gnu-toolchain -b 2024.11.22 /riscv/gcc
89+    cd /riscv/gcc

maflcko commented at 1:35 pm on December 5, 2024:

style-wise, it would be good to not use cd in the CI scripts, because it affects the global state for all remaining lines.

It is better to wrap it into (), for example ( cd a ; foo(); ) (or similar).

in ci/test/01_base_install.sh:107 in 4910ae5d0d outdated

102+        AR_FOR_TARGET=/opt/riscv-ilp32/bin/riscv32-unknown-elf-ar \
103+        RANLIB_FOR_TARGET=/opt/riscv-ilp32/bin/riscv32-unknown-elf-ranlib \
104+        CFLAGS_FOR_TARGET="-march=rv32gc -mabi=ilp32 -mcmodel=medlow "\
105+        CXXFLAGS_FOR_TARGET="-std=c++20 -march=rv32gc -mabi=ilp32 -mcmodel=medlow"
106+    make -j "$MAKEJOBS"
107+    make install

maflcko commented at 1:38 pm on December 5, 2024:

It would be good to rm -rf everything duplicate. Otherwise, you are using up space twice:

0$ podman image ls | grep riscv
1localhost/ci_native_riscv_bare  latest      dfa9b8223b62  5 minutes ago  13.9 GB

See the llvm build on how to do this.

sedited commented at 2:39 pm on December 6, 2024:

Should be better now:

0docker image ls | grep riscv
1ci_native_riscv_bare   latest    55627eefe6d7   3 minutes ago   2.8GB

sedited force-pushed on Dec 6, 2024

sedited commented at 3:39 pm on December 6, 2024: contributor

Updated 4910ae5d0d9954b8b811d7c52e3539979ea46016 -> f27760012b3e236933ea5d1b39a6ba74884df1a2 (bare_metal_support_0 -> bare_metal_support_1, compare)

Addressed @maflcko’s comment, removing sources after install step.
Addressed @maflcko’s comment, wrapped cd step into a subshell
Addressed @laanwj’s comment, use the installed lib
Addressed @laanwj’s comment, removing a bunch of unneeded/superfluous flags
Added a start section to the binary for the global pointer and returning an exit code. Though not needed, since we are only checking that it links, I feel like this makes the example a bit clearer.

in ci/test/03_test_script.sh:154 in f27760012b outdated

149+              la gp, __global_pointer$
150+              .option pop
151+
152+              call main
153+
154+              # Put Exit2 system call number into the a7 register

laanwj commented at 12:46 pm on December 10, 2024:

concept ACK on making it actually runnable somewhat i was confused for a moment here, as to what syscall numbers mean for bare-metal maybe add “Linux” to the comment

laanwj commented at 12:58 pm on December 10, 2024: member

Concept ACK, this looks like the minimum required to sanity check that libconsensus.a can be compiled for, and linked for bare-metal RISC-V. It is not enough to test that it actually works (this would require a lot more, quite nasty low-level code), but maybe that’s out of scope for this project. At the least it is for this PR.

hebasto commented at 10:53 am on December 14, 2024: member

Concept ACK.

in CMakeLists.txt:377 in f27760012b outdated

373@@ -372,8 +374,10 @@ if(BUILD_FUZZ_BINARY)
374   )
375 endif()
376 
377+if(NOT CMAKE_SYSTEM_NAME STREQUAL "Generic")

hebasto commented at 2:36 pm on February 3, 2025:

I imagine that expanding this build scenario might enable building boost and libevent in depends, no?

If so, the Boost check should have another option to be disabled.

Additionally, mind considering this comment: https://github.com/bitcoin/bitcoin/blob/1172bc4157eefe80d1aaf0b56459857ec651e535/cmake/module/AddBoostIfNeeded.cmake#L7-L8

which, in turn, was based on this legacy code:https://github.com/bitcoin/bitcoin/blob/32efe850438ef22e2de39e562af557872a402c31/configure.ac#L1247-L1251

DrahtBot added the label Needs rebase on Mar 31, 2025

sedited force-pushed on Apr 3, 2025

DrahtBot removed the label Needs rebase on Apr 3, 2025

DrahtBot added the label CI failed on Apr 3, 2025

DrahtBot commented at 6:00 am on April 4, 2025: contributor

CI failure:

0[12:50:33.743] gmake: *** No rule to make target 'install'.  Stop.

maflcko commented at 8:38 am on May 21, 2025: member

Coming from #32460 (comment), I noticed that https://www.sourceware.org/newlib/libc.html doesn’t list ftruncate, so I guess the build will fail once more code is compiled/linked?

fanquake commented at 8:41 am on May 21, 2025: member

I think the docs might be incomplete / outdated? See https://github.com/bminor/newlib/blob/b39b510c1ce68757e79410585262ca2cd48da839/newlib/libc/include/sys/unistd.h#L287.

DrahtBot added the label Needs rebase on Jul 29, 2025

sedited force-pushed on Aug 7, 2025

sedited commented at 7:59 pm on August 7, 2025: contributor

Rebased f27760012b3e236933ea5d1b39a6ba74884df1a2 -> ec7c86c732c995d2c38e2a3f93ad55a451a8cf81 (bare_metal_support_1 -> bare_metal_support_2, compare)

Fixed a bunch of silent and proper merge conflicts.

DrahtBot removed the label Needs rebase on Aug 7, 2025

DrahtBot removed the label CI failed on Aug 7, 2025

DrahtBot added the label Needs rebase on Sep 3, 2025

sedited force-pushed on Sep 4, 2025

sedited commented at 4:08 pm on September 4, 2025: contributor

Rebased ec7c86c732c995d2c38e2a3f93ad55a451a8cf81 -> a577bbe5df766a4e0e5a36fc997b7880eec45c0e (bare_metal_support_2 -> bare_metal_support_3, compare)

Fixed conflict with #32989

DrahtBot removed the label Needs rebase on Sep 4, 2025

sedited force-pushed on Sep 17, 2025

sedited commented at 11:44 am on September 17, 2025: contributor

Rebased a577bbe5df766a4e0e5a36fc997b7880eec45c0e -> e254de76ac0a390569bc5b561f61babb6d021dcb (bare_metal_support_3 -> bare_metal_support_4, compare)

?

added_to_project_v2 sedited

?

project_v2_item_status_changed sedited

DrahtBot added the label Needs rebase on Dec 22, 2025

build: Add option for building for bare metal envs

A bare metal build is now supported by setting CMAKE_SYSTEM_NAME=Generic

Skip the platform-dependent feature checks, such as threads and atomics,
which are typically not available on bare metal. Also only make the
boost headers mandatory if they exist for the target.

2a1aee564b

sedited force-pushed on Dec 23, 2025

sedited commented at 11:26 am on December 23, 2025: contributor

Rebased e254de76ac0a390569bc5b561f61babb6d021dcb -> dc53679c60c3a5758700002bbdca6a5ab112c24c (bare_metal_support_4 -> bare_metal_support_5, compare)

Fixed conflict with #33810

DrahtBot removed the label Needs rebase on Dec 23, 2025

fanquake commented at 11:03 am on February 9, 2026: member

Concept ACK - Similar to #34421, if there are reservations about having this run in the CI, we can at least add the config.

Add CI job for producing a static bare metal binary 7efc1fc1a3

Add CI job for riscv bare metal 774042bb0e

in ci/test/03_test_script.sh:214 in dc53679c60 outdated

209+        "${BASE_BUILD_DIR}"/src/secp256k1/lib/libsecp256k1.a \
210+        /opt/riscv-ilp32/riscv32-unknown-elf/lib/libstdc++.a \
211+        /opt/newlib/riscv32-unknown-elf/lib/libc.a \
212+        /opt/newlib/riscv32-unknown-elf/lib/libm.a \
213+        /opt/riscv-ilp32/lib/gcc/riscv32-unknown-elf/14.2.0/libgcc.a \
214+        -o test.elf

fanquake commented at 11:31 am on February 9, 2026:

It could be nice to call file/ldd or something similar, on the produced outputs, so it’s more clear what is being built. Otherwise the end-of run output is a bit hard to parse:

 0+ /opt/riscv-ilp32/bin/riscv32-unknown-elf-gcc -c start.s -o start.o
 1+ echo -e '#include <sys/stat.h>
 2          void _exit(int code) { while(1); }
 3          int _sbrk(int incr) { return 0; }
 4          int _write(int file, char *ptr, int len) { return 0; }
 5          int _close(int file) { return -1; }
 6          int _fstat(int file, struct stat *st) { st->st_mode = S_IFCHR; return 0; }
 7          int _isatty(int file) { return 1; }
 8          int _lseek(int file, int ptr, int dir) { return 0; }
 9          int _read(int file, char *ptr, int len) { return 0; }
10          int _kill(int pid, int sig) { return -1; }
11          int _getpid(void) { return -1; }'
12+ /opt/riscv-ilp32/bin/riscv32-unknown-elf-gcc -g -march=rv32i -mabi=ilp32 -c syscalls.c -o syscalls.o
13+ /opt/riscv-ilp32/bin/riscv32-unknown-elf-g++ -g -std=c++20 -march=rv32gc -mabi=ilp32 -nostdlib /opt/riscv-ilp32/lib/gcc/riscv32-unknown-elf/14.2.0/crtbegin.o test.o start.o syscalls.o /ci_container_base/ci/scratch/build-riscv32-unknown-elf-gcc/lib/libbitcoin_consensus.a /ci_container_base/ci/scratch/build-riscv32-unknown-elf-gcc/lib/libbitcoin_crypto.a /ci_container_base/ci/scratch/build-riscv32-unknown-elf-gcc/src/secp256k1/lib/libsecp256k1.a /opt/riscv-ilp32/riscv32-unknown-elf/lib/libstdc++.a /opt/newlib/riscv32-unknown-elf/lib/libc.a /opt/newlib/riscv32-unknown-elf/lib/libm.a /opt/riscv-ilp32/lib/gcc/riscv32-unknown-elf/14.2.0/libgcc.a -o test.elf
14+ '[' -n '' ']'
15+ '[' '' = true ']'
16+ '[' false = true ']'
17+ '[' false = true ']'
18+ '[' false = true ']'
19+ [[ '' == true ]]
20+ '[' false = true ']'
21Stop and remove CI container by ID
22+ docker container kill cfdbe4713beee3e4d396bd9d9226c99efafd3679bb04acdb89ec063eae8402d5
23Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
24cfdbe4713beee3e4d396bd9d9226c99efafd3679bb04acdb89ec063eae8402d5

sedited commented at 9:47 am on February 10, 2026:

Added file:

0+ file test.elf
1test.elf: ELF 32-bit LSB executable, UCB RISC-V, RVC, soft-float ABI, version 1 (SYSV), statically linked, with debug_info, not stripped

sedited force-pushed on Feb 10, 2026

sedited commented at 9:46 am on February 10, 2026: contributor

Updated dc53679c60c3a5758700002bbdca6a5ab112c24c -> 774042bb0eecee783ab04e0cb53f762b71984e77 (bare_metal_support_5 -> bare_metal_support_6, compare)

Took @fanquake’s suggestion, added file command to print final build artifact’s information.
Addressed @laanwj’s comment, added a small comment indicating that making the binary executable on linux serves debugging purposes.

RFC: Riscv bare metal CI job #31425

Code Coverage & Benchmarks

Reviews

Conflicts