ci: how to run native arm job on Apple silicon? #31344

issue Sjors openend this issue on November 22, 2024
  1. Sjors commented at 11:00 am on November 22, 2024: member

    I’m trying to run the arm64 worker job (ARM, unit tests, no functional tests) on my M4 mac, because that should be lot faster than emulating it in Qemu on my x86_64 machine.

    Unfortunately so far it fails:

     0[10:34:36.739] Test project /ci_container_base/ci/scratch/build-arm-linux-gnueabihf
     1[10:34:36.742]         Start   2: util_rpcauth_test
     2[10:34:36.744]         Start   4: univalue_object_test
     3[10:34:36.747]         Start   6: secp256k1_tests
     4[10:34:36.751]         Start   8: test_bitcoin-qt
     5[10:34:36.754]         Start  10: addrman_tests
     6[10:34:36.756]   2/141 Test   [#4](/bitcoin-bitcoin/4/): univalue_object_test .................***Failed    0.01 sec
     7[10:34:36.756] /ci_container_base/ci/scratch/build-arm-linux-gnueabihf/src/univalue/object: 1: Syntax error: word unexpected (expecting ")")
     8[10:34:36.756] 
     9[10:34:36.769]   9/141 Test   [#2](/bitcoin-bitcoin/2/): util_rpcauth_test ....................   Passed    0.03 sec
    10[10:34:36.888]  10/141 Test   [#1](/bitcoin-bitcoin/1/): util_test_runner .....................***Failed    0.15 sec
    11[10:34:36.888] 2024-11-22 10:34:36,767 - ERROR - OSError, Failed to execute /ci_container_base/ci/scratch/build-arm-linux-gnueabihf/src/bitcoin-util 
    

    https://cirrus-ci.com/task/4863394807808000

    This is running https://github.com/Sjors/bitcoin/pull/67 which is based on master @ 2638fdb4f934be96b7c798dbac38ea5ab8a6374a.

    My current setup:

    For .cirrus.yaml I use:

    0name: MBP24-I
    1token: ...
    2
    3labels:
    4  type: arm64
    

    I start the worker with:

    0RESTART_CI_DOCKER_BEFORE_RUN=1 cirrus worker run -f ~/.cirrus.yaml
    

    I also tried running the cirrus worker outside a VM, with cirrus installed via Homebrew. For setup that I used Podman Desktop with the Docker extension.

    It fails with the same error. Here’s the log for that:

    https://cirrus-ci.com/task/4838331626815488

    For comparison here’s the same job (still) running on x86_64 with qemu:

    https://cirrus-ci.com/task/4837454681735168

  2. maflcko commented at 11:24 am on November 22, 2024: member

    My recommendation would be to try without the cirrus overhead for debugging. Just copy-pasting the one-line bash command may be easier and faster for troubleshooting. See https://github.com/bitcoin/bitcoin/tree/master/ci#running-a-stage-locally

    As for the error itself: It is not possible to run 32-bit arm binaries on a CPU that only supports 64-bit mode.

    I don’t know how to fix this, but if nothing helps, you’ll have to select qemu-arm in UTM. (Is there a reason why you haven’t done this in the first place?)

    Relevant CI output:

    0[11:51:42.955] + lscpu
    1[11:51:42.957] Architecture:                         aarch64
    2[11:51:42.957] CPU op-mode(s):                       64-bit
    

    It should be 32-bit, or at least include it:

    0[20:50:34.606] + lscpu
    1[20:50:34.612] Architecture:                         aarch64
    2[20:50:34.612] CPU op-mode(s):                       32-bit, 64-bit
    
  3. maflcko added the label Tests on Nov 22, 2024
  4. maflcko added the label Questions and Help on Nov 22, 2024
  5. l0rinc commented at 12:09 pm on November 22, 2024: contributor

    Not sure it helps, but I have tried reproducing it on my M4 Max, emulating an s390x platform (as I’ve done in #31144 (comment)) - while it’s still very (very) slow:

    0brew install podman pigz
    1softwareupdate --install-rosetta
    2podman machine init
    3podman machine start
    4docker run --platform linux/s390x -it ubuntu:22.04 /bin/bash
    
    0apt update && apt install -y git build-essential cmake ccache pkg-config libevent-dev libboost-dev libssl-dev libsqlite3-dev python3 && \
    1cd /mnt && git clone https://github.com/bitcoin/bitcoin.git && cd bitcoin && git remote add Sjors https://github.com/Sjors/bitcoin.git && git fetch --all && git checkout sv2-transport && \
    2cmake -B build -DBUILD_TESTS=ON && cmake --build build --target test_bitcoin -j$(nproc) && \
    3ctest --test-dir build -j$(nproc)
    

    I think I’ve reproduced the error you got:

     0...
     1 util_test_runner .....................***Failed
     2...
     396% tests passed, 5 tests failed out of 136
     4
     5Total Test time (real) = 278.88 sec
     6
     7The following tests FAILED:
     8          1 - univalue_test (Not Run)
     9          2 - univalue_object_test (Not Run)
    10          3 - secp256k1_noverify_tests (Not Run)
    11          4 - secp256k1_tests (Not Run)
    12          5 - secp256k1_exhaustive_tests (Not Run)
    

    Maybe it’s easier to debug this way.

  6. Sjors commented at 12:25 pm on November 22, 2024: member

    It is not possible to run 32-bit arm binaries on a CPU that only supports 64-bit mode.

    Ah, in that case it’s probably pointless for a performance improvement.

    select qemu-arm in UTM. (Is there a reason why you haven’t done this in the first place?)

    My initial (and main) goal for that Ubuntu VM was to make Guix builds.

    I could spin up a separate machine in 32 bit mode, but that seems like a world of pain on its own. Closing for now.

  7. Sjors closed this on Nov 22, 2024

  8. Sjors commented at 12:27 pm on November 22, 2024: member
    I guess my confusion came from the use of arm64 as the label for cirrus, as well as its description “For an aarch64 machine”.
  9. maflcko commented at 12:49 pm on November 22, 2024: member
    Yes, the cirrus machine is intentionally an aarch64 (also referred to as arm64) machine, because arm64 is generally available as performant hardware, compared to 32-bit arm. It was also intentionally selected to support 32-bit mode. (If you are looking for a cloud solution, IIRC only Hetzner cloud ships compatible VMs). Otherwise, you’ll have to shop for compatible bare-metal yourself, or use qemu (or similar).
  10. Sjors commented at 2:04 pm on November 22, 2024: member

    It was also intentionally selected to support 32-bit mode.

    That’s useful to know.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-03 18:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me