guix: libsecp256k1 build failure on aarch64 (M1) #22441

issue fanquake openend this issue on July 14, 2021
  1. fanquake commented at 5:55 am on July 14, 2021: member

    Performing a Guix build of master (d8f1e1327f9c2f9fcc804468f6a981580acdf30a), for either x86_64-linux-gnu or x86_64-w64-mingw32 using:

    0time BASE_CACHE="/guix/base_cache" SOURCES_PATH="/guix/sources" SDK_PATH="/guix/SDKs" HOSTS="x86_64-linux-gnu" ./contrib/guix/guix-build
    

    inside an Alpine Docker container, running on an mac mini (M1) currently fails with the following:

      0=== configuring in src/secp256k1 (/distsrc-base/distsrc-d8f1e1327f9c-x86_64-linux-gnu/src/secp256k1)
      1configure: running /bin/sh ./configure --disable-option-checking '--prefix=/'  '--disable-ccache' '--disable-maintainer-mode' '--disable-dependency-tracking' '--enable-reduce-exports' '--disable-bench' '--disable-gui-tests' '--disable-fuzz-binary' '--disable-threadlocal' 'CFLAGS=-O2 -g -ffile-prefix-map=/bitcoin=.' 'CXXFLAGS=-O2 -g -ffile-prefix-map=/bitcoin=.' 'LDFLAGS=-Wl,--as-needed -Wl,--dynamic-linker=/lib64/ld-linux-x86-64.so.2 -static-libstdc++ -Wl,-O2' 'PKG_CONFIG_PATH=/root/.guix-profile/lib/pkgconfig' 'PYTHONPATH=/root/.guix-profile/lib/python3.8/site-packages' '--disable-shared' '--with-pic' '--enable-benchmark=no' '--enable-module-recovery' '--enable-module-schnorrsig' '--enable-experimental' --cache-file=/dev/null --srcdir=.
      2configure: loading site script /bitcoin/depends/x86_64-linux-gnu/share/config.site
      3checking build system type... aarch64-unknown-linux-gnu
      4checking host system type... x86_64-pc-linux-gnu
      5checking for a BSD-compatible install... /root/.guix-profile/bin/install -c
      6checking whether build environment is sane... yes
      7checking for x86_64-linux-gnu-strip... x86_64-linux-gnu-strip
      8checking for a thread-safe mkdir -p... /root/.guix-profile/bin/mkdir -p
      9checking for gawk... gawk
     10checking whether make sets $(MAKE)... yes
     11checking whether make supports nested variables... yes
     12checking how to print strings... printf
     13checking whether make supports the include directive... yes (GNU style)
     14checking for x86_64-linux-gnu-gcc... x86_64-linux-gnu-gcc
     15checking whether the C compiler works... yes
     16checking for C compiler default output file name... a.out
     17checking for suffix of executables... 
     18checking whether we are cross compiling... yes
     19checking for suffix of object files... o
     20checking whether we are using the GNU C compiler... yes
     21checking whether x86_64-linux-gnu-gcc accepts -g... yes
     22checking for x86_64-linux-gnu-gcc option to accept ISO C89... none needed
     23checking whether x86_64-linux-gnu-gcc understands -c and -o together... yes
     24checking dependency style of x86_64-linux-gnu-gcc... none
     25checking for a sed that does not truncate output... /root/.guix-profile/bin/sed
     26checking for grep that handles long lines and -e... /root/.guix-profile/bin/grep
     27checking for egrep... /root/.guix-profile/bin/grep -E
     28checking for fgrep... /root/.guix-profile/bin/grep -F
     29checking for ld used by x86_64-linux-gnu-gcc... /gnu/store/fzwlmgg54z0iqkhvs74v60zfx64x62gr-gcc-cross-x86_64-linux-gnu-8.4.0/libexec/gcc/x86_64-linux-gnu/ld
     30checking if the linker (/gnu/store/fzwlmgg54z0iqkhvs74v60zfx64x62gr-gcc-cross-x86_64-linux-gnu-8.4.0/libexec/gcc/x86_64-linux-gnu/ld) is GNU ld... yes
     31checking for BSD- or MS-compatible name lister (nm)... x86_64-linux-gnu-nm
     32checking the name lister (x86_64-linux-gnu-nm) interface... BSD nm
     33checking whether ln -s works... yes
     34checking the maximum length of command line arguments... 1572864
     35checking how to convert aarch64-unknown-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
     36checking how to convert aarch64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
     37checking for /gnu/store/fzwlmgg54z0iqkhvs74v60zfx64x62gr-gcc-cross-x86_64-linux-gnu-8.4.0/libexec/gcc/x86_64-linux-gnu/ld option to reload object files... -r
     38checking for x86_64-linux-gnu-objdump... x86_64-linux-gnu-objdump
     39checking how to recognize dependent libraries... pass_all
     40checking for x86_64-linux-gnu-dlltool... no
     41checking for dlltool... no
     42checking how to associate runtime and link libraries... printf %s\n
     43checking for x86_64-linux-gnu-ar... x86_64-linux-gnu-ar
     44checking for archiver [@FILE](/bitcoin-bitcoin/contributor/file/) support... @
     45checking for x86_64-linux-gnu-strip... (cached) x86_64-linux-gnu-strip
     46checking for x86_64-linux-gnu-ranlib... x86_64-linux-gnu-ranlib
     47checking command to parse x86_64-linux-gnu-nm output from x86_64-linux-gnu-gcc object... ok
     48checking for sysroot... no
     49checking for a working dd... /root/.guix-profile/bin/dd
     50checking how to truncate binary pipes... /root/.guix-profile/bin/dd bs=4096 count=1
     51checking for x86_64-linux-gnu-mt... no
     52checking for mt... no
     53checking if : is a manifest tool... no
     54checking how to run the C preprocessor... x86_64-linux-gnu-gcc -E
     55checking for ANSI C header files... yes
     56checking for sys/types.h... yes
     57checking for sys/stat.h... yes
     58checking for stdlib.h... yes
     59checking for string.h... yes
     60checking for memory.h... yes
     61checking for strings.h... yes
     62checking for inttypes.h... yes
     63checking for stdint.h... yes
     64checking for unistd.h... yes
     65checking for dlfcn.h... yes
     66checking for objdir... .libs
     67checking if x86_64-linux-gnu-gcc supports -fno-rtti -fno-exceptions... no
     68checking for x86_64-linux-gnu-gcc option to produce PIC... -fPIC -DPIC
     69checking if x86_64-linux-gnu-gcc PIC flag -fPIC -DPIC works... yes
     70checking if x86_64-linux-gnu-gcc static flag -static works... yes
     71checking if x86_64-linux-gnu-gcc supports -c -o file.o... yes
     72checking if x86_64-linux-gnu-gcc supports -c -o file.o... (cached) yes
     73checking whether the x86_64-linux-gnu-gcc linker (/gnu/store/fzwlmgg54z0iqkhvs74v60zfx64x62gr-gcc-cross-x86_64-linux-gnu-8.4.0/libexec/gcc/x86_64-linux-gnu/ld -m elf_x86_64) supports shared libraries... yes
     74checking dynamic linker characteristics... GNU/Linux ld.so
     75checking how to hardcode library paths into programs... immediate
     76checking whether stripping libraries is possible... yes
     77checking if libtool supports shared libraries... yes
     78checking whether to build shared libraries... no
     79checking whether to build static libraries... yes
     80checking whether make supports nested variables... (cached) yes
     81checking for x86_64-linux-gnu-pkg-config... /root/.guix-profile/bin/pkg-config --static
     82checking pkg-config is at least version 0.9.0... yes
     83checking for x86_64-linux-gnu-ar... /root/.guix-profile/bin/x86_64-linux-gnu-ar
     84checking for x86_64-linux-gnu-ranlib... /root/.guix-profile/bin/x86_64-linux-gnu-ranlib
     85checking for x86_64-linux-gnu-strip... /root/.guix-profile/bin/x86_64-linux-gnu-strip
     86checking for x86_64-linux-gnu-gcc option to accept ISO C89... (cached) none needed
     87checking dependency style of x86_64-linux-gnu-gcc... none
     88checking if x86_64-linux-gnu-gcc supports -std=c89 -pedantic -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-unused-function -Wno-long-long -Wno-overlength-strings... yes
     89checking if x86_64-linux-gnu-gcc supports -Wconditional-uninitialized... no
     90checking if x86_64-linux-gnu-gcc supports -fvisibility=hidden... yes
     91checking valgrind/memcheck.h usability... no
     92checking valgrind/memcheck.h presence... no
     93checking for valgrind/memcheck.h... no
     94checking for x86_64 assembly availability... yes
     95checking for libcrypto... no
     96checking openssl/crypto.h usability... no
     97checking openssl/crypto.h presence... no
     98checking for openssl/crypto.h... no
     99checking for gcc... gcc
    100checking whether we are using the GNU C compiler... (cached) yes
    101checking whether gcc accepts -g... yes
    102checking for gcc option to accept ISO C89... (cached) none needed
    103checking whether gcc understands -c and -o together... (cached) yes
    104checking dependency style of gcc... (cached) none
    105checking how to run the C preprocessor... gcc -E
    106checking if native gcc supports -Wall -Wextra -Wno-unused-function... yes
    107checking for working native compiler: gcc... yes
    108configure: ******
    109configure: WARNING: experimental build
    110configure: Experimental features do not have stable APIs or properties, and may not be safe for production use.
    111configure: Building extrakeys module: yes
    112configure: Building schnorrsig module: yes
    113configure: ******
    114checking that generated files are newer than configure... done
    115configure: creating ./config.status
    116config.status: creating Makefile
    117config.status: creating libsecp256k1.pc
    118config.status: creating src/libsecp256k1-config.h
    119config.status: executing depfiles commands
    120config.status: executing libtool commands
    121
    122Build Options:
    123  with ecmult precomp     = yes
    124  with external callbacks = no
    125  with benchmarks         = no
    126  with tests              = yes
    127  with openssl tests      = no
    128  with coverage           = no
    129  module ecdh             = no
    130  module recovery         = yes
    131  module extrakeys        = yes
    132  module schnorrsig       = yes
    133
    134  asm                     = x86_64
    135  ecmult window size      = 15
    136  ecmult gen prec. bits   = 4
    137
    138  valgrind                = no
    139  CC                      = x86_64-linux-gnu-gcc
    140  CFLAGS                  = -O2 -fvisibility=hidden -std=c89 -pedantic -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-unused-function -Wno-long-long -Wno-overlength-strings -W -pipe -O2 -O2 -g -ffile-prefix-map=/bitcoin=.
    141  CPPFLAGS                =  -I/bitcoin/depends/x86_64-linux-gnu/include/ 
    142  LDFLAGS                 = -L/bitcoin/depends/x86_64-linux-gnu/lib -Wl,--as-needed -Wl,--dynamic-linker=/lib64/ld-linux-x86-64.so.2 -static-libstdc++ -Wl,-O2
    143
    144  CC_FOR_BUILD            = gcc
    145  CFLAGS_FOR_BUILD        = -Wall -Wextra -Wno-unused-function -g -O2
    146  CPPFLAGS_FOR_BUILD      = 
    147  LDFLAGS_FOR_BUILD       = 
    148
    149# <snip>
    150
    151make[3]: Entering directory '/distsrc-base/distsrc-d8f1e1327f9c-x86_64-linux-gnu/src/secp256k1'
    152gcc -I. -I./src -Wall -Wextra -Wno-unused-function -g -O2 -c src/gen_context.c -o gen_context.o
    153  CXX      bitcoin_cli-bitcoin-cli.o
    154In file included from src/field_5x52_impl.h:19:0,
    155                 from src/field_impl.h:17,
    156                 from src/gen_context.c:19:
    157src/field_5x52_asm_impl.h: In function 'secp256k1_fe_mul_inner':
    158src/field_5x52_asm_impl.h:28:1: error: unknown register name '%rdx' in 'asm'
    159 __asm__ __volatile__(
    160 ^~~~~~~
    161src/field_5x52_asm_impl.h:28:1: error: unknown register name '%rcx' in 'asm'
    162src/field_5x52_asm_impl.h:28:1: error: unknown register name '%rax' in 'asm'
    163src/field_5x52_asm_impl.h:28:1: error: output number 0 not directly addressable
    164src/field_5x52_asm_impl.h:28:1: warning: asm operand 6 probably doesn't match constraints
    165src/field_5x52_asm_impl.h: In function 'secp256k1_fe_sqr_inner':
    166src/field_5x52_asm_impl.h:298:1: error: unknown register name '%rdx' in 'asm'
    167 __asm__ __volatile__(
    168 ^~~~~~~
    169
    170# <snip>
    171
    172src/field_5x52_asm_impl.h:28:1: error: unknown register name '%rcx' in 'asm'
    173src/field_5x52_asm_impl.h:28:1: error: unknown register name '%rax' in 'asm'
    174src/field_5x52_asm_impl.h:28:1: error: output number 0 not directly addressable
    175src/field_5x52_asm_impl.h:28:1: warning: asm operand 6 probably doesn't match constraints
    176make[3]: *** [Makefile:1775: gen_context.o] Error 1
    177make[3]: Leaving directory '/distsrc-base/distsrc-d8f1e1327f9c-x86_64-linux-gnu/src/secp256k1'
    178make[2]: *** [Makefile:17987: secp256k1/libsecp256k1.la] Error 2
    179make[2]: *** Waiting for unfinished jobs....
    180  CXX      lib/libunivalue_la-univalue_write.lo
    181  CXXLD    libunivalue.la
    182x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
    183make[3]: Leaving directory '/distsrc-base/distsrc-d8f1e1327f9c-x86_64-linux-gnu/src/univalue'
    184make[2]: Leaving directory '/distsrc-base/distsrc-d8f1e1327f9c-x86_64-linux-gnu/src'
    185make[1]: *** [Makefile:16160: all-recursive] Error 1
    186make[1]: Leaving directory '/distsrc-base/distsrc-d8f1e1327f9c-x86_64-linux-gnu/src'
    187make: *** [Makefile:820: all-recursive] Error 1
    

    Building libsecp256k1 directly in the Alpine container works as expected, and building when running a guix environment, and installing the needed packages (autoconf automake gcc libtool make pkg-config ) also seems to work ok. So I’m assuming the failure here is due to something we are doing in our Guix environment.

    Additional build output here. @real-or-random @jonasnick you might be interested here.

  2. fanquake added the label Bug on Jul 14, 2021
  3. fanquake added the label Build system on Jul 14, 2021
  4. sipa commented at 6:00 am on July 14, 2021: member
    Depending on how fast this needs to be resolved, with https://github.com/bitcoin-core/secp256k1/pull/956 (and a presumed follow-up to apply the same to the other table) we’ll remove cross-compilation issues by having the generated static context data be part of the source tree.
  5. real-or-random commented at 9:08 am on July 14, 2021: member

    What’s going on here:

    • configure detects that we’re compiling for x86_64 and enables the x86_64 ASM
    • Then make tries to build an executable for the aarch64 build machine (for doing precomputations during the build procress) but the x86_64 ASM is enabled…

    We fixed a similar thing for ARM assembly upstream in https://github.com/bitcoin-core/secp256k1/commit/99f47c20ec41279075d6b3ae64c9c1a84b40a6f8 but unfortunately that fix does not apply because we have different macros USE_EXTERNAL_ASM and USE_ASM_X86_64. @fanquake Can you test if adding #undef USE_ASM_X86_64 at the top of gen_context.c fixes this? Alternatively, you can configure with CPPFLAGS_FOR_BUILD=-UUSE_ASM_X86_64.

  6. fanquake referenced this in commit be68a78bd7 on Jul 14, 2021
  7. fanquake commented at 1:19 pm on July 14, 2021: member
    @real-or-random thanks for taking a look. I’ve done a Guix build using HOSTS="x86_64-linux-gnu x86_64-w64-mingw32" with this branch: https://github.com/fanquake/bitcoin/commit/be68a78bd7e149917ee8b48c7b09c25e1050d39d, and the builds have completed successfully. The full build log is here: https://gist.github.com/fanquake/ec23884762b843f7e911f0b2ae4734c4.
  8. real-or-random referenced this in commit be8d9c262f on Jul 14, 2021
  9. real-or-random commented at 5:03 pm on July 14, 2021: member
    Okay, we merged the fix upstream. So the easiest way to solve this issue is to update the secp256k1 tree.
  10. sipa commented at 5:06 pm on July 14, 2021: member
    On it.
  11. sipa commented at 5:22 pm on July 14, 2021: member
    See #22448.
  12. fanquake closed this on Aug 3, 2021

  13. sidhujag referenced this in commit ce5d5632a3 on Aug 4, 2021
  14. UdjinM6 referenced this in commit 54f3b66902 on Aug 10, 2021
  15. 5tefan referenced this in commit 235da293a2 on Aug 12, 2021
  16. DrahtBot locked this on Aug 18, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-29 04:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me