ci: USDT on mantic #28600

issue maflcko openend this issue on October 5, 2023
  1. maflcko commented at 3:23 pm on October 5, 2023: member

    The asan CI uses Ubuntu 23.04, which doesn’t have clang-17 and will also be EOL in January:

    https://github.com/bitcoin/bitcoin/blob/52c6904c789103a316253df4cb20aa785b2afc42/ci/test/00_setup_env_native_asan.sh#L20

    Thus, it would be good to bump to 23.10. However that fails, even outside the CI.

    Steps to reproduce on a fresh install of Ubuntu 23.04/10:

    0export DEBIAN_FRONTEND=noninteractive && apt update && apt install curl wget htop git vim ccache -y && git clone https://github.com/bitcoin/bitcoin.git  --depth=1 ./bitcoin-core && cd bitcoin-core && apt install build-essential libtool autotools-dev automake pkg-config bsdmainutils python3-zmq     libevent-dev libboost-dev systemtap-sdt-dev clang-16 llvm-16 libclang-rt-16-dev  bpfcc-tools "linux-headers-$(uname --kernel-release)"  -y   &&  ./autogen.sh && ./configure   --enable-usdt CC=clang-16 CXX=clang++-16 && make -j $(nproc) && ./test/functional/interface_usdt_mempool.py
    

    Output on 23.04: passes

    On 23.10:

     0# ./test/functional/interface_usdt_mempool.py 
     12023-10-05T15:22:55.501000Z TestFramework (INFO): PRNG seed is: 3896309821586557942
     22023-10-05T15:22:55.502000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_930vj2wl
     32023-10-05T15:22:55.852000Z TestFramework (INFO): Hooking into mempool:added tracepoint...
     4/virtual/main.c:97:24: error: call to undeclared function '_bpf_readarg_trace_removed_1'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     5  ({ u64 __addr = 0x0; _bpf_readarg_trace_removed_1(ctx, &__addr, sizeof(__addr));bpf_probe_read_user(&removed.hash, HASH_LENGTH, (void *)__addr);});
     6                       ^
     7/virtual/main.c:97:24: note: did you mean '_bpf_readarg_trace_added_1'?
     8/virtual/main.c:13:28: note: '_bpf_readarg_trace_added_1' declared here
     9static __always_inline int _bpf_readarg_trace_added_1(struct pt_regs *ctx, void *dest, size_t len) {
    10                           ^
    11/virtual/main.c:98:24: error: call to undeclared function '_bpf_readarg_trace_removed_2'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    12  ({ u64 __addr = 0x0; _bpf_readarg_trace_removed_2(ctx, &__addr, sizeof(__addr));bpf_probe_read_user(&removed.reason, MAX_REMOVAL_REASON_LENGTH, (void *)__addr);});
    13                       ^
    14/virtual/main.c:98:24: note: did you mean '_bpf_readarg_trace_added_2'?
    15/virtual/main.c:19:28: note: '_bpf_readarg_trace_added_2' declared here
    16static __always_inline int _bpf_readarg_trace_added_2(struct pt_regs *ctx, void *dest, size_t len) {
    17                           ^
    18/virtual/main.c:99:3: error: call to undeclared function '_bpf_readarg_trace_removed_3'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    19  _bpf_readarg_trace_removed_3(ctx, &removed.vsize, sizeof(*(&removed.vsize)));
    20  ^
    21/virtual/main.c:99:3: note: did you mean '_bpf_readarg_trace_added_3'?
    22/virtual/main.c:25:28: note: '_bpf_readarg_trace_added_3' declared here
    23static __always_inline int _bpf_readarg_trace_added_3(struct pt_regs *ctx, void *dest, size_t len) {
    24                           ^
    25/virtual/main.c:100:3: error: call to undeclared function '_bpf_readarg_trace_removed_4'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    26  _bpf_readarg_trace_removed_4(ctx, &removed.fee, sizeof(*(&removed.fee)));
    27  ^
    28/virtual/main.c:101:3: error: call to undeclared function '_bpf_readarg_trace_removed_5'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    29  _bpf_readarg_trace_removed_5(ctx, &removed.entry_time, sizeof(*(&removed.entry_time)));
    30  ^
    31/virtual/main.c:112:24: error: call to undeclared function '_bpf_readarg_trace_rejected_1'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    32  ({ u64 __addr = 0x0; _bpf_readarg_trace_rejected_1(ctx, &__addr, sizeof(__addr));bpf_probe_read_user(&rejected.hash, HASH_LENGTH, (void *)__addr);});
    33                       ^
    34/virtual/main.c:112:24: note: did you mean '_bpf_readarg_trace_added_1'?
    35/virtual/main.c:13:28: note: '_bpf_readarg_trace_added_1' declared here
    36static __always_inline int _bpf_readarg_trace_added_1(struct pt_regs *ctx, void *dest, size_t len) {
    37                           ^
    38/virtual/main.c:113:24: error: call to undeclared function '_bpf_readarg_trace_rejected_2'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    39  ({ u64 __addr = 0x0; _bpf_readarg_trace_rejected_2(ctx, &__addr, sizeof(__addr));bpf_probe_read_user(&rejected.reason, MAX_REJECT_REASON_LENGTH, (void *)__addr);});
    40                       ^
    41/virtual/main.c:113:24: note: did you mean '_bpf_readarg_trace_added_2'?
    42/virtual/main.c:19:28: note: '_bpf_readarg_trace_added_2' declared here
    43static __always_inline int _bpf_readarg_trace_added_2(struct pt_regs *ctx, void *dest, size_t len) {
    44                           ^
    45/virtual/main.c:124:24: error: call to undeclared function '_bpf_readarg_trace_replaced_1'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    46  ({ u64 __addr = 0x0; _bpf_readarg_trace_replaced_1(ctx, &__addr, sizeof(__addr));bpf_probe_read_user(&replaced.replaced_hash, HASH_LENGTH, (void *)__addr);});
    47                       ^
    48/virtual/main.c:124:24: note: did you mean '_bpf_readarg_trace_added_1'?
    49/virtual/main.c:13:28: note: '_bpf_readarg_trace_added_1' declared here
    50static __always_inline int _bpf_readarg_trace_added_1(struct pt_regs *ctx, void *dest, size_t len) {
    51                           ^
    52/virtual/main.c:125:3: error: call to undeclared function '_bpf_readarg_trace_replaced_2'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    53  _bpf_readarg_trace_replaced_2(ctx, &replaced.replaced_vsize, sizeof(*(&replaced.replaced_vsize)));
    54  ^
    55/virtual/main.c:125:3: note: did you mean '_bpf_readarg_trace_added_2'?
    56/virtual/main.c:19:28: note: '_bpf_readarg_trace_added_2' declared here
    57static __always_inline int _bpf_readarg_trace_added_2(struct pt_regs *ctx, void *dest, size_t len) {
    58                           ^
    59/virtual/main.c:126:3: error: call to undeclared function '_bpf_readarg_trace_replaced_3'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    60  _bpf_readarg_trace_replaced_3(ctx, &replaced.replaced_fee, sizeof(*(&replaced.replaced_fee)));
    61  ^
    62/virtual/main.c:127:3: error: call to undeclared function '_bpf_readarg_trace_replaced_4'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    63  _bpf_readarg_trace_replaced_4(ctx, &replaced.replaced_entry_time, sizeof(*(&replaced.replaced_entry_time)));
    64  ^
    65/virtual/main.c:128:24: error: call to undeclared function '_bpf_readarg_trace_replaced_5'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    66  ({ u64 __addr = 0x0; _bpf_readarg_trace_replaced_5(ctx, &__addr, sizeof(__addr));bpf_probe_read_user(&replaced.replacement_hash, HASH_LENGTH, (void *)__addr);});
    67                       ^
    68/virtual/main.c:129:3: error: call to undeclared function '_bpf_readarg_trace_replaced_6'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    69  _bpf_readarg_trace_replaced_6(ctx, &replaced.replacement_vsize, sizeof(*(&replaced.replacement_vsize)));
    70  ^
    71/virtual/main.c:130:3: error: call to undeclared function '_bpf_readarg_trace_replaced_7'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    72  _bpf_readarg_trace_replaced_7(ctx, &replaced.replacement_fee, sizeof(*(&replaced.replacement_fee)));
    73  ^
    7414 errors generated.
    752023-10-05T15:22:57.476000Z TestFramework (ERROR): Unexpected exception caught during testing
    76Traceback (most recent call last):
    77  File "/root/bitcoin-core/test/functional/test_framework/test_framework.py", line 131, in main
    78    self.run_test()
    79  File "/root/bitcoin-core/./test/functional/interface_usdt_mempool.py", line 317, in run_test
    80    self.added_test()
    81  File "/root/bitcoin-core/./test/functional/interface_usdt_mempool.py", line 146, in added_test
    82    bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0)
    83          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    84  File "/usr/lib/python3/dist-packages/bcc/__init__.py", line 479, in __init__
    85    raise Exception("Failed to compile BPF module %s" % (src_file or "<text>"))
    86Exception: Failed to compile BPF module <text>
    872023-10-05T15:22:57.528000Z TestFramework (INFO): Stopping nodes
    882023-10-05T15:22:57.580000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_930vj2wl
    892023-10-05T15:22:57.580000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_930vj2wl/test_framework.log
    902023-10-05T15:22:57.580000Z TestFramework (ERROR): 
    912023-10-05T15:22:57.580000Z TestFramework (ERROR): Hint: Call /root/bitcoin-core/test/functional/combine_logs.py '/tmp/bitcoin_func_test_930vj2wl' to consolidate all logs
    922023-10-05T15:22:57.580000Z TestFramework (ERROR): 
    932023-10-05T15:22:57.580000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
    942023-10-05T15:22:57.580000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
    952023-10-05T15:22:57.580000Z TestFramework (ERROR): 
    
  2. maflcko added the label Bug on Oct 5, 2023
  3. maflcko added the label Tests on Oct 5, 2023
  4. maflcko commented at 3:24 pm on October 5, 2023: member
  5. 0xB10C commented at 10:18 pm on October 5, 2023: contributor
    As discussed offline, it looks like these errors are coming from the bpfcc-tools package version 0.26.0+ds-1ubuntu2 in Ubuntu 23.10 (mantic). For 23.04 (lunar) it’s version 0.26.0+ds-1ubuntu1. As we need to upgrade to 23.10, maybe switching the USDT tests to a different job not using Ubuntu is a solution for now?
  6. maflcko commented at 3:31 pm on October 6, 2023: member
    If no one fixes this issue within the next two months, I’d suggest to remove the bpfcc-tools package and not run the tests, temporarily, for now (TM).
  7. willcl-ark commented at 9:23 am on October 9, 2023: member

    As discussed offline, it looks like these errors are coming from the bpfcc-tools package version 0.26.0+ds-1ubuntu2

    The changelog between these versions looks very small, and likely unrelated? It just adds riscv build:

     0diff -Nru bpfcc-0.26.0+ds/debian/changelog bpfcc-0.26.0+ds/debian/changelog
     1--- bpfcc-0.26.0+ds/debian/changelog	2023-02-14 13:52:10.000000000 +0000
     2+++ bpfcc-0.26.0+ds/debian/changelog	2023-07-31 11:51:23.000000000 +0000
     3@@ -1,3 +1,9 @@
     4+bpfcc (0.26.0+ds-1ubuntu2) mantic; urgency=medium
     5+
     6+  * Build for riscv64.
     7+
     8+ -- Matthias Klose <doko@ubuntu.com>  Mon, 31 Jul 2023 13:51:23 +0200
     9+
    10 bpfcc (0.26.0+ds-1ubuntu1) lunar; urgency=low
    11 
    12   * Merge from Debian unstable. Remaining changes:
    13diff -Nru bpfcc-0.26.0+ds/debian/control bpfcc-0.26.0+ds/debian/control
    14--- bpfcc-0.26.0+ds/debian/control	2023-02-13 09:16:10.000000000 +0000
    15+++ bpfcc-0.26.0+ds/debian/control	2023-07-31 10:43:48.000000000 +0000
    16@@ -38,7 +38,7 @@
    17 Rules-Requires-Root: no
    18 
    19 Package: libbpfcc
    20-Architecture: amd64 arm64 ppc64el ppc64 s390x armhf
    21+Architecture: amd64 arm64 ppc64el ppc64 s390x armhf riscv64
    22 Depends: ${misc:Depends}, ${shlibs:Depends}
    23 Multi-Arch: same
    24 Description: shared library for BPF Compiler Collection (BCC)
    25@@ -52,7 +52,7 @@
    26  to control BPF programs from userspace.
    27 
    28 Package: libbpfcc-dev
    29-Architecture: amd64 arm64 ppc64el ppc64 s390x armhf
    30+Architecture: amd64 arm64 ppc64el ppc64 s390x armhf riscv64
    31 Section: libdevel
    32 Depends: ${misc:Depends},
    33  libbpfcc (= ${binary:Version})
    34@@ -114,7 +114,7 @@
    35  At this time this package contains subset of tools from bpfcc-tools
    36 
    37 Package: bpfcc-introspection
    38-Architecture: amd64 arm64 ppc64el ppc64
    39+Architecture: amd64 arm64 ppc64el ppc64 riscv64
    40 Depends: ${misc:Depends},
    41  ${shlibs:Depends},
    42  libbpfcc (= ${binary:Version})
    43diff -Nru bpfcc-0.26.0+ds/debian/rules bpfcc-0.26.0+ds/debian/rules
    44--- bpfcc-0.26.0+ds/debian/rules	2023-02-13 09:16:10.000000000 +0000
    45+++ bpfcc-0.26.0+ds/debian/rules	2023-07-31 09:37:01.000000000 +0000
    46@@ -8,7 +8,7 @@
    47 
    48 # for building libbpf-tools on selective architecture
    49 include /usr/share/dpkg/architecture.mk
    50-libbpf_tool_skip_arch := armhf s390x ppc64
    51+libbpf_tool_skip_arch := armhf s390x ppc64 riscv64
    52 
    53 export BPFTOOL=$(shell echo /usr/lib/linux-tools-*/bpftool)
    54 # Disable tests as they require root to run
    

    source: https://launchpadlibrarian.net/679695359/bpfcc_0.26.0+ds-1ubuntu1_0.26.0+ds-1ubuntu2.diff.gz

  8. maflcko commented at 9:25 am on October 9, 2023: member

    The changelog between these versions looks very small, and likely unrelated? It just adds riscv build:

    Pretty sure the build system changes between the two Ubuntu versions, so the compiler used to compile is different.

    If the compiler is also embedded or used by bpfcc, it will also affect the package (indirectly) without anything in the package changing.

  9. maflcko commented at 9:26 am on October 9, 2023: member
    So somehow pinning the gcc version to the previous one could be a workaround?
  10. willcl-ark commented at 9:28 am on October 9, 2023: member

    The changelog between these versions looks very small, and likely unrelated? It just adds riscv build:

    Pretty sure the build system changes between the two Ubuntu versions, so the compiler used to compile is different.

    If the compiler is also embedded or used by bpfcc, it will also affect the package (indirectly) without anything in the package changing.

    Ah that makes sense. Will have a play around with it in a container and see if I can find a fix

  11. willcl-ark referenced this in commit 9027ed048c on Oct 9, 2023
  12. willcl-ark commented at 1:45 pm on October 9, 2023: member

    I think this can be stop-gap-fixed by setting -Wno-error=implicit-function-declaration on the trace point program build, as done here: https://github.com/bitcoin/bitcoin/compare/master...willcl-ark:bitcoin:mantis-usdt-fix

    As this will remain unsupported into the future, this doesn’t seem to be a particularly resilient fix, IMO. However I am unsure if the “correct” way of fixing is to actually re-declare all the functions (which seems undesirable) or to wait for another upstream fix… There is a similar issue in the bcc repo: https://github.com/iovisor/bcc/issues/4740#issuecomment-1721830957 where the advice given was to re-declare all the functions.

    What do you think @maflcko @0xB10C ?

    edit: my commit message from the commit above also contains more background:

    Recently usage of undeclared functions became an error, rather than a warning. https://reviews.llvm.org/D122983?id=420290

    This change seems to have migrated into the build tools of Ubuntu 23.10, which now causes the USDT tests to fail to compile, see #28600

    Fix this by setting -Wno-error=implicit-function-declaration for the traceing programs.

  13. maflcko commented at 1:49 pm on October 9, 2023: member
    Happy to test your commit if you submit a PR
  14. willcl-ark referenced this in commit 4077e43bf6 on Oct 9, 2023
  15. fanquake referenced this in commit 06d469c26b on Oct 12, 2023
  16. fanquake closed this on Oct 12, 2023

  17. Frank-GER referenced this in commit 74a1221e45 on Oct 13, 2023
  18. janus referenced this in commit 9f884136ea on Apr 1, 2024
  19. bitcoin locked this on Oct 11, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-30 15:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me