ci: only run USDT interface tests on CirrusCI #26592

pull 0xB10C wants to merge 1 commits into bitcoin:master from 0xB10C:2022-11-ci-only-run-usdt-tests-on-cirrus-ci changing 1 files +13 −6

0xB10C commented at 8:48 PM on November 28, 2022: contributor
As mentioned in #26571, the task running the USDT interface tests fail when run in docker. cc7335edc87c6ef34429b4df94f53973db520aac in #25528 added that the tests are run in a VM in Cirrus CI. Running them locally in docker containers might not work:
- We use bcc as tracing toolkit which requires the kernel headers to compile the BPF bytecode. As docker containers use the hosts kernel and don't run their own, there is a potential for mismatches between kernel headers available in the container and the host kernel. This results in a failure loading the BPF byte code.
- Privilges are required to load the BPF byte code into the kernel. Normally, the docker containers aren't run with these.
- We currently use an untrusted third-party PPA to install the bpfcc-tools package on Ubuntu 22.04. Using this on a local dev system could be a security risk.
To not hinder the ASan + LSan + UBSan part of the CI task, the USDT tests are disabled on non-CirrusCI runs.

ci: only run USDT interface tests on CirrusCI

As mentioned in #26571, the task running the USDT interface tests
fail when run in docker. cc7335edc87c6ef34429b4df94f53973db520aac
in #25528 added that the tests are run in a **VM** in Cirrus CI.
Running them locally in docker containers might not work:

- We use [bcc] as tracing toolkit which requires the kernel headers
  to compile the BPF bytecode. As docker containers use the hosts
  kernel and don't run their own, there is a potential for mismatches
  between kernel headers available in the container and the host
  kernel. This results in a failure loading the BPF byte code.
- Privilges are required to load the BPF byte code into the kernel.
  Normally, the docker containers aren't run with these.
- We currently use an untrusted third-party PPA to install the
  bpfcc-tools package on Ubuntu 22.04. Using this on a local dev
  system could be a security risk.

To not hinder the ASan + LSan + UBSan part of the CI task, the USDT
tests are disabled on non-CirrusCI runs.

[bcc]: https://github.com/iovisor/bcc

2811f40f30

DrahtBot commented at 8:48 PM on November 28, 2022: contributor


The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.



Reviews

See the guideline for information on the review process. A summary of reviews will appear here.



Conflicts

Reviewers, this pull request conflicts with the following ones:
- #25797 (build: Add CMake-based build system by hebasto)
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
DrahtBot added the label Tests on Nov 28, 2022
0xB10C commented at 8:49 PM on November 28, 2022: contributor

@MarcoFalke @hebasto let me know if that moves things in the right direction and if it helps you run the task locally.

in ci/test/00_setup_env_native_asan.sh:26 in 2811f40f30

  27 | -export DOCKER_NAME_TAG=ubuntu:22.04  # May not run in docker unless --enable-usdt is dropped
  28 | +export PACKAGES="systemtap-sdt-dev clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libqrencode-dev libsqlite3-dev ${BPFCC_PACKAGE}"
  29 | +export DOCKER_NAME_TAG=ubuntu:22.04
  30 |  export NO_DEPENDS=1
  31 |  export GOAL="install"
  32 |  export BITCOIN_CONFIG="--enable-usdt --enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' --with-sanitizers=address,integer,undefined CC=clang CXX=clang++"

maflcko commented at 8:13 AM on November 29, 2022:

Does --enable-usdt fail if the bpfcc tools are missing?

0xB10C commented at 8:17 AM on November 29, 2022:

No, this only depends on the systemtap sys/sdt.h used to insert the tracepoints. BCC (bpfcc-tools on Ubuntu) is only needed for the functional tests. If not available, the tests are skipped.

maflcko approved
maflcko commented at 8:13 AM on November 29, 2022: member

Thanks, lgtm
maflcko merged this on Dec 2, 2022
maflcko closed this on Dec 2, 2022
0xB10C deleted the branch on Dec 2, 2022
sidhujag referenced this in commit 76a7f60e74 on Dec 2, 2022
bitcoin locked this on Dec 2, 2023

Contributors

Labels

Linked (view graph)

#25528 ci: run USDT interface tests in the CI #26571 ci: Install required packages when running in container

ci: only run USDT interface tests on CirrusCI #26592

Reviews

Conflicts