CI, tracing: run tracepoint interface tests in the CI

0xB10C commented at 11:23 am on April 6, 2022: contributor

The first tracepoint tests were merged in #24358. However, they are skipped in the CI. I’ve already done some experiments running them in the CI in #23296, however I haven’t been successful with it yet. The main issue was the limited capabilities inside the CirrusCI docker container.

An option would be to ask CirrusCI for help, another would be to run this part in a VM, if it’s not to much overhead.

0xB10C added the label Feature on Apr 6, 2022

maflcko commented at 11:34 am on April 6, 2022: member

Did you try to “recycle” the arm or macos tasks? They might already be using a “full” VM. (ref #23296 (comment))

Otherwise it should be possible to turn an existing task into using a fuller VM.

maflcko added the label Tests on Apr 6, 2022

0xB10C commented at 12:05 pm on April 6, 2022: contributor

I’ve looked at and experimented with the ARM tasks. Our ARM tasks run in an arm_container. https://github.com/bitcoin/bitcoin/blob/f3e35633690c71ebb69ec2ed08998565cade1522/.cirrus.yml#L190-L196

The CirrusCI docs mention that arm_container tasks run on Graviton2 (ARM processor) instances in AWS via Kubernetes (container runs via Kubernetes in the Google Cloud). We can’t use the macOS tasks as the tracepoints are Linux only..

I’ll look into converting an existing task into a VM.

maflcko commented at 1:28 pm on April 6, 2022: member

Ah right. According to https://cirrus-ci.org/guide/writing-tasks/#execution-environment they are all docker. https://cirrus-ci.org/guide/custom-vms/ could work, though.

0xB10C commented at 10:56 pm on June 29, 2022: contributor

https://cirrus-ci.org/guide/custom-vms/ could work, though.

That worked! I’ve used a compute_engine_instance to build and test the tracepoints. We have the required privileges there.

0task:
1  name: '[usdt] [VM]'
2  << : *GLOBAL_TASK_TEMPLATE
3  compute_engine_instance:
4    image_project: ubuntu-os-cloud
5    image: family/ubuntu-minimal-2204-lts

However, the Ubuntu bpf-tools package is quite outdated (v0.18 (Jan 05, 2021) vs v0.24 (Jan 14, 2022)). It prints a warning about warning: '__HAVE_BUILTIN_BSWAP64__' macro redefined to stderr when compiling to BPF bytecode. This has been fixed with https://github.com/iovisor/bcc/pull/3391 (May 1, 2021).

Printing to stderr let’s interface tests in test_runner.py fail.

I don’t really want to build bcc from source in the CI, but also didn’t have any luck installing it from their PPA. Might give it another shot, though. This could probably be removed with the next Ubuntu release.

maflcko commented at 6:54 am on June 30, 2022: member

You could use Debian: https://packages.debian.org/bookworm/bpfcc-tools

0xB10C commented at 10:44 am on June 30, 2022: contributor

I think the debian bookworm image is not yet available in GCE. https://cirrus-ci.com/task/6440165211111424

edit: yes, debian-11 is the latest available image: https://cloud.google.com/compute/docs/images/os-details

maflcko closed this on Aug 1, 2022

0xB10C commented at 12:59 pm on April 1, 2023: contributor

The up coming Ubuntu 23.04 seems to be shipping with an up-to-date bpfcc: https://packages.ubuntu.com/lunar/python3-bpfcc

bitcoin locked this on Mar 31, 2024

CI, tracing: run tracepoint interface tests in the CI #24782