Porting bcc tools to libbpf #30298

1. 
   rob-scheepens commented at 8:14 am on June 18, 2024: none

   @0xB10C : reviving this old discussion from some two years ago: @rob-scheepens agree that BPF CO-RE and libbpf is the way to go for future tracing programs. I’ve looked into this a while ago, but it seems libbpf doesn’t support USDT tracepoints yet (see https://github.com/iovisor/bcc/issues/3007). However, there was a mention of a soon to be open sourced libusdt library developed and used in production at Meta by @anakryiko here. Looking forward to testing this once available!

   Originally posted by @0xB10C in #22006 (comment)

   Now that USDT has been implemented it would be great to give this a try. In this context, I also came across https://github.com/iovisor/bcc/issues/4404, which could help make things easier.

2. 
   0xB10C commented at 8:53 am on June 18, 2024: contributor

   I’ve been extensively using libbpf (& the libbpf-rs Rust crate) with the Bitcoin Core USDT tracepoints in https://github.com/0xB10C/peer-observer. I would be happy to see the tools being ported and would help to review a port. That being said, the tools (contrinb/tracing/*) could also be extracted to a tracing-tools repository to be maintained (and tested against Bitcoin Core) there.

   More important than the tools is IMO an elegant way of getting rid of the bcc Python dependency, which has been troublesome, start and using libbpf in our tests. I’m not aware of Python bindings for libbpf (but haven’t tried to look for them).

   Given that there seem to be a lack of reviewers (or interest) for tracing related PRs (e.g. #26593, #25832), I’m not planning to start porting it myself anytime soon.

   Additionally, we use the SystemTap macros to insert tracepoints. I think the x84_64 support of libbpf for these is good, aarch64 might not be fully ready yet (https://github.com/libbpf/libbpf/issues/796). But I don’t think there are many people using the tracepoints on aarch64 (besides myself).

3. willcl-ark added the label Brainstorming on Jun 24, 2024
4. 
   willcl-ark commented at 1:41 pm on June 24, 2024: member

   @rob-scheepens Thanks for opening this up for discussion again after the developments in libbpf.

   Is this something that you plan on implementing yourself and wanted to brainstorm on first?

   If not (and @0xB10C is not interested in implementing currently due to lack of review) then I’m not sure that keeping this issue open longer than the above discussion is useful and might recommend closing it again until developer interest in this feature picks up…

5. 
   rob-scheepens commented at 7:43 am on September 16, 2024: none

   @willcl-ark sorry for the delay in response. I’d love to implement this, but I doubt I currently have the necessary development skills (I’m just a performance engineer). I’ll give it some thought and experiment a bit in the near future, let’s see how far I get. @0xB10C I just saw your eBPF Summit session, thanks for the update on the state of things. Have you by any chance seen the session on bpftime (https://youtu.be/YZbCBaTTkeE?si=a7qOB-arjxXBuTJP)? That appears to be useful in order to reduce overhead for usdt further. What about Windows support, now that ebpf appears to be getting ready on that os too?

Contributors
rob-scheepens 0xB10C willcl-ark

Labels
Brainstorming