tracing: add block header and compact block tracepoints #35368

w0xlt commented at 9:04 PM on May 23, 2026: contributor

This PR adds two USDT tracepoints for compact block relay:

net:block_header: fires when a new valid header is received from a peer (via headers or cmpctblock).
net:compact_block_reconstructed: fires when a compact block is successfully reconstructed, just before validation.

The motivation is to expose structured, low-overhead observability for compact block relay behavior.

Compact block relay is performance-sensitive: whether a node reconstructs a block without round trips depends on mempool overlap, extra transaction pool usefulness, and peer behavior.

Today this information is mostly available only through debug logs, which are string-based, less precise for tooling, and not ideal for continuous measurement.

What they enable measuring:

Which peers announce new headers first, and whether headers arrived via compact block announcements.
Per-peer compact block reconstruction success.
Where each transaction came from — prefilled, mempool, extra pool, or a getblocktxn round trip.
How often compact blocks reconstruct with no round trip, and the bandwidth cost when they don't.
Whether high-bandwidth compact block peers actually improve relay, and how mempool divergence affects reconstruction quality.

Design notes.

All data is already available at the reconstruction point; nothing new is computed in the hot path. USDT tracepoints stay inactive unless a tracing tool is attached.
Mempool and extra-pool counts are reported as disjoint fields, so consumers can aggregate without double-counting. The reconstruction tracepoint includes the peer ID, making metrics attributable to the peer that supplied the compact block data.

DrahtBot commented at 9:04 PM on May 23, 2026: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35368.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	0xB10C, m4ycon

If your review is incorrectly listed, please copy-paste <code></code> into the comment that the bot should ignore.

w0xlt force-pushed on May 23, 2026

DrahtBot added the label CI failed on May 23, 2026

tracing: add block header tracepoint c45092fcd6

tracing: add compact block reconstruction tracepoint fbfeb8859c

w0xlt force-pushed on May 23, 2026

w0xlt commented at 10:49 PM on May 23, 2026: contributor

CI error unrelated. See #34731.

0xB10C commented at 2:14 PM on May 24, 2026: contributor

Generally in favor of having a way of exposing data about this in a non-logging way. Especially since fast compact block relay and reconstructions seem to be somewhat of a share priority by contributors. Concept ACK on exposing this data.

In https://github.com/peer-observer/peer-observer/pull/420, @m4ycon has been working on reading very similar information for the debug log. My understanding is that this works well, but is obviously brittle.

There's also the question about adding more tracepoints now, or investing time into seeing if #35142 is feasible...

in src/net_processing.cpp:3587 in fbfeb8859c

3583 |          via_compact_block ? "cmpctblock " : "",
3584 | -        index.GetBlockHash().ToString(),
3585 | +        block_hash.ToString(),
3586 |          index.nHeight,
3587 |          peer.LogPeer()
3588 |      );

0xB10C commented at 2:19 PM on May 24, 2026:

With tracepoints essentially matching the log statements, I wonder if there's some way of combine the two - but that's obviously out of scope for this PR.

0xB10C commented at 9:52 AM on May 25, 2026:

I think a structured logging interface as discussed in #35369 would be a good solution avoiding duplication between logging statements and tracepoints. We'd get the same information about compact block reconstructions in a machine-to-machine interface.

in src/net_processing.cpp:236 in fbfeb8859c

 231 | +        stats.mempool_txn_count,
 232 | +        stats.extra_pool_txn_count,
 233 | +        stats.requested_txn_count,
 234 | +        stats.requested_txn_bytes,
 235 | +        hash.data()
 236 | +    );

m4ycon commented at 9:05 PM on May 24, 2026:

While monitoring dashboards from https://github.com/peer-observer/peer-observer/pull/420, my node had a pretty weird period where it was always requesting txs from other nodes (stats.requested_txn_count > 0). The image below, this panels measures time taken to reconstruct a block, left panel shows reconstructions that didn't need to request txs, and the right one shows more than zero requested transactions.

One of the things that people suggested to me, it was that I look at what kind of transactions were being requested, if there was some pattern or things like that. Unfortunately, I didn't have -debug=net enabled which probably would have helped me to answer it.

But with your PR, it makes me think if this isn't an interesting information to pass through tracepoints: all the txids from vtx_missing. What do you guys think? Maybe this is too much data to send? As a real sample from my node, the highest "transactions requested" was 2041 in the last 30 days.

0xB10C commented at 10:04 AM on May 25, 2026:

It's rather impractical to pass lists or sets via tracaepoints, they don't really support it. While it might work by concatenating the txids and passing the length and passing the concatenated bytes, I don't think we should do these kind of hacks if we can avoid it.

I didn't have -debug=net enabled

Note that you only need compact block logging enabled. However, we only print the txids if there were less than 5 transactions missing. I've in the past ran a patched bitcoind that removed this limitation and logged all missing txids. To look up why your node didn't know about a certain transaction, you probably want to have -debug=net enabled. It might tell you that a transaction was replaced, evicted from orphanage, you never got an inv for it, ...

https://github.com/bitcoin/bitcoin/blob/de925455c8025fc1f75d65d981c28b9dfa20e9f7/src/blockencodings.cpp#L222-L232

m4ycon commented at 9:08 PM on May 24, 2026: none

Concept ACK

Also a suggestion on additional info.

stickies-v commented at 11:13 AM on May 26, 2026: contributor

There's also the question about adding more tracepoints now, or investing time into seeing if #35142 is feasible...

I agree it would make more sense to clarify that first before adding more tracepoints.

w0xlt commented at 3:46 AM on May 27, 2026: contributor

I don't think the IPC tracing discussion should block this functionality, assuming there is demand for it.

My understanding is that IPC tracing is a promising direction, but not a drop-in replacement for USDT today. With the current libmultiprocess-generated C++ proxy interfaces, calls are synchronous by default, so a naive tracing implementation where an instrumentation site directly invokes subscriber callbacks would block the Bitcoin Core thread until the subscriber replies.

That callback model is a poor fit for high-frequency tracing, especially for events like raw P2P messages or per-UTXO cache events, unless it is designed with non-blocking buffering, batching, filtering, and backpressure/drop handling.

So I see IPC tracing as a longer-term project with its own tradeoffs, while this PR is a small addition to the existing USDT tracing interface.

DrahtBot removed the label CI failed on May 28, 2026