RFC: Support for CMake `codegen` target #284

CMake allows different approaches for running static analysis tools over a codebase: co-compiling and using a compilation database.

Bitcoin Core uses the latter. This approach has many benefits: for example, the ability to filter entries, which allows us to maintain a list of sources where IWYU warnings are enforced. When processing the compilation database, IWYU expects to find every source and header. This is not the case when the database is generated solely during the configuration step, as it lacks the headers and sources that will be generated during the subsequent build step.

Modern CMake provides a way to generate expected headers and sources beforehand using the built-in codegen target. This approach is currently used in the Bitcoin Core IWYU CI job.

However, the Libmultiprocess project seems inherently incapable of supporting the codegen target. While it is almost trivial to amend add_custom_command in the target_capnp_sources function with the conditional CODEGEN option, doing so is not possible for the capnp_generate_cpp function shipped by libcapnp-dev or similar packages.

There are several ways we could tackle this issue:

1. Execute a normal build step: This guarantees that all files will be generated without relying on codegen at all.

2. Introduce a custom target: We could create a custom target to generate all files beforehand, though this might be challenging.

3. Use CMake's co-compilation: Switch to the co-compiling approach, though I am unsure about the usability of this method for the CI job.

4. Alternative solutions: Open to other ideas.

I am hoping to gather my colleagues' opinions on the best way forward.

cc @purpleKarrot

I'm not sure I understand the background for this request. Is it just related to https://github.com/bitcoin/bitcoin/issues/35361 or there other reasons for it?

Naively I would think a codegen target that generates files across an entire codebase with no control over what is generated would not be that useful, because I'd think linters & checkers would want to be able to check individual files and targets. Also needs of different checkers would might be different. Some might not care about generated files files at all, others might not care about generated files but still need them to check other files, and others might actually check generated files.

For https://github.com/bitcoin/bitcoin/issues/35361 specifically, I would just expect bitcoin build to declare an iwyu_codegen target or similar that generates whatever files the IWYU check needs. Maybe the capnp_generate_cpp function can assist with that though.

Commands like capnp_generate_cpp should be discouraged in current cmake code. Populating variables with lists of source files is the style of the CMake version 2 era, which ended about 10 years ago. Version 3 era (ie. the last decade) was about target properties. Version 4 era (ie. now) is about file sets. Capnproto is lagging behind pretty badly, unfortunately.

Naively I would think ... I would just expect ...

Clients may have a gazillion use cases why they want generated files to be available without unnecessary work. Static analysis is just one. Code completion is another. There are use cases that we don't know or understand. As library authors, we are not in a position to question downstream clients.

As @hebasto wrote, CMake supports different approaches for static analysis. A library should not prevent or question what approach is used downstream. There may be clients that want to use co-compilation, and others that prefer a compilation database. There should be no reason why a library does not support both cases.

@purpleKarrot I am unsure what takeaway I am supposed to have from your comments other than "everyone is stupid because they are using cmake wrong."

You also seem to be quoting my comment about the behavior I'd expect appearing to disagree with it, but then just agree and restate my concern.

Appreciate you trying to help here, but it would be great if you could be more concrete and provide feedback that's more obviously relevant to the issue at hand.

they are using cmake wrong

I would not say "wrong". There was a time when populating variables was the idiomatic approach. But better approaches for code generation have emerged over the years and will continue to do so. Supporting the codegen target is really just a matter of adding a single keyword in the right place. Unfortunately, this place is in the capnproto codebase.