ci, iwyu: skip subtrees in compilation database #36252

Problem: The IWYU CI job analyzes src/ipc/libmultiprocess/ even though its results are thrown away. Every run prints IWYU suggestions for 14 libmultiprocess files in the non-enforced pass, applies them, and then reverts them with the git restore from #36112. The suggestions also disagree with what libmultiprocess's own IWYU CI reports (it runs with -Xiwyu --error and passes), because the two jobs use different IWYU versions and mapping files. libmultiprocess is the only subtree in this situation; leveldb, crc32c, minisketch and secp256k1 are already excluded from compile_commands.json with EXPORT_COMPILE_COMMANDS OFF. #35361 asked for subtrees not to be linted at all.

Solution: Filter SHARED_EXCLUDED_SUBTREES (from test/lint/lint_ignore_dirs.py) out of compile_commands.json at the start of the IWYU section of ci/test/03_test_script.sh, so subtree sources are neither analyzed nor edited. This is done in the CI script rather than in cmake so that only IWYU is affected: clang-tidy keeps running on libmultiprocess, and no list of subtree targets has to be maintained in cmake/libmultiprocess.cmake (that approach broke -DBUILD_TESTS=OFF in #35418 and was reverted in #35523). One-file change to the CI script.

Testing note: You can confirm this works by checking at IWYU CI logs and confirming that output like /home/runner/work/_temp/src/ipc/libmultiprocess/src/mp/gen.cpp should add these lines: no longer appears.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/36252.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline and AI policy for information on the review process. A summary of reviews will appear here.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #31349 (ci: detect outbound internet traffic generated while running tests by vasild)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

Why not #35418#pullrequestreview-4428862711?

Why not #35418 (review)?

I think excluding all subtrees from IWYU is better than individually setting in CMAKE_EXPORT_COMPILE_COMMANDS inside individual subtrees because:

• It directly addresses the fact that IWYU's instability makes it inappropriate to run on subtrees. IWYU's output changes depending on compiler version, standard libary version, IWYU version, and local mappings so it's not realistic to expect subtrees will be able to pass IWYU checks in bitcoin core, and there's not much reason to care if they do, since benefits of IWYU are faster compile times and more consistent code style, not anything safety related.

• This approach works for all subtrees, instead of requiring every individual subtree to hardcode cmake options.

• This approach only disables IWYU, and does not disable all other tools or checks that use the compilation database. I don't think that turning off clang-tidy, bitcoin-tidy, and other tools for all subtrees is a good idea just to prevent IWYU noise, when IWYU noise can be prevented by just running it on the right files.