depends: Hash included makefiles in package checksums #34239

This PR fixes an issue where modifications in included files (e.g. packages/qt_details.mk) do not trigger a rebuild of the parent packages (native_qt and qt).

This addresses an oversight in 248613eb3ee034bf143821a51635e697dc114e6c from #30997.

Reproduction

On master (@ 595504a43209bead162da54a204df7d140a25f0e), modifying the included makefile does not change the build ID:

$ cd depends
$ gmake print-qt_build_id HOST=x86_64-w64-mingw32
qt_build_id=b2ce790473c
$ gmake print-native_qt_build_id HOST=x86_64-w64-mingw32
native_qt_build_id=70e1e5164c5
$ echo "" >> packages/qt_details.mk
$ gmake print-qt_build_id HOST=x86_64-w64-mingw32
qt_build_id=b2ce790473c
$ gmake print-native_qt_build_id HOST=x86_64-w64-mingw32
native_qt_build_id=70e1e5164c5

With this patch

The checksum calculation now parses include directives and adds those files to the hash. The IDs now update correctly:

$ cd depends
$ gmake print-qt_build_id HOST=x86_64-w64-mingw32
qt_build_id=9a6ebf79cb3
$ gmake print-native_qt_build_id HOST=x86_64-w64-mingw32
native_qt_build_id=6ad78a3f644
$ echo "" >> packages/qt_details.mk
$ gmake print-qt_build_id HOST=x86_64-w64-mingw32
qt_build_id=ca820665c52
$ gmake print-native_qt_build_id HOST=x86_64-w64-mingw32
native_qt_build_id=082e4cb2364

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--006a51241073e994b41acfe9ec718e94-->

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/34239.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process.

If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

Approach NACK. We've removed sed usage (in #19761 + some later changes) because its generally not portable, can fail silently, and previously its usage was either poorly, or completely undocumented (this change also lacks any inline documentation). I'd rather not reintroduce it.

Approach NACK. We've removed sed usage (in #19761 + some later changes)

Yes, I remember that. However, sed is still used in both depends and contrib/guix:

$ git grep -l -w sed -- depends/Makefile contrib/guix/
contrib/guix/INSTALL.md
contrib/guix/guix-attest
contrib/guix/guix-build
contrib/guix/libexec/build.sh
contrib/guix/manifest.scm
depends/Makefile

... because its generally not portable, can fail silently, and previously its usage was either poorly, or completely undocumented (this change also lacks any inline documentation). I'd rather not reintroduce it.

I'll look into alternative implementations though.

The single usage in depends is straightforward, and needed to substitute runtime values. The usage in Guix is in the docs, a workaround for a bug in CMake and for dealing with runtime values (sha256sums).

Reworked to avoid using sed.

Rebased.

Using grep seems a bit better, but still not sure how to judge this in comparison. That said, the change seems fine to me mechanically.