guix: Use DOS newlines for SHA256SUMS files #29147

luke-jr commented at 6:49 AM on December 28, 2023: member

OpenPGP specifies that plain text should use CR LF for newlines. By doing so, it becomes possible to include the hashes directly in the .asc file.

(Currently untested, looking for Concept ACKs)

DrahtBot commented at 6:50 AM on December 28, 2023: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

DrahtBot added the label Build system on Dec 28, 2023

maflcko added the label DrahtBot Guix build requested on Dec 28, 2023

in contrib/guix/guix-attest:185 in 767a0862ff outdated

 173 | @@ -174,6 +174,10 @@ basenameify_SHA256SUMS() {
 174 |      sed -E 's@(^[[:xdigit:]]{64}[[:space:]]+).+/([^/]+$)@\1\2@'
 175 |  }
 176 |  
 177 | +unix2dos() {

Sjors commented at 9:42 AM on December 28, 2023:

Maybe refer to https://www.rfc-editor.org/rfc/rfc4880.html section 5.2.4 here.

luke-jr commented at 5:15 PM on December 28, 2023:

Done

maflcko commented at 10:17 AM on December 28, 2023: member

By doing so, it becomes possible to include the hashes directly in the .asc file.

Not sure. Wasn't the goal the exact opposite, so that it is easier to cat the hashes once and as many signatures as one wants?

luke-jr commented at 5:11 PM on December 28, 2023: member

Not sure. Wasn't the goal the exact opposite, so that it is easier to cat the hashes once and as many signatures as one wants?

You can still do that with the content in the file. Furthermore, this change does not require us to produce an attached SHA256SUMS.asc, it only enables us to do so if desired.

guix: Use DOS newlines for SHA256SUMS files

OpenPGP specifies that plain text should use CR LF for newlines.
By doing so, it becomes possible to include the hashes directly in the .asc file.

78afc76a0c

luke-jr force-pushed on Dec 28, 2023

DrahtBot commented at 8:03 PM on December 28, 2023: contributor

Guix builds (on x86_64)

File	commit 4b1196a9855dcd188a24f393aa2fa21e2d61f061<br>(master)	commit 8074d6733d4b205269e8bcc4634cd31317836ea2<br>(master and this pull)
SHA256SUMS.part	`7aefd803a584a152...`	`65de4aa6613795c1...`
*-aarch64-linux-gnu-debug.tar.gz	`735f38fb37b239f8...`	`5568484d1825aebd...`
*-aarch64-linux-gnu.tar.gz	`83a257c74e184b96...`	`d726c6e4dc963817...`
*-arm-linux-gnueabihf-debug.tar.gz	`513acf435b18fab2...`	`a9f14cb4313f37a5...`
*-arm-linux-gnueabihf.tar.gz	`581f911fcca5ae02...`	`97174e1b0060a91d...`
*-arm64-apple-darwin-unsigned.tar.gz	`7a3f2b4e01f27896...`	`8f8881e6bb4dfd25...`
*-arm64-apple-darwin-unsigned.zip	`dff45b014236e990...`	`5315430b83034a34...`
*-arm64-apple-darwin.tar.gz	`0d9696b7bd76b36f...`	`d617ad3b95170ca3...`
*-powerpc64-linux-gnu-debug.tar.gz	`dfe6cfdcd6ad3c6f...`	`126ef87c72027d6d...`
*-powerpc64-linux-gnu.tar.gz	`54c013201daad1ff...`	`51e46d071b8eb911...`
*-powerpc64le-linux-gnu-debug.tar.gz	`8e944b002273b678...`	`ffad1474a9120169...`
*-powerpc64le-linux-gnu.tar.gz	`1ba8199984d7c43a...`	`8b3d038ab1cd8f9c...`
*-riscv64-linux-gnu-debug.tar.gz	`e9abe48aeb7acc68...`	`ae88367256db4103...`
*-riscv64-linux-gnu.tar.gz	`f4d1484dec65b967...`	`7306a9d578851999...`
*-x86_64-apple-darwin-unsigned.tar.gz	`915650aa35e6e322...`	`d99fa8722014a1e0...`
*-x86_64-apple-darwin-unsigned.zip	`6a6139cb415de743...`	`c5dd030eb163eb81...`
*-x86_64-apple-darwin.tar.gz	`c479d099d1df9a85...`	`bc5bb6ea1c2e3506...`
*-x86_64-linux-gnu-debug.tar.gz	`9e27161179f18a4a...`	`a373a579e8358461...`
*-x86_64-linux-gnu.tar.gz	`f69632ca3dc98896...`	`343710fa41163578...`
*.tar.gz	`c04997c9f9e51d80...`	`937a7d44a4f8119f...`
guix_build.log	`3bafc3c44e9b416c...`	`f69a602f08da224f...`
guix_build.log.diff		`5d9a6d1b90b59904...`

DrahtBot removed the label DrahtBot Guix build requested on Dec 28, 2023

luke-jr commented at 1:09 AM on December 29, 2023: member

It seems @DrahtBot stops short of making the full SHA256SUMS file?

maflcko commented at 11:46 AM on December 29, 2023: member

it only enables us to do so if desired.

It would be good to list at least one benefit, otherwise the benefits of this change are unclear.

luke-jr commented at 5:47 PM on December 29, 2023: member

It would be good to list at least one benefit, otherwise the benefits of this change are unclear.

Having a single file to download for the signatures is simpler for end users.

DrahtBot added the label CI failed on Jan 15, 2024

luke-jr marked this as ready for review on Feb 29, 2024

luke-jr commented at 1:54 AM on March 27, 2024: member

It seems this isn't enough - OpenPGP explicitly identifies the data as either binary or text: https://datatracker.ietf.org/doc/html/rfc4880#section-5.2.1

So I guess the only way to support a combined file would be to use --clearsign, detach it manually, then reattach them all together. And then they would only be valid attached. :(

luke-jr closed this on Mar 27, 2024

luke-jr commented at 2:16 AM on March 30, 2024: member

Apparently stripping the final newline would fix this, but a new issue arose: if the signature is detached, it will only verify a pre-canonicalized (DOS-encoded and final newline stripped) file. Some sha256sum -c tools seem to not like this (they treat the \r as part of the filename and can't find it). GPG has a --textmode option that seems to suggest it addresses this, but I can't seem to get it to work.

So IF we're okay with the combined file ONLY being clearsign format (which can then be UNIX newlines again), I can fix this PR. Or someone could spend more time to figure out why --textmode doesn't work as expected.

The UX for downloading two verification files sucks IMO. I think we should just go for the clearsign option, even if it's exclusive.

Thoughts?

luke-jr reopened this on Mar 30, 2024

luke-jr marked this as a draft on Mar 30, 2024

BenWestgate commented at 10:32 PM on May 22, 2024: contributor

The UX for downloading two verification files sucks IMO. I think we should just go for the clearsign option, even if it's exclusive.

Thoughts?

We have ./contrib/verify-binaries/verify.py to make verification UX better. I just reviewed a PR and made a PR to it that makes it able to let a single file be downloaded & verified. Changes to here require changes to that script, right?

DrahtBot commented at 1:38 AM on August 20, 2024: contributor

🤔 There hasn't been much activity lately and the CI seems to be failing.

If no one reviewed the current pull request by commit hash, a rebase can be considered. While the CI failure may be a false positive, the CI hasn't been running for some time, so there may be a real issue hiding as well. A rebase triggers the latest CI and makes sure that no silent merge conflicts have snuck in.

maflcko commented at 8:23 AM on August 20, 2024: member

Are you still working on this? There is outstanding review feedback, asking if this is a breaking change in the verification steps? #29147 (comment)

maflcko commented at 9:51 AM on September 25, 2024: member

The UX for downloading two verification files sucks IMO. I think we should just go for the clearsign option, even if it's exclusive.

Note: When verifying a cleartext signature, gpg verifies only what makes up the cleartext signed data and not any extra data outside of the cleartext signature or the header lines directly following the dash marker line. The option --output may be used to write out the actual signed data, but there are other pitfalls with this format as well. It is suggested to avoid cleartext signatures in favor of detached signatures.

maflcko commented at 8:19 AM on October 28, 2024: member

Closing for now due to lack of progress. Leave a comment if you want this reopened.

maflcko closed this on Oct 28, 2024

bitcoin locked this on Oct 28, 2025