Canonical signature tests contain an invalid signature #4625

The first test in src/test/data/sig_canonical.json is 300602010002010001 which equates to a signature of R=0 and S=0.

This is an invalid test since according to the ECDSA spec, a valid signature must be in the range [1, n-1]. Given 0 is not in that range, the signature in question is invalid and will be found as such during a verify.

While it's true that a canonical test doesn't necessarily need to check this condition since it will fail during verification anyways, at the very least it seems like a bad idea to intentionally test an invalid signature is considered canonical.

The following comments from @sipa on IRC concur:

12:33 < sipa> no, i mean, it's not necessary that the canonicality check verifies this, as normal signature verification would fail anyway 12:34 < sipa> but i agree it's weird to have a 'slightly relaxed' definition for canonicality in the code, and test for it 12:34 < sipa> thanks for reporting

I would personally go further and change IsCanonicalSignature to consider a signature with an R or S value outside of the valid range as defined in the ECDSA spec as non-canonical and move the test in question to sig_noncanonical.json. It is an extremely cheap check and, in my opinion, it's better to be paranoid.

So given the above, this issue can be resolved in one of two ways:

• Simply remove the invalid signature from sig_canonical.json
• Update IsCanonicalSignature to consider R=0, R >= N, S=0, and S >= N as a non-canonical signature and move the test to sig_noncanonical.json. Perhaps then also add a couple more tests which contain R >= N and S >= N.

@laanwj Given that the signature in question is no longer tested, after #5004, and that the raw files are going to be removed in #5491 I think we can close this.