BIP39: 'recommended size' -> 'allowed size' #472

#471 for background

Clarifies that the BIP only allows entropy between 128-256 bits. @dabura667 @prusnak @dcousens @rubensayshi

Perhaps put it more bluntly.

"This BIP restricts the allowed sizes of ENT to be a minimum of 128 bits, to a maximum of 256 bits. Sizes outside of this range are disallowed."

And perhaps @prusnak could clarify whether, within that range, if [128, 160, 192, 224, 256] are the only suitable options or not.

Since, as is, all we have is:

The following table describes the relation between the initial entropy length (ENT), the checksum length (CS) and the length of the generated mnemonic length

Which is by no means a specification of valid sizes, nor even a recommendation.

Yes, these 5 options are the only valid ones (meaning 12, 15, 18, 21 or 24 words respectively).

I am closing the PR since this has a greater impact than what I'd hoped to raise in #471 it should be @prusnak

Please reopen and accept.

@prusnak @luke-jr

regardless of your intentions of what this BIP should have had as restrictions I really don't feel it's right to just merge a BC break in like that.

do BIP authors just have the power to change their BIPs whenever they feel like it?
what if @sipa decides to change BIP42 so allow more than 21mil BTC?
or Gavin decides to change the P2SH address version byte just because he's tired of seeing 3xxxxx?

Go read my last comment in #471 and provide some facts instead of feelings.

@prusnak I don't think that it matters at all what you think this spec should and shouldn't be used for, you've put a spec out there that allowed more than 256 bits and people have started using it for that.

you now break backwards compatibility of this BIP, turning wallets that have advocated being "BIP39 compatible" (which unfortunately is a "thing" people consider to be good / important) into no longer being BIP39 compatible.

it's not a question of if you think people are using BIP39 for the wrong thing or in the wrong way, it's a spec that people have started using, without a restriction to the size of the entropy (except that at 1024 bytes it will break) and you're now breaking this, which I don't think is supposed to be done this carelessly with a BIP (that is wildly used).

Ok, so no facts again. I am really disappointed by your obnoxious behaviour.

Anyway, if you DO have legitimate usecase, just create BIP-139 which says "this is BIP-39 with entropy upper limit raised to 1024 bits" and use that. Nobody is stopping you from doing that.

fact: the restriction was not there before, only a recommendation.

fact: people have been using it with more than 256 bits of entropy, thinking that they are compatible with BIP39 and their seeds will (forever) be interoperable with other BIP39 compatible wallets.

fact: you just carelessly broke that, completely ignoring the fact you might actually fuck people over down the line who rely on BIPs not randomly changing.

Afaik, accepted BIPs can't be changed, even by the original authors. Trivial changes like fixing typos may be an exception, but this doesn't sound like a trivial change.

I tend to agree with @schildbach here. Changing a recommendation to a hard restriction so long after the fact is wrong.

If this was a new BIP, and problems come to light in the implementation phase, these kind of changes can be considered. But this one has been in this state for years.

The correct way to handle this would be to define a new BIP that adds the restriction.

(Also as there is an effective hard limit of 8192 bits after which the algorithm breaks down, it would IMO be better to just add that as #471 did)

Well, I think the wording of "recommended size" was unfortunate, but it was lost in translation - we're not a native speakers. Our intention was clearly to use 128-256 range; obviously missing the check for this is also our mistake. We even did not consider to use more entropy, as you can see there's limit of calculating checksum, which wasn't known before. Also, our motivation was wallet interoperability and although libraries does not handle the limit (yet), I assume many UIs and applications allow only 12-24 words to be used, like TREZOR does.

Also we don't think there's any real usecase for such huge mnemonics, because - as Stick stated well - people may have troubles to write even 24 words correctly. Using longer mnemonics is actually less fault-proof.

I see that as our mistake in specs, which should be fixed before bigger damage will happen. People may started asking "why some wallets generate only 24 words, when others have longer and more secure seeds? I want one, too!". So I appreciate the finding, maybe I would use more diplomatic wording in discussion, but I agree with this PR.

Thanks for owning it @slush0.

The difficulty for those of who make the library implementations is how we move forward. We can easily add the restrictions, but we risk stopping developers being able to allow users to "fall-back" on the old behaviour to recover their [now, incompatible] mnemonic.

Likely it'll end up being an API with the safety checks, and a "upgrade" method for those who need it that will exist for a year or so to allow developers to migrate users across without re-implementing the BIP themselves.

Not a fantastic situation by any means.

So far there was zero examples of users given that have mnemonics outside of this range. Are we talking about a real problem here or just a hypothetical one?

@prusnak it is a real problem, but it is also impossible to gauge the extent (unless the wallet provider logged that data). The issue is, it was allowed.

I personally know several users who use non-standard numbers of words outside of the accepted values of 12, 15, 18, 21 or 24 words.

So which wallet is it?

Not sure I like the idea of retroactively restrict values.

GreenAddress's wallets supports 27 words mnemonics.

@prusnak Two were listed in the other PR, lets not flog a dead horse. TREZOR has a strong interest in BIP39 being 128-256 bits only, but I'd suggest you search Github yourself and see how people interpreted your spec.

well I think @greenaddress alone is enough, but the real issue imo is not the people we can find, it's the people we can't find and someday will run into this problem when their mnemonic can't be imported into any (other) wallet.

some wallets and users rely on the mnemonic being portable to other wallets as a anti-vendor-lockin mechanism.

some wallets and users rely on the mnemonic being portable to other wallets as a anti-vendor-lockin mechanism

That's exactly what we are trying to achieve here, if that is still not obvious. 27 word mnemonic is quite the opposite.

I think wallets that are generating stuff outside of the range, should allow importing this stuff for foreseeable future, but should change the defaults to be in the range.

I see this as the only way how to fix this situation.

the only way to "fix" this is by not breaking backward compatibility!

Why not change this back to a recommendation, and additionally add a limit that's inclusive of what is in use already?

@luke-jr do you mean up to what the bip and reference implementation allowed as of before these PRs (checksum limit) or what people can manage to find by looking around (which imho as @rubensayshi rightly said is bound to miss out on some use) ?

For clarity I think the former is ok, the latter is problematic.

I don't think it was ever implied that users can make up phrases on their own, so I would personally be happy with supporting what existing implementations generate. The alternative is fine as well, though.

@luke-jr in which case I think there exist some implementation that generate up to the checksum limits.

And is not even just libraries that allow that, even apps like this one https://iancoleman.github.io/bip39/ support far more than 24 words

For instance I just generated:

lend zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo zoo vocal puzzle there

By simply ticking the "Supply my own source of entropy" and passing in a lot of 1

and this is more common than you would expect as some people may not trust hardware wallets generated mnemonics and hardware wallet manufacturers sometimes are actively suggesting to users to generate their own outside of the device (example https://www.reddit.com/r/Bitcoin/comments/5c1wat/finally_got_a_ledger_nano_s_its_great/d9t8ctd/) - that's not to say wallets support that necessarily but some wallets (Ledger, GreenAddress) support hex seeds obtained from any mnemonic of any length.