BIP-39: say what separates two words #2237

The BIP never says it. The rule exists — the Japanese wordlist page requires the ideographic space U+3000, added in #130 — but it is not in the BIP text, and the note there holds a caveat: ASCII and ideographic spaces are equivalent "as long as your code never shows the user an ASCII space separated phrase or tries to split the phrase input by the user".

The reference implementation splits the input, on U+0020, and so cannot read back the Japanese sentences it writes: to_entropy raises on all 24 Japanese vectors of its own vectors.json. The fix of the reference implementation is open there: https://github.com/trezor/python-mnemonic/pull/145

This adds three sentences to "From mnemonic to seed": the separator, the Japanese exception, and what it means for software that splits a sentence before normalizing it. No test vector changes, and nothing that is correct today becomes incorrect.

Found downstream in btclib: btclib-org/btclib#258.

🤖 Generated with Claude Code

If this lands and there is interest, I am happy to write the follow-up on the other half of the question: what a reader should do with whitespace that is not the separator — a leading or trailing space, the line break of a paper backup wrapped over two lines, a doubled space from a copy-paste.

The BIP mandates NFKD and says nothing more, so implementations differ and the failure mode is quiet: to_seed hashes whatever it is given, so a sentence that a checksum check would reject still derives a seed — a valid-looking wallet that is not the user's. Two answers look defensible: refuse anything but the canonical single-space form, or collapse any run of whitespace after NFKD. No test vector changes under either; btclib, which I maintain, collapses.

I left it out of this PR because it is new normative text rather than a clarification of what the wordlist page already requires, and it should not hold up the separator sentence. Glad to open it separately if the authors want it, or to drop it if the view is that a Deployed BIP should not grow new requirements.

cc authors: @prusnak, @ebfull, @slush0, @voisine

On several occasions I raised my will that we should drop non-English wordlists from BIP39, but I was always blocked by editors.

Since I am not able to edit my own standard, I don't really care about issues related to non-English use.

prusnak I do understand your position, and I agree with you about English only.

That said, I do care about fixing inconsistencies, and I would love it if your "I don't really care" could become an ACK.

Also, the possible follow-up — what to do with whitespace that is not the separator, a leading or trailing space and so on — concerns English too, not just Japanese.