bip85: require direct HMAC use when sufficient #2295

No description provided.

cc: @akarve

@scgbckbone Requiring derivation paths that consume <= 512 bits to use the raw HMAC output creates a few problems:

1. Inconsistent codepaths within the same application (harder to test and prove correct)
2. A "back and forth" correctness dance during application development where we realize things like "oh, powers of 2 do not require re-rolls for DICE". Or "my app consumes a variable number of bits (like RSA key generation), how do I partition this into cases under and above the 512 bit limit?" This will lead to unwarranted test vector complexity and more carve-outs for backward compatibility (as was proposed for DICE).

A more consistent and sustainable policy would be to require that any application that could, under any of its derivation paths, consume more than 512 bits of pseudorandomness MUST use the DRNG. All other apps MUST use the HMAC output. For instance PWD BASE85 was designed so that all of its paths obey the 512 bit ceiling. In exchange for the gain in consistency and clarity, it is true that we trade away hardware wallet reach for the time being. That is a fundamental limitation of the already finalized and accepted DRNG application. I would rather see us put energy into noting in the BIP the fact that apps that depend on the DRNG have little to no hw wallet support as of today. The RSA app is already in this category.

cACK: All other apps MUST use the HMAC output.

This is worth adding to the BIP. I agree with akarve an application intended to support every HWW should try to use <= 512-bits to avoid the DRNG.

However for the BIP93 app we run into a conflict where BIP93 can't be implemented with only 512-bits so its unfortunately a DRNG app. Future codex32 profiles like extended private keys or passwords would need even more bits (for shares).

@scgbckbone Requiring derivation paths that consume <= 512 bits to use the raw HMAC output creates a few problems:

1. Inconsistent codepaths within the same application (harder to test and prove correct)

2. A "back and forth" correctness dance during application development where we realize things like ["oh, powers of 2 do not require re-rolls for DICE"](/bitcoin-bips/1958/#issuecomment-5655138809). Or "my app consumes a variable number of bits (like RSA key generation), how do I partition this into cases under and above the 512 bit limit?" This will lead to unwarranted test vector complexity and more carve-outs for backward compatibility (as was proposed for DICE).

A more consistent and sustainable policy would be to require that any application that could, under any of its derivation paths, consume more than 512 bits of pseudorandomness MUST use the DRNG. All other apps MUST use the HMAC output. For instance PWD BASE85 was designed so that all of its paths obey the 512 bit ceiling. In exchange for the gain in consistency and clarity, it is true that we trade away hardware wallet reach for the time being. That is a fundamental limitation of the already finalized and accepted DRNG application. I would rather see us put energy into noting in the BIP the fact that apps that depend on the DRNG have little to no hw wallet support as of today. The RSA app is already in this category.

both are non-reasons, sorry.

1. you fear to add if statement to the code
2. is any app with at least one user (besides your reference implementation) using this DICE app ?

I have no idea what your're trying to optimize for - but definitely not usability.

cACK: All other apps MUST use the HMAC output.

This is worth adding to the BIP. I agree with akarve an application intended to support every HWW should try to use <= 512-bits to avoid the DRNG.

However for the BIP93 app we run into a conflict where BIP93 can't be implemented with only 512-bits so its unfortunately a DRNG app. Future codex32 profiles like extended private keys or passwords would need even more bits (for shares).

I really cared about the BIP93 app here - but in this current form I rather not implement it and do my own thing

both are non-reasons, sorry.

1. you fear to add if statement to the code
2. is any app with at least one user (besides your reference implementation) using this DICE app ?

I have no idea what your're trying to optimize for - but definitely not usability. @scgbckbone The tradeoff is between consistent and easier to get correct apps on one hand, and hw wallet reach on the other. If we can be reasonably certain that few if any hw wallets will support SHAKE256 or the DRNG in the near future (my quick survey showed almost no support) then your direction makes sense. What would be the point of apps that don't run on hw wallets? That's an earnest question. Any hard data on hw wallet support or lack thereof is appreciated. Include that in this thread wherever you find it. As a sanity check, do any wallets support the RSA application?

If we go this route (I'm inclined, pending some more data) then I think stochastic apps can fail for a given index if they run out of entropy. Stochastic apps are really the wrinkle as in RSA, DICE, etc. you don't know how much entropy you will need. I would further request that each app that could use the DRNG under some parameterization declare precisely which params and ranges will use the HMAC and which the DRNG. That should keep the test cases pretty clean.

For app-93, the precise declaration HMAC/DRNG boundary under this proposal can be specified exactly:

• Master Seed, profile 0: always use the HMAC output directly. Even a 512-bit seed consumes exactly 512 pseudorandom bits; any remaining payload bits are zero padding.
• Shares with payload_len <= 102: use the HMAC output directly (5 * payload_len <= 510 bits).
• Shares with payload_len >= 103: use BIP85-DRNG (5 * payload_len > 512 bits).

BIP85 HWW/DRNG follow-up

Following up on the HWW data @akarve asked for. I checked the devices currently supported by Bitcoin Core HWI, since that seems like a reasonable set of Bitcoin hardware wallets to evaluate.

The result is quite different from “few if any hw wallets will support SHAKE256 or the DRNG”:

• Blockstream Jade hardware wallet supports the BIP85 RSA application today. Jade derives the 64-byte BIP85 RSA entropy, seeds SHAKE256 with it, and uses the SHAKE256 stream as the RNG for mbedtls_rsa_gen_key(): source. Jade's public API documentation also explicitly describes BIP85 entropy initializing a SHAKE256 PRNG.
• Ledger Nano S already has SHAKE256 in its device-specific SDK. The API_LEVEL_LNS branch implements cx_shake256_init_no_throw() and arbitrary-length SHAKE256 output: header, implementation. The current Ledger Secure SDK for Nano X and newer devices exposes SHAKE256 as well: source.
• BitBox02 vendors the RustCrypto SHA3 implementation in its firmware tree, including the SHAKE256 XOF: source. SHA3/Keccak is an optional dependency in the firmware today, so this isn't the same as saying its Bitcoin app already exposes BIP85-DRNG, but the implementation is already there.
• Trezor does not appear to expose a SHAKE256 API today, but it already ships the SHA3/Keccak implementation and underlying Keccak permutation: source. So SHAKE256 would be an extension of crypto machinery already on the device, not introduction of an unrelated primitive.
• KeepKey and Passport likewise build Trezor's SHA3/Keccak crypto sources: source.
• For Coldcard and legacy BitBox01/Digital Bitbox, I did not find SHAKE256 or SHA3/Keccak support in the public firmware. Those look like the real cases where supporting BIP85-DRNG would require adding the primitive rather than wiring up functionality that already exists.

So among HWI's seven device families, three already have SHAKE256 itself available in their codebase (Jade, Ledger, BitBox02), two more already have the underlying SHA3/Keccak machinery (Trezor, KeepKey), and Jade already implements the BIP85 RSA application specifically.

That does not mean every HWW currently implements BIP85-DRNG. It means that “almost no hardware support” substantially overstates the implementation barrier. For most of these devices the remaining issue is integration, code size and audit work, not whether the hardware can reasonably support SHAKE256.

Given the HWW evidence above, I don't think present-day vendor integration is a strong enough reason to make a permanent per-path HMAC/DRNG split normative.

I prefer the application-level rule: if any valid path in an application can require >512 bits, use the DRNG consistently for that application; otherwise use the HMAC output directly. BIP85 should clarify this so new apps aren't left to decide.