BIP 85: Deterministic Entropy From BIP32 Keychains

ethankosakovsky commented at 5:21 pm on April 16, 2020: contributor

I would like to request a BIP number for this BIP

ethankosakovsky force-pushed on Apr 16, 2020

ethankosakovsky commented at 6:03 pm on April 16, 2020: contributor

Experimenting with splitting the applications to a separate BIP like 32 and 44 are separated. https://github.com/ethankosakovsky/bips/blob/entropy_apps/bip-entropy-applications.mediawiki

junderw commented at 1:57 am on April 17, 2020: contributor

So the problem you’re trying to solve is:

“I have way too many wallets/keys/bip39 and I want to back them all up with one key”

In which case we are back to the bip39 discussion (or retrospect) of:

“How do we do versioning?”
“Should we store data pertaining to wallets and encode it somewhere?”

etc.

In your examples, there’s a pretty complicated derivation scheme of language, word count, app number, etc.

App number could be hard coded into the app, sure… but how are you going to relay the language / word count?

Would it make sense to give an app a 12 word English BIP39, tell it to create a master key for a 24 word Japanese BIP39 phrase, then have it derive the wallet? Even if it did make sense? How would I relay that info? So do I enter my root BIP32 xprv as my backup and then select which language / word count to pass through?…

Then let’s take a few steps back.

As a wallet dev do I want to use a master key that someone is entering into every wallet they encounter?

I have seen a lot of instances where a theft occurs and the victim gets upset and threatens to sue the wallet dev only to find out that they wrote down their phrase into a shady website to get an airdrop (unrelated to the wallet)… So getting devs to support this scheme is going to take a lot more effort than even BIP39 (which isn’t perfect adoption even yet).

This needs a lot of major re-thinking, and to be honest I’m not sure the problem it is trying to solve is much of a problem.

ChristopherA commented at 4:41 am on April 17, 2020: none

I’m not sure I’m comfortable with your approach of using multiple keys to create each word of BIP39. Just used the derived key to create new master seed that is converted to BIP39, SLIP39, Lightning recovery, future encodings, etc..

– Christopher Allen

ChristopherA commented at 4:50 am on April 17, 2020: none

With our #LetheKit offline/airgap device, and our new bc-seedtool-cli project, we are doing HKDF-SHA256 to convert raw randomness of dice or other sources of randomness into a master seed.

That we’ve standardized on this means that both of the above tools, plus @iancoleman’s [BIP39] tool, you can enter the same dice on all three devices and end up with the same master seed and BIP39. This means you can independently confirm that all three devices do their encoding correctly.

– Christopher Allen

ChristopherA commented at 5:08 am on April 17, 2020: none

Blockchain Commons’ use case for this proposal is as follows:

You have a previously created titanium key (say generated as per our free #SmartCustody book) with BIP39 words stamped on it, in a safe deposit box in a vault. In that vault you also have a #LetheKit, which conveniently is designed to fit in your safety deposit box with your key. You previously confirmed that #LetheKit executes code correctly when you stored it away.

You enter the vault and bring a new 12 volt battery and adapter to charge your #LetheKit, and you load your words from the titanium blank (or from a SLIP39 recovery set of shards that you brought with you), which creates a master seed internally that allows you to confirm your BIP39 words are valid, or deal them to a deck of SLIP39 shards, or with a path, create a child zprv or zpub.

With this proposal, you could also enter a path to create either a new master seed, which can be saved as a BIP39 set of words (or SLIP39 or to or a QR code) that can be used as a child seed for a Ledger, Trezor, mobile phone, or some other device you take out of the bank vault with you.

You turn off #LetheKit, your master seed evaporates, and your master seed never leaves the vault. But a derived child (either as an zprv, zpub, QR on your phone, new deck of SLIP39 shards, etc.) goes out with you to be used.

We also have a new multisig iOS wallet called FullyNoded 2 which can go into the vault with you, which has a PSBT needing an offline key. You go to #LetheKit, restore the master, recreate the child needed for the PSBT, turn off #LetheKit, then go outside and transmit your fully signed PSBT. Again, the master key never leaves the vault or is on an networked device, but in 10 minutes you can sign an important transaction.

Offline master seeds, child keys, #LetheKit, FullyNoded 2, and clever use of multisigs so even if your bank is compromised you have alternatives, means that you have better than custodial bank class security for your keys if you want it. Currently this level o security maybe using the Thale’s class of equipment $250K. The #LetheKit and an iPhone and the rent on a couple of safe deposit boxes in different banks is several orders of magnitude less expensive.

I’m not recommending this level of security for everyone (see the risk modeling and adversary analysis section of the #SmartCustodby book), but that you can do it all is pretty powerful.

– Christopher Allen

doc-hex commented at 5:43 pm on April 17, 2020: none

I feel strongly this should be a single BIP, not multiple.

ethankosakovsky commented at 11:46 pm on April 18, 2020: contributor

@junderw There has been a bitcoin-dev discussion on it, I summarized my points previously here https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-March/017685.html

I disagree this is another BIP39 situation, although I would agree about BIP39, there is a lot wrong with it :-) But this BIP is not concerned with BIP39 specifically, but BIP32. Ultimately all derivation schemes, no matter how poorly conceived, boil down to a BIP32 root, and we’re concerned here about how to create child derivations for incompatible derivation (BIP39 vs Electrum, vs Bitcoin Core’s WIF as a root, SLIP39 etc.) schemes such that everything remains a child of the one root parent keychain.

Additionally, BIP is exactly an argument against funds loss because with the current need for many incompatible seeds for various wallets we may or may not need to use, or may need in the future, the more backups we need to maintain, the worse the problem gets. The reason for having standardized paths is to make it less difficult to forget lose your paths if you were to lose that information. However, BIP32 allows for arbitrary paths, so the problem is no better or worse than BIP44/49 etc, and users are still free for custom applications to chose their own custom paths, with all the caveats.

junderw commented at 11:42 am on April 19, 2020: contributor

this is another BIP39 situation

I never said this is another BIP39 solution, I merely said the considerations you need to think about are similar to those that were brought up during the BIP39 discussions.

Let me simplify my concerns into a few simple questions:

I have a BIP32 root key that I want to use with your protocol: xprv9s21ZrQH143K2vmu4Nkp4Q2MmH8Ptn2qwHqdZjaPgKFtjd6PTRK3FW4XNmsSEYLYaa6rBVcugaRbECQ3uQ4z4bfFFwAeYNGjq6GovTYCatw

How do I know it’s a key for your protocol and not just a normal BIP32 wallet?
How many apps am I using it with? If I only have the string above, how can I tell?
If the protocol adds a backwards incompatible change, how do I know which version of the protocol to use?
(Using your BIP39 example) How do I know which phrases I’ve used? (ex. In BIP44, there’s a “gap limit of 20 empty addresses before stopping checking of addresses)

The idea of adding a layer of abstraction to entropy generation is not a good or bad idea. As long as you’re willing to take the security hit, and you understand it, that’s fine.

The problem I am proposing is:

The fine details of how this protocol works and how it deals with recovery need to be clear. (Do we create a JSON file that continuously grows with metadata? Do we just tell the user to keep track of everything they use and an xprv key? Do we make every wallet derive every known app type including the BIP39 stuff with all languages all word counts, and give a gap limit for each phrase? (performance hit on startup) Do we leave it up to the wallet devs to decide (in which case you get a lot of interoperability problems.)

That is the most crucial part of this protocol and is noticeably absent.

ethankosakovsky force-pushed on Apr 29, 2020

ethankosakovsky commented at 1:22 am on April 29, 2020: contributor

Rebased and squashed.

ethankosakovsky commented at 1:30 am on April 29, 2020: contributor

@junderw The point of this BIP is not to redesign deterministic key derivation, but to act as a middleware to solve the problem proliferation (a problem which may not exist if everything was restarted with hindsight maybe). I agree that there is a need for wallet meta data but that seems to be out of scope for this proposal unless you have some suggestions?

It would seem to me that a “wallet metadata” system could be an extensive and independent BIP which would describe how to reconstruct any wallet. Given the scope of BIP32 it would seem just as relevant to a BIP32 as to this BIP or any other derivation scheme. Currently all derivation schemes boil down to BIP32, so it would be prudent to have a wallet descriptor to describe how a wallet was derived and the paths used in the derivation, regardless of the starting point scheme (e.g. BIP32, BIP39, SLIP32 etc).

in bip-entropy.mediawiki:228 in 97be6e609a outdated

223+===HEX===
224+Application number: 128169'
225+
226+The derivation path format is: <code>m/83696968'/128169'/{num_bytes}'/{index}'</code>
227+
228+`16 >= num_bytes <= 64`

doc-hex commented at 5:16 pm on April 29, 2020:

should be 16 <= num_bytes <= 64

luke-jr commented at 2:28 pm on April 30, 2020: member

This is missing a section for backward compatibility.

luke-jr added the label New BIP on Apr 30, 2020

junderw commented at 11:25 pm on April 30, 2020: contributor

The point of this BIP is not to redesign deterministic key derivation

That’s not what I am saying… the point of this BIP is to be standardize something that is useful so that everyone isn’t doing a million different things and things can be interoperable.

How would two wallet apps, without communicating with each other, exchange the Deterministic Entropy with each other, validate whether the data was meant to be used with this BIP and validate what features of this BIP it is using?

Since you won’t answer my question, please tell me if this answer is correct:

“You would need the user to present a BIP32 xprv string, we will assume any valid xprv could be used with this BIP, then we will derive entropy according to every possible feature laid out in this BIP, and show these pieces of entropy to the user and ask them for each “do you recognize this entropy? Would you like to use it?””

Or is it more like:

“You need to present a valid xprv. The application hard codes 1 or more uses of the xprv in accordance with this BIP directly into its source code, the end user may not choose what they want to use it for / what language BIP39 phrase they can pass through. If someone mistakenly inserts an xprv meant for use in BIP32, there’s no way to tell, so continue as if it was meant for our BIP anyways.”

These are the types of answers I was looking for, are either of them correct. If either of them are correct the utility of this BIP is extremely low and would need a major reworking. (a big one would be a new encoding format to prevent mistakenly using a BIP32 key that is just a BIP32 key)

luke-jr commented at 11:44 pm on April 30, 2020: member

the point of this BIP is to be standardize something that is useful so that everyone isn’t doing a million different things and things can be interoperable.

I’m not sure it actually does that. Wouldn’t the BIP need to itself define a PRNG to use the data with, and the outputs thereof?

in bip-entropy.mediawiki:9 in 6ef879aa53 outdated

0@@ -0,0 +1,257 @@
1+<pre>
2+  BIP: XXXX
3+  Layer: Applications
4+  Title: Deterministic Entropy From BIP32 Keychains
5+  Author: Ethan Kosakovsky <ethankosakovsky@protonmail.com>
6+  Comments-Summary: No comments yet.
7+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-XXXX
8+  Status: Proposed
9+  Type: Standards Track

luke-jr commented at 7:18 pm on June 1, 2020:

This looks more like Informational?

luke-jr renamed this:
~~Deterministic Entropy BIP~~
BIP 85: Deterministic Entropy From BIP32 Keychains
on Jun 1, 2020

luke-jr commented at 7:21 pm on June 1, 2020: member

Assigned BIP number 85

in bip-entropy.mediawiki:48 in 6ef879aa53 outdated

43+
44+==Specification==
45+
46+We assume a single BIP32 master root key. This specification is not concerned with how this was derived (e.g. directly or via a mnemonic scheme such as BIP39).
47+
48+For each application that requires its own wallet, a unique private key is derived from the BIP32 master root key using fully hardened derivation path. The resulting private key (k) is then processed with HMAC-SHA512, where the key is "bip-entropy-from-k", and the message payload is "k": <code>HMAC-SHA512(key="bip-entropy-from-k", msg=k)</code>. The result produces 512 bits of entropy. Each application SHOULD use up to the required number of bits necessary for their operation truncating the rest.

junderw commented at 4:28 am on June 4, 2020:

Remove quotes from “k” here to discourage confusion.

0For each application that requires its own wallet, a unique private key is derived from the BIP32 master root key using fully hardened derivation path. The resulting private key (k) is then processed with HMAC-SHA512, where the key is "bip-entropy-from-k", and the message payload is the private key k: <code>HMAC-SHA512(key="bip-entropy-from-k", msg=k)</code>. The result produces 512 bits of entropy. Each application SHOULD use up to the required number of bits necessary for their operation truncating the rest.

ethankosakovsky commented at 12:13 pm on June 11, 2020:

added

in bip-entropy.mediawiki:220 in 6ef879aa53 outdated

215+INPUT:
216+* MASTER BIP32 ROOT KEY: xprv9s21ZrQH143K2LBWUUQRFXhucrQqBpKdRRxNVq2zBqsx8HVqFk2uYo8kmbaLLHRdqtQpUm98uKfu3vca1LqdGhUtyoFnCNkfmXRyPXLjbKb
217+* PATH: m/83696968'/39'/0'
218+
219+OUTPUT
220+* DERIVED ENTROPY=7040bb53104f27367f317558e78a994ada7296c6fde36a364e5baf206e502bb1

junderw commented at 4:31 am on June 4, 2020:

This is the same DERIVED ENTROPY from HD-Seed WIF

in bip-entropy.mediawiki:226 in 6ef879aa53 outdated

221+* DERIVED WIF=xprv9s21ZrQH143K2srSbCSg4m4kLvPMzcWydgmKEnMmoZUurYuBuYG46c6P71UGXMzmriLzCCBvKQWBUv3vPB3m1SATMhp3uEjXHJ42jFg7myX
222+
223+===HEX===
224+Application number: 128169'
225+
226+The derivation path format is: <code>m/83696968'/128169'/{num_bytes}'/{index}'</code>

junderw commented at 4:32 am on June 4, 2020:

specify which bytes.

If I use 16, should I just the first 16 bytes or last 16 bytes? Or something else.

ethankosakovsky commented at 12:13 pm on June 11, 2020:

added

in bip-entropy.mediawiki:89 in 6ef879aa53 outdated

84+Derivation path uses the format <code>m/83696968/' + /app_no' + /index'</code> where ''app_no'' path for the application, and `index` in the index.
85+
86+===BIP39===
87+Application number: 39'
88+
89+Entropy is truncated to the number of bits required to map to the relevant word length 128 bits for 12 words, 256 bits for 24 words.

junderw commented at 4:38 am on June 4, 2020:

Truncate can refer to chopping off data from the beginning or end. Specify one or the other. Colloquially most people understand truncate to mean “from the end” which in the case of a byte array is that of least significance (or higher index from a coding perspective)

0Truncate trailing (least significant) bytes of the entropy to the number of bits required to map to the relevant word length 128 bits for 12 words, 256 bits for 24 words.

ethankosakovsky commented at 12:13 pm on June 11, 2020:

added.

ethankosakovsky force-pushed on Jun 11, 2020

Initial commit of entropy BIP 96927344e7

ethankosakovsky force-pushed on Jun 11, 2020

ethankosakovsky commented at 12:13 pm on June 11, 2020: contributor

@luke-jr ready for merge.

luke-jr merged this on Jun 12, 2020

luke-jr closed this on Jun 12, 2020

aido cross-referenced this on Jan 4, 2022 from issue BIP39 and SLIP39 generated wallets do not match by ksedgwic

BIP 85: Deterministic Entropy From BIP32 Keychains #910