BIP 388: Wallet Policies for Descriptor Wallets

bigspider commented at 3:34 pm on November 16, 2022: contributor

Initial version posted to bitcoin-dev in May.

Wallet policies are implemented in the Ledger bitcoin app (since version 2.1.0).

The following PR experimenting with an HWI integration might provide more context and an end-to-end demo:

https://github.com/bitcoin-core/HWI/pull/647

in bip-wallet-policies.mediawiki:30 in fd3b84da83 outdated

25+
26+This BIP is licensed under the BSD 2-clause license.
27+
28+== Motivation ==
29+
30+''[[bip-0380.mediawiki|Output Script Descriptors]]'' were introduced in bitcoin-core as a way to represent collections of output scripts. It is a very general and flexible language, designed to catch all the possible use-cases of bitcoin wallets (that is, if you know the script and you have the necessary keys, it will be possible to sign transactions with bitcoin-core's descriptor-based wallets).

darosior commented at 9:53 am on November 30, 2022:

Not only in Bitcoin Core?

bigspider commented at 9:20 am on January 24, 2023:

Good point, changed in 162c675415ef717a599cc899fc4a0b7617710a3e to “any descriptor-based software wallet”.

in bip-wallet-policies.mediawiki:42 in fd3b84da83 outdated

37+
38+Moreover, other limitations like the limited size of the screen might affect what design choices are available in practice. Therefore, minimizing the size of the information shown on-screen is important for a good user experience; that is crucial since the ability for the user to completely validate on-screen the kind of script used (and each of the involved keys) is a prerequisite for secure usage, as the machine that is interacting with the hardware signer (and running the software wallet) is considered untrusted.
39+
40+A more native, compact representation of the wallet receive/change might also benefit the UX of software wallets using descriptors to represent software wallets using descriptors (possibly with miniscript) for complex locking conditions.
41+
42+We remark that wallet policies are not related to the ''policy'' language, a higher level language that can be compiled to miniscript.

darosior commented at 9:55 am on November 30, 2022:

:)

bigspider commented at 9:17 am on January 24, 2023:

Still open to suggestions for a less-ambiguous name!

jgriffiths commented at 5:52 am on February 22, 2023:

If the suggestions below to reduce/remove malleability are taken, I’d suggest wallet patterns as these strings describe the pattern of derivation without including the keys.

joostjager commented at 2:38 pm on May 2, 2023:

I am wondering if signing policy/pattern as a name can be useful too. Thinking of constraining outputs to spend to a predefined wallet, which isn’t necessarily the wallet that is being spend from.

For example a policy that only allows spending to cold storage addresses derived from a pre-registered xpub.

darosior commented at 10:01 am on November 30, 2022: member

Good to see progress on this! Just a small comment about only mentioning Bitcoin Core: i know of several projects using descriptors, and even more of them using descriptors without Miniscript.

kallewoof added the label New BIP on Jan 4, 2023

bigspider force-pushed on Jan 24, 2023

bigspider commented at 9:25 am on January 24, 2023: contributor

In dfa26453fb1975c0c8b7909e8237a5ac55f4dd8c, I added the possibility of using unhardened derivation steps in the key origin information (while keeping the restriction for change/address_index in the key placeholders). While most wallets use hardened derivations for all but the last two steps, there are deployed use cases with unhardened derivations, and this only adds negligible implementation complexity to wallet policies.

jgriffiths referenced this in commit 873942a31c on Feb 13, 2023

jgriffiths referenced this in commit 9883c6ca10 on Feb 13, 2023

in bip-wallet-policies.mediawiki:232 in 162c675415 outdated

227+
228+Avoiding key reuse among different wallet accounts is also extremely important, but out of scope for this document.
229+
230+== Examples ==
231+
232+In the examples in this section, the vector of key information items is omitted. See the test vectors below for a complete

jgriffiths commented at 11:26 am on February 13, 2023:

nit: for a complete list. ?

bigspider commented at 8:32 am on February 21, 2023:

Fixed in da3e117d075f4428f14d4120124d840682e5587d with for complete examples.

in bip-wallet-policies.mediawiki:245 in 162c675415 outdated

240+Common multisignature schemes:
241+* <tt>wsh(multi(2,@0,@1))</tt> - SegWit 2-of-2 multisignature, keys in order.
242+* <tt>sh(sortedmulti(2,@0,@1,@3))</tt> - Legacy 2-of-3 multisignature, sorted keys.
243+
244+Some miniscript policies in <tt>wsh</tt>:
245+* <tt>wsh(and_v(v:pk(key_user),or_d(pk(key_service),older(12960))))</tt> - Trust-minimized second factor, degrading to a single signature after about 90 days.

jgriffiths commented at 11:27 am on February 13, 2023:

I think you need to change: key_user -> @0 key_service -> @1

bigspider commented at 8:33 am on February 21, 2023:

Fixed in da3e117d075f4428f14d4120124d840682e5587d.

in bip-wallet-policies.mediawiki:298 in 162c675415 outdated

293+
294+TBD: add examples with taproot scripts and miniscript.
295+
296+== Backwards Compatibility ==
297+
298+The <tt>@</tt> character used for key placeholders is not part of the syntax of output script descriptors, therefore any valid descriptor template is not a valid descriptor template. Vice versa, any descriptor template with at least one key placeholder is not a valid output script descriptor.

jgriffiths commented at 11:29 am on February 13, 2023:

therefore any valid descriptor template is not a valid descriptor template

I think you mean therefore any valid descriptor is not a valid descriptor template. Is this because a descriptor template must contain at least one key placeholder? You might want to put that into ‘Additional rules’ if so.

bigspider commented at 8:34 am on February 21, 2023:

Fixed the sentence and added the additional rule in da3e117d075f4428f14d4120124d840682e5587d.

in bip-wallet-policies.mediawiki:182 in 162c675415 outdated

177+** the string  <tt>/**</tt>, or
178+** a string of the form <tt>/<NUM;NUM>/*</tt>, for two distinct decimal numbers <tt>NUM</tt> representing unhardened derivations.
179+
180+TODO: reference BIP-389 for multipath descriptors (not yet merged into the bips repository at time of writing).
181+
182+The <tt>/**</tt> in the placeholder template represents commonly used paths for receive/change addresses, and is equivalent to <tt><0;1>/*</tt>.

jgriffiths commented at 11:40 am on February 13, 2023:

While /** is shorter, it also looks to be easy to mistype/confuse with /*. Compare with something like /+ or /++ for example.

bigspider commented at 8:44 am on February 21, 2023:

I prefer /** as it has a similar meaning in other languages (and was used in BIP-0129 for almost the same meaning).

As the /* alone is invalid for wallet policies, I don’t see the risk of confusing it as very dangerous.

It’s been pointed out in the past that /** is redundant since the /<M;N>/* expression is already more general, but in practice my expectation is that the vast majority of the use cases will just be happy with using /** for all the key expressions (which makes it a lot easier to inspect). Complex taproot scripts with many leaves might find the /<M;N>/* notation more interesting, perhaps.

jgriffiths commented at 5:42 am on February 22, 2023:

Ah, that’s fair enough, I wasn’t aware of BIP-0129. I’m OK with /** in that case. Given that:

It’s been pointed out in the past that /** is redundant since the /<M;N>/* expression is already more general

I think you should disallow <0;1>/* within wallet policies and require they be expressed as /**. Can’t recall if that’s already the case.

In fact I think you should go further and mandate the hardened indicator be only ' (or 'h, but only one or the other) in wallet policies. At the moment, the primary source of descriptor malleability (two textually different descriptors that describe the same thing, but have different checksums) is the key paths (plus no lexicographical sorting in the source sortedmulti key expression order). Given that you are replacing the keys with @n and enforcing monotonic numbering from left to right, doing the above will make it much simpler to identify standard policies/templates by not requiring all possible variants to be stored and compared. What do you think?

edit: Forgot about multi-path. You need additionally to state that multi-path extensions of length 3 and greater must be sorted in numerical order.

bigspider commented at 10:32 am on February 22, 2023:

Mandating ' instead of h sounds good to me, but completely fixing malleability is probably a lost battle. For example, impossible to make sure are not reused if descriptors are different because of multi/sortedmulti, as you pointed out. More crucially, you might or might not know the key origin info of some external xpub (although this is only within the key info vector, not in the descriptor template).

I’m not sure about disallowing the /<M;N>/* notation; for one thing, it’s already adopted for descriptors and used in many multisig wallets. Moreover, I suspect using <M;N> other than 0;1 will have use cases: you might have the same root xpub in different spending conditions, but using different values for M;N. One use case might be delegation: or(and(Alice1,Bob),and(Alice2,Carl)): Alice might want to allow Bob to sign, but not Carl, so she only signs with Alice1. If only the /** notations is present, then the only option for Alice is to use two different xpubs; instead, one could have or(and(Alice/<0;1>/*,Bob),and(Alice/<2;3>/*, Carl)) and have a single xpub for Alice.

jgriffiths commented at 11:13 am on February 22, 2023:

but completely fixing malleability is probably a lost battle.

Agreed, it wasn’t baked in at the start and can’t be reliably added now. My goal is just to make sure that the common cases are all represented with the same pattern as much as possible.

I’m not sure about disallowing the /<M;N>/* notation

To be clear, I’m only suggesting this when M=0 and N=1, not in the general case! its just a simple text substitution much as the hardening indicator would be, that would go a long way to making common patterns trivially identifiable. Agreed that there are use cases for other values here.

in bip-wallet-policies.mediawiki:222 in 162c675415 outdated

217+
218+<tt>pkh([d34db33f/44'/0'/0']xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL/<0;1>/*)</tt>
219+
220+=== Implementation guidelines ===
221+
222+Implementations must not necessarily implement all the possible wallet policies defined by this standard, but it is recommended to clearly document any limitation.

jgriffiths commented at 11:47 am on February 13, 2023:

Would it not be better to avoid listing the miniscript/descriptor elements that are supported, and just define this bip in terms of :

A change to bip32 KEY expressions against miniscript/descriptors; they must be replaced with @ placeholders
Non-bip32 keys (e.g. raw and private keys) are not allowed (not sure if this is specified here)
/** (or /+ or whatever is decided on) is used for <0;1>/*, and presumably multi-path is disallowed elsewhere.

Given that implementations may only support a subset of elements, you might as well open it up to future elements and avoid needing to update this bip.

bigspider commented at 8:53 am on February 21, 2023:

That’s an option, indeed.

I opted for a bottom-up definition (explicitly whitelisting the parts of the descriptor language that we adopt) because there are some inherent incompatibilities, plus it is hard to predict if more will come in the future. For example:

The combo descriptor seems inherently not fitting for wallet policies
Likewise, extensions like this would have to be “blacklisted” explicitly

On the other hand, the bottom-up approach leaves the exact spec for miniscript descriptors templates in the air, until there is no corresponding BIP…

jgriffiths commented at 5:30 am on February 22, 2023:

Yeah, we have to hope that at some point we stop shoe-horning in descriptor extensions for obsolete cases. Doing so makes a complete implementation prohibitively difficult, makes universal descriptor support less likely, and disincentivizes the eventual/priority movement of coins to modern UTXO/key derivation schemes.

combo, and the linked extension both have in common that they change the cardinality of the solved descriptor. I think you could get the benefit of forward compatibility for this BIP by just stating that final wildcards and multi-path are the only supported key expressions that change cardinality, and all others are forbidden. Then any future extensions to the path syntax, and any expression fragments that return multiple variants would automatically be excluded.

bigspider commented at 10:34 am on February 22, 2023:

My understanding is that the plan is for extending descriptors liberally, but then implementations can choose which subset to implement. So the bottom-up approach is imho more likely to stay well-defined (but it’s certainly more work and will need updates in the future).

jgriffiths commented at 11:17 am on February 22, 2023:

I’ll leave the approach here to your judgement, since there are arguments for both sides. My view is that descriptors become less useful the more incompatible implementations there are. In wally I’m likely just going to accept anything that looks like a correct pattern as long as it parses and doesn’t change the cardinality when solved (assuming this BIP is accepted).

in bip-wallet-policies.mediawiki:242 in 162c675415 outdated

237+* <tt>sh(wpkh(@0/**))</tt> (nested segwit).
238+* <tt>tr(@0/**)</tt> (taproot single-signature account).
239+
240+Common multisignature schemes:
241+* <tt>wsh(multi(2,@0,@1))</tt> - SegWit 2-of-2 multisignature, keys in order.
242+* <tt>sh(sortedmulti(2,@0,@1,@3))</tt> - Legacy 2-of-3 multisignature, sorted keys.

bucko13 commented at 11:41 pm on February 15, 2023:

Possible typo that the last key is @3 instead of @2?

bucko13 commented at 11:41 pm on February 15, 2023:

0* <tt>sh(sortedmulti(2,@0,@1,@2))</tt> - Legacy 2-of-3 multisignature, sorted keys.

bigspider commented at 8:53 am on February 21, 2023:

Fixed in da3e117d075f4428f14d4120124d840682e5587d, thanks!

bigspider force-pushed on Feb 21, 2023

bigspider commented at 8:55 am on February 21, 2023: contributor

Thanks @jgriffiths and @bucko13 for the reviews! I addressed most of the comments in da3e117d075f4428f14d4120124d840682e5587d.

In bb98f8017a883262e03127ab718514abf4a5e5f9 I deleted the examples coming from HTLC miniscripts; they don’t quite make sense as wallet policies.

in bip-wallet-policies.mediawiki:214 in bb98f8017a outdated

209+
210+From a wallet descriptor template (and the associated vector of key information items), one can therefore obtain the corresponding multipath descriptor by:
211+
212+* replacing each key placeholder with the corresponding key origin
213+information;
214+* replacing every <tt>/**</tt> with <tt>/<0;1>/*</tt>.

joostjager commented at 2:54 pm on May 2, 2023:

Could an alternative to using the /** suffix be to separate define the wallet policies for the inputs and outputs? The input policy can then specify /0/* and the output policy /1/*. Both would still reference the same entry in the key info vector. The added flexibility may also be useful to constrain outputs to a known-but-different wallet.

benma commented at 8:40 am on June 13, 2023:

Maybe I am misunderstanding you, but: inputs can be /0/* or /1/* (changes can be spent too)

in bip-wallet-policies.mediawiki:198 in bb98f8017a outdated

193+* Optionally, key origin information, consisting of:
194+** An open bracket <tt>[</tt>
195+** Exactly 8 hex characters for the fingerprint of the master key from which this key is derived from (see [[bip-0032.mediawiki|BIP-32]] for details)
196+** Followed by zero or more <tt>/NUM'</tt> or <tt>/NUM</tt> path elements to indicate hardened or unhardened derivation steps between the fingerprint and the xpub that follows
197+** A closing bracket <tt>]</tt>
198+* Followed by the actual key, which is a serialized extended public key (as defined in [[bip-0032.mediawiki|BIP-32]]).

joostjager commented at 3:02 pm on May 2, 2023:

Echoing the discussion in https://github.com/LedgerHQ/app-bitcoin-new/issues/153:

It would be great if a wallet policy can support 2-of-2 multisig transactions where one of the keys is ephemeral (completely random). This is useful when using presigned transactions to simulate covenants, for example to implement time-locked vaults.

in bip-wallet-policies.mediawiki:254 in bb98f8017a outdated

249+* <tt>wsh(or_d(pk(@0),and_v(v:multi(2,@1,@2,@3),older(65535))))</tt> - A singlesig wallet with automatic inheritance to a timelocked 2-of-3 multisig of family members.
250+
251+== Test Vectors ==
252+
253+[[bip-0044.mediawiki|BIP-44]], first account
254+ Descriptor template: pkh(@0)

benma commented at 8:49 am on June 13, 2023:

The spec above says that the @N must always be followed by /** or /<NUM:NUM>/*, but some examples and test vectors don’t have that suffix.

0 followed by a non-negative decimal number, with no leading zeros (except for <tt>@0</tt>)
1* ''always'' followed by either:
2** the string  <tt>/**</tt>, or
3...

Is @0 the same as @0/**? Would be good to fix either the specification or the examples/test vectors.

jgriffiths commented at 11:23 am on September 1, 2023:

We have to also allow /* following @n (i.e. a single asterisk instead of two), since not all wallets use 0/1 paths to differentiate change addresses.

I’m OK with @n alone expanding to @0/** but this does add even more malleability to expressions. Regardless, the always section should be updated to allow single asterisks.

bigspider commented at 3:22 pm on September 5, 2023:

The naked @N was an oversight, I don’t think it should be allowed as it would be confusing (Ledger’s implementation doesn’t permit it). I will fix it.

–

If we generalize the suffix to expressions other than <0;1>/*, then we have to make sure that no pubkey is repeated. In practice, for /*, that would mean that all the placeholders must be followed by the exact suffix /*.

Out of curiosity, what software wallets use that scheme?

jgriffiths commented at 10:36 pm on September 5, 2023:

If we generalize the suffix to expressions other than <0;1>/*

I’m not sure I follow. I’m not suggesting generalizing the suffix, at least in the sense of allowing general path expressions to be used as suffixes. I’m saying the only key expression suffixes that should be allowed are /**, /<m;n>/* and /*.

If we generalize the suffix to expressions other than <0;1>/*, then we have to make sure that no pubkey is repeated.

I assume you mean here that a key expression could be given as [/a/b/c]foo/** and then elsewhere as [/a/b/c/0]bar/* and/or [/a/b/c/1]bar/* giving rise to the same derived keys with different key identifiers.

In practice, for /, that would mean that all the placeholders must be followed by the exact suffix /

For a given key expression, yes. I’m fine with supporting /* being optional, and if present the implementation must ensure this.

Note also that at present this BIP does not state that implementations must ensure that KP expressions refer to distinct keys when substituting them into a policy, although that appears to be a requirement. I think it would also be clearer to state that all key expressions in a policy must be replaced by KP expressions (i.e. that no non-KP keys are allowed).

Also missing are that <m;n> expressions should be numerically sorted, and that explicit <0;1> should be disallowed.

I’m happy to make individual commits with suggested changes for the above for you to review/cherry-pick. Should I create a PR against your repo for this?.

Out of curiosity, what software wallets use that scheme?

Blockstream Green multisig wallets do not use bip44 style paths (the wallet predates the BIP). Also descriptors themselves do not enforce bip44 derivation.

bigspider commented at 9:17 am on September 6, 2023:

I’m not sure I follow. I’m not suggesting generalizing the suffix, at least in the sense of allowing general path expressions to be used as suffixes. I’m saying the only key expression suffixes that should be allowed are /**, /<m;n>/* and /*.

What I mean is that if we allow mixing things like @0/* with @1/<m;n>/*, then we for the xpub of @1 we would need to derive @1/m and @1/n and compare with the pubkey of the xpub in @0, which is quite expensive on a hardware wallet.

With no mixing, for any two placeholders like @0/<m;n>/* and @0/<p;q>/*, I can just check that the sets {m, n} and {p, q} are disjoint.

For a given key expression, yes. I’m fine with supporting /* being optional, and if present the implementation must ensure this.

Not just for a given key expression, I mean it across the entire wallet policy. That is, either the KPs all end with /*, or they all end with some /<NUM;NUM>/*.

Note also that at present this BIP does not state that implementations must ensure that KP expressions refer to distinct keys when substituting them into a policy, although that appears to be a requirement.

Good point, worth adding explicitly the “Additional rules”, with the motivation that it’s required in miniscript (and it’s simpler to add this restriction globally, rather than on the individual miniscript parts of the policy).

So basically:

all the public keys (deserialized from the corresponding xpubs in the keys information vector) must be distinct
all the derivations in each KP expression referring to the same index in the keys information vector must be distinct

In order to allow optional extensions like /*, perhaps I could mention that possibility in a separate section?

I think it would also be clearer to state that all key expressions in a policy must be replaced by KP expressions (i.e. that no non-KP keys are allowed).

A wallet policy is the pair (descriptor_template, keys_information_vector), so not sure what you mean here. The grammar seems to specify exactly what are the allowed KP and KEY expressions, as far as I can tell.

Also missing are that <m;n> expressions should be numerically sorted, and that explicit <0;1> should be disallowed.

I like suggesting to sort <m;n> numerically, but not sure about disallowing <0;1>. I expect people to prefer not mixing @0/** with @0/<2;3>/*, as it’s more explicit in that case to just write @0/<0;1>/*.

I’m happy to make individual commits with suggested changes for the above for you to review/cherry-pick. Should I create a PR against your repo for this?.

Sure, feel free to propose changes, but I’ll try to incorporate the comments now. I’m also hoping to find the time for a general pass over the whole document to improve the Motivation section, as that was written before the BtcPrague meeting and it can surely be improved in hindsight.

benma commented at 9:21 am on September 6, 2023:

I like suggesting to sort <m;n> numerically, but not sure about disallowing <0;1>. I expect people to prefer not mixing @0/** with @0/<2;3>/, as it’s more explicit in that case to just write @0/<0;1>/.

What’s the reason for sorting? It’s not mentioned in BIP-389.

+1 to keep allowing <0;1>/* for readability for the reason you stated.

jgriffiths commented at 10:10 am on September 6, 2023:

Not just for a given key expression, I mean it across the entire wallet policy.

OK, I understand where you are coming from now. Enforcing this across the entire policy is sensible (no-one should be mixing key expressions with different cardinalities), and its pretty easy to implement.

So basically:

Yes these 2 two points are whats missing IMO.

perhaps I could mention that possibility in a separate section?

Sure, that works, let me know if you’d like me to draft it or if you are happy to.

The grammar seems to specify exactly what are the allowed KP and KEY expressions, as far as I can tell.

This is true, I just prefer being more explicit. Feel free to ignore this if you disagree.

I expect people to prefer not mixing +1 to keep allowing <0;1>/* for readability

No doubt its slightly better for human readability in the rare case where alternate multipath derivations are given. But its bad for comparing two such policies (or checking against a whitelist of supported policies) because it’s another source of malleability. It also complicates logic to convert a descriptor to a human friendly policy automatically, because substitution of /<0;1>/* for /** as recommended in this BIP will not result in the human friendly format in this case. Complicated policies are far more likely to be automatically generated than human written, and so in practice I suspect the large majority will end up mixing /** and /<M;N>/* anyway. However its your decision.

What’s the reason for sorting?

Malleability and ease of comparing multipaths. Unfortunately 389 states that the meaning of all n=2 multipaths is fixed (receive addresses and change), and so sorting is probably not possible to enforce given some hypothetical wallet out there could use descending numbers for this purpose.

It’s not mentioned in BIP-389

True, but we limit multipaths to length 2 in policies which is also not part of 389.

benma commented at 10:28 am on September 6, 2023:

Unfortunately 389 states that the meaning of all n=2 multipaths is fixed (receive addresses and change), and so sorting is probably not possible

I was actually thinking primarily about this case and wondered why one would restrict people from using receive and change indices to be ordered.

Malleability and ease of comparing multipaths.

/<1;2;3>* is not the same as <3;2;1>/*, so why should they be compared or treated to be the same? Applications may assign meaning to the position (similar to receive and change with two elements).

In practice it probably does not matter much, but I wouldn’t impose this restriction here.

Blockstream Green multisig wallets do not use bip44 style paths (the wallet predates the BIP).

Out of curiosity, how does Green identify which UTXOs are change?

bigspider commented at 1:34 pm on September 6, 2023:

As per this discussion:

3ade41fc6bc12461388f34d5aebe9ba60a008704 makes the rules on distinct pubkeys explicit;
0d6e07781e17a4ee8b208b3aea5985fed7ac2cbc adds a section for the optional additional derivations, pointing out the potential footguns I could think of.

I didn’t add any requirement/suggestion to sort <m;n> expressions. Thanks for the comments, and feel free to suggest further changes, of course!

jgriffiths commented at 8:54 pm on September 6, 2023:

@bigspider Thanks!

Comments on the current state:

under ** a string of the form there should be a new bullet point stating that any path expression as listed in the Optional derivation paths is optionally allowed.
There still appear to be some bare @n references in the test vectors (e.g. multisig).
The Why must public keys be distinct? reference doesn’t appear to point anywhere. Do you have a reference to any docs/discussion on pubkey reuse in miniscript this can link to?
Some invalid test vectors would be helpful for implementors to check against (see below).

I’m fully implementing the BIP in wally now, and may have some further comments once that work is complete.

Suggested invalid test vectors:

pkh(@0) Key with no following path
pkh(@0/0/*) Key with an explicit path present
sh(multi(1,@1/**,@0/**)) Keys used out of order
sh(multi(1,@0/**,@0/**)) Repeated keys w/same path expression
sh(multi(1,@0/<0;1>/*,@0/<1;2>/*)) Non-disjoint multipath expressions
sh(multi(1,@0/**,xpub6AHA9hZDN11k2ijHMeS5QqHx2KP9aMBRhTDqANMnwVtdyw2TDYRmF8PjpvwUFcL1Et8Hj59S3gTSMcUQ5gAqTz3Wd8EsMTmF3DChhqPQBnU/<0;1>/*)) Expression with a non KP key present
pkh(@0/<0;1;2>/*) Solved cardinality > 2
combo(@0/**) Disallowed expression type/Solved cardinality > 2

jgriffiths commented at 9:29 pm on September 6, 2023:

@benma

note slightly O/T to this BIP.

why one would restrict people from using receive and change indices to be ordered. /<1;2;3>* is not the same as <3;2;1>/*, so why should they be compared or treated to be the same? Applications may assign meaning to the position (similar to receive and change with two elements).

They are the same assuming no other paths appear in the same expression; they both result in the same set of solutions, just in a different order. Of course this is not true with multiple keys: <1;2;3> and <4;5;6> have different solutions to <3;2;1> and <4;5;6> due to the stepwise resolution through path elements when generating the solutions. But in the single case its a cause of expression malleability.

Note that ordering of expressions is not explicitly specified anywhere other than this stepwise derivation, and no meaning can be inferred except in the n=2 case (where all n=2 cases must be considered as receive/change even if this is not the intention). Consider expressions like combo or new extensions like explicit key lists. In what order is combo(<key>/<1;2;3>/*) solved, paths iterated first or the combo variants for each path?

As per my comments on the BIP the addition of extensions that change cardinality and the non-specification of ordering for N>3 means that we have the worst of all worlds: proscribed meaning for one case that prevents general use of N=2, and no ability to state anything about all the meaning of solutions when other expressions that change cardinality are present. Enforcing ordering and describing the expansion order of expressions would not have been onerous but would have allowed implementations to process them consistently instead of needing whitelists/understanding specific patterns.

In practice it probably does not matter much, but I wouldn’t impose this restriction here.

Agreed, this ship along with other malleability concerns has unfortunately sailed. Although it would be nice to enforce ' rather than h for hardening in policies at least, @bigspider?

Out of curiosity, how does Green identify which UTXOs are change?

For singlesig wallets we use bip44 like everyone else, although you can’t rely on the internal/external paths only for change detection, since its possible to create txs using either. For multisig we don’t explicitly track change, the backend computes limits etc based on the net effect on the wallet, and displaying change is based on a simple check of the txs outputs (one output only to the wallet = redeposit, non-wallet output plus wallet output = payment and change etc).

bigspider commented at 3:29 pm on September 7, 2023:

Agreed, this ship along with other malleability concerns has unfortunately sailed. Although it would be nice to enforce ' rather than h for hardening in policies at least, @bigspider?

I agree; Ledger’s implementation only allows ' (choice mostly because it looks better on a small screen). The current specs indeed don’t mention h.

I’ll go over the other comments (and add the failure cases, thanks!) next week.

bigspider commented at 8:22 am on September 12, 2023:

4b1f826217ab8f99c9484fbd6f1b1f88fa0c3bc0 should address all the remaining comments from this thread, thanks @jgriffiths!

Suggested invalid test vectors:

sh(multi(1,@0/<0;1>/*,@1/<1;2>/*)) Non-disjoint multipath expressions

This is actually valid as @0 and @1 are independent; added an example where both are @0 instead.

bigspider commented at 8:56 am on September 12, 2023:

For the miniscript reference, I suppose https://bitcoin.sipa.be/miniscript/ is the only one, although the mention of separate keys is quite buried in the page in the section on malleability.

jgriffiths commented at 9:01 pm on September 12, 2023:

@bigspider Thanks!

The wally implementation (https://github.com/ElementsProject/libwally-core/pull/369) is now complete and is being tested for addition to Jade. Once merged it would be good to add to the implementations section. At this stage it looks like this BIP has support in 3 HWW so getting a BIP number assigned should not be an issue.

It would be good IMO to squash all commits at this stage. I’ll do a final review following wally code review and Jade testing. Many thanks for your responsiveness in addressing review comments.

jgriffiths commented at 8:17 pm on September 25, 2023:

FYI policy support has been merged into wally now and support in Jade is now being tested.

benma referenced this in commit 0884576354 on Jun 20, 2023

benma referenced this in commit 996cb79f35 on Jun 20, 2023

benma referenced this in commit 59b047a96f on Jun 20, 2023

benma referenced this in commit 78671aec91 on Jun 20, 2023

benma referenced this in commit 0468931b93 on Jun 20, 2023

benma referenced this in commit 0a88519fe2 on Jun 20, 2023

benma referenced this in commit 4069cb5c35 on Jun 20, 2023

benma referenced this in commit 808dc7c3ff on Jun 20, 2023

benma referenced this in commit 5196443f10 on Jun 29, 2023

benma referenced this in commit 5625aec043 on Jun 29, 2023

darosior commented at 4:27 pm on August 9, 2023: member

@luke-jr can this be assigned a BIP number?

jgriffiths referenced this in commit 2dff714959 on Sep 6, 2023

jgriffiths referenced this in commit 8e4fc85ab5 on Sep 7, 2023

dr-orlovsky commented at 6:40 am on September 11, 2023: contributor

After @benma suggestion I propose to merge my recent proposal https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2023-September/021946.html with this standard.

Specifically, I think we can remove unnecessary remaining ambiguity, when multiple keys need to repeat the same data (BIP44 purpose, coin, change derivation path segments) and make descriptors even shorter, more readable and standards, removing key list as a separate structure.

What can be done in this regard:

Descriptor is equipped with prefix and suffix containing information to reconstruct all shared key components: wsh/testnet(...)/<0;1>/*#checksum
Keys do not list shared components, skipping them: [f149e757//0h]xpub (where 0h is the account number and “terminal derivation” from the suffix is appended to all keys, such that they all follow uniform derivation);
Instead of a separate key list the keys go into the descriptor referenced in full only the first time they appear;
Each key has an alphanumeric alias, separated using @: alice@[f149e757//0h]xpub; the same key can be referred to in other descriptor places as @alice;
Keys in different spending paths, when necessary (since in Taproot it is not necessary) are distinguished by an additional derivation path segment called “branch”. It is a non-hardened index going before the change index: alice@[f149e757//0h]xpub/1 (which after expansion with prefix and suffix becomes [f149e757/89h/1h/0h]xpub/1/<0;1>/*;

Overall, with these changes, the descriptors will look like

0wsh/test(or(
1    and(alice@[fe569a81//1']xpub1..., bob@[8871bad9//1']xpub2..., carol@[beafcafe//1']xpub3...), 
2    and(older(1000), thresh(2, [@alice](/bitcoin-bips/contributor/alice/), [@bob](/bitcoin-bips/contributor/bob/), [@carol](/bitcoin-bips/contributor/carol/)))
3))/<0;1>/*

The benefits of the proposal are:

Remove separate key list;
Use human-readable aliaces;
Avoid conflicting descriptors using different BIP44 purposes, networks or terminal indexes;
Separate from previous descriptor standard using new BIP44 purpose value;
Do not assign new meaning for the change index and make the standard compatible with RGB protocol (which uses change index for specifying derivations containing assigned client-side state or tapret commitments). This also makes wallet implementation easier, helping to clearly understand which indexes to use when deriving external or internal (change) addresses.

I can work on a PR to this PR to put this proposals into the text.

benma commented at 7:40 am on September 11, 2023: none

Remove separate key list;

The separate key list is very nice though. On the hardware wallets, being able to verify the descriptor without all the xpub/fingerprint/path-prefix noise is much better UX and I think the main purpose of this BIP.

pythcoiner commented at 7:43 am on September 11, 2023: none

Descriptor is equipped with prefix and suffix containing information to reconstruct all shared key components: wsh/testnet(...)/<0;1>/*#checksum

In my mind including the network identifier outside of the xpub is good from a global pov but bad on the UX pov: you asking all signer vendors/user to change they way of export xpubs…..

Instead of a separate key list the keys go into the descriptor referenced in full only the first time they appear;

I personally feel that a separate list of keys is more clear/clean to read/use

Each key has an alphanumeric alias, separated using @: alice@[f149e757//0h]xpub; the same key can be referred to in other descriptor places as @alice;

In my mind the key should have a numeric alias, this numeric alias can be replaced by some kind of alphanumeric ‘mnemonic’ by the hardware that handles the display

Keys in different spending paths, when necessary (since in Taproot it is not necessary) are distinguished by an additional derivation path segment called “branch”. It is a non-hardened index going before the change index: alice@[f149e757//0h]xpub/1 (which after expansion with prefix and suffix becomes [f149e757/89h/1h/0h]xpub/1/<0;1>/*;

The ‘branch’ should be allowed with @, it’s needed for actual miniscripts.

Looks like you forgot to talk about not limit (or increase limit) the number of change segments to allow you to use <0;1;9;10> with RGB

dr-orlovsky commented at 8:46 am on September 11, 2023: contributor

@benma:

The separate key list is very nice though. On the hardware wallets, being able to verify the descriptor without all the xpub/fingerprint/path-prefix noise is much better UX and I think the main purpose of this BIP.

You are right that displaying that in the UI is terrible. Instead, my understanding is that the descriptor will be parsed by the wallet and the keys and policy will be present in the UI in separate form @pythcoiner:

I personally feel that a separate list of keys is more clear/clean to read/use

I am not insisting on merging them; however, to import/export descriptors and pass them between software wallets it will be desirable to have a joined form.

In my mind including the network identifier outside of the xpub is good from a global pov but bad on the UX pov: you asking all signer vendors/user to change they way of export xpubs…..

No, the xpubs are exported the same way. However, when they are used in the descriptor, their shared parts are moved to prefix/suffix, ensuring they can be combined into the same descriptor (i.e. do not belong to different networks, which is hard to check otherwise).

In my mind the key should have a numeric alias, this numeric alias can be replaced by some kind of alphanumeric ‘mnemonic’ by the hardware that handles the display

Putting additional requirements on having ints and having them in a strictly incremental manner just bloats validation code with no clear benefits

The ‘branch’ should be allowed with @, it’s needed for actual miniscripts.

Optionally. If not provided with a branch, then each time the key appears the branch number is automatically incremented. This reduces the visual load and size of the descriptor validation code (you can’t write it in the wrong way).

pythcoiner commented at 9:11 am on September 11, 2023: none

No, the xpubs are exported the same way. However, when they are used in the descriptor, their shared parts are moved to prefix/suffix, ensuring they can be combined into the same descriptor (i.e. do not belong to different networks, which is hard to check otherwise).

so it’s mean the key/xpub a user supply to a wallet/coordinator will look differently than the one in the descriptor, how they can check accurately?

Putting additional requirements on having ints and having them in a strictly incremental manner just bloats validation code with no clear benefits

should we allow arabic/chinese/japanese/koreans/cyrillic/…. alphabets? the benefit i found is just KISS, let the fancy stuff optionnaly on the display side, not on the communication side.

Optionally. If not provided with a branch, then each time the key appears the branch number is automatically incremented. This reduces the visual load and size of the descriptor validation code (you can’t write it in the wrong way).

i wonder if the order the key are passed to the miniscript ‘compiler_that_is_not_compiler’ are the same order in the output descriptor?

jgriffiths commented at 9:58 pm on September 11, 2023: none

@dr-orlovsky I feel like your suggestion would be better as a separate BIP as it appears to have slightly different goals and tradeoffs than this one. I don’t think it would be feasible to merge these two proposals without morphing one into the other. If you do this and link it here I’d be happy to provide feedback on that issue or PR.

bitcoin deleted a comment on Sep 11, 2023

bigspider commented at 8:46 am on September 12, 2023: contributor

Descriptor is equipped with prefix and suffix containing information to reconstruct all shared key components: wsh/testnet(...)/<0;1>/*#checksum

Keys do not list shared components, skipping them: [f149e757//0h]xpub (where 0h is the account number and “terminal derivation” from the suffix is appended to all keys, such that they all follow uniform derivation);

Instead of a separate key list the keys go into the descriptor referenced in full only the first time they appear;

Each key has an alphanumeric alias, separated using @: alice@[f149e757//0h]xpub; the same key can be referred to in other descriptor places as @alice;

Keys in different spending paths, when necessary (since in Taproot it is not necessary) are distinguished by an additional derivation path segment called “branch”. It is a non-hardened index going before the change index: alice@[f149e757//0h]xpub/1 (which after expansion with prefix and suffix becomes [f149e757/89h/1h/0h]xpub/1/<0;1>/*;

The proposed changes undo most of the design choices of this BIP proposal (like separating the actual keys from the “descriptor template” − something I consider a core feature and benefit!), while making the resulting language a lot more incompatible with output descriptor; moreover, it would make parsing a lot more complicated, as parsing a key expression is no longer context-free, because aliases are back-references to the previously-parsed part of the string.

In wallet policies, aliases for keys could easily be introduced as additional metadata associated to the elements of the keys information vector.

This BIP proposal wants to model in the most minimal way the object that software/hardware wallets (and their users) think of as “accounts”. Application-specific use cases could be implemented either by building on top of it (if there is a desire / need to stay compatible), or with completely independent approaches.

in bip-wallet-policies.mediawiki:309 in 4b1f826217 outdated

300@@ -298,6 +301,21 @@ Miniscript: A singlesig wallet with automatic inheritance to a timelocked 2-of-3
301 
302 TBD: add examples with taproot scripts and miniscript.
303 
304+=== Invalid policies ===
305+
306+The following descriptor templates are invalid:
307+
308+* <tt>pkh(@0)</tt>: Key placeholder with no path following it
309+* <tt>pkh(@0/0/*)</tt>: Key placeholder with an explicit path present

jgriffiths commented at 9:36 am on September 12, 2023:

This needs to change to ** from * to be a valid case for the non-extended BIP (my bad, sorry).

bigspider commented at 9:49 am on September 12, 2023:

Right! Fixed in 6f98026a2fd27069446fdfcabdfbaad6450b3e23.

dr-orlovsky commented at 5:18 pm on September 13, 2023: contributor

@jgriffiths:

I feel like your suggestion would be better as a separate BIP as it appears to have slightly different goals and tradeoffs than this one.

That was what I began with:

After @benma suggestion I propose to merge my recent proposal https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2023-September/021946.html with this standard.

and was suggested to merge into this standard

Anyway, I see that it doesn’t fit the hardware wallet devs, so I rest my case. The only part which is essential is to remember that RGB and other client-side-validation may require more derivation indexes in the change segment.

jgriffiths commented at 8:15 pm on September 25, 2023: none

RGB and other client-side-validation may require more derivation indexes in the change segment.

If this doesn’t fit the existing patterns or the single derivation case (for example if the solved cardinality of such descriptors is > 2, and/or they contain more than one multi-path expression), I think this can be added as an optional extension as the single derivation case (non-bip44-style) was. As long as every key expression in a policy has the same solved cardinality this should work (disclaimer: I’m not following RGB development).

jgriffiths commented at 10:37 am on November 20, 2023: none

@bigspider Policy support for the Jade HWW is now released as of firmware version 1.0.24 (https://github.com/Blockstream/Jade/releases/tag/1.0.24), if you’d like to update the Reference Implementation section.

Jade support is implemented via libwally-core (C/C++/Python/Java/JS) v1.0.0 (https://github.com/ElementsProject/libwally-core/releases/tag/release_1.0.0) if you want to link a general purpose implementation. Both Jade and wally implement the single-path derivation /* extension for non-bip44 wallets.

From my POV review is otherwise complete, ACK for this to receive a BIP number and be merged.

bigspider commented at 10:51 am on November 21, 2023: contributor

Thanks for the update, @jgriffiths! I added Jade/libwally support in 3c97885, and did a couple of small improvements to the python demo.

bigspider commented at 10:58 am on November 21, 2023: contributor

@luke-jr, please let me know if this can be assigned a BIP number, or if something is missing.

There are now three hardware signers already using these specs in production.

bigspider force-pushed on Nov 21, 2023

in bip-wallet-policies.mediawiki:9 in 07f0fcf5f4 outdated

0@@ -0,0 +1,343 @@
1+<pre>
2+  BIP: wallet-policies
3+  Layer: Applications
4+  Title: Wallet Policies for Descriptor Wallets
5+  Author: Salvatore Ingala <salvatoshi@protonmail.com>
6+  Comments-Summary: No comments yet.
7+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-wallet-policies
8+  Status: Draft
9+  Type: Informational

luke-jr commented at 6:50 pm on December 26, 2023:

I think this is supposed to be Standards Track.

bigspider commented at 4:46 pm on January 11, 2024:

Changed in d4c650bad3ac190c9106d059b1ed5b110f2660bb.

luke-jr commented at 6:55 pm on December 26, 2023: member

Assigned BIP 388

luke-jr renamed this:
~~[New BIP] Wallet Policies~~
BIP 388: Wallet Policies for Descriptor Wallets
on Dec 26, 2023

bigspider force-pushed on Jan 8, 2024

bigspider commented at 10:51 am on January 8, 2024: contributor

I squashed the history, rebased and updated BIP number and type in 3bb2cfe4751313470b98c9aaaff9322078fb8e3c d4c650bad3ac190c9106d059b1ed5b110f2660bb.

bigspider force-pushed on Jan 8, 2024

bigspider commented at 4:21 pm on February 5, 2024: contributor

@luke-jr, is there any other further change that I missed?

Otherwise, it would be great to have this merged so I can start linking to it in docs.

darosior commented at 1:59 pm on April 2, 2024: member

@luke-jr could you merge this when you’ve got a minute?

in bip-0388.mediawiki:67 in d4c650bad3 outdated

62+
63+Therefore, there are two fundamental design goals to strive for:
64+* Minimize the amount of information that is shown on screen - so that the user can actually validate it.
65+* Minimize the number of times the user has to validate such information.
66+
67+Designing a secure protocol for the coordination of a descriptor wallet among distant parties is also a challenging problem that is out of scope in this document. See [[bip-00129.mediawiki|BIP-129 (Bitcoin Secure Multisig Setup)]] for an approach designed for multisignature wallets. Regardless the approach, the ability for the user to carefully verify all the details of the spending policies using the hardware signer's screen is a prerequisite for security in adversarial environments.

murchandamus commented at 6:29 pm on April 22, 2024:

0Designing a secure protocol for the coordination of a descriptor wallet among distant parties is also a challenging problem that is out of scope in this document. See [[bip-00129.mediawiki|BIP-129 (Bitcoin Secure Multisig Setup)]] for an approach designed for multisignature wallets. Regardless of the approach, the ability for the user to carefully verify all the details of the spending policies using the hardware signer's screen is a prerequisite for security in adversarial environments.

in bip-0388.mediawiki:140 in d4c650bad3 outdated

135+          xpubE/<22;23>/*)
136+      }
137+    }
138+  }
139+})
140+</pre>

murchandamus commented at 7:19 pm on April 22, 2024:

This example is hard to access since the format it uses is only introduced later in the Specification section.

bigspider commented at 9:04 am on May 7, 2024:

Agreed; moreover, plus this is using musig which is not even part of the standard. A text description is probably sufficient: 95cf53916113a44487e0381029d3602e5bb1db6a.

in bip-0388.mediawiki:323 in d4c650bad3 outdated

318+
319+== Backwards Compatibility ==
320+
321+The <tt>@</tt> character used for key placeholders is not part of the syntax of output script descriptors, therefore any valid output descriptor with at least one `KEY` expression is not a valid descriptor template. Vice versa, any descriptor template with at least one key placeholder is not a valid output script descriptor.
322+
323+Adoption of wallet policies in software and harder wallets is opt-in. Conversion from wallet policies to the corresponding descriptors is programmatically extremely easy, and conversion from descriptors to wallet policies (when respecting the required patterns) can be automated. See the reference implementation below for some examples of conversion.

murchandamus commented at 7:20 pm on April 22, 2024:

0Adoption of wallet policies in software and hardware wallets is opt-in. Conversion from wallet policies to the corresponding descriptors is programmatically extremely easy, and conversion from descriptors to wallet policies (when respecting the required patterns) can be automated. See the reference implementation below for some examples of conversion.

in bip-0388.mediawiki:302 in d4c650bad3 outdated

297+ Descriptor template: wsh(or_d(pk(@0/**),and_v(v:multi(2,@1/**,@2/**,@3/**),older(65535))))
298+ Keys info: ["[6738736c/48'/0'/0'/100']xpub6FC1fXFP1GXQpyRFfSE1vzzySqs3Vg63bzimYLeqtNUYbzA87kMNTcuy9ubr7MmavGRjW2FRYHP4WGKjwutbf1ghgkUW9H7e3ceaPLRcVwa", "[b2b1f0cf/44'/0'/0'/100']xpub6EYajCJHe2CK53RLVXrN14uWoEttZgrRSaRztujsXg7yRhGtHmLBt9ot9Pd5ugfwWEu6eWyJYKSshyvZFKDXiNbBcoK42KRZbxwjRQpm5Js", "[a666a867/44'/0'/0'/100']xpub6Dgsze3ujLi1EiHoCtHFMS9VLS1UheVqxrHGfP7sBJ2DBfChEUHV4MDwmxAXR2ayeytpwm3zJEU3H3pjCR6q6U5sP2p2qzAD71x9z5QShK2", "[bb641298/44'/0'/0'/100']xpub6Dz8PHFmXkYkykQ83ySkruky567XtJb9N69uXScJZqweYiQn6FyieajdiyjCvWzRZ2GoLHMRE1cwDfuJZ6461YvNRGVBJNnLA35cZrQKSRJ"]
299+ Descriptor:wsh(or_d(pk([6738736c/48'/0'/0'/100']xpub6FC1fXFP1GXQpyRFfSE1vzzySqs3Vg63bzimYLeqtNUYbzA87kMNTcuy9ubr7MmavGRjW2FRYHP4WGKjwutbf1ghgkUW9H7e3ceaPLRcVwa),and_v(v:multi(2,[b2b1f0cf/44'/0'/0'/100']xpub6EYajCJHe2CK53RLVXrN14uWoEttZgrRSaRztujsXg7yRhGtHmLBt9ot9Pd5ugfwWEu6eWyJYKSshyvZFKDXiNbBcoK42KRZbxwjRQpm5Js,[a666a867/44'/0'/0'/100']xpub6Dgsze3ujLi1EiHoCtHFMS9VLS1UheVqxrHGfP7sBJ2DBfChEUHV4MDwmxAXR2ayeytpwm3zJEU3H3pjCR6q6U5sP2p2qzAD71x9z5QShK2,[bb641298/44'/0'/0'/100']xpub6Dz8PHFmXkYkykQ83ySkruky567XtJb9N69uXScJZqweYiQn6FyieajdiyjCvWzRZ2GoLHMRE1cwDfuJZ6461YvNRGVBJNnLA35cZrQKSRJ),older(65535))))
300+<br>
301+
302+TBD: add examples with taproot scripts and miniscript.

murchandamus commented at 7:26 pm on April 22, 2024:

This looks like an open todo.

bigspider commented at 9:02 am on May 7, 2024:

Indeed; added a taproot example in 95cf53916113a44487e0381029d3602e5bb1db6a.

in bip-0388.mediawiki:11 in d4c650bad3 outdated

 6+  Comments-Summary: No comments yet.
 7+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0388
 8+  Status: Draft
 9+  Type: Standards Track
10+  Created: 2022-11-16
11+  License: BSD-2-Clause

murchandamus commented at 7:30 pm on April 22, 2024:

Please include the post history (please add other references if there were more threads):

0  License: BSD-2-Clause
1  Post-History: 2022-05-10: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-May/020423.html

murchandamus commented at 7:33 pm on April 22, 2024: contributor

This looks ready for merge to me, although I have a few nits that you might want to address.

ACK d4c650bad3ac190c9106d059b1ed5b110f2660bb

murchandamus commented at 9:32 pm on April 22, 2024: contributor

I also noticed that this BIP does not have a Rationale section, although it seems to me that the rationale is sufficiently covered in Motivation and Specification (and therefore no change is necessary in that regard).

murchandamus changes_requested

murchandamus commented at 3:02 pm on May 6, 2024: contributor

One of the automated checks is failing, because the BIP is not added to the overview table. Please add your BIP to the table in README.mediawiki. You can find the expected table entry in the error message of the failed check.

jonatack commented at 3:59 pm on May 6, 2024: contributor

Yes, it’s nice that the check is working now. Here’s the entry it suggests:

0+> | [[bip-0388.mediawiki|388]]
1+> | Applications
2+> | Wallet Policies for Descriptor Wallets
3+> | Salvatore Ingala
4+> | Standard
5+> | Draft
6+> |-

New BIP: Wallet Policies 44798a2a9e

Update assigned BIP number; change type to "Standards Track" 25657cbee6

Apply suggestions from code review

Co-authored-by: Mark "Murch" Erhardt <murch@murch.one>

40c7760d78

Improvements from PR review.

- Removed large example of taproot policy; replaced with the textual description
- Added an example of a taproot wallet policy containing miniscript

95cf539161

Added BIP-388 to README a0c8501f96

bigspider force-pushed on May 7, 2024

bigspider commented at 9:06 am on May 7, 2024: contributor

Rebased, and addressed the final comments (thank you!). Please let me know if you prefer me to squash the new commits.

in bip-0388.mediawiki:164 in a0c8501f96 outdated

159+
160+==== Additional rules ====
161+
162+A wallet policy must have at least one key placeholder and the corresponding key.
163+
164+The public keys obtained by deserializing elements of the keys information vector must be pairwise distinct<ref>'''Why must public keys be distinct?''' Reusing pubkeys could be insecure in the conext of wallet policies containing [https://bitcoin.sipa.be/miniscript/ miniscript]. Avoiding repeated public keys altogether avoids the problem at the source.</ref>.

murchandamus commented at 1:52 pm on May 7, 2024:

0The public keys obtained by deserializing elements of the keys information vector must be pairwise distinct<ref>'''Why must public keys be distinct?''' Reusing pubkeys could be insecure in the context of wallet policies containing [https://bitcoin.sipa.be/miniscript/ miniscript]. Avoiding repeated public keys altogether avoids the problem at the source.</ref>.

murchandamus commented at 2:30 pm on May 7, 2024:

0The public keys obtained by deserializing elements of the key information vector must be pairwise distinct<ref>'''Why must public keys be distinct?''' Reusing pubkeys could be insecure in the conext of wallet policies containing [https://bitcoin.sipa.be/miniscript/ miniscript]. Avoiding repeated public keys altogether avoids the problem at the source.</ref>.

bigspider commented at 8:25 pm on May 7, 2024:

Fixed in 7d0c08e38acac3ef14095d0e8664c7332b7be381, together with few other occurrences of the same in other parts of the document.

in bip-0388.mediawiki:17 in a0c8501f96 outdated

12+  Post-History: 2022-05-10: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-May/020423.html
13+</pre>
14+
15+== Abstract ==
16+
17+Wallet policies build on top of output descriptors to represent in a compact, easier to inspect way the types of descriptors that are typically used to represent "accounts" in a software wallet, or a hardware signing device. A wallet policy always represents exactly two descriptors, which produce the receive and change addresses that are logically part of the same account.

murchandamus commented at 1:58 pm on May 7, 2024:

I found the first sentence a bit long and hard to parse. Perhaps the following is better, or it would be easier to read if it were split into two sentences.

0Wallet policies build on top of output descriptors to represent the types of descriptors that are typically used to represent "accounts" in a software wallet, or a hardware signing device, in a compact, reviewable way. A wallet policy always represents exactly two descriptors, which produce the receive and change addresses that are logically part of the same account.

murchandamus commented at 2:04 pm on May 7, 2024:

Also, you use sometimes “output script descriptor”, “output descriptor”, and just “descriptor” to mean the same thing. Perhaps you should introduce it once as “output script descriptor (abbreviated with ‘descriptor’ in the following)”.

bigspider commented at 8:28 pm on May 7, 2024:

In 7d0c08e38acac3ef14095d0e8664c7332b7be381, I made sure that only the forms “output script descriptor” and “descriptor” appear. I think it’s generally clear from context that both expression refers to the same.

in bip-0388.mediawiki:19 in a0c8501f96 outdated

14+
15+== Abstract ==
16+
17+Wallet policies build on top of output descriptors to represent in a compact, easier to inspect way the types of descriptors that are typically used to represent "accounts" in a software wallet, or a hardware signing device. A wallet policy always represents exactly two descriptors, which produce the receive and change addresses that are logically part of the same account.
18+
19+Reducing the generality of descriptors to just the essential features, and separating the extended pubkeys and other key information from the descriptor, allows to simplify the language in a way that suits devices with limited memory, where even keeping the entire descriptor in memory could be a major hurdle.

murchandamus commented at 2:00 pm on May 7, 2024:

How about using the active voice and switching around the two subsentences?

0We simplify the language to suit devices with limited memory, where even keeping the entire descriptor in memory could be a major hurdle, by reducing the generality of descriptors to just the essential features and by separating the extended pubkeys and other key information from the descriptor.

in bip-0388.mediawiki:21 in a0c8501f96 outdated

16+
17+Wallet policies build on top of output descriptors to represent in a compact, easier to inspect way the types of descriptors that are typically used to represent "accounts" in a software wallet, or a hardware signing device. A wallet policy always represents exactly two descriptors, which produce the receive and change addresses that are logically part of the same account.
18+
19+Reducing the generality of descriptors to just the essential features, and separating the extended pubkeys and other key information from the descriptor, allows to simplify the language in a way that suits devices with limited memory, where even keeping the entire descriptor in memory could be a major hurdle.
20+
21+Moreover, together with the gain in compactness, this simplifies user's inspection of the policy.

murchandamus commented at 2:01 pm on May 7, 2024:

How about:

0This results in a more compact representation and simplifies the inspection of the policy by the user.

in bip-0388.mediawiki:23 in a0c8501f96 outdated

18+
19+Reducing the generality of descriptors to just the essential features, and separating the extended pubkeys and other key information from the descriptor, allows to simplify the language in a way that suits devices with limited memory, where even keeping the entire descriptor in memory could be a major hurdle.
20+
21+Moreover, together with the gain in compactness, this simplifies user's inspection of the policy.
22+
23+Finally, by keeping the language extremely close to that of output script descriptors, the compilation of wallet policies to the corresponding descriptor is extremely easy, and even the reverse process is not too difficult for supported descriptors.

murchandamus commented at 2:03 pm on May 7, 2024:

Seems easier to read to me:

0The compilation of wallet policies to the corresponding descriptor is trivial, and the reverse process is easy for supported descriptors, because the language is kept similar to that of output script descriptors.

in bip-0388.mediawiki:31 in a0c8501f96 outdated

26+
27+This BIP is licensed under the BSD 2-clause license.
28+
29+== Motivation ==
30+
31+''[[bip-0380.mediawiki|Output Script Descriptors]]'' were introduced in bitcoin-core as a way to represent collections of output scripts. It is a very general and flexible language, designed to catch all the possible use-cases of bitcoin wallets (that is, if you know the script and you have the necessary keys, it will be possible to sign transactions with any descriptor-based software wallet).

murchandamus commented at 2:06 pm on May 7, 2024:

0''[[bip-0380.mediawiki|Output Script Descriptors]]'' were introduced in Bitcoin Core as a way to represent collections of output scripts. It is a general and flexible language, designed to catch all the possible use-cases of bitcoin wallets (that is, if you know the script and you have the necessary keys, it will be possible to sign transactions with any descriptor-based software wallet).

in bip-0388.mediawiki:33 in a0c8501f96 outdated

28+
29+== Motivation ==
30+
31+''[[bip-0380.mediawiki|Output Script Descriptors]]'' were introduced in bitcoin-core as a way to represent collections of output scripts. It is a very general and flexible language, designed to catch all the possible use-cases of bitcoin wallets (that is, if you know the script and you have the necessary keys, it will be possible to sign transactions with any descriptor-based software wallet).
32+
33+Unfortunately, descriptors are not a perfect match for the typical usage of hardware signing devices (often also called ''hardware wallets''). Most of them have some of the following limitations when compared to a general-purpose machine running bitcoin-core:

murchandamus commented at 2:06 pm on May 7, 2024:

0Unfortunately, descriptors are not a perfect match for the typical usage of hardware signing devices (often also called ''hardware wallets''). Most of them have some of the following limitations when compared to a general-purpose machine running Bitcoin Core:

in bip-0388.mediawiki:39 in a0c8501f96 outdated

34+
35+* they are embedded devices with limited RAM, and computational power;
36+* they cannot import additional private keys (that is, they can only sign with keys derived from a single seed via [[bip-0032.mediawiki|BIP-32]]);
37+* they have limited storage, or they might not have persistent storage at all (''stateless design'').
38+
39+Moreover, other limitations like the limited size of the screen might affect what design choices are available in practice. Therefore, minimizing the size of the information shown on-screen is important for a good user experience; that is crucial since the ability for the user to completely validate on-screen the kind of script used (and each of the involved keys) is a prerequisite for secure usage, as the machine that is interacting with the hardware signer (and running the software wallet) is considered untrusted.

murchandamus commented at 2:07 pm on May 7, 2024:

0Moreover, other limitations like the limited size of the screen might affect what design choices are available in practice. Therefore, minimizing the amount of information shown on-screen is important for a good user experience. The ability for the user to completely validate on-screen the kind of script used (and each of the involved keys) is crucial for secure usage, as the machine that is interacting with the hardware signer (and running the software wallet) is considered untrusted.

in bip-0388.mediawiki:41 in a0c8501f96 outdated

36+* they cannot import additional private keys (that is, they can only sign with keys derived from a single seed via [[bip-0032.mediawiki|BIP-32]]);
37+* they have limited storage, or they might not have persistent storage at all (''stateless design'').
38+
39+Moreover, other limitations like the limited size of the screen might affect what design choices are available in practice. Therefore, minimizing the size of the information shown on-screen is important for a good user experience; that is crucial since the ability for the user to completely validate on-screen the kind of script used (and each of the involved keys) is a prerequisite for secure usage, as the machine that is interacting with the hardware signer (and running the software wallet) is considered untrusted.
40+
41+A more native, compact representation of the wallet receive/change might also benefit the UX of software wallets using descriptors to represent software wallets using descriptors (possibly with miniscript) for complex locking conditions.

murchandamus commented at 2:11 pm on May 7, 2024:

Missing word, and “the UX of software wallets using descriptors to represent software wallets using descriptors” feels like you accidentally repeated something here.

0A more native, compact representation of the wallet receive and change addresses might also benefit the UX of software wallets when they use descriptors (possibly with miniscript) for representing complex locking conditions.

in bip-0388.mediawiki:47 in a0c8501f96 outdated

42+
43+We remark that wallet policies are not related to the ''policy'' language, a higher level language that can be compiled to miniscript.
44+
45+=== Security and UX concerns for hardware signing devices ===
46+
47+For a hardware signing device, allowing the usage of complex scripts presents challenges in terms of both security and user experience.

murchandamus commented at 2:12 pm on May 7, 2024:

0The usage of complex scripts presents challenges in terms of both security and user experience for a hardware signing device.

in bip-0388.mediawiki:51 in a0c8501f96 outdated

46+
47+For a hardware signing device, allowing the usage of complex scripts presents challenges in terms of both security and user experience.
48+
49+==== Security issues ====
50+
51+One of the security properties that hardware signing devices strive to guarantee is the following: as long as the user correctly verifies the information that is shown on the device's screen before approving, no action can be performed without the user's consent.

murchandamus commented at 2:14 pm on May 7, 2024:

This sentence could be simplified as follows:

0Hardware signing devices strive to guarantee that no action can be performed without the user’s consent as long as the user correctly verifies the information that is shown on the device’s screen before approving.

in bip-0388.mediawiki:62 in a0c8501f96 outdated

57+
58+This makes it impossible for an attacker to surreptitiously modify the policy, therefore stealing or burning the user's funds.
59+
60+==== UX issues ====
61+
62+With miniscript (and taproot trees) allowing substantially more complex spending policies to be used, it becomes more challenging to make sure that the user is practically able to verify the information on the screen.

murchandamus commented at 2:18 pm on May 7, 2024:

0Miniscript (and taproot trees) allow substantially more complex spending policies. It is a challenge to ensure that the user can practically verify such spending policies per the screen.

in bip-0388.mediawiki:64 in a0c8501f96 outdated

59+
60+==== UX issues ====
61+
62+With miniscript (and taproot trees) allowing substantially more complex spending policies to be used, it becomes more challenging to make sure that the user is practically able to verify the information on the screen.
63+
64+Therefore, there are two fundamental design goals to strive for:

murchandamus commented at 2:25 pm on May 7, 2024:

0We set two fundamental design goals:

in bip-0388.mediawiki:104 in a0c8501f96 outdated

 99+** 10 different scripts using a 3-of-3 MuSig2 aggregated key, plus
100+** a final leaf with a fallback 3-of-5 multisignature using <tt>multi_a</tt> (in case interactive signing is not available).
101+
102+With each xpub being 118 bytes long, the repetition of xpubs makes the descriptor become extremely large.
103+
104+Replacing the common part of the key with a short key placeholder and moving the key expression separately helps to keep the size of the wallet policy small, which is crucial to allow human inspection during the registration flow.

murchandamus commented at 2:28 pm on May 7, 2024:

“Moving” feels odd here, how about:

0Replacing the common part of the key with a short key placeholder and presenting the key expression helps to keep the size of the wallet policy small, which is crucial to allow human inspection during the registration flow.

bigspider commented at 8:26 pm on May 7, 2024:

In 7d0c08e38acac3ef14095d0e8664c7332b7be381, I further replaced the sentence to:

Replacing the common part of the key with a short key placeholder and organizing all the key expressions in a separate list helps to keep the size of the wallet policy small, which is crucial to allow human inspection during the registration flow.

in bip-0388.mediawiki:147 in a0c8501f96 outdated

142+
143+Note that while [[bip-0389.mediawiki|BIP-389]] allows multipath `/<NUM;NUM;...;NUM>` expressions with an arbitrary number of options, this specification restricts it to exactly 2 choices (with the typical meaning of receive/change addresses).
144+
145+The placeholder <tt>@i</tt> for some number ''i'' represents the ''i''-th key in the vector of key information items (which must be of size at least ''i + 1'', or the wallet policy is invalid).
146+
147+Note: while descriptor templates for miniscript are not formally defined in this version of the document (pending standardization) it is straightforward to adapt this approach by adding additional <tt>SCRIPT</tt> expressions.

murchandamus commented at 2:29 pm on May 7, 2024:

0Note: while descriptor templates for miniscript are not formally defined in this version of the document (pending standardization), it is straightforward to adapt this approach by adding additional <tt>SCRIPT</tt> expressions.

in bip-0388.mediawiki:168 in a0c8501f96 outdated

163+
164+The public keys obtained by deserializing elements of the keys information vector must be pairwise distinct<ref>'''Why must public keys be distinct?''' Reusing pubkeys could be insecure in the conext of wallet policies containing [https://bitcoin.sipa.be/miniscript/ miniscript]. Avoiding repeated public keys altogether avoids the problem at the source.</ref>.
165+
166+If two key placeholders are <tt>@i/<M;N>/*</tt> and <tt>@i/<P;Q>/*</tt> for the same index <tt>i</tt>, then the sets <tt>{M, N}</tt> and <tt>{P, Q}</tt> must be disjoint.
167+
168+The key information vector should be ordered so that placeholder <tt>@i</tt> never appear for the first time before an occurrence of <tt>@j</tt>  for some j < i</tt>; for example, the first placeholder is always <tt>@0</tt>, the next one is <tt>@1</tt>, etc.

murchandamus commented at 2:32 pm on May 7, 2024:

Grammar, and I think there is a missing <tt> here:

0The key information vector should be ordered so that placeholder <tt>@i</tt> never appears for the first time before an occurrence of <tt>@j</tt>  for some <tt>j < i</tt>; for example, the first placeholder is always <tt>@0</tt>, the next one is <tt>@1</tt>, etc.

in bip-0388.mediawiki:186 in a0c8501f96 outdated

181+
182+<tt>pkh([d34db33f/44'/0'/0']xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL/<0;1>/*)</tt>
183+
184+=== Implementation guidelines ===
185+
186+Implementations must not necessarily implement all the possible wallet policies defined by this standard, but it is recommended to clearly document any limitation.

murchandamus commented at 2:37 pm on May 7, 2024:

I find this sentence confusing because in specifications following RFC 2119 “MUST NOT” is used to express a prohibition. It seems to me that you mean “do not need to” here. How about:

0It is acceptable to implement only a subset of the possible wallet policies defined by this standard. It is recommended that any limitations are clearly documented.

in bip-0388.mediawiki:277 in a0c8501f96 outdated

272+* <tt>pkh(@0/0/**)</tt>: Key placeholder with an explicit path present
273+* <tt>sh(multi(1,@1/**,@0/**))</tt>: Key placeholders out of order
274+* <tt>sh(multi(1,@0/**,@2/**))</tt>: Skipped key placeholder <tt>@1</tt>
275+* <tt>sh(multi(1,@0/**,@0/**))</tt>: Repeated keys with the same path expression
276+* <tt>sh(multi(1,@0/<0;1>/*,@0/<1;2>/*))</tt>: Non-disjoint multipath expressions (<tt>@0/1/*</tt> appears twice)
277+* <tt>sh(multi(1,@0/**,xpub6AHA9hZDN11k2ijHMeS5QqHx2KP9aMBRhTDqANMnwVtdyw2TDYRmF8PjpvwUFcL1Et8Hj59S3gTSMcUQ5gAqTz3Wd8EsMTmF3DChhqPQBnU/<0;1>/*))</tt>: Expression with a non KP key present

murchandamus commented at 2:46 pm on May 7, 2024:

0* <tt>sh(multi(1,@0/**,xpub6AHA9hZDN11k2ijHMeS5QqHx2KP9aMBRhTDqANMnwVtdyw2TDYRmF8PjpvwUFcL1Et8Hj59S3gTSMcUQ5gAqTz3Wd8EsMTmF3DChhqPQBnU/<0;1>/*))</tt>: Expression with a non-KP key present

in bip-0388.mediawiki:280 in a0c8501f96 outdated

275+* <tt>sh(multi(1,@0/**,@0/**))</tt>: Repeated keys with the same path expression
276+* <tt>sh(multi(1,@0/<0;1>/*,@0/<1;2>/*))</tt>: Non-disjoint multipath expressions (<tt>@0/1/*</tt> appears twice)
277+* <tt>sh(multi(1,@0/**,xpub6AHA9hZDN11k2ijHMeS5QqHx2KP9aMBRhTDqANMnwVtdyw2TDYRmF8PjpvwUFcL1Et8Hj59S3gTSMcUQ5gAqTz3Wd8EsMTmF3DChhqPQBnU/<0;1>/*))</tt>: Expression with a non KP key present
278+* <tt>pkh(@0/<0;1;2>/*)</tt>: Solved cardinality > 2
279+
280+Remark: some of the descriptor templates above might be valid if optional extensions allowing them are added in the implementation.

murchandamus commented at 2:49 pm on May 7, 2024:

This sentence feels a redundant, it seems obvious that extensions are added and those extensions would need to allow something to become valid if it’s invalid without the extensions. How about:

0Remark: some of the examples of invalid descriptor templates may be valid via optional extensions.

murchandamus approved

murchandamus commented at 2:54 pm on May 7, 2024: contributor

I did another thorough read and noticed a few style and grammar nits. I did not review the examples or technical details of the proposal. I do not consider any of these comments to be blockers, please feel free to adopt or ignore them at your leisure.

It is not necessary to squash the commits, please follow your own preference.

Apply suggestions from code review

Co-authored-by: Mark "Murch" Erhardt <murch@murch.one>

cf2250e27c

More nits from PR review 7d0c08e38a

bigspider commented at 8:30 pm on May 7, 2024: contributor

Thanks @murchandamus for the thorough suggestions! I accepted almost all directly, and slightly modified the remaining ones in 7d0c08e38acac3ef14095d0e8664c7332b7be381.

murchandamus approved

murchandamus commented at 12:53 pm on May 8, 2024: contributor

ACK 7d0c08e

murchandamus merged this on May 8, 2024

murchandamus closed this on May 8, 2024

bigspider deleted the branch on Jul 10, 2024

BIP 388: Wallet Policies for Descriptor Wallets #1389