doc/descriptors.md tweaks

ryanofsky commented at 2:05 pm on September 6, 2018: member

Add some implementation details, and tweak phrasing in examples section to be more explicit about how expressions are used for matching.

Add some implementation details, and tweak phrasing in examples section to be
more explicit about how script expressions are used for matching.

eb49412562

fanquake added the label Docs on Sep 6, 2018

ryanofsky force-pushed on Sep 6, 2018

laanwj requested review from sipa on Sep 6, 2018

laanwj added this to the milestone 0.17.0 on Sep 6, 2018

practicalswift commented at 9:25 pm on September 6, 2018: contributor

Concept ACK

in doc/descriptors.md:136 in bce4db1857 outdated

134+P2PKH, P2WPKH, or P2SH-P2WPH scripts for that key. In case the key is
135+uncompressed, it only matches the P2PK or P2PKH scripts.
136+
137+### Output Descriptors vs. Addresses
138+
139+Like a bitcoin address, a bitcoin output descriptor can be used as a human

sipa commented at 9:38 pm on September 6, 2018:

I think this is not a similarity but a difference: an address does not (generally) convey information on how to spend it; a descriptor does (unless raw or addr are used).

ryanofsky commented at 9:09 pm on September 10, 2018:

Yeah, this sentence would probably be clearer dropping “that a given wallet knows how to spend” from the end. The difference you are describing is basically difference #1 in the list below.

But I think it might be a good idea to drop this section entirely. Basically I was trying to describe the properties of descriptors by contrasting them with addresses. But I think it might be better to have a section just describing some ways descriptors can be used (internally or externally), since this is something actually I’m fuzzy on.

sipa commented at 10:59 pm on October 12, 2018:

I suspect this will clear itself up when more uses for descriptors appear.

ryanofsky commented at 4:46 pm on October 17, 2018:

Ok, just dropped this section. Pushed to https://github.com/ryanofsky/bitcoin/commit/9592a1bd912f3bd8542b061b79567ed55be13a32 for reference.

in doc/descriptors.md:25 in bce4db1857 outdated

33-- `wsh(multi(2,03a0434d9e47f3c86235477c7b1ae6ae5d3442d49b1943c2b752a68e2a47e247c7,03774ae7f858a9411e5ef4246b70c65aac5649980be5c17891bbec17895da008cb,03d01115d548e7561b15c38f004d734633687cf4419620095bc5b0f47070afe85a))` represents a P2WSH *2-of-3* multisig.
34-- `sh(wsh(multi(1,03f28773c2d975288bc7d1d205c3748651b075fbc6610e58cddeeddf8f19405aa8,03499fdf9e895e719cfd64e67f07d38e3226aa7b63678949e6e49b241a60e823e4,02d7924d4f7d43ea965a465ae3095ff41131e5946f3c85f79e44adbcf8e27e080e)))` represents a P2SH-P2WSH *1-of-3* multisig.
35-- `pk(xpub661MyMwAqRbcFtXgS5sYJABqqG9YLmC4Q1Rdap9gSE8NqtwybGhePY2gZ29ESFjqJoCu1Rupje8YtGqsefD265TMg7usUDFdp6W1EGMcet8)` refers to a single P2PK output, using the public key part from the specified xpub.
36-- `pkh(xpub68Gmy5EdvgibQVfPdqkBBCHxA5htiqg55crXYuXoQRKfDBFA1WEjWgP6LHhwBZeNK1VTsfTFUHCdrfp1bgwQ9xv5ski8PX9rL2dZXvgGDnw/1'/2)` refers to a single P2PKH output, using child key *1'/2* of the specified xpub.
37-- `wsh(multi(1,xpub661MyMwAqRbcFW31YEwpkMuc5THy2PSt5bDMsktWQcFF8syAmRUapSCGu8ED9W6oDMSgv6Zz8idoc4a6mr8BDzTJY47LJhkJ8UB7WEGuduB/1/0/*,xpub69H7F5d8KSRgmmdJg2KhpAK8SR3DjMwAdkxj3ZuxV27CprR9LgpeyGmXUbC6wb7ERfvrnKZjXoUmmDznezpbZb7ap6r1D3tgFxHmwMkQTPH/1/0/*))` refers to a chain of *1-of-2* P2WSH multisig outputs, using public keys taken from two HD chains with corresponding derivation paths.
38+- `pk(0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798)` matches a P2PK output with the specified public key.

sipa commented at 9:41 pm on September 6, 2018:

I haven’t really thought of descriptors as “filters” that match certain sPK, more as compact descriptions for specifically instantiated scripts.

Perhaps it makes sense to describe it this way (especially in the context of scantxoutset), but perhaps it is also confusing. It may make people wonder why a range needs to be explicitly specified, or why an addr(1...) can’t match a P2PK output.

What do you think?

ryanofsky commented at 8:54 pm on September 10, 2018:

Well, these edits are just suggestions. I’m happy to revert anything or change it however you’d like.

But I guess I don’t really get the point about BIP32 ranges and addr() with P2PK, because it seems like you have to document the behavior either way. And in theory if we wanted to match P2PK keys & hashes, it wouldn’t actually complicate the current implementation very much.

I like saying “matches” here just because “matches” is concrete. It seems clearer to say that a descriptor either “matches” or “doesn’t match” a script than to say that it “represents,” or “refers,” or sometimes “constructs” one. But if you wanted to replace “matches” with a different concrete term like “generates” or “constructs” everywhere, I think that would be good, too.

sipa commented at 10:58 pm on October 12, 2018:

Ah, perhaps this isn’t clear from the current implementation yet.

One of the original goals was to have an explicit representation for the actual set of scriptPubKeys that match the wallet. This would permit much faster rescans, as it avoids the need to feed every scriptPubKey seen in the chain through the IsMine machinery, and instead just test for inclusion in a set. Alternatively, it permits even faster structures such as using a bloom filter for the wallet, or using the BIP158 filters for blocks.

Having functionality in the descriptors language which just specifies a pubkey hash but encompasses P2PK outputs is not compatible with that intended feature.

Does that clear things up? If you think using the term ‘matches’ doesn’t convey the wrong meaning taking that into account I’m fine with it.

ryanofsky commented at 4:20 pm on October 17, 2018:

Replaced “matches” with a “describes” in a followup commit.

After hearing about plans for the wallet, I think I now agree “matches scriptingPubKeys” is too narrow, since we’re not only going to use descriptors as a way of matching tx outputs, but also as a way of generating outputs and addresses (by replacing keypools in the wallet, and serving as a bridge between wallet and the “policy language” you described at coredev).

Having functionality in the descriptors language which just specifies a pubkey hash but encompasses P2PK outputs is not compatible with that intended feature.

I didn’t know about the bloom filtering idea, and I see how this does impose constraints on the types of scriptPubKeys that could be matched efficiently. I think it doesn’t really have bearing on how you document the descriptor language, though. For example, we talk about matching regexes despite the fact that regex languages are limited and won’t match many string patterns.

in doc/descriptors.md:94 in bce4db1857 outdated

90@@ -91,6 +91,14 @@ on Bitcoin's OP_CHECKMULTISIG opcode. To support these, we introduce the
91 multisig policy, where any *k* out of the *n* provided public keys must
92 sign.
93 
94+Key order is significant. A `multi()` expression will only match multisig

sipa commented at 9:42 pm on September 6, 2018:

Big +1 on this section; that information was really missing.

fanquake added the label Needs backport on Sep 11, 2018

MarcoFalke removed this from the milestone 0.17.0 on Sep 30, 2018

MarcoFalke added this to the milestone 0.17.1 on Sep 30, 2018

MarcoFalke commented at 1:13 am on September 30, 2018: member

Is this for 0.17.0? It seems it still needs some more work or feedback addressed?

sipa commented at 7:59 am on October 17, 2018: member

@ryanofsky Ping, what do you think about my comments above?

descriptors.md: Refer to descriptors as describing instead of matching eeeaa29214

ryanofsky force-pushed on Oct 17, 2018

ryanofsky commented at 5:05 pm on October 17, 2018: member

Updated bce4db1857c3907b7a8e87dcbcb82f658fc8478a -> eeeaa29214ab99e7f491b21d21bf362a86a48305 (pr/ddoc.3 -> pr/ddoc.4) dropping address section and replacing “matches” with “describes”.

sipa commented at 7:07 pm on October 17, 2018: member

utACK

jonasschnelli commented at 7:36 pm on October 17, 2018: contributor

utACK eeeaa29214ab99e7f491b21d21bf362a86a48305

DrahtBot commented at 10:29 am on October 20, 2018: member

#14477 (Add ability to convert solvability info to descriptor by sipa)
#14150 (Add key origin support to descriptors by sipa)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

MarcoFalke merged this on Oct 21, 2018

MarcoFalke closed this on Oct 21, 2018

MarcoFalke referenced this in commit 24e6c80668 on Oct 21, 2018

MarcoFalke referenced this in commit 33c65723a6 on Oct 21, 2018

MarcoFalke referenced this in commit 5c25409d68 on Oct 21, 2018

MarcoFalke removed the label Needs backport on Oct 21, 2018

MarcoFalke referenced this in commit 5f51fd6d59 on Oct 25, 2018

MarcoFalke referenced this in commit 703a24418c on Oct 25, 2018

toxeus referenced this in commit a83846edce on Nov 28, 2018

toxeus referenced this in commit 5dd3919f6b on Nov 28, 2018

jasonbcox referenced this in commit 0d6142d8ec on Oct 8, 2020

MarcoFalke locked this on Sep 8, 2021

doc/descriptors.md tweaks #14161