wallet: add seeds argument to importdescriptors #27351

pull apoelstra wants to merge 6 commits into bitcoin:master from apoelstra:2023-03--codex32 changing 13 files +1338 −45
  1. apoelstra commented at 0:57 am on March 28, 2023: contributor

    This PR introduces the ability to import BIP32 master seeds alongside xpubs. It currently expects seeds to be provided in BIP 93/codex32 either as a single seed or as a list of shares which can be assembled via Shamir Secret Sharing.

    It could be generalized to other seed formats, e.g. a hex-encoding or the old sethdseeds WIF format, easily enough by just attempting to parse the input in those formats and retrying until we find one that works. (Though we must be extremely careful that these formats are unambiguous! If the user imports a seed they don’t expect, which fails to match the one they’ve backed up, the result could be coin loss. For formats with no prefix or checksum, e.g. hex-encoded bytes, we may need to require the user provide a prefix or a separate flag.)

    The motivation here is that users who are importing their coins from another wallet likely have them in the form of a public descriptor (probably one of the standard pkh(BIP44/48/84/whatever) ones) along with a master seed (probably in some seed word format, but this is easy enough to convert to a straight BIP32 seed with an external tool). The seed, presumably, they want to minimize handling of, while the descriptor they are likely to be copy/pasting from somewhere and generally being less careful with.

    To import this into Core, they currently need to construct a version of their descriptor in which xpubs are replaced by xprivs, and import that. This is cumbersome and potentially dangerous. With this extra RPC flag they are able to separately tweak their descriptor however they need, and then adding their seed data, unmodified, at the last minute in one place.

    My specific use case is: I have an estate planning document which contains careful instructions for reconstructing seed data from shares which are stored separately and externally to the planning document. Meanwhile, the document itself contains a public descriptor, with checksum already computed, which is stored in several locations accessible by an open-ended set of people. If I were forced to mix the secret data with the public data, it would increase the risk of leaking the secrets as well as the risk of losing the public part.

    Edit: Other approaches, that I considered but chose not to take:

    • Extend the existing sethdseed RPC. This RPC is legacy-only and not supported by descriptor wallets, because it depends on the legacy “bag of keys” model where you could stick keys into the wallet and it’d “just work”, often in surprising ways.
    • Add a new descriptor-wallet-based importseed RPC or something. This doesn’t work for more-or-less the same reason; the descriptor wallet associates all its keys with ScriptPubKeyManagers, so we would need to add support for “global keys” somehow. This is a lot of work, lots of room for bikeshedding, and I worry that @achow101 will NACK anything that looks too much like a “bag of keys” model. (The situation would be much less bad than the legacy one, where 3rd parties could do stuff like mapping p2wpkh addresses to p2pkh ones and the wallet would just accept it … but it would still potentially lead to user confusion about what exact conditions the wallet could sign for what descriptors, and it may constrain future wallet changes.)
    • Adding a codex32(...)/1/2/3 xpriv format to the descriptor import format. This was suggested to me by @sipa and has the benefit that very closely matches the existing import model.

    Personally I don’t like the latter

    • for the reasons stated in the PR description – I like my descriptors to be public, and my secret data to be limited to 16 bytes, and I think this is a common mode of operation for people coming from other wallets
    • …and maybe worse, a common model for other wallet authors, who would refuse to nicely interoperate with this);
    • because it felt like enough of a change to deserve a change to one of the descriptor BIPs, but those are big enough already and it’s hard to get tiny changes to propagate to all implementors of standards. Plus it felt wrong to couple codex32 with descriptors like this.i
    • because if you have multiple descirptors that use the same seed, it would force you to copy the seed multiple times, which is dangerous. (In some cases BIP 389 will make this better, but it doesn’t cover everything and anyway is not accepted or implemented yet. One thing at a time :))

    But what cinched it, and made me switch to the approach in the PR, was that @roconnor-blockstream pointed out to me that the codex32 model means that you can’t predict, at writing-down-the-descriptor time, exactly what shares you’ll be inputting. And because descriptors have a checksum which covers their entire string encoding, this means that you can’t predict ahead-of-timei what will go into the codex32(...) slot and therefore what the total checksum ought to be. And this means that either (a) you lose the descriptor checksum, which is a Bad Idea and also inconvenient for importers who’ will have to construct a checksum with getdescriptorinfo to get importdescriptors to accept it, even though said checksum had never been used to protect data in storage; or (b) you modify the descriptor checksum algorithm to be aware of codex(...) and to skip checksumming the data inside of that. This means even more implementation complexity that other wallets are unlikely to bring in, plus the result generalizes less easily to other seed formats (e.g. hex) which don’t have checksums of their own.

  2. DrahtBot commented at 0:57 am on March 28, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK josibake, S3RK

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #29607 (refactor: Reduce memory copying operations in bech32 encoding/decoding by paplorinc)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  3. DrahtBot added the label Wallet on Mar 28, 2023
  4. apoelstra commented at 0:57 am on March 28, 2023: contributor
  5. apoelstra force-pushed on Mar 28, 2023
  6. josibake commented at 1:41 pm on March 28, 2023: member

    Concept ACK on adding BIP93 support to core.

    Regarding the approach, my first thought was why not extend sethdseed to support BIP93, but I’m guessing this doesn’t work out of the box per your findings in #27337 ?

  7. apoelstra commented at 1:59 pm on March 28, 2023: contributor
    Edit: moved the content of this comment into the PR description (was a description of alternate approaches)
  8. josibake commented at 2:12 pm on March 28, 2023: member

    This RPC (sethdseed) is legacy-only and not supported by descriptor wallets

    Great point, I had forgotten about this

    for the reasons stated in the PR description – I like my descriptors to be public, and my secret data to be limited to 16 bytes, and I think this is a common mode of operation for people coming from other wallets

    This is a very compelling argument, and your approach seems like the path of least resistance towards supporting this use case.

    the descriptor wallet associates all its keys with ScriptPubKeyManagers, so we would need to add support for “global keys” somehow. This is a lot of work, lots of room for bikeshedding

    Longterm, I think it would be great to look into a more comprehensive solution which addresses the above

    EDIT: I’d suggest adding your comment to the description; it really helps in understanding why you’re taking this approach

  9. in test/functional/wallet_importseed.py:5 in 88ee33895b outdated
    0@@ -0,0 +1,268 @@
    1+#!/usr/bin/env python3
    2+# Copyright (c) 2013 The Bitcoin Core developers
    3+# Distributed under the MIT software license, see the accompanying
    4+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
    5+"""Test the importseed RPC.
    


    instagibbs commented at 8:23 pm on March 28, 2023:
    I don’t think that RPC exists? :)

    apoelstra commented at 9:34 pm on March 28, 2023:
    Lol. Updated the doccomment.
  10. apoelstra force-pushed on Mar 28, 2023
  11. S3RK commented at 7:03 am on March 29, 2023: contributor

    Concept ACK. I think we should definitely support the described use case.

    For the approach, I think we already have some implementation ready for the second alternative you mentioned. #26728 adds support for a wallet master key (I think this is what you meant by “global keys”). And #23544 by @Sjors, built on top of it, adds a RPC for descriptor wallets similar to sethdseed.

    I like the alternative better because it support more use cases:

    1. storing wallet master key allows you to generate new descriptors for the existing master key, e.g. to add tr descriptors to existing wallet
    2. better support for multisig between core and a hardware wallet (described in #23544)
  12. apoelstra commented at 2:46 pm on March 29, 2023: contributor

    @S3RK I agree that Sjors’ PR does what we want (and more), up to the specific import format, but

    • It has not been updated in 18 months and now has many conflicts
    • It depends on two other unmerged PRs, one of which is marked draft
    • It is itself marked draft
    • It is 30 commits long and requires significant reviewer attention

    Meanwhile this PR is narrow in scope, doesn’t even touch the actual wallet code and therefore won’t conflict with ongoing mega-refactors, and is currently up to date.

    So I don’t think we should put this on hold because it may be made redundant by a future addseeds RPC. We may even want both, since the functionality here is slightly different (rather than importing global seeds, it allows importing a specific set of descriptors which share one seed, without affecting others).

  13. luke-jr referenced this in commit 083450c7ad on Aug 16, 2023
  14. luke-jr referenced this in commit 16f7501afd on Aug 16, 2023
  15. luke-jr referenced this in commit 9341c72747 on Aug 16, 2023
  16. luke-jr referenced this in commit 379d336bc0 on Aug 16, 2023
  17. luke-jr referenced this in commit 87030b20ec on Aug 16, 2023
  18. luke-jr referenced this in commit 34c15b459e on Aug 16, 2023
  19. DrahtBot added the label Needs rebase on Aug 17, 2023
  20. bech32: expose the character conversion functionality
    In the next commit we will implement a new checksum, codex32, which uses
    the same encoding and HRP rules as bech32 and bech32m, but has a
    substantially different checksum verification procedure. To minimize
    duplicated code, we expose the character conversion in a new
    bech32::internals module.
    7a57cda8e4
  21. codex32: implement encoding and decoding fd6975d631
  22. codex32: introduce Lagrange interpolation and derived shares 7eced20085
  23. codex32: provide user-readable error types 1fa04b11b2
  24. wallet: add ability for `importdescriptors` to import a seed e29e656cbd
  25. codex32: add functional test for seed import 91771366a3
  26. apoelstra force-pushed on Aug 17, 2023
  27. apoelstra commented at 4:27 pm on August 17, 2023: contributor
    Rebased.
  28. DrahtBot removed the label Needs rebase on Aug 17, 2023
  29. DrahtBot added the label CI failed on Aug 18, 2023
  30. DrahtBot removed the label CI failed on Aug 18, 2023
  31. luke-jr referenced this in commit a1adcf8b2a on Aug 29, 2023
  32. luke-jr referenced this in commit e94863bd27 on Aug 29, 2023
  33. luke-jr referenced this in commit 203725def9 on Aug 29, 2023
  34. luke-jr referenced this in commit dba29e603a on Aug 29, 2023
  35. luke-jr referenced this in commit 6b585a09fa on Aug 29, 2023
  36. luke-jr referenced this in commit eb78737ad4 on Aug 29, 2023
  37. in src/codex32.cpp:2 in 91771366a3
    0@@ -0,0 +1,423 @@
    1+// Copyright (c) 2017, 2021 Pieter Wuille
    2+// Copyright (c) 2021-2022 The Bitcoin Core developers
    


    rot13maxi commented at 0:15 am on September 10, 2023:
    is this copyright correct? doesn’t match the header file
  38. rot13maxi commented at 1:04 am on September 10, 2023: none

    Would love to have support for codex32 shares/seeds!

    Thanks for the detailed comments in codex32.cpp! Really made it easier to follow

    It looks like this presupposes that you already have an xpub to use with importdescriptors and lets you reconstruct the seed with codex32-encoded shares. It seems like you’d also need something like sethdseed to load secret shares into a blank descriptor wallet in the first place? I read and understand your rationale in the description, but it seems like there’s a missing step between “generate your seed/shares” and “import them along with the xpub”

  39. apoelstra commented at 4:34 pm on September 11, 2023: contributor
    @rot13maxi right – though I was imagining that there would (later) be an external tool to compute the master xpub from the seed data, if you didn’t already know it.
  40. DrahtBot added the label CI failed on Nov 2, 2023
  41. DrahtBot removed the label CI failed on Nov 7, 2023
  42. DrahtBot added the label CI failed on Jan 13, 2024
  43. luke-jr referenced this in commit 44f7483b32 on Mar 5, 2024
  44. luke-jr referenced this in commit e94669ef17 on Mar 5, 2024
  45. luke-jr referenced this in commit 160e18783c on Mar 5, 2024
  46. luke-jr referenced this in commit 2f308ebff5 on Mar 5, 2024
  47. luke-jr referenced this in commit 7ab76491c0 on Mar 5, 2024
  48. luke-jr referenced this in commit 8721d6f228 on Mar 5, 2024
  49. achow101 commented at 2:38 pm on April 9, 2024: member
    The approach in this PR does not seem to have support.
  50. achow101 closed this on Apr 9, 2024

  51. Sjors commented at 2:55 pm on April 16, 2024: member

    Now that we have a createwalletdescriptor RPC call #29130, this could be expanded to take private key material, as suggested by @ryanofsky: #29130 (review)

    codex32 could then be one of the import formats.

  52. apoelstra commented at 3:07 pm on April 16, 2024: contributor
    Thanks! The next time I am working on codex32 wallet support I will try that approach instead.

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-03 07:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me