BIP329: add optional data fields, fix a JSON type #1750

pull doc-hex wants to merge 19 commits into bitcoin:master from doc-hex:pr329 changing 1 files +53 −4
  1. doc-hex commented at 6:54 pm on January 23, 2025: contributor

    If the goal is solely to move labels between cooperating wallets, then previously defined values are the absolute minimum needed. However, wallet data exports can serve other purposes and many values associated with addresses, transactions and outputs are already on-hand for the wallet generating the export, and yet would be hard or impossible for importing tools to reconstruct.

    This PR proposes a bunch of optional fields that could be useful to users of this data. All are optional.

  2. Correct type error in sample JSON 58d1ef568c
  3. first pass: moar data 71ac9922a4
  4. more fiat and edits e712e991c1
  5. more edits c90d2acd0d
  6. murchandamus commented at 8:30 pm on January 23, 2025: contributor
    cc: @craigraw
  7. nvk commented at 3:05 pm on January 24, 2025: none

    ACK

    If there was appetite would go even further with added WAC optional field and attempt to cover as much common accounting needs.

  8. xavierfiechter commented at 9:45 pm on January 24, 2025: none

    Below are my thoughts on specific fields and suggestions for improving the proposal while keeping it practical:

    Block Height and Timestamps

    Including the block height is very useful, especially for understanding transaction confirmation status. The timestamp, while useful in some cases, feels less critical. It can be derived if needed, and excluding it could simplify the format.

    Transaction and Fees

    Fees are a tricky field to standardize, especially for batched or multi-party ownership transactions. Shifting the focus to inputs and outputs rather than whole transactions and handling the fee on its own makes it a lot easier for me.

    Fair Market Value and Rate

    The FMV field is probably the attribute I wanted the most. The problem with exchange rate is, that there are many of them and I would stick to one (your own) value.

    Keypath

    Okay.

    Value (in sats)

    While value (in sats) can be looked up using tools like Electrum, having it as part of the format is still a helpful convenience.

    Heights for Addresses

    Nah. Address reuse is a bad practice.

    While it’s tempting to include some additional fields (mainly confirmation block height, MFV and value in sats) for richer metadata, I believe the BIP should stick to the smallest common denominator for simplicity and broad compatibility. Generally, I’m okay to store and parse metadata from the text label field and delegating more advanced data aggregation, management, and lookups to external tools and services, like labelbase.space.

    That said, if @craigraw is open to merging changes into BIP-329, I am happy to adapt the python-bip329 library to support these extensions and make them accessible on the Labelbase platform as well.

  9. add per-txn value f91e5dccb0
  10. jonatack added the label Proposed BIP modification on Jan 29, 2025
  11. jonatack added the label Pending acceptance on Jan 29, 2025
  12. in bip-0329.mediawiki:142 in f91e5dccb0 outdated
    138 </pre>
    139 
    140+==Additional Fields==
    141+
    142+If the goal is solely to move labels between cooperating wallets,
    143+then above values are the minimum needed. However, wallet data
    


    jonatack commented at 5:40 pm on January 29, 2025:
    0then the above values are the minimum needed. However, wallet data
    
  13. in bip-0329.mediawiki:144 in f91e5dccb0 outdated
    140+==Additional Fields==
    141+
    142+If the goal is solely to move labels between cooperating wallets,
    143+then above values are the minimum needed. However, wallet data
    144+exports can serve other purposes and many values associated with
    145+addresses, transactions and outputs are already on-hand for the
    


    jonatack commented at 5:46 pm on January 29, 2025:
    0addresses, transactions and outputs are already on hand for the
    
  14. in bip-0329.mediawiki:143 in f91e5dccb0 outdated
    139 
    140+==Additional Fields==
    141+
    142+If the goal is solely to move labels between cooperating wallets,
    143+then above values are the minimum needed. However, wallet data
    144+exports can serve other purposes and many values associated with
    


    jonatack commented at 5:47 pm on January 29, 2025:

    Hard to follow run-on sentence. Best to write separate sentences.

    0exports can serve other purposes. Many values associated with
    
  15. in bip-0329.mediawiki:148 in f91e5dccb0 outdated
    144+exports can serve other purposes and many values associated with
    145+addresses, transactions and outputs are already on-hand for the
    146+wallet generating the export, and yet would be hard or impossible
    147+for importing tools to reconstruct.
    148+
    149+All of the following values are optional for exporter to provide,
    


    jonatack commented at 5:48 pm on January 29, 2025:
    0All of the following values are optional for the exporter to provide,
    
  16. in bip-0329.mediawiki:153 in f91e5dccb0 outdated
    149+All of the following values are optional for exporter to provide,
    150+but should be given if they are readily available.
    151+
    152+=== Transactions ===
    153+
    154+* <tt>height</tt>: An integer giving the block height where this fully-confirmed transaction can be found. Omit for transactions that are confirmed by less than 6 blocks, or use value zero.
    


    jonatack commented at 5:49 pm on January 29, 2025:
    0* <tt>height</tt>: An integer giving the block height where this fully confirmed transaction can be found. For transactions that are confirmed by less than 6 blocks, omit this field or provide a value of zero.
    
  17. in bip-0329.mediawiki:165 in f91e5dccb0 outdated
    161+
    162+* <tt>rate</tt>: Exchange rate at time of transaction. This is the fiat value of a Bitcoin at the time of the transaction, based on user preferences for data source. Example: <tt>"rate": { "USD":  105620.00 }</tt>
    163+
    164+=== Address, Inputs, and Outputs ===
    165+
    166+* <tt>keypath</tt>: The data needed to build full descriptor down to the specific address.  This extends <tt>origin</tt> with the final two components are are unhardened (in the typical case, assuming BIP-84).  Provide string <tt>/1/123</tt> for <tt>wpkh([d34db33f/84'/0'/0'/1/123])</tt>. If the first character is not <tt>/</tt>, then it should be interpreted as a full descriptor, independant of <tt>origin</tt> (if any).
    


    jonatack commented at 5:52 pm on January 29, 2025:
    “are are” -> “that are”?

    doc-hex commented at 6:17 pm on January 29, 2025:
    yes.
  18. in bip-0329.mediawiki:167 in f91e5dccb0 outdated
    163+
    164+=== Address, Inputs, and Outputs ===
    165+
    166+* <tt>keypath</tt>: The data needed to build full descriptor down to the specific address.  This extends <tt>origin</tt> with the final two components are are unhardened (in the typical case, assuming BIP-84).  Provide string <tt>/1/123</tt> for <tt>wpkh([d34db33f/84'/0'/0'/1/123])</tt>. If the first character is not <tt>/</tt>, then it should be interpreted as a full descriptor, independant of <tt>origin</tt> (if any).
    167+
    168+=== Inputs, and Outputs ===
    


    jonatack commented at 5:52 pm on January 29, 2025:
    0=== Inputs and Outputs ===
    
  19. in bip-0329.mediawiki:155 in f91e5dccb0 outdated
    151+
    152+=== Transactions ===
    153+
    154+* <tt>height</tt>: An integer giving the block height where this fully-confirmed transaction can be found. Omit for transactions that are confirmed by less than 6 blocks, or use value zero.
    155+
    156+* <tt>time</tt>: ISO-8601 formatted timestamp of the block given by height, preferably in UTC, although ISO-8601 can represent local times. Example: <tt>2025-01-23T11:40:35-05:00</tt>.
    


    jonatack commented at 5:54 pm on January 29, 2025:

    It’s not clear that “height” here refers to a field name.

    0* <tt>time</tt>: ISO-8601 formatted timestamp of the block given by the "height" field, preferably in UTC, although ISO-8601 can represent local times. Example: <tt>2025-01-23T11:40:35-05:00</tt>.
    
  20. in bip-0329.mediawiki:177 in f91e5dccb0 outdated
    173+
    174+* <tt>height</tt> and <tt>time</tt>: Same definition as defined in Transactions. To save space, if the time value is already shared in a transaction, only the height is required.
    175+
    176+=== Address ===
    177+
    178+* <tt>heights</tt>: a list of block heights which contain inputs which deposit to the address.
    


    jonatack commented at 5:58 pm on January 29, 2025:
    0* <tt>heights</tt>: a list of block heights that contain inputs that deposit to the address.
    
  21. jonatack commented at 6:02 pm on January 29, 2025: member
    A few fixups and suggestions.
  22. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    ecc1634fe2
  23. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    26cc309bd8
  24. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    599c83ad14
  25. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    99fca36161
  26. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    10420c6adb
  27. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    a1b98d47ff
  28. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    9872825549
  29. Update bip-0329.mediawiki
    Co-authored-by: Jon Atack <jon@atack.com>
    2263a4bb4a
  30. edit e63bfa45ac
  31. doc-hex commented at 6:19 pm on January 29, 2025: contributor

    A few fixups and suggestions.

    Thanks, all accepted and merged.

  32. jonatack commented at 7:09 pm on January 31, 2025: member
    Note that commit 58d1ef568c7cde56f499e2c6f351306b40da6ddc in this PR is a fix proposed separately in #1749.
  33. jonatack commented at 7:19 pm on January 31, 2025: member

    Latest push e63bfa45 indeed took all my feedback per git diff f91e5dcc e63bfa45 @xavierfiechter any specific changes/removals you would strongly request?

    Pinging BIP author @craigraw for feedback.

  34. jonatack renamed this:
    Add more data fields to BIP-329 labels
    BIP329: add optional data fields, fix a JSON type
    on Jan 31, 2025
  35. xavierfiechter commented at 7:57 am on February 2, 2025: none
    @jonatack I’ve already shared my points here, on Twitter, and in previous discussions. I also believe that having nothing left to remove has its own value and justification, just as incorporating (some) of the requested changes does. IMHO, ensuring a vendor-agnostic standard that is widely adopted by both the industry and community is more important to me than the exact specifications. I think we should prioritize this aspect while still considering the details.
  36. in bip-0329.mediawiki:153 in e63bfa45ac outdated
    149+All of the following values are optional for the exporter to provide,
    150+but should be given if they are readily available.
    151+
    152+=== Transactions ===
    153+
    154+* <tt>height</tt>: An integer giving the block height where this fully confirmed transaction can be found. For transactions that are confirmed by less than 6 blocks, omit this field or provide a value of zero.
    


    craigraw commented at 2:19 pm on February 6, 2025:
    Is 6 block confirmation requirement really necessary? It makes the addition of this field more complicated, and seems overly defensive. A wallet label export is a “point-in-time” export since a label can change at any time. If the export is viewed as an event stream, when a reorg occurs a new event with the same txid can be sent. There may be a need I haven’t understood here though.

    doc-hex commented at 1:53 pm on February 24, 2025:
    Until it is fully confirmed, the “height” of a transaction is in flux and may vary due to chain reorgs. However, the consumer of this file format does not know the current block height, so they cannot know if the height is “real” (firm, fixed) or just transitory, so I think it’s important not to include the height unless the generating wallet considers the txn confirmed.

    doc-hex commented at 2:02 pm on February 24, 2025:
    added above justification
  37. in bip-0329.mediawiki:155 in e63bfa45ac outdated
    151+
    152+=== Transactions ===
    153+
    154+* <tt>height</tt>: An integer giving the block height where this fully confirmed transaction can be found. For transactions that are confirmed by less than 6 blocks, omit this field or provide a value of zero.
    155+
    156+* <tt>time</tt>: ISO-8601 formatted timestamp of the block given by the "height" field, preferably in UTC, although ISO-8601 can represent local times. Example: <tt>2025-01-23T11:40:35-05:00</tt>.
    


    craigraw commented at 2:23 pm on February 6, 2025:
    The ISO-8601 defined time zone should be required if this field is present. Given the preference for UTC, a better example would be 2025-01-23T11:40:35Z.

    doc-hex commented at 1:50 pm on February 24, 2025:
    as it is written, the example is for the localtime case. probably better to show UTC example tho.
  38. in bip-0329.mediawiki:157 in e63bfa45ac outdated
    153+
    154+* <tt>height</tt>: An integer giving the block height where this fully confirmed transaction can be found. For transactions that are confirmed by less than 6 blocks, omit this field or provide a value of zero.
    155+
    156+* <tt>time</tt>: ISO-8601 formatted timestamp of the block given by the "height" field, preferably in UTC, although ISO-8601 can represent local times. Example: <tt>2025-01-23T11:40:35-05:00</tt>.
    157+
    158+* <tt>fee</tt>: Integer giving the number of Satoshis that went to the miner for this transaction.
    


    craigraw commented at 2:24 pm on February 6, 2025:
    No change needed here, but just noting this is generally only known for outgoing transactions, where all the inputs are from the wallet.
  39. in bip-0329.mediawiki:159 in e63bfa45ac outdated
    155+
    156+* <tt>time</tt>: ISO-8601 formatted timestamp of the block given by the "height" field, preferably in UTC, although ISO-8601 can represent local times. Example: <tt>2025-01-23T11:40:35-05:00</tt>.
    157+
    158+* <tt>fee</tt>: Integer giving the number of Satoshis that went to the miner for this transaction.
    159+
    160+* <tt>value</tt>: Signed integer giving the number of Satoshis that came into the wallet by this transaction. Will be negative when sats leave the wallet. Could be zero if it is a consolidation transaction that moves from old UTXO to new. 
    


    craigraw commented at 2:26 pm on February 6, 2025:
    Referring to the second sentence - in the case of a consolidation transaction this will reflect the transaction fee, which is unlikely to be zero, so this should be amended to be clearer.

    doc-hex commented at 1:55 pm on February 24, 2025:
    I agree it’s unlikely, but a zero could happen if other UTXO are used to pay the miner’s fee.

    jonatack commented at 8:16 pm on February 25, 2025:
    perhaps s/to new./to new (possibly modulo the transaction fee)."

    doc-hex commented at 9:07 pm on February 25, 2025:
    if a fee is paid in a consolidation, then it cannot be zero value; it would be net negative… might be better to delete the zero case here as it’s just confusing.
  40. in bip-0329.mediawiki:161 in e63bfa45ac outdated
    157+
    158+* <tt>fee</tt>: Integer giving the number of Satoshis that went to the miner for this transaction.
    159+
    160+* <tt>value</tt>: Signed integer giving the number of Satoshis that came into the wallet by this transaction. Will be negative when sats leave the wallet. Could be zero if it is a consolidation transaction that moves from old UTXO to new. 
    161+
    162+* <tt>rate</tt>: Exchange rate at time of transaction. This is the fiat value of a Bitcoin at the time of the transaction, based on user preferences for data source. Example: <tt>"rate": { "USD":  105620.00 }</tt>
    


    craigraw commented at 2:30 pm on February 6, 2025:
    I don’t think this needs to be restricted to fiat currencies, but can be “the equivalent value of a Bitcoin in another currency at the time of the transaction”. It should be noted in the BIP that multiple currencies can be specified in the mapping, and ISO 4217 currency codes should be preferred as keys where possible.

    doc-hex commented at 2:06 pm on February 24, 2025:
    agreed
  41. in bip-0329.mediawiki:173 in e63bfa45ac outdated
    169+
    170+* <tt>value</tt>: Integer with the number of Satoshis (<tt>nValue</tt>) of the input or output.
    171+
    172+* <tt>fmv</tt>: Fair market value of the input/output relative to some other currency, typically fiat. The value should be a mapping, from currency code to decimal number. Example: <tt>"fmv": { "USD":  1233.45 }</tt>. Most situations will have only a single currency value, and it represents the real price of the goods/services expressed in some fiat currency. This is not an exchange *rate*, but an absolute value. By dividing by the <tt>value</tt> (above), it is possible to calculate an effective change rate for the transaction.
    173+
    174+* <tt>height</tt> and <tt>time</tt>: Same definition as defined in Transactions. To save space, if the time value is already shared in a transaction, only the height is required.
    


    craigraw commented at 2:32 pm on February 6, 2025:
    Regarding the second sentence: neither field is required, so this is confusing. Besides, if you are expecting the receiver to do a lookup based on received transactions for the time field, why not the height as well? I think it would be better to omit this sentence altogether, since BIP329 can be used in an event stream in which case previous events should not be relied upon.

    doc-hex commented at 1:58 pm on February 24, 2025:
    I will delete the last sentence.
  42. in bip-0329.mediawiki:177 in e63bfa45ac outdated
    173+
    174+* <tt>height</tt> and <tt>time</tt>: Same definition as defined in Transactions. To save space, if the time value is already shared in a transaction, only the height is required.
    175+
    176+=== Address ===
    177+
    178+* <tt>heights</tt>: a list of block heights that contain inputs that deposit to the address.
    


    craigraw commented at 3:10 pm on February 6, 2025:
    Should be amended to “that contain outputs that deposit to the address”. Should also include the block heights of transactions with inputs that spend from the address. It should be noted that it can be an empty array to indicate the address is unused.
  43. in bip-0329.mediawiki:134 in e63bfa45ac outdated
    130@@ -131,14 +131,63 @@ The following fragment represents a wallet label export:
    131 { "type": "addr", "ref": "bc1q34aq5drpuwy3wgl9lhup9892qp6svr8ldzyy7c", "label": "Address" }
    132 { "type": "pubkey", "ref": "0283409659355b6d1cc3c32decd5d561abaac86c37a353b52895a5e6c196d6f448", "label": "Public Key" }
    133 { "type": "input", "ref": "f91d0a8a78462bc59398f2c5d7a84fcff491c26ba54c4833478b202796c8aafd:0", "label": "Input" }
    134-{ "type": "output", "ref": "f91d0a8a78462bc59398f2c5d7a84fcff491c26ba54c4833478b202796c8aafd:1", "label": "Output" , "spendable" : "false" }
    135+{ "type": "output", "ref": "f91d0a8a78462bc59398f2c5d7a84fcff491c26ba54c4833478b202796c8aafd:1", "label": "Output", "spendable": false }
    


    craigraw commented at 3:19 pm on February 6, 2025:
    @sethforprivacy Note the type correction here.
  44. craigraw changes_requested
  45. craigraw commented at 3:21 pm on February 6, 2025: contributor
    Regarding the origin field, the line “The origin property should only appear where type is tx, and the spendable property only where type is output” should be amended to just “The spendable property should only appear where type is output” since origin can now be used on all types.
  46. craigraw commented at 3:33 pm on February 6, 2025: contributor

    I’m generally ok with all the proposed fields, since they reference publicly available information, or at least information that should be publicly available.

    A note on implementation: Historical fiat exchange rate information is becoming more difficult for open source wallet clients to retrieve, with Coingecko and Coinbase now requiring paid subscriptions for historical rate lookups. Only mempool.space provides an open API for historical rates, and then only for the last couple of years in a handful of major currencies. However, this presents an interesting opportunity for server-based implementations, which can add the exchange rate fields by processing this format as an event stream, and may have less need for historical rate access.

  47. jonatack added the label PR Author action required on Feb 6, 2025
  48. include @craigraw feedback cc55f29e6d
  49. typo fix c77783bac3
  50. add comment 8f2b29bf59
  51. doc-hex commented at 2:20 pm on February 24, 2025: contributor
    Thanks for the feedback @craigraw … I’ve put it in as best I can.
  52. further edits 473b8ca82c
  53. doc-hex commented at 4:08 pm on February 24, 2025: contributor

    Regarding the origin field, the line “The origin property should only appear where type is tx, and the spendable property only where type is output” should be amended to just “The spendable property should only appear where type is output” since origin can now be used on all types.

    I made this edit just now.

  54. broaden uses for address.heights 795255ba5b
  55. craigraw commented at 7:23 am on February 25, 2025: contributor
    @doc-hex Thanks for the edits. @jonatack LGTM, happy to merge.
  56. jonatack removed the label Pending acceptance on Feb 25, 2025
  57. jonatack removed the label PR Author action required on Feb 25, 2025
  58. jonatack assigned jonatack on Feb 25, 2025
  59. in bip-0329.mediawiki:177 in 795255ba5b
    173+
    174+* <tt>height</tt> and <tt>time</tt>: Same definition as defined in Transactions.
    175+
    176+=== Address ===
    177+
    178+* <tt>heights</tt>: a list of block heights that contain any activity related to this address, include outputs that deposit to the address, and transactions that spend UTXO of this address. An empty array indicates the address is unused.
    


    jonatack commented at 8:12 pm on February 25, 2025:

    Either “a UTXO” or “UTXOs” here…

    0* <tt>heights</tt>: a list of block heights that contain any activity related to this address, include outputs that deposit to the address, and transactions that spend UTXOs of this address. An empty array indicates the address is unused.
    

    doc-hex commented at 9:04 pm on February 25, 2025:
    UTXOs
  60. in bip-0329.mediawiki:184 in 795255ba5b
    180+== Comment on Types in JSON ==
    181+
    182+JSON can serialize a number of basic types, including string, integer
    183+and boolean (true/false).  Decimal values (<tt>123.45</tt>) can
    184+also be serialized, but some parsing libraries may interpret them as floating
    185+point values which is generally not what we want in financial applications.
    


    jonatack commented at 8:19 pm on February 25, 2025:
    0point values, which is generally not what we want in financial applications.
    

    doc-hex commented at 9:04 pm on February 25, 2025:
    sure
  61. in bip-0329.mediawiki:153 in 795255ba5b
    149+All of the following values are optional for the exporter to provide,
    150+but should be given if they are readily available.
    151+
    152+=== Transactions ===
    153+
    154+* <tt>height</tt>: An integer giving the block height where this fully confirmed transaction can be found. For transactions that are confirmed by less than 6 blocks, omit this field or provide a value of zero.  (Background: Until it is fully confirmed, the "height" of a transaction is in flux and may vary due to chain reorgs. However, the consumer of the labels, may not know the current block height, so it cannot know if the height is "real" (firm, fixed) or just transitory, so it's important not to include the height unless the generating wallet considers the transaction to be confirmed.)
    


    jonatack commented at 8:23 pm on February 25, 2025:
    0* <tt>height</tt>: An integer giving the block height where this fully confirmed transaction can be found. For transactions that are confirmed by less than 6 blocks, omit this field or provide a value of zero.  (Background: Until it is fully confirmed, the "height" of a transaction is in flux and may vary due to chain reorgs. However, the consumer of the labels may not know the current block height, so it cannot know if the height is "real" (firm, fixed) or just transitory. Therefore, it is important to omit the height unless the generating wallet considers the transaction to be confirmed.)
    

    doc-hex commented at 9:05 pm on February 25, 2025:
    yes, I like
  62. jonatack commented at 8:25 pm on February 25, 2025: member

    Thank you @craigraw for reviewing.

    ACK 795255ba5bc3abf509ca0d211d3fb5f2e2e0fc5f modulo a few editorial changes for a follow-up (edit: done in #1776).

  63. jonatack merged this on Feb 25, 2025
  64. jonatack closed this on Feb 25, 2025


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bips. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2025-03-29 07:10 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me