rpc: Auto-format RPCResult #17809

pull MarcoFalke wants to merge 2 commits into bitcoin:master from MarcoFalke:1906-rpcResult changing 11 files +1515 −1248
  1. MarcoFalke commented at 9:43 pm on December 27, 2019: member

    This enforces most syntax rules of the RPCResult at compile time (or some at run time during unit and functional tests)

    Apart from normalizing the syntax, by separating stylistic formatting from the structure, we could in theory directly generate the html for e.g. https://bitcoincore.org/en/doc/0.19.0/rpc/wallet/importmulti/

  2. fanquake added the label RPC/REST/ZMQ on Dec 27, 2019
  3. fanquake added the label Docs on Dec 27, 2019
  4. MarcoFalke commented at 9:44 pm on December 27, 2019: member
    Currently based on #17804, will squash after that was merged
  5. DrahtBot commented at 10:28 pm on December 27, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #18164 (rpc: generalize address in decoderawtransaction RPCResult by theStack)
    • #17812 (config, net, test: asmap feature refinements and functional tests by jonatack)
    • #17034 (Bip174 extensions by achow101)
    • #16807 (Let validateaddress locate error in Bech32 address by meshcollider)
    • #16795 (rpc: have raw transaction decoding infer output descriptors by instagibbs)
    • #16490 (rpc: Report reason for replaceable txpool transactions by MarcoFalke)
    • #16463 ([BIP 174] Implement serialization support for GLOBAL_XPUB field. by achow101)
    • #16440 (BIP-322: Generic signed message format by kallewoof)
    • #12911 (wallet: Show fee in results for signrawtransaction* for segwit inputs by kallewoof)
    • #12677 (RPC: Add ancestor{count,size,fees} to listunspent output by luke-jr)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  6. promag commented at 10:32 pm on December 27, 2019: member
    Concept ACK 👏
  7. in src/wallet/rpcwallet.cpp:848 in 9ebdc9848d outdated
    844@@ -843,8 +845,8 @@ static UniValue sendmany(const JSONRPCRequest& request)
    845             "       \"CONSERVATIVE\""},
    846                 },
    847                  RPCResult{
    848-            "\"txid\"                   (string) The transaction id for the send. Only 1 transaction is created regardless of \n"
    849-            "                                    the number of addresses.\n"
    850+                     RPCResult::Type::STR, "txid", "The transaction id for the send. Only 1 transaction is created regardless of\n"
    


    paymog commented at 10:21 am on December 28, 2019:

    Can this be of type HEX_STR?

    I’m still new to Bitcoin, my current understanding is that transactions are represented in hex.


    MarcoFalke commented at 2:53 pm on December 29, 2019:
    Thanks. Fixed in 6817c3af8ca13153bf5f85be26eb1db3845b8cac
  8. in src/wallet/rpcwallet.cpp:974 in 9ebdc9848d outdated
    968@@ -967,10 +969,11 @@ static UniValue addmultisigaddress(const JSONRPCRequest& request)
    969                     {"address_type", RPCArg::Type::STR, /* default */ "set by -addresstype", "The address type to use. Options are \"legacy\", \"p2sh-segwit\", and \"bech32\"."},
    970                 },
    971                 RPCResult{
    972-            "{\n"
    973-            "  \"address\":\"multisigaddress\",    (string) The value of the new multisig address.\n"
    974-            "  \"redeemScript\":\"script\"         (string) The string value of the hex-encoded redemption script.\n"
    975-            "}\n"
    976+                    RPCResult::Type::OBJ, "", "",
    


    paymog commented at 10:23 am on December 28, 2019:
    It might be worthwhile to add a constructor where these two empty strings aren’t needed. It’s quite confusing as a reader to see these everywhere.

    MarcoFalke commented at 2:57 pm on December 29, 2019:

    The first one is the name and it is only used when it is needed to name it (i.e. as a key in a dictionary). Not sure if it makes sense to omit it from the constructor when it is not needed. I’ll wait for others to chime in.

    The second string is the description, which should never be empty. Or at least it shouldn’t be made easy to leave empty. I prefer if new code is explicit about not providing a description.

  9. in src/wallet/rpcwallet.cpp:975 in 9ebdc9848d outdated
    974-            "  \"redeemScript\":\"script\"         (string) The string value of the hex-encoded redemption script.\n"
    975-            "}\n"
    976+                    RPCResult::Type::OBJ, "", "",
    977+                    {
    978+                        {RPCResult::Type::STR, "address", "The value of the new multisig address"},
    979+                        {RPCResult::Type::STR, "redeemScript", "The string value of the hex-encoded redemption script"},
    


    paymog commented at 10:23 am on December 28, 2019:
    can this be of type HEX_STR? The comment/description mentions hex-econded

    MarcoFalke commented at 3:00 pm on December 29, 2019:
    Fixed in 2fe4fda6ec
  10. in src/wallet/rpcwallet.cpp:1212 in 9ebdc9848d outdated
    1221+                            {RPCResult::Type::STR_AMOUNT, "amount", "The total amount in " + CURRENCY_UNIT + " received by the address"},
    1222+                            {RPCResult::Type::NUM, "confirmations", "The number of confirmations of the most recent transaction included"},
    1223+                            {RPCResult::Type::STR, "label", "The label of the receiving address. The default label is \"\""},
    1224+                            {RPCResult::Type::ARR, "txids", "",
    1225+                            {
    1226+                                {RPCResult::Type::STR, "txid", "The ids of transactions received with the address"},
    


    paymog commented at 10:26 am on December 28, 2019:
    Note that you use STR_HEX at https://github.com/bitcoin/bitcoin/pull/17809/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR1383. Should STR_HEX be used here too?

    MarcoFalke commented at 3:01 pm on December 29, 2019:
    Thanks. Fixed in 6817c3a
  11. in src/wallet/rpcwallet.cpp:1427 in 9ebdc9848d outdated
    1436+                    RPCResult::Type::ARR, "", "",
    1437+                    {
    1438+                        {RPCResult::Type::OBJ, "", "", Cat(Cat<std::vector<RPCResult>>(
    1439+                        {
    1440+                            {RPCResult::Type::BOOL, "involvesWatchonly", "Only returns true if imported addresses were involved in transaction."},
    1441+                            {RPCResult::Type::STR, "address", "The bitcoin address of the transaction."},
    


    paymog commented at 10:28 am on December 28, 2019:
    should addresses be STR_HEX?

    MarcoFalke commented at 3:02 pm on December 29, 2019:
    Addresses are not hex strings, they are base58-check encoded or bech32 formatted strings
  12. in src/wallet/rpcwallet.cpp:1429 in 9ebdc9848d outdated
    1438+                        {RPCResult::Type::OBJ, "", "", Cat(Cat<std::vector<RPCResult>>(
    1439+                        {
    1440+                            {RPCResult::Type::BOOL, "involvesWatchonly", "Only returns true if imported addresses were involved in transaction."},
    1441+                            {RPCResult::Type::STR, "address", "The bitcoin address of the transaction."},
    1442+                            {RPCResult::Type::STR, "category", "The transaction category.\n"
    1443+                                "\"send\"                  Transactions sent.\n"
    


    paymog commented at 10:28 am on December 28, 2019:
    Not in scope for this MR but should these categories by an enum?

    MarcoFalke commented at 3:03 pm on December 29, 2019:
    Good idea for a follow up
  13. in src/wallet/rpcwallet.cpp:3303 in 9ebdc9848d outdated
    3312+                        {
    3313+                            {RPCResult::Type::OBJ, "", "",
    3314+                            {
    3315+                                {RPCResult::Type::STR_HEX, "txid", "The hash of the referenced, previous transaction"},
    3316+                                {RPCResult::Type::NUM, "vout", "The index of the output to spent and used as input"},
    3317+                                {RPCResult::Type::STR, "scriptSig", "The hex-encoded signature script"},
    


    paymog commented at 10:34 am on December 28, 2019:
    STR_HEX?

    MarcoFalke commented at 4:03 pm on December 29, 2019:
    Thanks, fixed in fe3e2f1575
  14. in src/rpc/blockchain.cpp:384 in 9ebdc9848d outdated
    410+    RPCResult{RPCResult::Type::NUM, "size", "(DEPRECATED) same as vsize. Only returned if bitcoind is started with -deprecatedrpc=size\n"
    411+                                            "size will be completely removed in v0.20."},
    412+    RPCResult{RPCResult::Type::NUM, "weight", "transaction weight as defined in BIP 141."},
    413+    RPCResult{RPCResult::Type::STR_AMOUNT, "fee", "transaction fee in " + CURRENCY_UNIT + " (DEPRECATED)"},
    414+    RPCResult{RPCResult::Type::STR_AMOUNT, "modifiedfee", "transaction fee with fee deltas used for mining priority (DEPRECATED)"},
    415+    RPCResult{RPCResult::Type::NUM, "time", "local time transaction entered pool in seconds since 1 Jan 1970 GMT"},
    


    paymog commented at 10:42 am on December 28, 2019:
    NUM_TIME?

    MarcoFalke commented at 4:03 pm on December 29, 2019:
    Thanks, fixed in fe3e2f1575
  15. in src/rpc/blockchain.cpp:972 in 9ebdc9848d outdated
    978+                        {RPCResult::Type::NUM, "transactions", "The number of transactions with unspent outputs"},
    979+                        {RPCResult::Type::NUM, "txouts", "The number of unspent transaction outputs"},
    980+                        {RPCResult::Type::NUM, "bogosize", "A meaningless metric for UTXO set size"},
    981+                        {RPCResult::Type::STR_HEX, "hash_serialized_2", "The serialized hash"},
    982+                        {RPCResult::Type::NUM, "disk_size", "The estimated size of the chainstate on disk"},
    983+                        {RPCResult::Type::NUM, "total_amount", "The total amount"},
    


    paymog commented at 10:45 am on December 28, 2019:
    NUM_AMOUNT?

    MarcoFalke commented at 4:03 pm on December 29, 2019:
    Thanks, fixed in fe3e2f1575
  16. in src/rpc/blockchain.cpp:1019 in 9ebdc9848d outdated
    1031+                        {RPCResult::Type::NUM, "confirmations", "The number of confirmations"},
    1032+                        {RPCResult::Type::STR_AMOUNT, "value", "The transaction value in " + CURRENCY_UNIT},
    1033+                        {RPCResult::Type::OBJ, "scriptPubKey", "",
    1034+                            {
    1035+                                {RPCResult::Type::STR_HEX, "asm", ""},
    1036+                                {RPCResult::Type::STR_HEX, "hex", ""},
    


    paymog commented at 10:46 am on December 28, 2019:
    Still learning how this community operates, should this MR also aim to update/fix descriptions?

    MarcoFalke commented at 3:11 pm on December 29, 2019:

    I want to make this change as easy to review as possible. So I am for a minimal rendered diff, assuming --ignore-all-space.

    So for example, the diff for this RPC is:

     0$ git diff --ignore-all-space  --word-diff ./gettxout
     1diff --git a/gettxout b/gettxout
     2index 1d186cd..90fa947 100644
     3--- a/gettxout
     4+++ b/gettxout
     5@@ -8,21 +8,21 @@ Arguments:
     63. include_mempool    (boolean, optional, default=true) Whether to include the mempool. Note that an unspent output that is spent in the mempool won't appear.
     7
     8Result:
     9{                         {+(json object)+}
    10  "bestblock" : [-"hash",-]{+"hex",+}    (string) The hash of the block at the tip of the chain
    11  "confirmations" : n,    (numeric) The number of confirmations
    12  "value" : [-x.xxx,-]{+n,+}            (numeric) The transaction value in BTC
    13  "scriptPubKey" : {      (json object)
    14    "asm" : [-"code",-]{+"hex",+}        (string)
    15    "hex" : "hex",        (string)
    16    "reqSigs" : [-n,          (numeric)-]{+"hex",    (string)+} Number of required signatures
    17    "type" : [-"pubkeyhash",-]{+"hex",+}       (string) The type, eg pubkeyhash
    18    "addresses" : [       [-(array of string)-]{+(json array)+} array of bitcoin addresses
    19      [-"address"-]{+"str",+}              (string) bitcoin address
    20      [-,...-]
    21[-     ]-]{+...+}
    22{+    ],+}
    23  },
    24  "coinbase" : [-true|false   (boolean)-]{+"hex",     (string)+} Coinbase or not
    25}
    26
    27Examples:
    
  17. in src/rpc/blockchain.cpp:1206 in 9ebdc9848d outdated
    1240+                                {RPCResult::Type::STR, "type", "one of \"buried\", \"bip9\""},
    1241+                                {RPCResult::Type::OBJ, "bip9", "status of bip9 softforks (only for \"bip9\" type)",
    1242+                                {
    1243+                                    {RPCResult::Type::STR, "status", "one of \"defined\", \"started\", \"locked_in\", \"active\", \"failed\""},
    1244+                                    {RPCResult::Type::NUM, "bit", "the bit (0-28) in the block version field used to signal this softfork"},
    1245+                                    {RPCResult::Type::NUM, "start_time", "the minimum median time past of a block at which the bit gains its meaning"},
    


    paymog commented at 10:47 am on December 28, 2019:
    NUM_TIME?

    MarcoFalke commented at 4:03 pm on December 29, 2019:
    Thanks, fixed in fe3e2f1575
  18. in src/rpc/blockchain.cpp:1706 in 9ebdc9848d outdated
    1739+                        {RPCResult::Type::NUM, "75th_percentile_feerate", "The 75th percentile feerate"},
    1740+                        {RPCResult::Type::NUM, "90th_percentile_feerate", "The 90th percentile feerate"},
    1741+                    }},
    1742+                {RPCResult::Type::NUM, "height", "The height of the block"},
    1743+                {RPCResult::Type::NUM, "ins", "The number of inputs (excluding coinbase)"},
    1744+                {RPCResult::Type::NUM, "maxfee", "Maximum fee in the block"},
    


    paymog commented at 10:48 am on December 28, 2019:
    NUM_AMOUNT?

    MarcoFalke commented at 4:02 pm on December 29, 2019:
    Unfortunately not. This should be fixed in a follow-up pull request
  19. paymog commented at 10:50 am on December 28, 2019: none

    This is a heroic PR. I left some comments but I didn’t finish going through all the code. I’ll take a look again after the comments are responded to - I don’t want to provide bad comments and I’ll calibrate based on how these are received.

    However, concept ACK. This is awesome.

  20. MarcoFalke force-pushed on Jan 2, 2020
  21. jonasschnelli commented at 9:07 pm on January 5, 2020: contributor
    Concept ACK. Great to see progress on the RPC output as well..
  22. DrahtBot added the label Needs rebase on Jan 7, 2020
  23. MarcoFalke force-pushed on Jan 9, 2020
  24. DrahtBot removed the label Needs rebase on Jan 9, 2020
  25. emilengler commented at 7:20 pm on January 24, 2020: contributor
    Concept ACK
  26. MarcoFalke commented at 7:24 pm on January 24, 2020: member
    Please review #18098 first
  27. in src/rpc/util.h:112 in 37fc4d6d30 outdated
    101+ * dictionaries can be nested in json. The top-level outer type is "NONE".
    102+ */
    103+enum class OuterType {
    104+    ARR,
    105+    OBJ,
    106+    NONE, // Only set on first recursion
    


    emilengler commented at 7:24 pm on January 24, 2020:
    nit: You can drop this comma

    MarcoFalke commented at 7:28 pm on January 24, 2020:
    I want to keep it

    Sjors commented at 12:18 pm on February 14, 2020:
    Trailing comma’s prevent diff headaches later on, so we usually keep them.
  28. emilengler commented at 10:57 am on January 25, 2020: contributor

    I just ran #17804 and this PR on:

    0clang version 7.0.1-8 (tags/RELEASE_701/final)
    1Target: x86_64-pc-linux-gnu
    2Thread model: posix
    3InstalledDir: /usr/bin
    

    The base PR worked fine, however this PR gave me more than 20 errors. I post the error log here: https://pastebin.com/G5G6fNMJ

  29. DrahtBot added the label Needs rebase on Jan 29, 2020
  30. laanwj commented at 3:06 pm on January 31, 2020: member
    Concept ACK
  31. jonatack commented at 9:07 am on February 7, 2020: member
    Concept ACK
  32. MarcoFalke force-pushed on Feb 9, 2020
  33. MarcoFalke force-pushed on Feb 9, 2020
  34. MarcoFalke force-pushed on Feb 9, 2020
  35. DrahtBot removed the label Needs rebase on Feb 9, 2020
  36. Sjors commented at 12:02 pm on February 14, 2020: member
    Concept ACK, will review #18098 first.
  37. Sjors approved
  38. Sjors commented at 1:18 pm on February 14, 2020: member

    ACK a6db5efce95262671edba832eb2f8f2c95761b3c

    A handy incantation that prints all RPC docs in a row (except the hidden ones):

    0src/bitcoin-cli help | grep -Eo '^[^ ]+' | grep -v "=" |  while read -r line ; do `echo src/bitcoin-cli help $line`; echo "\n\n\n\n\n\n\n\n\n\n\n\n\n"; done
    

    Put that output in before.txt and after.txt and compare using vimdiff before.txt after.txt -c 'set diffopt+=iwhite' (install via Homebrew on macOS Catalina), and enjoy pretty diff:

    I did not use a toothcomb while looking through that diff; smaller mistakes can be fixed in followups, or appended commits, imo.

  39. MarcoFalke force-pushed on Feb 17, 2020
  40. DrahtBot added the label Needs rebase on Feb 25, 2020
  41. rpc: Move OuterType enum to header
    This is needed so that it can be used by RPCResult
    
    Also,
    * rename NAMED_ARG to NONE for generalization.
    * change RPCArg constructors to initialize the members by moving values
    fa7d0503d3
  42. rpc: Auto-format RPCResult fa6b061fc1
  43. MarcoFalke force-pushed on Feb 25, 2020
  44. MarcoFalke commented at 3:46 pm on February 25, 2020: member
    Rebased due to trivial conflict in the includes. Should be easy to re-ACK
  45. DrahtBot removed the label Needs rebase on Feb 25, 2020
  46. Sjors commented at 7:29 pm on February 25, 2020: member
    Indeed, re-ACK fa6b061fc118995eec41766519a11bc0dd1a901d
  47. laanwj added this to the "Blockers" column in a project

  48. in src/wallet/rpcwallet.cpp:3918 in fa6b061fc1
    3918-            "}\n"
    3919+                    RPCResult::Type::OBJ_DYN, "", "json object with addresses as keys",
    3920+                    {
    3921+                        {RPCResult::Type::OBJ, "address", "json object with information about address",
    3922+                        {
    3923+                            {RPCResult::Type::STR, "purpose", "Purpose of address (\"send\" for sending address, \"receive\" for receiving address)"},
    


    ajtowns commented at 9:59 am on March 2, 2020:
    “(json object) json object with ….” seems a bit redundant. Maybe “(json object) keyed by address” and “(json object) information about the address” would be better?

    MarcoFalke commented at 2:34 pm on March 4, 2020:
    Seems like a good follow-up issue: #18258
  49. in src/wallet/rpcwallet.cpp:2866 in fa6b061fc1
    2877+                            {RPCResult::Type::BOOL, "spendable", "Whether we have the private keys to spend this output"},
    2878+                            {RPCResult::Type::BOOL, "solvable", "Whether we know how to spend this output, ignoring the lack of keys"},
    2879+                            {RPCResult::Type::BOOL, "reused", "(only present if avoid_reuse is set) Whether this output is reused/dirty (sent to an address that was previously spent from)"},
    2880+                            {RPCResult::Type::STR, "desc", "(only when solvable) A descriptor for spending this output"},
    2881+                            {RPCResult::Type::BOOL, "safe", "Whether this output is considered safe to spend. Unconfirmed transactions"
    2882             "                              from outside keys and unconfirmed replacement transactions are considered unsafe\n"
    


    ajtowns commented at 10:08 am on March 2, 2020:
    Not following why the leading spaces here are being kept?

    MarcoFalke commented at 1:54 pm on March 2, 2020:
    To preserve the output of git blame. Our style-guideline is to keep the current style unless the line is changed for other reasons.

    ajtowns commented at 4:09 am on March 6, 2020:
    Hmm, perhaps I was unclear: I meant the leading spaces inside the quoted string

    MarcoFalke commented at 7:43 pm on March 6, 2020:
    Even those are stylistic and should be trimmed at the beginning of a newline

    MarcoFalke commented at 7:45 pm on March 6, 2020:
    Oh wait you are saying in the line above this line I missed a \n character? I think I did. Someone should fix that.

    MarcoFalke commented at 7:38 pm on March 13, 2020:
    Fixed in #18346
  50. ajtowns approved
  51. ajtowns commented at 10:13 am on March 2, 2020: member
    ACK fa6b061fc118995eec41766519a11bc0dd1a901d – skimmed code changes and differences to rpc help output
  52. MarcoFalke merged this on Mar 4, 2020
  53. MarcoFalke closed this on Mar 4, 2020

  54. MarcoFalke deleted the branch on Mar 4, 2020
  55. fanquake removed this from the "Blockers" column in a project

  56. sidhujag referenced this in commit bb71c12bbd on Mar 5, 2020
  57. domob1812 referenced this in commit fea9d3cd52 on Mar 13, 2020
  58. domob1812 referenced this in commit c667dc1b05 on Mar 13, 2020
  59. domob1812 referenced this in commit 080a36336d on Mar 13, 2020
  60. jasonbcox referenced this in commit 36af7ba910 on Oct 26, 2020
  61. jasonbcox referenced this in commit 1197985073 on Oct 27, 2020
  62. sidhujag referenced this in commit 91bda62ece on Nov 10, 2020
  63. ftrader referenced this in commit 3c10dd6534 on Apr 14, 2021
  64. DrahtBot locked this on Feb 15, 2022

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-07-05 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me