rest: add verbose and mempool_sequence query params for mempool/contents #26207

pull andrewtoth wants to merge 3 commits into bitcoin:master from andrewtoth:rest-verbose-mempool changing 3 files +48 −3
  1. andrewtoth commented at 7:04 pm on September 29, 2022: contributor

    The verbose mempool json response can get very large. This adds an option to return the non-verbose response of just the txids. It is identical to the rpc response so the diff here is minimal. This also adds the mempool_sequence parameter for rpc consistency. Verbose defaults to true to remain backwards compatible.

    It uses query parameters to be compatible with the efforts in #25752.

  2. DrahtBot added the label RPC/REST/ZMQ on Sep 29, 2022
  3. glozow requested review from brunoerg on Sep 30, 2022
  4. glozow requested review from luke-jr on Sep 30, 2022
  5. glozow requested review from stickies-v on Sep 30, 2022
  6. glozow removed review request from luke-jr on Sep 30, 2022
  7. in src/rest.cpp:622 in e462602152 outdated
    618+                return RESTERR(req, HTTP_BAD_REQUEST, "The \"mempool_sequence\" query parameter must be either \"true\" or \"false\".");
    619+            }
    620+            bool verbose = raw_verbose == "true";
    621+            bool mempool_sequence = raw_mempool_sequence == "true";
    622+            if (verbose && mempool_sequence) {
    623+                return RESTERR(req, HTTP_BAD_REQUEST, "Verbose results cannot contain mempool sequence values. (hint: set \"verbose=false\")");
    


    brunoerg commented at 11:40 am on September 30, 2022:
    This error and the previous ones could be covered in the functional test
  8. brunoerg commented at 11:56 am on September 30, 2022: contributor

    Concept ACK

    You should update the doc (REST-interface.md) as well

  9. in src/rest.cpp:611 in e462602152 outdated
    607@@ -608,7 +608,20 @@ static bool rest_mempool(const std::any& context, HTTPRequest* req, const std::s
    608     case RESTResponseFormat::JSON: {
    609         std::string str_json;
    610         if (param == "contents") {
    611-            str_json = MempoolToJSON(*mempool, true).write() + "\n";
    612+            std::string raw_verbose = req->GetQueryParameter("verbose").value_or("true");
    


    brunoerg commented at 1:29 pm on September 30, 2022:

    maybe use a const here?

    0            const std::string raw_verbose = req->GetQueryParameter("verbose").value_or("true");
    
  10. andrewtoth commented at 4:18 pm on October 1, 2022: contributor

    You should update the doc (REST-interface.md) as well @brunoerg would it make sense to update that file after there has been a release with this patch? I think that might confuse users who see those options in the doc but are unable to use them in any release.

  11. andrewtoth force-pushed on Oct 1, 2022
  12. in doc/REST-interface.md:124 in fd4c6dcae7 outdated
    120@@ -121,11 +121,13 @@ Returns various information about the transaction mempool.
    121 Only supports JSON as output format.
    122 Refer to the `getmempoolinfo` RPC help for details.
    123 
    124-`GET /rest/mempool/contents.json`
    125+`GET /rest/mempool/contents.json?verbose=false`
    


    stickies-v commented at 11:50 am on October 3, 2022:
    0`GET /rest/mempool/contents.json?verbose=<true|false>&mempool_sequence=<true|false>`
    
  13. in doc/REST-interface.md:131 in fd4c6dcae7 outdated
    127 Returns the transactions in the mempool.
    128 Only supports JSON as output format.
    129-Refer to the `getrawmempool` RPC help for details.
    130+Refer to the `getrawmempool` RPC help for details. Supports the `verbose` and
    131+`mempool_sequence` arguments as query parameters. Defaults to setting
    132+`verbose=true`.
    


    stickies-v commented at 11:54 am on October 3, 2022:
    We already refer to the getrawmempool RPC, as long as these arguments mirror the RPC I think we can leave out this extra documentation.
  14. in src/rest.cpp:618 in fd4c6dcae7 outdated
    614+                return RESTERR(req, HTTP_BAD_REQUEST, "The \"verbose\" query parameter must be either \"true\" or \"false\".");
    615+            }
    616+            const std::string raw_mempool_sequence = req->GetQueryParameter("mempool_sequence").value_or("false");
    617+            if (raw_mempool_sequence != "true" && raw_mempool_sequence != "false") {
    618+                return RESTERR(req, HTTP_BAD_REQUEST, "The \"mempool_sequence\" query parameter must be either \"true\" or \"false\".");
    619+            }
    


    stickies-v commented at 1:14 pm on October 3, 2022:
    I think the most elegant solution here would be to introduce a HTTPRequest::GetQueryParameter<bool> specialization that converts to and returns a bool, or throws an err if the string can’t be parsed. However, that would require some more fundamental changes to how we deal with errors (which unfortunately we return instead of throw) here. Flagging it here for follow-up, feel free to grab it if you’re keen.

    brunoerg commented at 2:07 pm on October 3, 2022:
    Strong concept ACK on introducing a HTTPRequest::GetQueryParameter<bool>, it would be more elegant than using string (“true”/“false”), but I know it would require a lot of more work and maybe could be done in a follow-up.

    andrewtoth commented at 3:34 am on October 4, 2022:
    I can definitely do that in a follow-up PR after this is merged.
  15. in src/rest.cpp:619 in fd4c6dcae7 outdated
    615+            }
    616+            const std::string raw_mempool_sequence = req->GetQueryParameter("mempool_sequence").value_or("false");
    617+            if (raw_mempool_sequence != "true" && raw_mempool_sequence != "false") {
    618+                return RESTERR(req, HTTP_BAD_REQUEST, "The \"mempool_sequence\" query parameter must be either \"true\" or \"false\".");
    619+            }
    620+            const bool verbose = raw_verbose == "true";
    


    stickies-v commented at 1:15 pm on October 3, 2022:

    {} list initialization is preferred (+ in other places):

    0            const bool verbose{raw_verbose == "true"};
    
  16. stickies-v commented at 1:37 pm on October 3, 2022: contributor

    Concept ACK - bringing it further in line with the getrawmempool RPC seems reasonable.

    would it make sense to update that file after there has been a release with this patch?

    To minimize maintenance I’d suggest labeling this pull with the 25.0 milestone and adding a small disclaimer to the rest-interface.md documentation that these 2 parameters are available only from 25.0, similar to https://github.com/bitcoin/bitcoin/blob/3baa0f5a603cedd66442492346583f2b3d776db7/doc/REST-interface.md?plain=1#L56 That way when this PR doesn’t make it into 25.0 it’ll be easy to track and update the docs.

  17. andrewtoth force-pushed on Oct 4, 2022
  18. andrewtoth commented at 3:35 am on October 4, 2022: contributor

    I’d suggest labeling this pull with the 25.0 milestone @stickies-v I don’t think I have permissions to add labels. Otherwise I have addressed all feedback from yourself and @brunoerg .

  19. in doc/REST-interface.md:130 in c35c7d091b outdated
    127+`GET /rest/mempool/contents.json?verbose=<true|false>&mempool_sequence=<true|false>`
    128 
    129 Returns the transactions in the mempool.
    130 Only supports JSON as output format.
    131-Refer to the `getrawmempool` RPC help for details.
    132+Refer to the `getrawmempool` RPC help for details. Defaults to setting 
    


    stickies-v commented at 10:59 am on October 4, 2022:

    trailing whitespace is failing the linter:

    0Refer to the `getrawmempool` RPC help for details. Defaults to setting
    
  20. in doc/REST-interface.md:131 in c35c7d091b outdated
    128 
    129 Returns the transactions in the mempool.
    130 Only supports JSON as output format.
    131-Refer to the `getrawmempool` RPC help for details.
    132+Refer to the `getrawmempool` RPC help for details. Defaults to setting 
    133+`verbose=true`.
    


    stickies-v commented at 11:00 am on October 4, 2022:
    0`verbose=true` and `mempool_sequence=false`.
    
  21. in doc/REST-interface.md:126 in c35c7d091b outdated
    122@@ -123,11 +123,15 @@ Returns various information about the transaction mempool.
    123 Only supports JSON as output format.
    124 Refer to the `getmempoolinfo` RPC help for details.
    125 
    126-`GET /rest/mempool/contents.json`
    127+`GET /rest/mempool/contents.json?verbose=<true|false>&mempool_sequence=<true|false>`
    


    stickies-v commented at 11:01 am on October 4, 2022:

    Actually, since false is default, perhaps better to set that as the first option?

    0`GET /rest/mempool/contents.json?verbose=<true|false>&mempool_sequence=<false|true>`
    
  22. stickies-v approved
  23. stickies-v commented at 11:20 am on October 4, 2022: contributor

    ACK c35c7d091 (for 25.0)

    I don’t think I have permissions to add labels.

    The maintainers should be able to add the 25.0 label when they see this, otherwise you can always send a message on #bitcoin-core-dev IRC. It’s not super urgent though, just would be a bit easier to track I think.

  24. andrewtoth force-pushed on Oct 4, 2022
  25. in doc/REST-interface.md:131 in b534c541c8 outdated
    128 
    129 Returns the transactions in the mempool.
    130 Only supports JSON as output format.
    131-Refer to the `getrawmempool` RPC help for details.
    132+Refer to the `getrawmempool` RPC help for details. Defaults to setting
    133+`verbose=true` and `mempool_sequence=false.
    


    brunoerg commented at 1:01 pm on October 4, 2022:
    0`verbose=true` and `mempool_sequence=false`.
    

    andrewtoth commented at 1:40 pm on October 4, 2022:
    Ahh sorry should have caught that.
  26. andrewtoth force-pushed on Oct 4, 2022
  27. stickies-v commented at 11:33 am on October 5, 2022: contributor
    Can you please rebase to latest master to unblock the (unrelated) failing Win64 CI test?
  28. rest: add verbose and mempool_sequence query params for mempool/contents a518fff0f2
  29. tests: mempool/contents verbose and mempool_sequence query params tests 52a31dccc9
  30. doc: add mempool/contents rest verbose and mempool_sequence args 1ff5d61dfd
  31. andrewtoth force-pushed on Oct 5, 2022
  32. stickies-v approved
  33. stickies-v commented at 1:47 pm on October 5, 2022: contributor

    re-ACK 1ff5d61

    I verified that the only changes since c35c7d0 are suggested documentation updates to REST-interface.md.

  34. pablomartin4btc approved
  35. pablomartin4btc commented at 6:47 am on March 10, 2023: member

    tested ACK 1ff5d61dfdaf8987e5619162662e4c760af76a43.

    I’ve performed manual tests getting the node started with -rest, comparing results between the rest api call using curl with the different combinations of params and values vs. calling the getrawtransacion rpc command (using bitcoin-cli), which is essentially what the python functional tests do.

    It works as expected and it’s consistent with rpc.

  36. DrahtBot commented at 6:47 am on March 10, 2023: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK stickies-v, pablomartin4btc, achow101
    Concept ACK brunoerg

    If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

  37. achow101 commented at 11:33 pm on March 15, 2023: member
    ACK 1ff5d61dfdaf8987e5619162662e4c760af76a43
  38. achow101 merged this on Mar 15, 2023
  39. achow101 closed this on Mar 15, 2023

  40. sidhujag referenced this in commit 0dea134c10 on Mar 16, 2023
  41. sidhujag referenced this in commit 595099b00b on Mar 16, 2023
  42. bitcoin deleted a comment on Mar 16, 2023
  43. andrewtoth deleted the branch on Jun 6, 2023
  44. bitcoin locked this on Jun 5, 2024

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-30 00:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me