[REST] add a rest endpoint for estimatesmartfee, docs, and test #11770

pull joemphilips wants to merge 1 commits into bitcoin:master from joemphilips:rest_fee changing 3 files +80 −0
  1. joemphilips commented at 2:48 PM on November 27, 2017: contributor

    This could be useful if other clients want to use the core's fee estimation logic via REST.

  2. joemphilips force-pushed on Nov 27, 2017
  3. in test/functional/test_framework/rest.py:5 in 5ee1c95b4a outdated
       0 | @@ -0,0 +1,23 @@
       1 | +import http.client
       2 | +
       3 | +
       4 | +#allows simple http get calls
       5 | +def http_get_call(host, port, path, response_object = 0):
    


    MarcoFalke commented at 3:17 PM on November 27, 2017:

    nit: Instead of moving them to a separate file, you could add convenience wrappers to the test_node class? Thus, it would not be required to pass in host and port all the time.


    joemphilips commented at 7:29 PM on November 29, 2017:

    :+1:

  4. joemphilips force-pushed on Nov 27, 2017
  5. in src/rest.cpp:5 in e485e5f864 outdated
       1 | @@ -2,7 +2,6 @@
       2 |  // Copyright (c) 2009-2016 The Bitcoin Core developers
       3 |  // Distributed under the MIT software license, see the accompanying
       4 |  // file COPYING or http://www.opensource.org/licenses/mit-license.php.
       5 | -
    


    promag commented at 3:38 PM on November 27, 2017:

    Unrelated change, please revert.


    joemphilips commented at 7:40 AM on November 28, 2017:

    done.

  6. in src/rest.cpp:286 in e485e5f864 outdated
     282 | @@ -283,6 +283,7 @@ static bool rest_chaininfo(HTTPRequest* req, const std::string& strURIPart)
     283 |      case RF_JSON: {
     284 |          JSONRPCRequest jsonRequest;
     285 |          jsonRequest.params = UniValue(UniValue::VARR);
     286 | +
    


    promag commented at 3:38 PM on November 27, 2017:

    Unrelated change, please revert.


    joemphilips commented at 7:40 AM on November 28, 2017:

    Thanks, done. I will make sure will not to include these kinds of changes from next time.

  7. in src/rest.cpp:597 in e485e5f864 outdated
     596 | +    };
     597 | +
     598 | +
     599 | +    switch (rf) {
     600 | +    case RF_JSON: {
     601 | +
    


    promag commented at 3:41 PM on November 27, 2017:

    Remove empty line.


    joemphilips commented at 8:28 AM on November 28, 2017:

    done

  8. in src/rest.cpp:598 in e485e5f864 outdated
     593 | +    FeeEstimateMode mode;
     594 | +    if (!FeeModeFromString(modestr, mode)){
     595 | +        return RESTERR(req, HTTP_BAD_REQUEST, "<MODE> must be one of <unset|economical|conservative>");
     596 | +    };
     597 | +
     598 | +
    


    promag commented at 3:42 PM on November 27, 2017:

    Remove empty line.


    joemphilips commented at 8:29 AM on November 28, 2017:

    done

  9. in src/rest.cpp:611 in e485e5f864 outdated
     606 | +        params.push_back(conf_target);
     607 | +        params.push_back(mode);
     608 | +        jsonRequest.params = params;
     609 | +
     610 | +        UniValue estimatedfee;
     611 | +        estimatedfee = estimatesmartfee(jsonRequest);
    


    promag commented at 3:49 PM on November 27, 2017:

    Sounds weird to "redirect" to RPC handler (redo the mode parse etc). Maybe factor out what is needed from there?

    Maybe just call ::feeEstimator.estimateSmartFee() here?


    joemphilips commented at 7:02 PM on November 29, 2017:

    :+1:

  10. in doc/REST-interface.md:107 in e485e5f864 outdated
     102 | +`GET /rest/fee/<MODE>/<TARGET>.json`
     103 | +
     104 | +Returns proper fee estimated by bitcoind. `<MODE>` should be one of `<unset|conservative|economical>`.
     105 | +`<TARGET>` is the desired confirmation time (in block height).
     106 | +
     107 | +
    


    promag commented at 3:50 PM on November 27, 2017:

    Remove 2nd empty line.


    joemphilips commented at 7:39 AM on November 28, 2017:

    done.

  11. in src/rest.cpp:587 in e485e5f864 outdated
     582 | +
     583 | +    std::vector<std::string> path;
     584 | +    boost::split(path, param, boost::is_any_of("/"));
     585 | +    path.erase(path.begin());
     586 | +
     587 | +    if (path.size() != 2)
    


    promag commented at 3:50 PM on November 27, 2017:

    Missing {}.


    joemphilips commented at 7:42 AM on November 28, 2017:

    done.

  12. in src/rest.cpp:578 in e485e5f864 outdated
     570 | @@ -570,6 +571,55 @@ static bool rest_getutxos(HTTPRequest* req, const std::string& strURIPart)
     571 |      }
     572 |  }
     573 |  
     574 | +// A bit of a hack - dependency on a function defined in rpc/mining.cpp
     575 | +UniValue estimatesmartfee(const JSONRPCRequest& request);
     576 | +
     577 | +static bool rest_getfee(HTTPRequest* req, const std::string& strURIPart) {
     578 | +    if (!CheckWarmup(req))
    


    promag commented at 3:50 PM on November 27, 2017:

    Missing {}.


    joemphilips commented at 7:41 AM on November 28, 2017:

    done.

  13. in doc/REST-interface.md:104 in e485e5f864 outdated
      97 | @@ -98,6 +98,13 @@ Only supports JSON as output format.
      98 |  Returns transactions in the TX mempool.
      99 |  Only supports JSON as output format.
     100 |  
     101 | +#### Fees
     102 | +`GET /rest/fee/<MODE>/<TARGET>.json`
     103 | +
     104 | +Returns proper fee estimated by bitcoind. `<MODE>` should be one of `<unset|conservative|economical>`.
    


    promag commented at 3:54 PM on November 27, 2017:

    Returns proper fee estimated by bitcoind, this is bitcoind. 😄


    joemphilips commented at 7:38 AM on November 28, 2017:

    Thanks, done.

  14. promag commented at 4:00 PM on November 27, 2017: member

    Light concept ACK. Maybe too much for REST?

    At least the the URI could be improved. This looks weird:

    • /rest/fee/economical/6.json
    • /rest/fee/unset/3.json
  15. joemphilips commented at 6:35 PM on November 27, 2017: contributor

    Thanks for reviewing. I will update according to the review soon.

    Maybe too much for REST?

    Rationale for this PR is that some light client(or other services) wants to use information about block or mempool for estimating fee. This is pretty close to the motivation of getutxo in terms of it places some trust on the full node. core's fee estimation logic is likely to update to a more sophisticated scheme in the near future so many clients may want to use core's logic directly.

    At least the the URI could be improved. This looks weird:

    You are right. Can I use &foo=bar style query parameter here? I didn't because other endpoint didn't. in that case, what about /rest/fee.json?target=<TARGET>&mode=<MODE> and make mode optional? Though I'm not sure how easy this is to implement.

    Otherwise all I can think is enabling to omit <MODE> . and query in the form like /rest/fee/5.json

  16. promag commented at 7:03 PM on November 27, 2017: member

    My vote goes to GET /rest/fee?target=<TARGET>&mode=<MODE>, with query parameters and without extension. IMO all endpoints should return JSON, unless Accept: headers requests other format.

  17. in src/rest.cpp:575 in e485e5f864 outdated
     570 | @@ -570,6 +571,55 @@ static bool rest_getutxos(HTTPRequest* req, const std::string& strURIPart)
     571 |      }
     572 |  }
     573 |  
     574 | +// A bit of a hack - dependency on a function defined in rpc/mining.cpp
     575 | +UniValue estimatesmartfee(const JSONRPCRequest& request);
    


    jonasschnelli commented at 8:02 PM on November 27, 2017:

    I think it's not worth passing through the RPC call estimatesmartfee. Better directly use feeEstimator.estimateFee(nBlocks);.


    joemphilips commented at 7:03 PM on November 29, 2017:

    you mean feeEstimator::estimateSmartFee ?

  18. jonasschnelli commented at 8:05 PM on November 27, 2017: contributor

    Concept ACK. Please no query string. The in this PR proposed URL scheme seems correct and in align with other calls: https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md#query-utxo-set

  19. fanquake added the label RPC/REST/ZMQ on Nov 27, 2017
  20. in test/functional/smartfees.py:126 in e485e5f864 outdated
     119 | @@ -117,7 +120,13 @@ def check_estimates(node, fees_seen, max_invalid, print_estimates = True):
     120 |          if e >= 0:
     121 |              valid_estimate = True
     122 |              if i >= 13:  # for n>=14 estimatesmartfee(n/2) should be at least as high as estimatefee(n)
     123 | -                assert(node.estimatesmartfee((i+1)//2)["feerate"] > float(e) - delta)
     124 | +                rpcfeerate = node.estimatesmartfee((i+1)//2)["feerate"]
     125 | +                assert(rpcfeerate > float(e) - delta)
     126 | +
     127 | +                json_string = http_get_call(url.hostname, url.port, '/rest/fee/' + 'unset/' + "{}".format((i+1)//2) + '.json')
    


    jb55 commented at 12:54 AM on November 28, 2017:

    nit. why not:

    "/rest/fee/unset/{}.json".format((i+1)//2)
    

    joemphilips commented at 7:42 AM on November 28, 2017:

    :+1:

  21. laanwj commented at 11:28 AM on November 28, 2017: member

    Please no query string. The in this PR proposed URL scheme seems correct and in align with other calls:

    Yep, agree. Concept ACK.

  22. joemphilips force-pushed on Nov 29, 2017
  23. joemphilips force-pushed on Nov 30, 2017
  24. joemphilips commented at 1:08 PM on November 30, 2017: contributor

    updated according to review by @promag , @jb55 and @MarcoFalke . Ready for next review.

  25. promag commented at 3:03 PM on February 3, 2018: member

    Needs rebase.

  26. joemphilips force-pushed on Feb 5, 2018
  27. joemphilips commented at 2:30 PM on February 5, 2018: contributor

    rebased.

  28. joemphilips commented at 8:55 AM on February 15, 2018: contributor

    umm... sorry, does "Needs rebase" imply needs for squashing? or just rebasing onto master? I only did rebasing onto master but it needs squashing before merge. (Sorry for silly question. Forgive me, this is my first PR.) @promag

  29. fanquake deleted a comment on Feb 15, 2018
  30. fanquake commented at 10:40 AM on February 15, 2018: member

    @joemphilips You'll need to rebase onto master, as this currently has merge conflicts in test/functional/feature_fee_estimation.py.

    At the same time, you can also squash your changes into a single commit. See Squashing Commits. Use a descriptive commit message i.e "rest: add an endpoint for estimatesmartfee".

  31. joemphilips force-pushed on Feb 18, 2018
  32. joemphilips force-pushed on Feb 18, 2018
  33. joemphilips force-pushed on Feb 18, 2018
  34. joemphilips force-pushed on Feb 18, 2018
  35. joemphilips force-pushed on Feb 18, 2018
  36. joemphilips commented at 5:09 PM on February 18, 2018: contributor

    rebased (and squashed)

  37. in test/functional/test_framework/test_node.py:17 in 5c548c1e5a outdated
      12 | @@ -13,6 +13,8 @@
      13 |  import re
      14 |  import subprocess
      15 |  import time
      16 | +import urllib.parse
      17 | +import http.client
    


    fanquake commented at 12:55 AM on February 19, 2018:

    http.client is already imported above.


    joemphilips commented at 3:08 AM on February 19, 2018:

    Thanks, done.

  38. fanquake deleted a comment on Feb 19, 2018
  39. joemphilips force-pushed on Feb 19, 2018
  40. joemphilips force-pushed on Mar 9, 2018
  41. MarcoFalke added the label Needs rebase on Jun 6, 2018
  42. MarcoFalke removed the label Needs rebase on Nov 8, 2018
  43. DrahtBot added the label Needs rebase on Nov 8, 2018
  44. jgarzik approved
  45. jgarzik commented at 3:15 AM on November 9, 2018: contributor

    concept ACK

  46. joemphilips force-pushed on Nov 9, 2018
  47. MarcoFalke commented at 4:54 PM on November 9, 2018: member

    Still needs rebase.

    Could do something like this:

    git checkout rest_fee
    git fetch bitcoin
    git merge bitcoin/master
    git reset --soft bitcoin/master
    git commit -m '[REST] add a rest endpoint for estimatesmartfee, docs, and test'
    git push origin rest_fee -f
    
  48. joemphilips force-pushed on Nov 9, 2018
  49. joemphilips force-pushed on Nov 9, 2018
  50. joemphilips force-pushed on Nov 9, 2018
  51. joemphilips force-pushed on Nov 9, 2018
  52. joemphilips force-pushed on Nov 9, 2018
  53. joemphilips force-pushed on Nov 9, 2018
  54. joemphilips force-pushed on Nov 9, 2018
  55. DrahtBot removed the label Needs rebase on Nov 9, 2018
  56. joemphilips commented at 6:53 PM on November 9, 2018: contributor

    rebased

  57. DrahtBot commented at 7:42 PM on November 9, 2018: member

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    No conflicts as of last run.

  58. in src/rest.cpp:614 in 5ed237a062 outdated
     609 | +            conservative = true;
     610 | +        }
     611 | +        int64_t conf_target;
     612 | +        try {
     613 | +             conf_target = atoi64(path[1]);
     614 | +        } catch (std::invalid_argument& e) {
    


    practicalswift commented at 10:27 AM on November 10, 2018:

    Remove e since unused :-)

  59. in src/rest.cpp:610 in 5ed237a062 outdated
     605 | +        bool conservative;
     606 | +        if (mode == FeeEstimateMode::ECONOMICAL) {
     607 | +            conservative = false;
     608 | +        } else {
     609 | +            conservative = true;
     610 | +        }
    


    practicalswift commented at 10:29 AM on November 10, 2018:

    Nit: bool conservative = mode != FeeEstimateMode::ECONOMICAL; instead? :-)

  60. in src/rest.cpp:646 in 5ed237a062 outdated
     597 | +        auto modestr = path[0];
     598 | +        std::transform(modestr.cbegin(), modestr.cend(), modestr.begin(), ToUpper);
     599 | +        FeeEstimateMode mode;
     600 | +        if (!FeeModeFromString(modestr, mode)){
     601 | +            return RESTERR(req, HTTP_BAD_REQUEST, "<MODE> must be one of <unset|economical|conservative>");
     602 | +        };
    


    practicalswift commented at 10:29 AM on November 10, 2018:

    Drop ; :-)

  61. in src/rest.cpp:655 in 5ed237a062 outdated
     611 | +        int64_t conf_target;
     612 | +        try {
     613 | +             conf_target = atoi64(path[1]);
     614 | +        } catch (std::invalid_argument& e) {
     615 | +            return RESTERR(req, HTTP_BAD_REQUEST, strprintf("Uneable to parse confirmation target to int"));
     616 | +        };
    


    practicalswift commented at 10:30 AM on November 10, 2018:

    Drop ; :-)

  62. in src/rest.cpp:659 in 5ed237a062 outdated
     615 | +            return RESTERR(req, HTTP_BAD_REQUEST, strprintf("Uneable to parse confirmation target to int"));
     616 | +        };
     617 | +        unsigned int max_target = ::feeEstimator.HighestTargetTracked(FeeEstimateHorizon::LONG_HALFLIFE);
     618 | +        if (conf_target < 1 || (unsigned int)conf_target > max_target) {
     619 | +            return RESTERR(req, HTTP_BAD_REQUEST, strprintf("Invalid confirmation target, must be in between %u - %u", 1, max_target));
     620 | +        };
    


    practicalswift commented at 10:30 AM on November 10, 2018:

    Drop ; :-)

  63. in test/functional/interface_rest.py:302 in 5ed237a062 outdated
     298 | @@ -299,5 +299,17 @@ def run_test(self):
     299 |          json_obj = self.test_rest_request("/chaininfo")
     300 |          assert_equal(json_obj['bestblockhash'], bb_hash)
     301 |  
     302 | +        # Prepate for Fee estimation
    


    practicalswift commented at 10:31 AM on November 10, 2018:

    "Prepare" :-)

  64. in src/rest.cpp:615 in 5ed237a062 outdated
     610 | +        }
     611 | +        int64_t conf_target;
     612 | +        try {
     613 | +             conf_target = atoi64(path[1]);
     614 | +        } catch (std::invalid_argument& e) {
     615 | +            return RESTERR(req, HTTP_BAD_REQUEST, strprintf("Uneable to parse confirmation target to int"));
    


    practicalswift commented at 10:31 AM on November 10, 2018:

    "Unable" :-)

  65. practicalswift changes_requested
  66. joemphilips force-pushed on Nov 10, 2018
  67. DrahtBot added the label Needs rebase on Jan 23, 2019
  68. rest: add endpoint for estimatesmartfee
    * write REST interface for getting estimated fee
    * update docs about REST interface for fee estimation
    * add test
    eff1b3e201
  69. joemphilips force-pushed on Jan 23, 2019
  70. DrahtBot removed the label Needs rebase on Jan 23, 2019
  71. practicalswift commented at 4:06 PM on May 7, 2019: contributor

    This PR does not compile when rebased on master. Is it still active or should it be closed? :-)

  72. joemphilips commented at 4:11 PM on May 7, 2019: contributor

    This PR does not compile when rebased on master. Is it still active or should it be closed? :-)

    I will close. sorry for leaving it here for so long.

  73. joemphilips closed this on May 7, 2019

  74. MarcoFalke locked this on Dec 16, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-13 15:15 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me