[WIP] Add a basic python REST API Server Wrapper #23309

1. 
   JeremyRubin commented at 7:03 pm on October 19, 2021: contributor

   Related to #23259, a draft of what a simple external REST API Server would look like. Benefits of this approach would be that we can drop the messy HTTP/Rest code and it should be easier to extend this code for other public information RPCs. Further, using whitelistrpc’d credentials, it improves the security of the REST API to not be able to access any internal state (e.g., a buffer overflow can’t write out memory from our core process).

   I copied the Authproxy code from the test framework.

   The rest_getutxos is not implemented as I don’t think it has valid REST semantics, so we could discuss making it valid rest (using query parameters for multiple args) or hacking it another way.

2. JeremyRubin force-pushed on Oct 19, 2021
3. JeremyRubin force-pushed on Oct 19, 2021
4. Add a basic python REST API Server Wrapper 3d060feddf
5. JeremyRubin force-pushed on Oct 19, 2021
6. DrahtBot added the label Scripts and tools on Oct 19, 2021
7. 
   practicalswift commented at 3:26 pm on October 20, 2021: contributor

   Strong concept ACK on providing a REST API which is built with the intention to be secure.

   The current situation where we don’t want to vouch for the security of our REST API and instead suggest that our users should roll their own sanitisation via an nginx proxy if they want to use it in settings where security might be needed feels really weird for a security critical project. We can do better! :)

8. 
   sipa commented at 4:17 pm on October 20, 2021: member

   @practicalswift That seems entirely orthogonal here. The concern with REST (and also RPC, though RPC has additional exposure) is that it isn’t designed to be DoS resistant, so it’s assumed to only be exposed to clients that are trusted to not DoS attack the node. Putting in a Python wrapper that translates REST to RPC seems like that would only worsen this.
9. TDO1988 approved
10. 
    JeremyRubin commented at 4:39 pm on December 20, 2021: contributor

    @sipa it’s not worse because a python rest server can enforce things like rate limiting or be configured to use a pool of nodes. If attacked directly, the python server’s slowdowns can also function as rate limiting.
11. Make rest server use tornado's async HTTP client 11f14b69bf
12. Add a basic Round Robin load balancer 229844a3df
13. 
    JeremyRubin commented at 8:32 pm on December 26, 2021: contributor

    Made a few mods here:

    • Use Tornado’s Async HTTP Client
    • Add a load balancer if you want to be backed by >1 node (-1 on consistency across calls, but you can mitigate this by having 1 external node connected to a fan out of mirror listener nodes serving RPCs)
14. 
    JeremyRubin commented at 5:52 pm on January 8, 2022: contributor

    @stickies-v you might be interested in helping with this as well :)
15. 
    DrahtBot commented at 3:03 pm on September 23, 2022: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    No conflicts as of last run.

16. achow101 marked this as a draft on Oct 12, 2022
17. JeremyRubin closed this on Dec 16, 2022

    ---

18. bitcoin locked this on Dec 16, 2023

Contributors
JeremyRubin practicalswift sipa TDO1988 DrahtBot

Labels
Scripts and tools