Multithreaded JSON-RPC with HTTP 1.1 Keep-Alive support #1101

pull jgarzik wants to merge 2 commits into bitcoin:master from jgarzik:http11 changing 4 files +103 −40
  1. jgarzik commented at 1:05 am on April 15, 2012: contributor

    This is a cleaned up version of JoelKatz’ work, originally found in pull request #568.

    A few minor didnt-check-error-return bugs were fixed. All other changes were comment- or coding style-related changes unrelated to behavior.

    Should be upstream-ready at this point. Is in use at a few pools and other places.

  2. luke-jr commented at 1:09 am on April 15, 2012: member
    Note this is missing some important thread-safety issues I had cleaned up in #568 (which has been upstream-ready for months)
  3. jgarzik commented at 2:40 am on April 15, 2012: contributor
    Nothing is -missing-. There is a lock across the entirety of the RPC command execution.
  4. luke-jr commented at 4:09 am on April 15, 2012: member
    Ah, okay… that’s what I missed when comparing the two.
  5. gavinandresen commented at 7:37 pm on April 15, 2012: contributor
    Any limit to the number of threads spawned? Could somebody out-of-memory DoS if they just keep connecting to the RPC port a gazillion times and never closing the connections?
  6. luke-jr commented at 7:45 pm on April 15, 2012: member
    I suspect you’d hit PID limits before memory, and socket limits before that. And since bitcoind uses select(), it would randomly corrupt memory. But that’s already a problem anyway… :/
  7. jgarzik commented at 9:10 pm on April 15, 2012: contributor

    There is no limit to the threads spawned – but note that threads are spawned only after checking the IP filter list.

    The first resource likely to be exhausted is a thread-group or systemwide thread limit, not memory. But yes, if you pass the IP filter list, that can be DoS’d.

    Should not be a problem to add a simple simultaneous-threads limit here, though.

  8. gavinandresen commented at 11:46 pm on April 16, 2012: contributor
    ACK. I assume the main benefit right now is the keepalive to save constant connection setup/teardown, since RPC calls will still be essentially single-threaded due to obtaining the cs_main and wallet locks. And the secondary benefit is we can eventually move to more fine-grained locks…
  9. gavinandresen commented at 11:43 pm on April 21, 2012: contributor
    Anybody else getting a compiler warning: src/bitcoinrpc.cpp:2618: warning: suggest a space before ‘;’ or explicit braces around empty body in ‘while’ statement [-Wempty-body]
  10. jgarzik commented at 5:06 am on April 22, 2012: contributor
    Rebased. @gavinandresen do you still get a warning? If yes, can you paste the code line as well as the warning, just for double-checking?
  11. gavinandresen commented at 6:44 pm on April 23, 2012: contributor

    Still getting an error:

    bitcoinrpc.cpp: In function ‘void ThreadRPCServer3(void*)’: bitcoinrpc.cpp:2610: warning: suggest a space before ‘;’ or explicit braces around empty body in ‘while’ statement

    Line 2610 is: while (0);

    … which looks to me like it should be deleted, since loop is defined as: for(;;) in util.h

  12. jgarzik commented at 5:10 am on April 24, 2012: contributor
    @gavinandresen fixed. Merge error added some useless code.
  13. gavinandresen commented at 2:43 pm on April 24, 2012: contributor
    Testing this, I got a crash on shutdown of Bitcoin-Qt; see https://gist.github.com/2480177
  14. Support multi-threaded JSON-RPC
    Change internal HTTP JSON-RPC server from single-threaded to
    thread-per-connection model.  The IP filter list is applied prior to starting
    the thread, which then processes the RPC.
    
    A mutex covers the entire RPC operation, because not all RPC operations are
    thread-safe.
    
    [minor modifications by jgarzik, to make change upstream-ready]
    e9205293bd
  15. RPC: Support HTTP/1.0 and HTTP/1.1, including the proper use of keep-alives 96c5269511
  16. in src/bitcoinrpc.cpp: in 8f4f8f28b5 outdated
    2599             }
    2600         }
    2601         catch (Object& objError)
    2602         {
    2603-            ErrorReply(stream, objError, id);
    2604+            ErrorReply(conn->stream, objError, id);
    


    luke-jr commented at 4:46 pm on May 1, 2012:
    Shouldn’t this be setting fRun to false also? (#1075 removes the first try block entirely, letting this handle errors)
  17. in src/bitcoinrpc.cpp: in 8f4f8f28b5 outdated
    2605+            break;
    2606         }
    2607         catch (std::exception& e)
    2608         {
    2609-            ErrorReply(stream, JSONRPCError(-32700, e.what()), id);
    2610+            ErrorReply(conn->stream, JSONRPCError(-32700, e.what()), id);
    


    luke-jr commented at 4:47 pm on May 1, 2012:
    Same fRun thing here too… someone should probably note whether -32700 vs -1 is the correct response?
  18. jgarzik commented at 0:29 am on May 9, 2012: contributor
    Rebased, and updated for recent table-driven RPC dispatch. That change simplified our error handling, here.
  19. jgarzik referenced this in commit b34c5f3c0f on May 11, 2012
  20. jgarzik merged this on May 11, 2012
  21. jgarzik closed this on May 11, 2012

  22. coblee referenced this in commit 75d9717e4b on Jul 17, 2012
  23. lateminer referenced this in commit ed708a6f67 on Jan 22, 2019
  24. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-11-17 21:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me