rpc: parse each batch element into its own request #36642

pull fametrano wants to merge 1 commits into bitcoin:master from fametrano:rpc-batch-reset-id changing 2 files +27 −5
  1. fametrano commented at 6:49 AM on October 24, 2026: contributor

    In a JSON-RPC batch, an element that is not an object gets an error reply with the id and version of the element before it.

    ExecuteHTTPRPC parses every element of a batch into the same request object, and parse() throws on a non-object before it sets id and m_json_version. For [{"jsonrpc":"2.0","method":"getblockcount","id":"X"}, 2] the reply has two objects with "id":"X", one with a result and one with an error. If the first element is a notification, the error is dropped. I give each element its own copy of the request, which holds only the HTTP fields, so the error is the one the first element of a batch already gets. The new case in interface_rpc.py fails on master and passes with this change.

  2. DrahtBot added the label RPC/REST/ZMQ on Oct 24, 2026
  3. DrahtBot commented at 6:50 AM on October 24, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK hodlinator, janb84, pinheadmz
    Stale ACK AndreaDiazCorreia

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. AndreaDiazCorreia commented at 2:05 AM on October 26, 2026: none

    ACK 0811092e117f08ac9ad0e608f796cd8ac014abd0

    Reviewed and tested, looks good to me.

    Small nit: the existing m_json_version reset before the jsonrpc check is now redundant and could be removed.

  5. in src/rpc/request.cpp:206 in 0811092e11 outdated
     199 | @@ -200,6 +200,11 @@ std::vector<UniValue> JSONRPCProcessBatchReply(const UniValue& in)
     200 |  
     201 |  void JSONRPCRequest::parse(const UniValue& valRequest)
     202 |  {
     203 | +    // Start from the defaults, so that an error reply to a request that is
     204 | +    // not an object does not carry the id or version of a previous request
     205 | +    id = UniValue::VNULL;
     206 | +    m_json_version = JSONRPCVersion::V1_LEGACY;
    


    pinheadmz commented at 8:01 PM on October 26, 2026:

    remove the copy of this statement on line 221?


    hodlinator commented at 3:25 PM on November 6, 2026:

    While it does make for a more surgical fix, it doesn't feel robust to reset individual fields.

    Would prefer something closer to https://github.com/hodlinator/bitcoin/tree/pr/36642_suggestions where in commit db01f652d2e092440a90f9339e86b804327b0faf a fresh JSONRPCRequest is created for each element in a batch. Maybe it could be simplified a bit (further), curious what others think.


    fametrano commented at 6:59 AM on November 7, 2026:

    Yes, a fresh request per element is sturdier, and I'm glad to take your commits, here or in a follow-up, whichever you and others prefer.

    One condition: your first commit leaves three throws outside any try, so they lose their JSON-RPC error replies. A body of 2 now gets HTTP 500 with a plain unknown error instead of the -32700 "Top-level object parse error". I'd take the commits once the refactor keeps those three replies as they are.


    sedited commented at 7:59 AM on November 7, 2026:

    @fametrano another reminder to write your comments yourself. Both comments left here so far were not nits, but requests to change your code in this pull request. If you find problems with them, just improve them, don't ask for exact copies from reviewers.


    fametrano commented at 5:36 PM on November 7, 2026:

    Pushed 5861dd29a7 with your suggestion: each batch element now gets its own request. request.cpp is back to master.

    this text is written by a human being ;-)


    hodlinator commented at 8:17 AM on November 9, 2026:

    Good catch on the escaping throws in my refactor! Thanks for switching to the more robust approach for the batch requests.

  6. in test/functional/interface_rpc.py:185 in 0811092e11
     180 | +        self.log.info("Testing that an invalid batch element does not reuse the id or version of the previous one...")
     181 | +        invalid_element = {"result": None, "error": {"code": RPC_INVALID_REQUEST, "message": "Invalid Request object"}, "id": None}
     182 | +
     183 | +        response, status = send_json_rpc(self.nodes[0], [{"jsonrpc": "2.0", "method": "getblockcount", "id": "X"}, 2])
     184 | +        assert_equal(status, 200)
     185 | +        assert_equal(response, [{"jsonrpc": "2.0", "result": 0, "id": "X"}, invalid_element])
    


    pinheadmz commented at 8:03 PM on October 26, 2026:

    this is a little confusing to read, here and below. what if you just assert the response to the valid request is correct and the second response exists, and doesn't have the same id?

  7. pinheadmz commented at 8:05 PM on October 26, 2026: member

    concept ACK

    This is a reasonable follow up to #27101 and fixes something we missed from the jsonrpc 2.0 spec that could confuse consumers of the interface expecting unique IDs in batch responses.

  8. fametrano force-pushed on Nov 1, 2026
  9. fametrano commented at 6:41 PM on November 1, 2026: contributor

    Done: I removed the second m_json_version reset and simplified the test's asserts. The batch case now checks the valid reply, then that the error reply has another id and no jsonrpc field, which fails if the version reset stays where it was; the notification case checks that the error reply is there. @AndreaDiazCorreia, the removed reset is the one from your nit; would you mind taking another look at 2ab7c47f60?

  10. in test/functional/interface_rpc.py:182 in 2ab7c47f60
     175 | @@ -176,6 +176,22 @@ def test_batch_requests(self):
     176 |              request_fields={"jsonrpc": "2.1"},
     177 |              response_fields={"result": None, "error": {"code": RPC_INVALID_REQUEST, "message": "JSON-RPC version not supported"}}))
     178 |  
     179 | +    def test_batch_invalid_element_reply(self):
     180 | +        self.log.info("Testing that an invalid batch element does not reuse the id or version of the previous one...")
     181 | +
     182 | +        response, status = send_json_rpc(self.nodes[0], [{"jsonrpc": "2.0", "method": "getblockcount", "id": "X"}, 2])
    


    pinheadmz commented at 7:17 PM on November 2, 2026:

    The 2 here is an invalid request because its not JSON but that's not obvious for review. Especially because there is another 2 a few lines down that is a critical integer value. If I wrote this test I'd probably use something like a literal string "invalid json" there.


    fametrano commented at 10:44 PM on November 3, 2026:

    Done in 325e600c02: both batches now send "not an object" instead of 2, since the element parses as JSON and is rejected only for not being an object.

  11. fametrano force-pushed on Nov 3, 2026
  12. pinheadmz approved
  13. pinheadmz commented at 2:48 PM on November 4, 2026: member

    ACK 325e600c02b0f2dcb28848d66e53e6e053f0b499

    <details><summary>Show Signature</summary>

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    ACK 325e600c02b0f2dcb28848d66e53e6e053f0b499
    -----BEGIN PGP SIGNATURE-----
    
    iQJPBAEBCAA5FiEE5hdzzW4BBA4vG9eM5+KYS2KJyToFAmrrRaIbFIAAAAAABAAO
    bWFudTIsMi41KzEuMTIsMCwzAAoJEOfimEtiick6IioP/jEZHsTmwDSmS3GvMOYw
    qNPF7kCAXObPLr2JP7J3Ewx0aN09itYXoYYUkmp94NSbzRNdwWC7e2Ak5wKYsVud
    AflT+cqT2Xoo0G8fZJBKcPAgqM+4u+DavuHpLuMfbPVoSYnt0rB5pTvZMuutLxYx
    edBFiHkTjPrtGy4SyESz+QBFlxf6c4kt++B3KXNwsyZwpgd3gNnjPrqcGIXfECBE
    rGuIEhFj/2L9T+I7oeMW/+XtgZKNLP+PA3AEhK2StXaK0mzc2Y8K7kvpt7GE30TV
    XWBWF7oyQLeBL+qs22bDy+m/acUcM5GWz00dzjCF+P6oG/N85hJAVq3Qp+skGJ45
    YxjKC0/BY3s1em0CYjLhboqozMjnEdIu+Rw6awbgXyaSLus1Nq5far9n0mlDMErD
    umYgaY0sJZ0lfGwyB7twnTS8tlI09f/8HMWjG8EJJeeaHsguY0VWYt/QxSxb82pq
    OfL5vK/0xf5KVnoyYrQKCix+9ToOb2yMaD8GEu/4HdPlnzjGBF+81kwbU4F0GGq5
    cc6RKufK9ejMZalmdbd+p2ccE1QseD7vMmu6Sy3n9V58O1xi6JuGJ5U107ylLiga
    JZkKgAHzj4XL0tHlWuYf9M8L4OwxiQPPSyKZhnlyE+Tjyj48X5xebxWkTCfv+hCu
    u6rmS9Kx4djQ6WtyCgyj/HOV
    =xEBx
    -----END PGP SIGNATURE-----
    

    pinheadmz's public key is on openpgp.org

    </details>

  14. fanquake requested review from hodlinator on Nov 4, 2026
  15. in test/functional/interface_rpc.py:193 in 325e600c02
     188 | +
     189 | +        # The error is not dropped as if it were the reply to the notification before it
     190 | +        response, status = send_json_rpc(self.nodes[0], [{"jsonrpc": "2.0", "method": "getblockcount"}, "not an object"])
     191 | +        assert_equal(status, 200)
     192 | +        assert_equal(len(response), 1)
     193 | +        assert_equal(response[0]["error"]["code"], RPC_INVALID_REQUEST)
    


    janb84 commented at 9:23 AM on November 5, 2026:

    Suggestion to rewrite these tests to remove all the separate asserts (and remove the new include for assert_not_equal) and condense them into two, one line asserts:

            response, status = send_json_rpc(self.nodes[0], [{"jsonrpc": "2.0", "method": "getblockcount", "id": "X"}, "not an object"])
            assert_equal(status, 200)
            assert_equal(response, [
                {"jsonrpc": "2.0", "result": 0, "id": "X"},
                {"result": None, "error": {"code": RPC_INVALID_REQUEST, "message": "Invalid Request object"}, "id": None},
            ])
    
            # The error is not dropped as if it were the reply to the notification before it
            response, status = send_json_rpc(self.nodes[0], [{"jsonrpc": "2.0", "method": "getblockcount"}, "not an object"])
            assert_equal(status, 200)
            assert_equal(response, [
                {"result": None, "error": {"code": RPC_INVALID_REQUEST, "message": "Invalid Request object"}, "id": None},
            ])
    

    fametrano commented at 10:38 PM on November 5, 2026:

    Thanks, I'll take this if I need to push again.


    fametrano commented at 5:36 PM on November 7, 2026:

    Done in 5861dd29a7: each batch is now checked with one assert_equal on the whole response.

  16. janb84 commented at 9:27 AM on November 5, 2026: contributor

    Concept ACK,

    Have validated that the supplied patch fixes the bug and removing it will fail the supplied tests.

    Have added a small NIT to condense the functional tests a bit.

  17. DrahtBot requested review from AndreaDiazCorreia on Nov 5, 2026
  18. hodlinator commented at 3:31 PM on November 6, 2026: contributor

    Concept ACK 325e600c02b0f2dcb28848d66e53e6e053f0b499

    Thanks for finding and working on a fix for this!

  19. rpc: parse each batch element into its own request
    In a JSON-RPC batch, an element that is not an object gets an error
    reply with the id and version of the element before it. A client that
    matches replies by id can take that error for the result of the
    previous call. If the previous element was a 2.0 notification, the
    error is dropped.
    
    Every element was parsed into the same JSONRPCRequest. I give each
    element its own copy of the request before parsing, which holds only
    the HTTP fields at that point. The functional test covers the two
    cases.
    5861dd29a7
  20. fametrano force-pushed on Nov 7, 2026
  21. fametrano renamed this:
    rpc: reset id and version per batch element
    rpc: parse each batch element into its own request
    on Nov 7, 2026
  22. in test/functional/interface_rpc.py:186 in 5861dd29a7
     181 | +
     182 | +        response, status = send_json_rpc(self.nodes[0], [{"jsonrpc": "2.0", "method": "getblockcount", "id": "X"}, "not an object"])
     183 | +        assert_equal(status, 200)
     184 | +        assert_equal(response, [
     185 | +            {"jsonrpc": "2.0", "result": 0, "id": "X"},
     186 | +            {"result": None, "error": {"code": RPC_INVALID_REQUEST, "message": "Invalid Request object"}, "id": None},
    


    hodlinator commented at 8:30 AM on November 9, 2026:

    Tangential remark: It's seems unfortunate that our RPC implementation inserts result in the error case here, https://www.jsonrpc.org/specification#response_object states:

    result ... This member MUST NOT exist if there was an error invoking the method.

    Guess it's expected behavior since we don't yet default to JSON RPC 2.0, but wonder when we should make the switch. Also annoying when the batch request contains proper 2.0 elements.

  23. hodlinator approved
  24. hodlinator commented at 8:36 AM on November 9, 2026: contributor

    ACK 5861dd29a73a9f0a21851a83adfa21d4637282b5

  25. DrahtBot requested review from janb84 on Nov 9, 2026
  26. DrahtBot requested review from pinheadmz on Nov 9, 2026
  27. janb84 commented at 10:17 AM on November 9, 2026: contributor

    ACK 5861dd29a73a9f0a21851a83adfa21d4637282b5

    Thanks for taking my suggestions.

    EDIT: after further contemplation I think the change should be a followup:

    I'm conflicted on one issue if this PR is complete or not, the behavior is different if bitcoind is started with --rpcwhitelist or not. Normally I would say the PR is incomplete, whitelist or not the response should be the same, but given this is already different on master I'm hesitant.

    Batch Without a whitelist With a whitelist
    Only valid elements 200, array of replies 200, array of replies
    Contains "not an object" 200, array, error only for that element 400, one error object, nothing runs
    Contains an object without method 200, array, "Missing method" for that element 500, one error object, nothing runs

    This can also be (partially) correct, if something from the batch cannot be checked by the whitelist -> fail everything. Given the batch spec I would say we have to treat every call on it's own and failing everything for one failed call is not the correct response.

    The 500 is in any case not correct

  28. pinheadmz commented at 8:02 PM on November 9, 2026: member

    @janb84 see #36753 for the rpcwhitelist fix 🤣 There's now three jsonrpc PRs circling around each other

  29. pinheadmz approved
  30. pinheadmz commented at 8:10 PM on November 9, 2026: member

    ACK 5861dd29a73a9f0a21851a83adfa21d4637282b5

    built and tested on macos/arm64. the patch is very lean, correct and the best approach (thanks @hodlinator).

    <details><summary>Show Signature</summary>

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    ACK 5861dd29a73a9f0a21851a83adfa21d4637282b5
    -----BEGIN PGP SIGNATURE-----
    
    iQJPBAEBCAA5FiEE5hdzzW4BBA4vG9eM5+KYS2KJyToFAmryKDgbFIAAAAAABAAO
    bWFudTIsMi41KzEuMTIsMCwzAAoJEOfimEtiick6hjkP/0LuARbf5pjU8wGQmtW5
    gFDP5qB+8p90+TcqHY/6IbeCdnyYB+XFtWfgCQPH0QwZQVI1kU1VSByE24cDWXTi
    rJbu0OXGPcdy7ha0fRwm4OJw1yFxXR1slgEPYmnLGIUkvXKcUIuw/qNigS/qGYF5
    2jtufqXEiTvQdz0f9I/1Hx4LTv7YR1lmH1z03kVsZv9PRjsDj4Gbw6DUswiYOmef
    WzEHZ9nBjlMLoEvdH3//YaVhnPSsoNj8N+vgu7Cp/7ik36RGmhZ/ANRa/h6uW3sV
    bKAeo2iHZthgj1q07fpcKDCaKRJrsacr7ALQvUa4eOOxfm4oP5q9Zr9LTn2AuMAL
    F2HtGHSulD0v6lX5toGjmiAPKLGPxGo2pepkiZSnCTpaXyYua8Dq7f+BDhJJm3cO
    y08eSmNXFlN0EntsCc/Y7T0RSRB7mmA37IBgpcWB+FXIePO0O3kTWzjQSzj0xF7g
    GNm/3+7acdKHdGwo/ph9p2hPV77bZAd/scMbuuDM9r1Gfemulw3Je3OMSzedUMwg
    zdJPweD+/l1UaXigUEMnf7aWEW6LHNDk5r4IEXx0PuTJLg07JcNOOziq7SpGOuoZ
    exg8ij68GTIedIn/8TrqX8txqDnQZvqGF5/McWtaJqA62f96Ji5T31eo1ZRoBzrA
    l3lSEdu/xBQbyB6YKhctsmwt
    =KFjP
    -----END PGP SIGNATURE-----
    

    pinheadmz's public key is on openpgp.org

    </details>

  31. ryanofsky assigned ryanofsky on Nov 10, 2026
  32. ryanofsky commented at 2:24 PM on November 10, 2026: contributor

    This makes sense to merge before the related PRs #36753 and #36758. It makes them easier to understand and fixes a bug where when a batch contains an element that is not a JSON object, the error reply for that element gets the id of the element before it. A client matching replies by id then sees two replies for one call, or, if the element before was a notification, the error is dropped.

    Things for followups:

    1. A JSON-RPC 2.0 batch element without an id that fails to parse, like {"jsonrpc": "2.0", "method": 1}, is treated as a notification and gets no reply. The spec's examples reply to that request with an "Invalid Request" error and "id": null. #36758 currently fixes this for single requests only, and pinheadmz plans to fix batches too after rebasing on this PR, so both follow the same rule (discussion).

    2. Batch replies should be the same with and without -rpcwhitelist, as janb84 pointed out. With a whitelist, a non-object element currently fails the whole batch with HTTP 400, and an element without a method fails it with HTTP 500. #36753 fixes this.

    3. JSONRPCRequest holds both fields describing the client connection (URI, authUser, peerAddr, context) and fields parsed from each request. Splitting these into two structs would let each batch element start from a fresh request, as hodlinator suggested, without copying anything, and would make it easy to share the per-element parse and reply code between batch loops. I think I will also do this in #36753.

    4. Error replies in the legacy format include "result": null, which JSON-RPC 2.0 doesn't allow, and an element that isn't an object can only be answered in the legacy format. hodlinator asked when 2.0 should become the default. It seems like we should have a comment somewhere explaining current choice and any future plans to change

  33. ryanofsky merged this on Nov 10, 2026
  34. ryanofsky closed this on Nov 10, 2026


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-11-20 15:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me