RFC: Support a format description of our JSON-RPC interface #34683

pull willcl-ark wants to merge 8 commits into bitcoin:master from willcl-ark:json-rpc-schema changing 13 files +466 −1
  1. willcl-ark commented at 3:22 pm on February 26, 2026: member

    Ref #29912.

    This draft PR adds a machine-readable OpenRPC 1.3.2 specification of out JSON-RPC interface, auto-generated from existing RPCHelpMan metadata.

    There is currently no formal, machine-readable specification of the RPC API. As discussed in #29912, this has knock-on consequences:

    • Client libraries re-implement the API manually, leading to bugs like unit mistakes (sats vs BTC, vB vs kvB) and missing/incorrect argument types. No existing client library fully and correctly implements the API in a type-safe manner.
    • When the API changes, every downstream client must manually discover and adapt, creating downstream maintenance burden. There is no artifact they can diff between releases.
    • Implementing a new client in a new language requires reading C++ source or help text and transcribing it, which is error-prone and tedious, and represents an on-going porting cost.
    • Existing documentation is either stale or not machine-readable. The developer.bitcoin.org docs are wrong/outdated in places, and the bitcoincore.org/en/doc/ pages are rendered from help output but not in a standard schema format.
    • (new/extra) AI/LLM tooling increasingly builds on structured API specifications. A standard spec format enables AI-assisted client generation and integration without the ambiguity of parsing human-readable help text.

    This draft builds on prior art by @casey and the observations by @laanwj, @stickies-v, @kilianmh, @hodlinator, and @cdecker in #29912. Casey’s work demonstrated that RPCHelpMan already contains all the structured information needed, which makes this feasible without duplicating any API definitions.

    This differs from Casey’s branches in that it uses the OpenRPC standard rather than an ad-hoc format or raw JSON Schema.

    Why OpenRPC

    I seletced OpenRPC for a number of reasons:

    • It’s purpose-built for JSON-RPC APIs (suggested by @stickies-v, @nflatrea, and @kilianmh).
    • It wraps JSON schema for params/results, so consumers get both the method-level structure and the type-level schemas.
    • Unlike OpenAPI, it is not path-centric, which better fits our single-endpoint JSON-RPC model (concern raised by @hodlinator).
      • Although it therefore does not cover our REST interface.
    • It’s kind of a standard format with (some) existing tooling for type generation (TypeScript, Rust, Python, Go) and client scaffolding, though maturity varies by language. More importantly though, IMO, a ~standardised format is inherently more useful than any ad-hoc one: any JSON Schema validator works, any LLM can consume it directly, and anyone can write a bespoke generator against a known schema rather than parsing help text.

    Approach

    RPCHelpMan metadata → getopenrpcinfo RPC → OpenRPC JSON

    Tradeoffs

    vs an ad-hoc format OpenRPC gives us interoperability with the (admittedly surprisingly limited) tooling, documentation generators, code generators, and validators, at the cost of needing x-bitcoin-* extensions for Bitcoin-specific concepts. As Casey noted after trying both approaches, JSON Schema “is probably not a great fit”. OpenRPC’s method-level framing on top of JSON Schema addresses the ergonomic issues while keeping the schema benefits. After testing both, I think I agree.

    Types: JSON Schema cannot natively express units like “BTC/kvB” or semantic distinctions like “this hex string is a txid.” These are preserved in description text and x-bitcoin-type-str extensions, but are not machine-enforceable from the schema alone. This was recognized as a fundamental limitation by several commenters in the issue. Future work could enrich the x-bitcoin-* extensions with a more structured unit vocabulary.

    Some RPCs return different types depending on argument values (e.g. verbosity levels). These are represented as oneOf in the result schema with free-text condition descriptions. This is accurate but not fully machine-parseable — a code generator cannot automatically determine which result variant corresponds to which argument value without parsing the description. I still we have enough information to satisfy humans an agents alike though.

    Regenerating the spec

    The current functional test simply tests that the RPC runs, and produces valid JSON. We might want to consider extending this..

    The RPC output documents which RPCs are available for any given built binary.

    Discussion questions

    • Is this valuable/wanted?
    • Do we like openrpc format? (less relevant if we don’t want this in this repo, as another repo could generate one or many definitions).
    • Should we cover “hidden” RPCs? They are currently hidden, but don’t have to be…

    My personal thoughts are that this is very nice to have.

  2. DrahtBot commented at 3:23 pm on February 26, 2026: contributor

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Concept ACK sedited, janb84, nervana21, stickies-v, hodlinator, w0xlt

    If your review is incorrectly listed, please copy-paste <!–meta-tag:bot-skip–> into the comment that the bot should ignore.

    Conflicts

    No conflicts as of last run.

  3. sedited commented at 3:30 pm on February 26, 2026: contributor
    Concept ACK
  4. janb84 commented at 3:56 pm on February 26, 2026: contributor
    Concept ACK. This was already useful for bitcoin-tui
  5. in test/functional/rpc_openrpc.py:36 in 2846fab06d
    31+
    32+        openrpc_path = srcdir / "doc" / "openrpc.json"
    33+        committed = json.loads(openrpc_path.read_text(encoding="utf-8"))
    34+
    35+        if generated != committed:
    36+            self.log.info("Generated doc/openrpc.json:\n%s",
    


    maflcko commented at 4:48 pm on February 26, 2026:

    I guess it could be distracting to have to see every rpc change twice. Maybe the test could just check that generating the json does not crash?

    In theory, there could be a diff output like #26706 (comment) somewhere for each pull request, but that seems better to be optional and hidden.

    the doc/openrpc.json could have the same This is a placeholder file. content like the manpages.

    (Of course it is fine to keep both commits in this pull for now, so that the json is visible here, and only drop them before merge)


    willcl-ark commented at 11:47 am on February 27, 2026:

    Thank you for this comment.

    Having the json checked in as a skeleton (like manpages) is a nice approach. It’s kind of the opposite to stickies thoughts, so I guess I’ll see which folks seem to generally prefer (and will leave the file in this PR for now for reference).

    I guess it could be distracting to have to see every rpc change twice. Maybe the test could just check that generating the json does not crash?

    This is also a nice idea/possibility. If we are not worrying about having a live artifact in this repo, then I agree this is almost certainly how it should work.


    maflcko commented at 12:17 pm on February 27, 2026:

    I am also thinking about devs on platforms that do not support a full build (IIRC external signer was not present on Windows for some time?). I guess they could manually undo the hunk that removes the $feature-related RPC from the json, or copy-paste the full json output from the CI log somehow.

    In any case, either way seems fine and should be trivial to adjust post-merge. This is just a nit.

  6. nervana21 commented at 11:56 pm on February 26, 2026: contributor
    Concept ACK
  7. stickies-v commented at 4:18 am on February 27, 2026: contributor

    Big concept ACK, very nice work. If we want to cram less functionality in Bitcoin Core, we need to make it easier for people to build on top of it. A change like this will make it much easier for consumers to use the RPC. In addition, I think it will also help with our RPC stability guarantees and testing, and to make it easier for consumers to quickly find and incorporate any changes that have been made to the interface.

    it could also cause more work for devs who change an rpc, as the workflow may involve: …

    This seems like a feature, not a bug? As long as devs can compile the binaries they need to test/dev, I think this doesn’t impose any unnecessary overhead?

    If so, should it live in this repo?

    If OpenRPC is the best choice for our repo (from quick glance, it seems to work well), and we have enough support to commit to keeping our RPC OpenRPC compatible, I think it absolutely should live in this repo, so that we can quickly and strictly enforce continued compatibility with the spec.

    I’m less opinionated on whether the output json should live in this repo. I think it’s fine to do so - the changes should be a LOT less frequent than e.g. doxygen changes, and it’s nice having it easily accessible. However, I think it’s also acceptable for users to generate this on-the-fly, or use any other hosted mirror of this documentation, which should be trivial to do.

  8. ajtowns commented at 12:15 pm on February 27, 2026: contributor

    → Python gen-openrpc.py → doc/openrpc.json (the artifact that gets committed)

    I don’t really see why it wouldn’t be better to have a getopenrpcinfo command that returns the json directly, giving you a direct report of the interface your bitcoind supports (whether the wallet or zmq is compiled in or not). Is there any reason you need python code to do the conversion rather than having it be built in to the bitcoind binary?

    If people want to grab the artifact from the web instead of their node, then providing a url on bitcoincore.org (like https://bitcoincore.org/en/doc/30.0.0/rpc/network/disconnectnode/ eg) seems plausible enough.

    Having the json output in the repo just seems annoying to me; more data, more conflicts, more diffs to check, more things you have to manually mess around with that could be automated.

    For the test suite, picking a rarely updated but somewhat interesting RPC, and checking that the “getopenrpcinfo” result for that command matches a fixed value, and that the commands listed match the output of “help” is probably fine, or at least seems like it would be to me.

    OpenRPC seems fine to me; doesn’t seem like there’s much benefit in worrying about the details, beyond getting something that’s moderately machine readable.

  9. willcl-ark commented at 1:32 pm on February 27, 2026: member

    I don’t really see why it wouldn’t be better to have a getopenrpcinfo command that returns the json directly, giving you a direct report of the interface your bitcoind supports (whether the wallet or zmq is compiled in or not). Is there any reason you need python code to do the conversion rather than having it be built in to the bitcoind binary?

    Good question, sorry I didn’t answer in OP initially, I had meant to provide a comparison of sorts…

    I tried this approach, and basically my reasoning was that you end up with a much larger c++ change (which I thought might be less desirable), although you get the nice benefits you note.

    I think it could be useful to codify the possible approaches I see, and their impact on this codebase. So roughly in ascending order of LoC needed in this repo:

    1. a single dump_all_command_descriptions RPC command (and basic functional test), ~ the first commit here. Doesn’t emit json. up to downstream to write their own transformer.
    2. The above + a transformer contrib/ script + skeleton openrpc.json file
    3. RPC to return fully-structured JSON (This needed a fair amount of cpp code, in my attempt)
    4. Something like this PR being proposed (including a living openrpc.json file)

    I am going to rework my branch which outputs the json directly from RPC, and will post an update here with a comparison.

    If people want to grab the artifact from the web instead of their node, then providing a url on bitcoincore.org (like bitcoincore.org/en/doc/30.0.0/rpc/network/disconnectnode eg) seems plausible enough.

    agree, this would be fine.

    Having the json output in the repo just seems annoying to me; more data, more conflicts, more diffs to check, more things you have to manually mess around with that could be automated.

    Yeah I think I agree here too. I did test breaking an RPC to see how annoying it was to detect and fix. Quite annoying, was the answer.

    OpenRPC seems fine to me; doesn’t seem like there’s much benefit in worrying about the details, beyond getting something that’s moderately machine readable.

    👍🏼 Thank you for articulating this, as I did spend a fair bit of time “worrying” about which format to select, but I see now you’re correct; as long as it’s structured (i.e valid json) and machine readable, it should be OK, and people can easily transform it into their own formats.

  10. natobritto commented at 10:08 pm on February 27, 2026: none
    Hey @willcl-ark, have you checked my comment on corepc? Is something like that you wanted to build?
  11. willcl-ark force-pushed on Mar 2, 2026
  12. DrahtBot added the label CI failed on Mar 2, 2026
  13. DrahtBot commented at 12:09 pm on March 2, 2026: contributor

    🚧 At least one of the CI tasks failed. Task Windows native, fuzz, VS: https://github.com/bitcoin/bitcoin/actions/runs/22572396443/job/65383261925 LLM reason (✨ experimental): Fuzz test failed because RPC command “getopenrpcinfo” is not listed in allowed RPC commands for fuzzing.

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  14. willcl-ark force-pushed on Mar 2, 2026
  15. willcl-ark force-pushed on Mar 2, 2026
  16. rpc: add missing string_view include to server.h
    CRPCTable::help() takes std::string_view but server.h relies on
    transitive includes for it.
    
    Add the direct include (probably makes iwyu happier too?)
    fd04999840
  17. rpc: erase empty map entry in removeCommand
    After removing the last CRPCCommand pointer for a given name,
    erase the now-empty vector from mapCommands. Without this,
    listCommands() returns the name of a fully removed command
    because it iterates mapCommands keys unconditionally.
    
    For example, when unloading the wallet the RPCs are deregistered, and
    this prevents getopenrpcinfo from returning non-existant RPCs.
    c1a46066f2
  18. willcl-ark force-pushed on Mar 2, 2026
  19. DrahtBot removed the label CI failed on Mar 2, 2026
  20. willcl-ark commented at 9:49 am on March 3, 2026: member

    I don’t really see why it wouldn’t be better to have a getopenrpcinfo command that returns the json directly @ajtowns I have updated the draft here now to work like this. It was not as much extra c++ code in sever.cpp as I thought, which seems nice, and I agree it’s a better approach. Comparing the python output to the c++ output only differed on formatting. New output file from a --preset dev-mode build here.

    Hey @willcl-ark, have you checked my https://github.com/rust-bitcoin/corepc/issues/4#issuecomment-3935551647? Is something like that you wanted to build? @natobritto I had not heard of this one, no. I knew there were one or two? unmaintained rust bitcoin core json rpc libs, but pretty sure neither was this one. I will check it out.

  21. janb84 commented at 1:01 pm on March 3, 2026: contributor

    I like the new C++ approach!

    I found one conundrum:

    As per openRPC output the command getopenrpcinfo should return an empty info object (explicit “additionalProperties”: false):

     0
     1    {
     2      "name": "getopenrpcinfo",
     3      "description": "Returns an OpenRPC document for currently available RPC commands.",
     4      "params": [
     5      ],
     6      "result": {
     7        "name": "result",
     8        "schema": {
     9          "type": "object",
    10          "properties": {
    11            "openrpc": {
    12              "type": "string",
    13              "description": "OpenRPC specification version."
    14            },
    15            "info": {
    16              "type": "object",
    17              "properties": {
    18              },
    19              "additionalProperties": false,
    20              "description": "Metadata about this JSON-RPC interface."
    21            },
    22            "methods": {
    23              "type": "array",
    24              "items": {
    25                "type": "object",
    26                "properties": {
    27                },
    28                "additionalProperties": false
    29              },
    30              "description": "Documented RPC methods."
    31            }
    32          },
    33          "additionalProperties": false,
    34          "required": [
    35            "openrpc",
    36            "info",
    37            "methods"
    38          ]
    39        }
    40      },
    41      "x-bitcoin-category": "control"
    42    },
    

    The output is enriched with an info object filled (so it has additional properties) :

    0  "info": {
    1    "title": "Bitcoin Core JSON-RPC",
    2    "version": "v30.99.0-dev",
    3    "description": "Autogenerated from Bitcoin Core RPC metadata."
    4  },
    

    This invalidates the generated openRPC spec

  22. willcl-ark force-pushed on Mar 3, 2026
  23. willcl-ark commented at 5:02 pm on March 3, 2026: member

    This invalidates the generated openRPC spec

    Doh! You’re right that the getopenrpcinfo schema was indeed self-invalidating!

    I’ve updated the RPCResult to fully describe the info fields (title, version, description) and the method object structure (name, description, params, result, x-bitcoin-category), including the param and result Content Descriptor shapes. Dynamic JSON Schema fields within those use unconstrained schemas since their structure varies per method. I don’t think there’s anything we can do about that. There are also a few bits related to optional args before required, but those are legacy RPC artefacts.

    I also added a sample helper script I used to validate that it’s now actually valid, per the meta-schema (you can run this yourself with uv run contrib/devtools/check-openrpc.py <path_to_generated_schema>). Could be useful to have something like this to avoid regressions, although if it’s not running in the test suite, then I’m not so sure…

  24. DrahtBot added the label CI failed on Mar 3, 2026
  25. DrahtBot commented at 6:20 pm on March 3, 2026: contributor

    🚧 At least one of the CI tasks failed. Task Windows native, fuzz, VS: https://github.com/bitcoin/bitcoin/actions/runs/22633855642/job/65591195202 LLM reason (✨ experimental): Fuzz test failure: assertion failed in fuzz/rpc.cpp (trigger_internal_bug not found) during RPC fuzz processing.

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

  26. rpc: render Type::ANY in help text instead of aborting
    RPCResult::Type::ANY triggers NONFATAL_UNREACHABLE() in ToSections(),
    which crashes the help() RPC when a command uses Type::ANY in a
    nested result field.
    
    Previously this was never hit because Type::ANY was only used as a
    top-level alternate result type, filtered out before ToSections() is
    called.
    
    getopenrpcinfo() will use this result type, so render it like other
    types allowing it to be used in nested result definitions like schema.
    1f0d50fafc
  27. rpc: add getopenrpcinfo command
    There is no machine-readable description of the Bitcoin Core JSON-RPC
    interface, making it difficult for tooling and clients to discover
    available methods, parameter types, and result schemas programmatically.
    
    Add an RPC that returns an OpenRPC 1.3.2 document describing all
    registered RPC methods. The schema is generated directly from the
    existing RPCArg and RPCResult C++ types.
    
    Store the RpcMethodFnType function pointer on CRPCCommand so the schema
    generator can obtain RPCHelpMan metadata. Add minimal accessors
    (GetDescription, GetArgs, GetResults) to RPCHelpMan for external read
    access to its private fields.
    
    Add a basic unit test on an empty table.
    1072d9067b
  28. test: add functional test for getopenrpcinfo
    Verify the RPC returns a serializable document (valid JSON) and
    spot-check the generated schema for getblock, which exercises
    conditional result handling (verbosity-dependent `oneOf` with
    `additionalProperties`).
    92bd714d58
  29. doc: add openrpc skeleton document
    Having a checked-in copy makes the schema diffable across releases
    without needing a running node.
    
    This commit is optional as the schema is always available from a running
    bitcoind.
    229e6c0497
  30. guix: add openrpc.json to release tarball share/
    Users and downstream packagers cannot access the RPC schema without
    building and running bitcoind.
    
    Include openrpc.json in the release tarball so it ships alongside other
    documentation.
    
    This commit is optional as the schema is always available from a running
    bitcoind.
    b3a6bc35d5
  31. contrib: dummy openrpc schema verify script d4a754688d
  32. willcl-ark force-pushed on Mar 3, 2026
  33. DrahtBot removed the label CI failed on Mar 3, 2026
  34. hodlinator commented at 2:41 pm on March 4, 2026: contributor

    Concept ACK

    My gut says we should be maintaining a shared spec and generate stub C++ binding code from it, as is done for .capnp files. However, the current reverse approach of generating the spec from the C++ declarations has the following benefits:

    • Doesn’t add more codegen complexity to the build.
    • Doesn’t add complexity of keeping generated C++ code and version controlled code in sync.
    • Doesn’t touch existing C++ RPC declarations.
    • If we were to decide we want to switch to non-C++ spec + codegen, the current approach gives us the raw material for that spec.
  35. in doc/openrpc.json:6 in d4a754688d
    0@@ -0,0 +1,9 @@
    1+{
    2+  "openrpc": "1.3.2",
    3+  "info": {
    4+    "title": "Bitcoin Core JSON-RPC",
    5+    "version": "v30.99.0-dev",
    6+    "description": "Placeholder OpenRPC skeleton. Regenerate this file for release candidates using `bitcoin-cli getopenrpcinfo`."
    


    ajtowns commented at 5:06 pm on March 5, 2026:
    Not clear to me if this should be in this repo, or done like the rpc help at https://github.com/bitcoin-core/bitcoincore.org/blob/master/contrib/doc-gen/generate.go

    willcl-ark commented at 8:44 pm on March 5, 2026:

    Yes, I think the final 4 commits here are up for conceptual-inclusion review; we could… drop them all. The other extreme is to vendor the schema into this repo and use that in the functional test. Or we could keep something like the basic sanity test I have here and drop the final 3.

    IMO if we want to distribute in guix tarballs keeping the skeleton file checked in isn’t too bad. But I’m not strongly wedded to the idea either. If we didn’t want to distribute a static file, we could drop both the skeleton file and guix commits.


    casey commented at 0:32 am on March 6, 2026:
    I think it’s very useful to be able to interrogate bitcoind for its schema, it makes the schema much more discoverable, and makes it much more likely that the code will be kept up to date. It’s also probably the path to integrate it with the least code, since there’s minimal additional code for a new API call, and the C++ objects which represent the RPC API are all available to the schema generator.
  36. w0xlt commented at 8:38 pm on March 6, 2026: contributor
    Concept ACK

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-03-09 09:13 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me