util: Set Univalue to null after read failure #36088

pull maflcko wants to merge 2 commits into bitcoin:master from maflcko:2608-util-uni-read-fail-null changing 6 files +36 −11
  1. maflcko commented at 7:38 AM on August 26, 2026: member

    Currently, UniValue::read() may leave the value in a dirty/corrupt state after a read failure.

    This is perfectly fine, because all production code-paths check the read return value and exit early.

    However, it seems nicer and safer to discard the dirty and corrupt state. So do that here.

    This refactor doesn't change any production behavior. However, it fixes a fuzz failure in the rpc target, which was recently reworked in commit fa895bb77a8a061734d6626fde40090ad231ad5d. Later, adding new fuzz inputs (e.g. fuzz_corpora/rpc/fa1b0eeaa948a091f022c1ff2d0002a3fa6a631f ) and commit 747cff842481153357199bf9a81b5a4d82ea91fb made it hit this invalid UniValue code path.

  2. DrahtBot renamed this:
    util: Set Univalue to null after read failure
    util: Set Univalue to null after read failure
    on Aug 26, 2026
  3. DrahtBot added the label Utils/log/libs on Aug 26, 2026
  4. DrahtBot commented at 7:38 AM on August 26, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    Code Coverage & Benchmarks

    For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/36088.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #36087 (util: Add and use AssertUnreachable by maflcko)
    • #29409 (multiprocess: Add capnp wrapper for Chain interface by ryanofsky)
    • #10102 (Multiprocess bitcoin by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  5. maflcko force-pushed on Aug 26, 2026
  6. DrahtBot added the label CI failed on Aug 26, 2026
  7. DrahtBot commented at 7:49 AM on August 26, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/32943725277/job/98099900958</sub> <sub>LLM reason (✨ experimental): CI failed because IWYU detected an include change and deliberately returned a failure (“Failure generated from IWYU”).</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  8. maflcko added this to the milestone 32.0 on Aug 26, 2026
  9. maflcko removed the label CI failed on Aug 26, 2026
  10. DrahtBot added the label CI failed on Aug 26, 2026
  11. maflcko commented at 8:59 AM on August 26, 2026: member

    Can be manually tested via:

    curl --location --fail 'https://github.com/bitcoin-core/qa-assets/raw/ca855e6f7fa3e0f5f5ab291b43b26a74f2b3b765/fuzz_corpora/rpc/fa1b0eeaa948a091f022c1ff2d0002a3fa6a631f' --output '/tmp/rpc_fa1b0eeaa' 
    FUZZ=rpc ./bld-cmake/bin/fuzz -runs=1 /tmp/rpc_fa1b0eeaa 
    
  12. in src/univalue/test/object.cpp:456 in fa50b80854 outdated
     451 | @@ -452,6 +452,16 @@ void univalue_readwrite()
     452 |      BOOST_CHECK(!v.read("[]{}"));
     453 |      BOOST_CHECK(!v.read("{}[]"));
     454 |      BOOST_CHECK(!v.read("{} 42"));
     455 | +
     456 | +    // Failed read clears value
    


    hodlinator commented at 9:10 AM on August 26, 2026:

    fa50b80 test: Add UniValue failed read test:

    First commit states:

    // Failed read clears value
    

    But that commit does not have that behavior. Would be more correct with:

    // Verify state after failed read
    

    maflcko commented at 10:36 AM on August 26, 2026:

    thx, added (XXX, TODO, FIXME)

  13. in src/univalue/include/univalue.h:2 in 8888d1a0ae


    hodlinator commented at 9:24 AM on August 26, 2026:

    nanonit: Outdated


    maflcko commented at 10:36 AM on August 26, 2026:

    Heh, I thought all of those were fixed long ago. Thx, fixed up manually.

  14. hodlinator approved
  15. hodlinator commented at 9:24 AM on August 26, 2026: contributor

    ACK 8888d1a0ae031e5a40150daf3c15b52aec90781c

    Agree with fixing this bad API globally instead of patching up each consequence as in #36086.

  16. DrahtBot removed the label CI failed on Aug 26, 2026
  17. test: Add UniValue failed read test
    Just to document the current behavior, fixed in the next commit.
    fa7786592d
  18. in src/univalue/include/univalue.h:103 in 8888d1a0ae
      98 | @@ -99,6 +99,7 @@ class UniValue {
      99 |      std::string write(unsigned int prettyIndent = 0,
     100 |                        unsigned int indentLevel = 0) const;
     101 |  
     102 | +    /// Parse JSON; set value to null on failure.
     103 |      bool read(std::string_view raw);
    


    hodlinator commented at 10:16 AM on August 26, 2026:

    Realized that maybe there could be some case like allowing a partially valid settings.json without aborting, but we do error out in that case:

    https://github.com/bitcoin/bitcoin/blob/031175197f1b7f90397b838a381f0892a74ca62a/src/common/settings.cpp#L86-L92

    (SettingsValue is a typedef of UniValue).

    Might merit a [[nodiscard]] tag on the return value just to be safe.


    maflcko commented at 10:37 AM on August 26, 2026:

    thx, added nodiscard

  19. maflcko force-pushed on Aug 26, 2026
  20. maflcko force-pushed on Aug 26, 2026
  21. DrahtBot added the label CI failed on Aug 26, 2026
  22. DrahtBot commented at 10:50 AM on August 26, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task macOS-cross to arm64: https://github.com/bitcoin/bitcoin/actions/runs/32959090522/job/98147226271</sub> <sub>LLM reason (✨ experimental): CI failed because the C++ build stopped on a Clang -Werror error: ignoring a [[nodiscard]] return value in common-types.h (value.read(...)).</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  23. maflcko force-pushed on Aug 26, 2026
  24. maflcko removed the label CI failed on Aug 26, 2026
  25. maflcko commented at 11:17 AM on August 26, 2026: member

    heh, maybe I should compile with werror locally, instead of just with warnings ...

  26. util: Set Univalue to null after read failure fa72de78a9
  27. maflcko force-pushed on Aug 26, 2026
  28. DrahtBot added the label CI failed on Aug 26, 2026
  29. DrahtBot commented at 11:18 AM on August 26, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task Alpine (musl): https://github.com/bitcoin/bitcoin/actions/runs/32961204083/job/98153802371</sub> <sub>LLM reason (✨ experimental): CI failed during compilation because UniValue::read()’s [[nodiscard]] return value was ignored in script_standard_tests.cpp, and warnings were treated as errors (-Werror).</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  30. hodlinator approved
  31. hodlinator commented at 11:46 AM on August 26, 2026: contributor

    re-ACK fa72de78a931e9fd5708f95adae389d90de3ee9b

  32. rustaceanrob commented at 12:55 PM on August 26, 2026: member

    Looks like there is a new sanitization issue if m_height is 64 bit max and we add 1. Not like it is possible to see an m_height over 32 bits but here is a fix:

    diff --git a/src/policy/fees/mempool_estimator.cpp b/src/policy/fees/mempool_estimator.cpp
    index 88290ba6f2..5f384925fd 100644
    --- a/src/policy/fees/mempool_estimator.cpp
    +++ b/src/policy/fees/mempool_estimator.cpp
    @@ -18,6 +18,7 @@
     #include <util/feefrac.h>
     #include <util/fees.h>
     #include <util/fs.h>
    +#include <util/overflow.h>
     #include <util/syserror.h>
     #include <validation.h>
    
    @@ -207,10 +208,11 @@ bool MemPoolFeeRateEstimator::Read(AutoFile& file)
                 return false;
             }
             for (size_t i = 1; i < blocks.size(); ++i) {
    -            if (blocks[i].m_height != blocks[i - 1].m_height + 1) {
    +            const uint64_t expected_height{SaturatingAdd(blocks[i - 1].m_height, uint64_t{1})};
    +            if (blocks[i].m_height != expected_height) {
                     LogWarning("%s: Non-consecutive block heights read, expected height %s but found %s; ignoring file",
                                FeeRateEstimatorTypeToString(FeeRateEstimatorType::MEMPOOL_POLICY),
    -                           blocks[i - 1].m_height + 1, blocks[i].m_height);
    +                           expected_height, blocks[i].m_height);
                     return false;
                 }
             }
    

    opened #36092

  33. jeanpablojp commented at 2:04 PM on August 26, 2026: contributor

    tACK fa72de78a931e9fd5708f95adae389d90de3ee9b

    Built and ran the univalue tests plus the qa-assets input on the rpc target, and checked the new test fails without the change.

  34. maflcko referenced this in commit c0fdbf48d9 on Aug 26, 2026
  35. in src/ipc/capnp/common-types.h:128 in fa72de78a9
     123 | @@ -123,7 +124,9 @@ decltype(auto) CustomReadField(TypeList<UniValue>, Priority<1>, InvokeContext& i
     124 |  {
     125 |      return read_dest.update([&](auto& value) {
     126 |          auto data = input.get();
     127 | -        value.read(std::string_view{data.begin(), data.size()});
     128 | +        if (!value.read(std::string_view{data.begin(), data.size()})) {
     129 | +            throw std::runtime_error{"invalid JSON received over IPC"};
    


    Sjors commented at 2:49 PM on August 26, 2026:

    In fa72de78a931e9fd5708f95adae389d90de3ee9b util: Set Univalue to null after read failure: could add a test:

    diff --git a/src/ipc/test/ipc_tests.cpp b/src/ipc/test/ipc_tests.cpp
    index e353a7ee7c..6695024412 100644
    --- a/src/ipc/test/ipc_tests.cpp
    +++ b/src/ipc/test/ipc_tests.cpp
    @@ -97,4 +97,20 @@ void IpcPipeTest()
         BOOST_CHECK_EQUAL(uni1.write(), uni2.write());
    
    +    // Test: invalid JSON is rejected with a call error. Bypass the client
    +    // wrapper and send a raw capnp request, like a non-libmultiprocess client
    +    // could.
    +    std::promise<std::string> read_error;
    +    foo->m_context.loop->sync([&] {
    +        auto request{foo->m_client.passUniValueRequest()};
    +        request.setArg("invalid json");
    +        foo->m_context.loop->m_task_set->add(request.send().then(
    +            [&](auto&&) { read_error.set_value(""); },
    +            [&](kj::Exception&& e) { read_error.set_value(e.getDescription().cStr()); }));
    +    });
    +    const std::string err{read_error.get_future().get()};
    +    BOOST_CHECK(err.find("invalid JSON received over IPC") != std::string::npos);
    +    // The connection survives the failed call.
    +    BOOST_CHECK_EQUAL(foo->add(4, 5), 9);
    +
         CMutableTransaction mtx;
         mtx.version = 2;
    

    Sjors commented at 2:55 PM on August 26, 2026:

    Equivalent functional test, maybe easier to read:

    diff --git a/test/functional/interface_ipc.py b/test/functional/interface_ipc.py
    index 2c11f88305..119591a675 100755
    --- a/test/functional/interface_ipc.py
    +++ b/test/functional/interface_ipc.py
    @@ -5,4 +5,5 @@
     """Test the IPC (multiprocess) interface."""
     import asyncio
    +import json
    
     from contextlib import ExitStack
    @@ -10,4 +11,5 @@ from test_framework.test_framework import BitcoinTestFramework
     from test_framework.util import assert_equal
     from test_framework.ipc_util import (
    +    assert_capnp_failed,
         load_capnp_modules,
         make_capnp_init_ctx,
    @@ -52,4 +54,22 @@ class IPCInterfaceTest(BitcoinTestFramework):
             asyncio.run(capnp.run(async_routine()))
    
    +    def run_rpc_invalid_json_test(self):
    +        self.log.info("Running RPC invalid JSON test")
    +        async def async_routine():
    +            ctx, init = await make_capnp_init_ctx(self)
    +            self.log.debug("Create Rpc proxy object")
    +            rpc = init.makeRpc(ctx).result
    +            self.log.debug("Invalid JSON in an executeRpc request must be rejected")
    +            try:
    +                await rpc.executeRpc(ctx, "invalid json", "/", "")
    +                raise AssertionError("executeRpc unexpectedly succeeded")
    +            except capnp.KjException as e:
    +                assert_capnp_failed(e, "remote exception: std::exception: invalid JSON received over IPC")
    +            self.log.debug("The connection still works after the failed call")
    +            request = json.dumps({"method": "getblockcount", "params": [], "id": 1})
    +            response = json.loads((await rpc.executeRpc(ctx, request, "/", "")).result)
    +            assert_equal(response["result"], self.nodes[0].getblockcount())
    +        asyncio.run(capnp.run(async_routine()))
    +
         def run_mining_test(self):
             self.log.info("Running mining test")
    @@ -165,4 +185,5 @@ class IPCInterfaceTest(BitcoinTestFramework):
         def run_test(self):
             self.run_echo_test()
    +        self.run_rpc_invalid_json_test()
             self.run_mining_test()
             self.run_deprecated_mining_test()
    

    maflcko commented at 2:58 PM on August 26, 2026:
    like a non-libmultiprocess client
    

    I wonder if this is a real use-case? Is there a client out there without using libmultiprocess?

    In any case, it could be done in a later pull, as this one has 4 acks already?


    Sjors commented at 3:12 PM on August 26, 2026:

    Is there a client out there without using libmultiprocess?

    Yes, SRI can connect directly to mining interface (as an alternative to sv2-tp).

    And since we allow general RPC calls via IPC, I would expect different software to do this. Hopefully none that actually produces invalid JSON, though our parser could have bugs too.


    Sjors commented at 3:12 PM on August 26, 2026:

    (it's fine to leave as a followup, getting CI green is more important)


    maflcko commented at 3:44 PM on August 26, 2026:

    Yeah, right. I guess the functional test can't hurt. In the worst case, it adds code coverage for one line.

    Maybe you can submit it, given that the pull was merged?


    Sjors commented at 3:51 PM on August 26, 2026:

    Done in #36093

  36. Sjors commented at 2:50 PM on August 26, 2026: member

    ACK fa72de78a931e9fd5708f95adae389d90de3ee9b

  37. rustaceanrob commented at 2:50 PM on August 26, 2026: member

    ACK fa72de78a931e9fd5708f95adae389d90de3ee9b

  38. DrahtBot removed the label CI failed on Aug 26, 2026
  39. l0rinc approved
  40. l0rinc commented at 3:06 PM on August 26, 2026: contributor

    code review ACK fa72de78a931e9fd5708f95adae389d90de3ee9b

  41. fanquake merged this on Aug 26, 2026
  42. fanquake closed this on Aug 26, 2026

  43. maflcko referenced this in commit 6fe23857e8 on Aug 26, 2026
  44. maflcko deleted the branch on Aug 26, 2026

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-08-31 18:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me