http: Improve HTTPRemoteClient::MaybeDisconnect() #36159

pull hodlinator wants to merge 5 commits into bitcoin:master from hodlinator:2026/09/should_disconnect changing 3 files +159 −50
  1. hodlinator commented at 10:41 AM on September 3, 2026: contributor

    Issues

    #35182 had 1 case where we flag a client for disconnect where we do not log the reason.

    #35829 introduced the somewhat poorly named non-const HTTPRemoteClient::MaybeDisconnect(). It also lacks testing.

    The control-flow in MaybeDisconnect()/ShouldDisconnect() is somewhat complex. #35829 (review)

    Commits solving these issues in same order

    • http: Add log message for remaining case where we set HTTPRemoteClient::m_disconnect = true
    • refactor: Rename MaybeDisconnect() to ShouldDisconnect() and make it const
    • refactor: Enable mocking the time around ShouldDisconnect()
    • test: Add test for behavior of ShouldDisconnect()
    • refactor: Flatten control flow in ShouldDisconnect()
  2. DrahtBot added the label RPC/REST/ZMQ on Sep 3, 2026
  3. DrahtBot commented at 10:41 AM on September 3, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--006a51241073e994b41acfe9ec718e94-->

    External sites

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline and AI policy for information on the review process.

    Type Reviewers
    ACK pinheadmz, winterrdog
    Concept ACK jeanpablojp

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #36204 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36204.svg"></sub> (http: disconnect clients that never finish a request by janb84)
    • #36074 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/36074.svg"></sub> (scripted-diff: [test] Add util/check.h includes for assertions by maflcko)
    • #35713 <sub><img src="https://drahtbot.space/ack_count/bitcoin/bitcoin/35713.svg"></sub> (Remove boost as a unit test runner by rustaceanrob)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

    LLM Linter (✨ experimental)

    Possible typos and grammar issues:

    • src/test/httpserver_tests.cpp: claiming its able to send everything → claiming it's able to send everything [typo: its should be it's]

    <sup>2026-10-06 19:40:23</sup>

  4. jeanpablojp commented at 2:19 PM on September 4, 2026: contributor

    Concept ACK

    Built the four commits and mutation-tested the changed branches.

    now is a whole second, taken before the clients exist and compared against an m_idle_since they stamp later, so the checks expecting a disconnect at now + timeout + 1s only hold while both fall in the same second. About 0.6ms sits in between on busy_slow, and a rollover there makes the comparison 30s > 30s. Forced one to confirm. Worth taking now per block, after WriteReply() for busy_slow?

  5. in src/test/httpserver_tests.cpp:857 in 332f4e48c5
     852 | +            }
     853 | +        };
     854 | +
     855 | +        // Allow disconnecting directly after WriteReply() when it is able to
     856 | +        // complete the optimistic send of the full response due to the FastSocket.
     857 | +        auto busy_fast{std::make_shared<Client>(std::make_unique<FastSocket>())};
    


    jeanpablojp commented at 2:19 PM on September 4, 2026:

    ZeroSock::Send() already returns len, so could this just be a ZeroSock?

            // Allow disconnecting directly after WriteReply() when it is able to
            // complete the optimistic send of the full response due to the ZeroSock.
            auto busy_fast{std::make_shared<Client>(std::make_unique<ZeroSock>())};
    
  6. in src/httpserver.cpp:1115 in 332f4e48c5 outdated
    1118 | +bool HTTPRemoteClient::ShouldDisconnect(std::chrono::time_point<SteadyClock> now, std::chrono::seconds rpcservertimeout, bool disconnect_all) const
    1119 |  {
    1120 | -    // First check for idle timeout. We reset the timer when we send and receive data,
    1121 | +    // Disconnect this client due to error or end of communication
    1122 | +    // May drop unsent data if we are closing due to error.
    1123 | +    if (m_disconnect) {
    


    jeanpablojp commented at 2:19 PM on September 4, 2026:

    This also stops the idle line being logged when the client is already flagged. Before, is_idle was worked out first and logged inside if (m_disconnect || is_idle), so a client already flagged and past the timeout got both lines. Looks like an improvement, since that line was claiming a timeout for a client leaving for another reason. Worth a mention in the commit message, since it says refactor:?

  7. in src/test/httpserver_tests.cpp:798 in 332f4e48c5 outdated
     793 | +    {
     794 | +        // Nothing to disconnect for yet.
     795 | +        auto idle{std::make_shared<Client>()};
     796 | +        BOOST_CHECK(!idle->ShouldDisconnect(now, timeout, /*disconnect_all=*/false));
     797 | +        // A shutdown waits for a connection that is still busy, and it starts busy.
     798 | +        BOOST_CHECK(!idle->ShouldDisconnect(now, timeout, /*disconnect_all=*/true));
    


    jeanpablojp commented at 2:19 PM on September 4, 2026:

    A connection that is accepted and never sends anything keeps m_connection_busy at its initial true, so the disconnect_all branch never picks it up. Inside ShouldDisconnect() only the idle timeout is left, and with -rpcservertimeout=0 there is no idle timeout either. On regtest shutdown goes from 0.7s to 30.7s and ends in a Force-disconnecting warning. At the default of 30s that timeout and StopHTTPServer()'s own deadline are the same number. Releasing them at once would give up the 503 a client can still get today by sending after stop. Is the wait deliberate?


    hodlinator commented at 7:34 PM on September 5, 2026:

    re #36159 (review): You are pointing out behavior on master, right? Maybe we could disconnect clients quicker if they just open the connection and don't send anything, within 5 seconds or so? Not sure I want to increase the complexity of this PR though. That would probably introduce another timeout-setting.


    jeanpablojp commented at 10:27 PM on September 5, 2026:

    Yes, master, but also measured at the merge-base and on today's master, the same both times, under a second to shut down with no such connection open and 30.7s with one at -rpcservertimeout=0.

    The slots are the sharper half. With -rpcmaxconnections=4 and four sockets that connect and send nothing, a normal call has to wait for the idle timeout to free one, and -rpcservertimeout=0 means nothing frees them. Agreed it does not belong here though, and your five seconds sounds like the right direction. Happy to open a follow-up.

  8. in src/test/httpserver_tests.cpp:869 in 332f4e48c5 outdated
     864 | +        }
     865 | +        BOOST_CHECK( busy_fast->ShouldDisconnect(now + timeout - 1s, timeout, /*disconnect_all=*/false));
     866 | +    }
     867 | +
     868 | +    {
     869 | +        // A malformed request flags the client, and that outranks the rest.
    


    jeanpablojp commented at 2:19 PM on September 4, 2026:

    Every client here is HTTP/1.0, so m_keep_alive is never true and the branch for a healthy persistent connection is only ever asserted not firing. Replacing that condition with false leaves this test green. Worth one more client?

        {
            // A keep-alive client whose reply is out is neither flagged nor idle,
            // so the shutdown branch is the only thing that can release it.
            auto keepalive{std::make_shared<Client>(std::make_unique<ZeroSock>())};
            keepalive->MutateRecvBuffer().append("GET / HTTP/1.1\nHost: 127.0.0.1\n\n");
            {
                std::unique_ptr request{HTTPRemoteClient::TryReadRequest(keepalive)};
                BOOST_REQUIRE(request);
                request->WriteReply(HTTP_OK);
            }
            const auto now{Now<SteadySeconds>()};
            BOOST_CHECK(!keepalive->ShouldDisconnect(now, timeout, /*disconnect_all=*/false));
            BOOST_CHECK( keepalive->ShouldDisconnect(now, timeout, /*disconnect_all=*/true));
        }
    
        {
            // A malformed request flags the client, and that outranks the rest.
    
  9. hodlinator force-pushed on Sep 5, 2026
  10. hodlinator commented at 8:39 PM on September 5, 2026: contributor

    Thanks for the feedback @jeanpablojp (https://github.com/bitcoin/bitcoin/pull/36159#pullrequestreview-5114081037)!

    Addressed all of it to some extent in the latest push (352fdc279751694348c6b406322de5582f59ad49) + inline comment. Opted for mocking the clock and freeze it rather than trying to keep up with the current time of the actual clock as it drifts throughout the added test case.

  11. DrahtBot added the label CI failed on Sep 5, 2026
  12. DrahtBot commented at 9:31 PM on September 5, 2026: contributor

    <!--85328a0da195eb286784d51f73fa0af9-->

    🚧 At least one of the CI tasks failed. <sub>Task MSan: https://github.com/bitcoin/bitcoin/actions/runs/33990407720/job/101371509013</sub> <sub>LLM reason (✨ experimental): CI failed during the build link step (bitcoin-wallet) due to a ld.bfd error: “bad reloc symbol index … in section .debug_addr,” causing the final link to fail.</sub>

    <details><summary>Hints</summary>

    Try to run the tests locally, according to the documentation. However, a CI failure may still happen due to a number of reasons, for example:

    • Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.

    • A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.

    • An intermittent issue.

    Leave a comment here, if you need help tracking down a confusing failure.

    </details>

  13. winterrdog commented at 12:19 AM on September 6, 2026: contributor

    tACK 352fdc279751694348c6b406322de5582f59ad49

    successfully built and tested on this toolchain: Debian/clang++ 18/x86_64.


    P.S: the CI failure is completely independent of the changes introduced by this PR

  14. DrahtBot requested review from jeanpablojp on Sep 6, 2026
  15. DrahtBot removed the label CI failed on Sep 17, 2026
  16. in src/httpserver.cpp:1138 in 352fdc2797
    1147 | +    // need to disconnect all clients... Unless we still have data for
    1148 | +    // this client, in which case we'd rather continue the I/O loop
    1149 | +    // until all data is sent or an error is encountered.
    1150 | +    } else if (disconnect_all && !m_connection_busy) {
    1151 | +        // This is a healthy persistent connection (e.g. keep-alive)
    1152 | +        // but it's time to say goodbye.
    


    pinheadmz commented at 3:43 PM on September 24, 2026:

    352fdc279751694348c6b406322de5582f59ad49

    tiny nit, comment placement: I think this comment can just be removed or something added to the above comment that this is where even keep-alive clients get dropped, because we are shutting down.

    your other comments all describe a logic decision about to be made on the next line and so I think its a bit confusing to have this written here.

  17. in src/httpserver.cpp:1101 in 352fdc2797
    1103 | -                                                                     m_rpcservertimeout,
    1104 | -                                                                     /*disconnect_all=*/m_disconnect_all_clients);
    1105 | -                                  });
    1106 | +    const auto now{MockableSteadyClock::now()};
    1107 | +    size_t erased = std::erase_if(m_connected, [&](auto& client) {
    1108 | +        return client->ShouldDisconnect(now,
    


    pinheadmz commented at 5:04 PM on September 24, 2026:

    Possible impovement: move the Log("Disconnecting client...") here inside the lambda. Then ShouldDisconnect() could be clarified by filling those blank spaces with return statements

  18. pinheadmz approved
  19. pinheadmz commented at 5:38 PM on September 24, 2026: member

    ACK 352fdc279751694348c6b406322de5582f59ad49

    Built and tested on macos/arm64. Reviewed the changes and checked with both meatball brain and LLM that the refactor didn't change behavior. Great new text coverage helps with that a lot as well. Left a few non blocking suggestions.

    <details><summary>Show Signature</summary>

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    ACK 352fdc279751694348c6b406322de5582f59ad49
    -----BEGIN PGP SIGNATURE-----
    
    iQJPBAEBCAA5FiEE5hdzzW4BBA4vG9eM5+KYS2KJyToFAmq1X5EbFIAAAAAABAAO
    bWFudTIsMi41KzEuMTIsMCwzAAoJEOfimEtiick6nf8QALaqDEDS8I6yCKfKkJ9L
    xEpZ46pQu9Ip04KhRfa2oHzplxr7Zc+0UQC0IiPqfgsVOOg0lH9jrzIgttdyEa/Z
    xWnfpxjW/BKKrBGpdp9DcDeNZqQ36IknKMvDDLCtSv677XcxysQqxHNDUh9XNmyy
    CgGJkCKF0y3kzPs/Zu/IUZxLEzdE6Uzr+fz4PphwbO8ewAqygDRa9Sl2a9m7Dp5W
    50UG6exRmMsrjWZF+oO1CWHvfEC5OTJPIK4u+BGpCuTjZzGvCGrncnbTyrxDlZBh
    rE5Tk+/mmWjlE1NMoNf3quTjYDb0TwEp+jdNoaHEn3ivHgWig33H/4Y5yXxZh4Y4
    5jhMzekBo7P2NcsTkuin3g72V/ZxYWbJbC9y3Vxs86mbBwqVDmGiHH1HL9JRI0Kt
    DiM1F+VGMx0kc/ZrzY64397CwuwZAuYPFlM7aVMsQlLtqlex6JiY7lAYFuTttm5R
    u7OHtCVFZ3RFkR/M+LZm2GwLiMQN4GVfdIfO5bEQCfXMoVyLvzZ8r6kb8s1uwgvE
    DSmvwnOgEizVbp3uo3GmaFItpUxbY2bG8dkx37VPwyWBpU2TuJ1Y4S2AsmcCRriV
    ccDrOqY45LV7E19y3alKNLM7mwQVcw27TyIvT/XU4Rs63dcNOneRQ8at4mCB1WM/
    JZzZqJl1uPRsV77e2IWOAxzJ
    =k0v8
    -----END PGP SIGNATURE-----
    

    pinheadmz's public key is on openpgp.org

    </details>

  20. sedited commented at 8:07 AM on October 6, 2026: contributor

    @hodlinator do you want to take @pinheadmz's nits? Asking because this is already a followup/refactoring pull request.

  21. hodlinator force-pushed on Oct 6, 2026
  22. http: Add log message for remaining case where we set m_disconnect = true cb8f4b0a30
  23. refactor: Rename MaybeDisconnect() to ShouldDisconnect() and make it const
    Also moves the "Disconnecting ..." log message as it is a consequence of the client reporting that it should disconnect.
    
    Co-authored-by: Matthew Zipkin <pinheadmz@pm.me>
    5fba330833
  24. refactor: Switch client idle timeout to mockable clock
    Enables precision in later tests.
    59a3e8803c
  25. test: Verify behavior of HTTPRemoteClient::ShouldDisconnect()
    Best to nail this down before the refactor in the next commit.
    
    Co-authored-by: JP <jeanpablo.jp@hotmail.com>
    5b98d56c93
  26. http: Flatten control flow in HTTPRemoteClient::ShouldDisconnect()
    There's a slight behavior change in that clients already flagged with m_disconnect = true no longer also get logged as passing the idle timeout if that condition would also be met.
    
    Co-authored-by: winterrdog <winterrdog@protonmail.ch>
    2abf671d2e
  27. hodlinator force-pushed on Oct 6, 2026
  28. hodlinator commented at 7:52 PM on October 6, 2026: contributor

    re #36159 (comment) & #36159#pullrequestreview-5306713554:

    Thanks for the review @pinheadmz, applied your neat nits. Also attempted to improve some comments. git range-diff 352fdc27~4...352fdc27 2abf671d~4...2abf671d

  29. pinheadmz approved
  30. pinheadmz commented at 6:01 AM on October 14, 2026: member

    ACK 2abf671d2e14b5ceb91fb570258945b41ef78299

    Built, tested. Re-reviewed. Thanks for taking my suggestions!

    <details><summary>Show Signature</summary>

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    ACK 2abf671d2e14b5ceb91fb570258945b41ef78299
    -----BEGIN PGP SIGNATURE-----
    
    iQJPBAEBCAA5FiEE5hdzzW4BBA4vG9eM5+KYS2KJyToFAmrPGqMbFIAAAAAABAAO
    bWFudTIsMi41KzEuMTIsMCwzAAoJEOfimEtiick612wP/3qL+Lw+FuCh+T8oAdMc
    x8E2M59R7lM1npICusZ3HMYxdx8+Umk1OmQolhnlcEk9SWni5skPIpnFhem+pLUt
    Zq/6Fl6rHEphzLnRDgBzynhndmjnGWMivpOc8yZRjfJG6x3pKqow5yhCP3x+wiQm
    qBIPKKiVrQbEVha0SKlM68quBRl6oKxv7cBkQ4GJYQ7AVe9assdXoVWq9vnfnk9I
    bz8HZ8kfRsXDXuBBUa6IfHZ+hc4e9uA8UuVwK2KzTtF8pwBkF9ipPc1sHY37eXqe
    02A/2jwMCPAp24NTsZ3QMFsf+7kvDkwqfLzUOJim72SHsNzze7d3pKSC/5qh2ana
    3ljZA9HervZizi8n7CEYXnzFc+1jN/ocs6vzYSYAjK9/+QtxUvSMDIH00xkFQWFu
    Nr/yjYIVHo4f4EIIKckIcHiDHqSuUnhewj39ges2RSTMvjLsSry0KnuKzlUt5hqT
    e8SO+mlul43W4cxmd2Bp8J39BwbHwc2eT9a47BTmgdLXnRYweT0DupA/NhjKSz4h
    UmGmihT/nG2n/tq9FaQm9D03Llw0ROAHAZoGZ7Sxc4YVSwrIOP0C7xmqTgxcMRPx
    c/LAo6JTsu/1/r5DY+jiw3CCt1NTfkKH2EIgUC5rkUWVUFI5Meen/LigsOq0z33k
    9MBfzFPboMW63Feqc6zGifAW
    =Bwbg
    -----END PGP SIGNATURE-----
    

    pinheadmz's public key is on openpgp.org

    </details>

  31. in src/httpserver.cpp:1171 in 2abf671d2e
    1174 |          m_connected_size.fetch_sub(erased, std::memory_order_relaxed);
    1175 |      }
    1176 |  }
    1177 |  
    1178 | -bool HTTPRemoteClient::MaybeDisconnect(std::chrono::time_point<SteadyClock> now, std::chrono::seconds rpcservertimeout, bool disconnect_all)
    1179 | +bool HTTPRemoteClient::ShouldDisconnect(MockableSteadyClock::time_point now, std::chrono::seconds rpcservertimeout, bool disconnect_all) const
    


    winterrdog commented at 3:04 PM on October 14, 2026:

    cosmetic nit: now that every branch returns, i think plain ifs (without else) would read a bit nicer here. right now, the comments sit between a return true; and the next } else if, so they look like they belong to the previous block (kind of puzzled me at first glance but later on i realised what you meant). with separate ifs each comment lands right above the condition it explains, and the behavior would be identical. sth like:

    diff --git a/src/httpserver.cpp b/src/httpserver.cpp
    index dc2b8e8310..86696d2c69 100644
    --- a/src/httpserver.cpp
    +++ b/src/httpserver.cpp
    @@ -1175,30 +1175,31 @@ bool HTTPRemoteClient::ShouldDisconnect(MockableSteadyClock::time_point now, std
         if (m_disconnect) {
             // We should already have logged the reason when we set m_disconnect.
             return true;
    +    }
         // Check for idle timeout. We reset the timer when we send and receive data,
         // but if the server is busy handling a request we should ignore the timeout until
         // the reply is sent. If we did erase the shared_ptr<HTTPRemoteClient> reference in m_connected
         // while the server is busy with a request, it might be prematurely dropped before
         // the response has been sent, or if the HTTPRequest was holding a temporary shared_ptr
         // client on a worker thread - it would keep the socket open even after "disconnecting".
    -    } else if (rpcservertimeout.count() > 0 &&
    -               now - m_idle_since.load() > rpcservertimeout &&
    -               !m_req_busy) {
    +    if (rpcservertimeout.count() > 0 &&
    +        now - m_idle_since.load() > rpcservertimeout &&
    +        !m_req_busy) {
             LogDebug(BCLog::HTTP,
                      "HTTP client idle timeout %s (id=%llu)",
                      m_origin,
                      m_id);
             return true;
    +    }
         // Disconnect this client because the server is shutting down and we need to
         // disconnect all clients... Unless the client is still being handled by a
         // worker thread, in which case we'd rather continue the I/O loop (but might
         // not complete sending back the full response).
    -    } else if (disconnect_all && !m_connection_busy) {
    +    if (disconnect_all && !m_connection_busy) {
             return true;
    -    } else {
    -        // No reason to disconnect.
    -        return false;
         }
    +    // No reason to disconnect.
    +    return false;
     }
    
     void HTTPServer::ClearConnectedClients()
    

    non-blocking, it is :)

  32. winterrdog commented at 3:11 PM on October 14, 2026: contributor

    reACK 2abf671d2e14b5ceb91fb570258945b41ef78299


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-10-14 17:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me