proxy: add local connection limit to ListenConnections #269

pull enirox001 wants to merge 4 commits into bitcoin-core:master from enirox001:04-26-ipc-local-connection-limit changing 8 files +351 −20
  1. enirox001 commented at 10:00 AM on April 8, 2026: contributor

    This adds an optional local connection limit toListenConnections().

    Previously, ListenConnections() would accept incoming connections indefinitely. This branch adds an optional max_connections parameter so a listener can stop accepting new connections once a per-listener cap is reached, and resume accepting when an existing connection disconnects.

    The limit is local to the listener instead of global to the EventLoop. This keeps the state and behavior scoped to the listening socket, and is closer to the direction discussed downstream for per--ipcbind limits.

    This also adds a test covering the behavior with max_connections=1, verifying that:

    • the first client is accepted normally
    • a second client is not accepted while the first remains connected
    • the second client is accepted after the first disconnects

    This is a follow-up to the IPC FD reservation work

  2. DrahtBot commented at 10:01 AM on April 8, 2026: none

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    Approach ACK ryanofsky
    Stale ACK xyzconstant

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #288 (Create support branch for CI scripts, documentation, and examples by ryanofsky)
    • #287 (Split repository into master (library source) and support (CI, docs, examples) branches. by ryanofsky)
    • #274 (Add nonunix platform support by ryanofsky)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  3. enirox001 marked this as a draft on Apr 8, 2026
  4. enirox001 force-pushed on Apr 8, 2026
  5. enirox001 force-pushed on Apr 8, 2026
  6. enirox001 force-pushed on Apr 8, 2026
  7. in include/mp/proxy-io.h:829 in 84ed607b39
     821 | @@ -820,8 +822,19 @@ std::unique_ptr<ProxyClient<InitInterface>> ConnectStream(EventLoop& loop, int f
     822 |  //! handles requests from the stream by calling the init object. Embed the
     823 |  //! ProxyServer in a Connection object that is stored and erased if
     824 |  //! disconnected. This should be called from the event loop thread.
     825 | +template <typename InitInterface, typename InitImpl, typename OnDisconnect>
     826 | +void _Serve(EventLoop& loop, kj::Own<kj::AsyncIoStream>&& stream, InitImpl& init, OnDisconnect&& on_disconnect);
     827 | +
     828 |  template <typename InitInterface, typename InitImpl>
     829 |  void _Serve(EventLoop& loop, kj::Own<kj::AsyncIoStream>&& stream, InitImpl& init)
    


    ryanofsky commented at 2:02 PM on April 8, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (84ed607b390e24dd0d41cd95d0159bfeadfaf5d8)

    Since _Serve is an internal function only called in a few places, I think it would be less confusing if it was not overloaded, and just always required an on_disconnect parameter, even if it requires a little extra verbosity at some call sites.


    enirox001 commented at 9:58 AM on April 9, 2026:

    Done. Simplified this now so _Serve always takes an on_disconnect callback instead of overloading it. Since it is only used internally

  8. in include/mp/proxy-io.h:865 in 84ed607b39 outdated
     861 | +        : listener(kj::mv(listener_)), max_connections(max_connections_) {}
     862 | +
     863 | +    kj::Own<kj::ConnectionReceiver> listener;
     864 | +    std::optional<size_t> max_connections;
     865 | +    size_t active_connections{0};
     866 | +    bool accept_pending{false};
    


    ryanofsky commented at 2:08 PM on April 8, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (84ed607b390e24dd0d41cd95d0159bfeadfaf5d8)

    Curious if this accept_pending variable is actually necessary or if code could just compare active_connections and max_connections when deciding whether to listen. Would prefer to avoid redundancy in the state representation if possible even if makes individual checks more a little more verbose.

    If accept_pending really is necessary would be a good to have a short comment about why.


    enirox001 commented at 9:59 AM on April 9, 2026:

    I kept accept_pending, but added a short comment explaining why it is needed here. active_connections only counts accepted connections, so without a separate flag nested _Listen() calls could post multiple pending accept() calls before active_connections is incremented.


    xyzconstant commented at 7:47 PM on May 20, 2026:

    FWIW I tested the code without accept_pending field and hence its check at _Listen removed, the tests passed with no issues. Then, asked Claude to generate a test that exercises it and produced this:

    diff --git a/test/mp/test/listen_tests.cpp b/test/mp/test/listen_tests.cpp
    index b367938..78298c7 100644
    --- a/test/mp/test/listen_tests.cpp
    +++ b/test/mp/test/listen_tests.cpp
    @@ -24,8 +24,10 @@
     #include <string>
     #include <sys/socket.h>
     #include <sys/un.h>
    +#include <kj/exception.h>
     #include <thread>
     #include <unistd.h>
    +#include <vector>
     
     namespace mp {
     namespace test {
    @@ -112,11 +114,39 @@ public:
         std::thread thread;
     };
     
    +//! kj::ExceptionCallback that captures KJ_LOG output into an external sink.
    +//! Must be instantiated on the thread whose KJ logs you want to capture; it
    +//! installs itself onto that thread's ExceptionCallback stack via its base
    +//! constructor and removes itself in the destructor.
    +class CaptureLogCallback : public kj::ExceptionCallback
    +{
    +public:
    +    CaptureLogCallback(std::mutex& mu, std::string& sink) : m_mu(mu), m_sink(sink) {}
    +
    +    void logMessage(kj::LogSeverity severity, const char* file, int line, int contextDepth,
    +                    kj::String&& text) override
    +    {
    +        {
    +            std::lock_guard<std::mutex> lock(m_mu);
    +            m_sink.append(text.cStr(), text.size());
    +            m_sink.push_back('\n');
    +        }
    +        // Still let the default callback emit to stderr so test debug output
    +        // isn't silenced for other observers.
    +        kj::ExceptionCallback::logMessage(severity, file, line, contextDepth, kj::mv(text));
    +    }
    +
    +private:
    +    std::mutex& m_mu;
    +    std::string& m_sink;
    +};
    +
     class ListenSetup
     {
     public:
         explicit ListenSetup(std::optional<size_t> max_connections = std::nullopt)
             : capped_listener(max_connections.has_value()), thread([this, max_connections] {
    +              CaptureLogCallback log_capture(captured_log_mutex, captured_log);
                   EventLoop loop("mptest-server", [this](mp::LogMessage log) {
                       if (log.level == mp::Log::Raise) throw std::runtime_error(log.message);
                       if (log.message.find("IPC server: socket connected.") != std::string::npos) {
    @@ -144,6 +174,18 @@ public:
     
         ~ListenSetup()
         {
    +        forceShutdown();
    +        thread.join();
    +    }
    +
    +    //! Synchronously tear down the event loop's task set so any pending accept
    +    //! promises are destroyed now (rather than when the destructor runs later).
    +    //! This makes it possible to assert on captured KJ log output before the
    +    //! ListenSetup goes out of scope. Idempotent.
    +    void forceShutdown()
    +    {
    +        if (shutdown_done) return;
    +        shutdown_done = true;
             if (capped_listener) {
                 EventLoop* loop;
                 {
    @@ -152,7 +194,6 @@ public:
                 }
                 if (loop) loop->sync([&] { loop->m_task_set.reset(); });
             }
    -        thread.join();
         }
     
         size_t ConnectedCount()
    @@ -184,11 +225,15 @@ public:
         UnixListener listener;
         std::promise<void> ready_promise;
         bool capped_listener{false};
    +    bool shutdown_done{false};
         std::mutex counter_mutex;
         std::condition_variable counter_cv;
         EventLoop* event_loop{nullptr};
         size_t connected_count{0};
         size_t disconnected_count{0};
    +    //! KJ log output captured from the server thread via CaptureLogCallback.
    +    std::mutex captured_log_mutex;
    +    std::string captured_log;
         std::thread thread;
     };
     
    @@ -245,6 +290,34 @@ KJ_TEST("ListenConnections keeps capped listeners alive before reaching the limi
         KJ_EXPECT(client2->client->add(2, 3) == 5);
     }
     
    +// Without `accept_pending`, cascaded close handlers each post a duplicate
    +// accept(). KJ silently serializes them so the cap isn't exceeded, but the
    +// extra pending promises are destroyed at cleanup and logged as
    +// "PromiseFulfiller was destroyed without fulfilling the promise."
    +// This test fails when accept_pending is removed and passes when it's intact.
    +KJ_TEST("ListenConnections does not leak accept promises during disconnect burst")
    +{
    +    constexpr size_t kCap = 2;
    +    ListenSetup setup(/*max_connections=*/kCap);
    +
    +    std::vector<std::unique_ptr<ClientSetup>> filling;
    +    filling.reserve(kCap);
    +    for (size_t i = 0; i < kCap; ++i) {
    +        filling.push_back(std::make_unique<ClientSetup>(setup.listener.Connect()));
    +    }
    +    setup.WaitForConnectedCount(kCap);
    +
    +    filling.clear();
    +    setup.WaitForDisconnectedCount(kCap);
    +
    +    // Trigger m_task_set.reset() now so any leaked accept promises get destroyed
    +    // before we read captured_log.
    +    setup.forceShutdown();
    +
    +    std::lock_guard<std::mutex> lock(setup.captured_log_mutex);
    +    KJ_EXPECT(setup.captured_log.find("PromiseFulfiller was destroyed") == std::string::npos);
    +}
    +
     } // namespace
     } // namespace test
     } // namespace mp
    

    Basically what this does is install a kj::ExceptionCallback in the server thread to capture the log "PromiseFulfiller was destroyed" generated by KJ runtime if cascaded disconnects each call _Listen and post a fresh accept() promise. The test fails with accept_pending removed and pass with it back.


    xyzconstant commented at 7:50 PM on May 20, 2026:

    IMO it's not so obvious why this field is needed here, this patch basically guarantee the same outcome and also pass the test generated by Claude above:

    diff --git a/include/mp/proxy-io.h b/include/mp/proxy-io.h
    index 78924c6..c002811 100644
    --- a/include/mp/proxy-io.h
    +++ b/include/mp/proxy-io.h
    @@ -851,11 +851,6 @@ struct ListenState
         kj::Own<kj::ConnectionReceiver> listener;
         std::optional<size_t> max_connections;
         size_t active_connections{0};
    -    //! Tracks whether accept() has already been posted. This is needed because
    -    //! active_connections only counts accepted connections, so without a
    -    //! separate flag, nested _Listen() calls could queue multiple pending
    -    //! accepts before active_connections increases.
    -    bool accept_pending{false};
     };
     
     template <typename InitInterface, typename InitImpl>
    @@ -866,9 +861,12 @@ void _ServeAccepted(EventLoop& loop, InitImpl& init, const std::shared_ptr<Liste
     {
         ++state->active_connections;
         _Serve<InitInterface>(loop, kj::mv(stream), init, [&loop, &init, state] {
    +        const bool was_at_cap = state->max_connections && state->active_connections == *state->max_connections;
             assert(state->active_connections > 0);
             --state->active_connections;
    -        _Listen<InitInterface>(loop, init, state);
    +        if (was_at_cap) {
    +            _Listen<InitInterface>(loop, init, state);
    +        }
         });
     }
     
    @@ -885,15 +883,12 @@ inline std::unique_ptr<EventLoopRef> _MakeCappedListenerRef(EventLoop& loop, con
     template <typename InitInterface, typename InitImpl>
     void _Listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<ListenState>& state)
     {
    -    if (state->accept_pending) return;
         if (_ListenAtCapacity(*state)) return;
     
    -    state->accept_pending = true;
         auto* ptr = state->listener.get();
         auto accept_ref{_MakeCappedListenerRef(loop, *state)};
         loop.m_task_set->add(ptr->accept().then(
             [&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable {
    -            state->accept_pending = false;
                 _ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream));
                 _Listen<InitInterface>(loop, init, state);
             }));
    

    enirox001 commented at 11:39 AM on May 21, 2026:

    I think this is a better invariant than tracking accept_pending.

    There should already be one pending accept whenever the capped listener is below capacity, and disconnects only need to post a new accept when they transition the listener from full to below full, because that is the only state where nocaccept was pending.

    So checking this before decrementing active_connections avoids the duplicate-accept case without adding another state variable.

    Taken and decided to use resume_accept for the local boolean instead

  9. in test/mp/test/test.cpp:51 in 84ed607b39
      46 |  namespace mp {
      47 |  namespace test {
      48 |  
      49 | +namespace {
      50 | +
      51 | +class UnixListener
    


    ryanofsky commented at 2:43 PM on April 8, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (84ed607b390e24dd0d41cd95d0159bfeadfaf5d8)

    Would be good to introduce the test in a separate commit and maybe separate file if it doesn't share much code with existing test.

    IMO would be nice if it test was introduced in an intial commit then updated after connection limits are added so it's easier to see how connection limits are tested separately from connection setup.


    enirox001 commented at 10:02 AM on April 9, 2026:

    Split the listener coverage out into a dedicated listen_tests.cpp file and reordered the history so the baseline ListenConnections() test is introduced first, then extended in follow-up commit with the local connection limit coverage.

  10. ryanofsky commented at 2:44 PM on April 8, 2026: collaborator

    Approach ACK 84ed607b390e24dd0d41cd95d0159bfeadfaf5d8. Implementation of local connection limit here looks almost exactly like I would have expected.

    I do think it would be helpful to see a draft PR in the bitcoin repo using this API (it should be fine to make libmultiprocess changes there and let the lint CI job fail so you don't need to mess with subtrees) because the approach in https://github.com/bitcoin/bitcoin/pull/34978 of adding a global connection limit option isn't exactly compatible with the implementation here of implementing a per-address connection limit.

    I'd personally prefer using per-address limits over introducing a global limit but both approaches seem reasonable

  11. enirox001 force-pushed on Apr 9, 2026
  12. enirox001 force-pushed on Apr 9, 2026
  13. enirox001 force-pushed on Apr 9, 2026
  14. enirox001 force-pushed on Apr 9, 2026
  15. enirox001 commented at 10:29 AM on April 9, 2026: contributor

    Thanks for the review @ryanofsky .

    I addressed the cleanup points in the latest push:

    • _Serve is now explicit at all call sites
    • accept_pending is documented,
    • listener coverage now lives in a dedicated listen_tests.cpp file introduced in a separate commit before the local-limit extension.

    I’m also planning to put together a draft Bitcoin Core PR using this API so the per-address approach can be evaluated downstream against the current global-limit direction.

  16. enirox001 commented at 1:48 PM on April 9, 2026: contributor

    I put together the downstream draft using this API here bitcoin/bitcoin#35037

    It uses per--ipcbind max-connections=<n> options, threads the parsed per-address limit into ListenConnections(), and adds downstream coverage for the local-limit behavior.

  17. enirox001 marked this as ready for review on May 4, 2026
  18. enirox001 commented at 9:43 AM on May 4, 2026: contributor

    This PR is now ready for review

  19. in test/mp/test/listen_tests.cpp:133 in 8511c68f8b outdated
     128 | +                      ++disconnected_count;
     129 | +                      counter_cv.notify_all();
     130 | +                  }
     131 | +              });
     132 | +              FooImplementation foo;
     133 | +              ListenConnections<messages::FooInterface>(loop, listener.release(), foo, max_connections);
    


    xyzconstant commented at 2:58 AM on May 13, 2026:

    nit: This will fail to compile mptest at commit 8c47a3aba8baf145d7ef9d0efe3a1a0f67be2331 since the 4th parameter max_connections is not introduced in ListenConnections's signature until next commit. I'd fix the call for this commit and then adjust the setup accordingly in 8511c68.


    enirox001 commented at 8:41 AM on May 14, 2026:

    Fixed, thanks

  20. in test/mp/test/listen_tests.cpp:190 in 8511c68f8b outdated
     185 | +
     186 | +    setup.WaitForConnectedCount(1);
     187 | +    KJ_EXPECT(client->client->add(1, 2) == 3);
     188 | +}
     189 | +
     190 | +KJ_TEST("ListenConnections enforces a local connection limit")
    


    xyzconstant commented at 3:06 AM on May 13, 2026:

    nit: Unlike in ListenConnections's call issue (compilation error), this will still fail for this commit (8c47a3aba8baf145d7ef9d0efe3a1a0f67be2331). Following ryanosfky's reasoning (https://github.com/bitcoin-core/libmultiprocess/pull/269#discussion_r3052126251) I believe this suite would be better introduced in 8511c68f8b6827f4e9b306e0245929965a363930 so the test lands with the feature it exercises.


    enirox001 commented at 8:42 AM on May 14, 2026:

    Done

  21. xyzconstant commented at 3:15 AM on May 13, 2026: contributor

    Code review ACK 8511c68f8b6827f4e9b306e0245929965a363930

    Reviewed each commit separately, compiled and ran tests. The changes look good to me, only left a couple of inline nits noting a compilation error + failing tests in the first commit.

  22. DrahtBot requested review from ryanofsky on May 13, 2026
  23. enirox001 force-pushed on May 14, 2026
  24. enirox001 commented at 8:43 AM on May 14, 2026: contributor

    Thanks for the review @xyzconstant . Fixed both commit-structure issues: the first test commit now only adds baseline ListenConnections() coverage using the existing 3-argument API, and the max_connections setup/test now lands with the feature commit that introduces the new parameter

  25. enirox001 commented at 8:44 AM on May 14, 2026: contributor

    Also tightened the capped listener behavior. It now stops posting accepts once the limit is reached, and keeps capped pending accepts alive so later clients can connect after an idle gap, including before the cap has been reached.

    Added coverage for the reconnect cases as well

  26. enirox001 force-pushed on May 14, 2026
  27. in include/mp/proxy-io.h:898 in 19e1386bca
     899 | -            _Serve<InitInterface>(loop, kj::mv(stream), init);
     900 | -            _Listen<InitInterface>(loop, kj::mv(listener), init);
     901 | +        [&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable {
     902 | +            state->accept_pending = false;
     903 | +            _ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream));
     904 | +            if (_ListenAtCapacity(*state)) return;
    


    xyzconstant commented at 3:13 AM on May 19, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (19e1386bca3cee7e2d5cc775b74eac064d522fed)

    Not sure if I'm missing something but this check here seems redundant with the same _ListenAtCapacity check at the start of _Listen (line 889).

    I tested commenting this line out and left the upper-level check (and vice-versa) and the tests passed with no issues. I'd suggest dropping any of these duplicates.


    enirox001 commented at 9:06 AM on May 19, 2026:

    Good catch, thanks. Dropped the lower _ListenAtCapacity() check since _Listen() already handles the capacity check before posting another accept.

  28. enirox001 force-pushed on May 19, 2026
  29. in include/mp/proxy-io.h:898 in b0207dd406 outdated
     899 | -            _Serve<InitInterface>(loop, kj::mv(stream), init);
     900 | -            _Listen<InitInterface>(loop, kj::mv(listener), init);
     901 | +        [&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable {
     902 | +            state->accept_pending = false;
     903 | +            _ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream));
     904 | +            _Listen<InitInterface>(loop, init, state);
    


    xyzconstant commented at 7:56 PM on May 20, 2026:

    With the accept_pending check in place, this _Listen call will never be reached.

    NOTE: if you apply this patch here (comment), then it will be needed here because we can't rely on the second _Listen call in _ServeAccepted which is executed conditionally after active_connections reached the cap.

  30. xyzconstant commented at 8:13 PM on May 20, 2026: contributor

    Thanks for the update @enirox001!

    I've been playing around with the PR code more throughly this time and have a different take on accept_pending now, so left a few comments.

    Overall the code is factually correct and works as expected. And I think it could be merged (despite my latest thoughts on accept_pending). So I'll just re-ACK b0207dd406b206e0cf2835f10e36dd0ad5a34c96

  31. enirox001 force-pushed on May 21, 2026
  32. enirox001 force-pushed on May 21, 2026
  33. enirox001 commented at 11:43 AM on May 21, 2026: contributor

    I've been playing around with the PR code more throughly this time and have a different take on accept_pending now, so left a few comments.

    Thanks for taking another look @xyzconstant

    I dropped accept_pending and switched to the simpler invariant you suggested.

  34. xyzconstant commented at 4:20 PM on May 21, 2026: contributor

    re-ACK 7e51ab235330d1a198b910871fac09a23ffdd5f4

    Thanks for the updates @enirox001!

  35. in test/mp/test/listen_tests.cpp:93 in d0cff019a9 outdated
      88 | +class ClientSetup
      89 | +{
      90 | +public:
      91 | +    explicit ClientSetup(int fd)
      92 | +        : thread([this, fd] {
      93 | +              EventLoop loop("mptest-client", [](mp::LogMessage log) {
    


    ryanofsky commented at 3:51 PM on June 2, 2026:

    In commit "test: add dedicated ListenConnections coverage" (d0cff019a9b543b0f1fb7db86fad465aa0cc9d2e)

    Here and below, it might be good to add KJ_LOG(INFO, log.level, log.message); to show debug logs when --verbose is used, like the other test file.

    I think it could also be good to add a more generic test setup class that can be reused in tests and deduplicate the EventLoop thread/join/promise code that is now repeated in 3 classes. It could have std::function hooks for running code before loop.loop(), or when messages are logged. But this is more of an idea for a followup, current code seems fine.


    enirox001 commented at 3:42 PM on June 3, 2026:

    Done. Added KJ_LOG(INFO, log.level, log.message); to the logging callbacks of EventLoop in both ClientSetup and ListenSetup to ensure debug logs print correctly under --verbose .

    Regarding the generic test setup helper, that is a good suggestion. I agree it would be a clean follow-up improvement.

  36. in test/mp/test/listen_tests.cpp:121 in d0cff019a9
     116 | +public:
     117 | +    ListenSetup()
     118 | +        : thread([this] {
     119 | +              EventLoop loop("mptest-server", [this](mp::LogMessage log) {
     120 | +                  if (log.level == mp::Log::Raise) throw std::runtime_error(log.message);
     121 | +                  if (log.message.find("IPC server: socket connected.") != std::string::npos) {
    


    ryanofsky commented at 3:57 PM on June 2, 2026:

    In commit "test: add dedicated ListenConnections coverage" (d0cff019a9b543b0f1fb7db86fad465aa0cc9d2e)

    Maybe it would be good to add a testing hook for this (grep for testing_hook_ for examples), instead of checking for a log message.

    Checking for a log message does seem ok though, and may be even better if the goal is to catch regressions since it doesn't require changing test and non-test code at the same time.


    enirox001 commented at 3:42 PM on June 3, 2026:

    Done. I added a testing_hook_connected member to EventLoop and invoked it inside _Serve .

    ListenSetup now hooks into this callback to count connections, which makes the tests more robust and consistent with the other testing_hook hooks in the codebase.

  37. in test/mp/test/listen_tests.cpp:111 in d0cff019a9 outdated
     106 | +        thread.join();
     107 | +    }
     108 | +
     109 | +    std::promise<std::unique_ptr<ProxyClient<messages::FooInterface>>> client_promise;
     110 | +    std::unique_ptr<ProxyClient<messages::FooInterface>> client;
     111 | +    std::thread thread;
    


    ryanofsky commented at 3:59 PM on June 2, 2026:

    In commit "test: add dedicated ListenConnections coverage" (d0cff019a9b543b0f1fb7db86fad465aa0cc9d2e)

    Other test class has a comment that may be useful here and below as well

        //! Thread variable should be after other struct members so the thread does
        //! not start until the other members are initialized.
    

    enirox001 commented at 3:42 PM on June 3, 2026:

    Thanks, added this comment where appropriate.

  38. in test/mp/test/test.cpp:8 in d0cff019a9
       4 | @@ -5,14 +5,15 @@
       5 |  #include <mp/test/foo.capnp.h>
       6 |  #include <mp/test/foo.capnp.proxy.h>
       7 |  
       8 | +#include <string.h> // NOLINT(modernize-deprecated-headers)
    


    ryanofsky commented at 4:02 PM on June 2, 2026:

    In commit "test: add dedicated ListenConnections coverage" (d0cff019a9b543b0f1fb7db86fad465aa0cc9d2e)

    Unclear what reason is for this change. Probably should be a separate commit if it's necessary


    enirox001 commented at 3:42 PM on June 3, 2026:

    Thanks, removed this header.

  39. in include/mp/version.h:27 in 7e51ab2353
      23 | @@ -24,7 +24,7 @@
      24 |  //! pointing at the prior merge commit. The /doc/versions.md file should also be
      25 |  //! updated, noting any significant or incompatible changes made since the
      26 |  //! previous version.
      27 | -#define MP_MAJOR_VERSION 10
      28 | +#define MP_MAJOR_VERSION 11
    


    ryanofsky commented at 4:16 PM on June 2, 2026:

    In commit "test: add dedicated ListenConnections coverage" (d0cff019a9b543b0f1fb7db86fad465aa0cc9d2e)

    Looks like this needs to be rebased since current version is already 11, and this should be bumping from 11>12.

    Also would suggest bumping version in an initial new commit before the other changes, instead of alongside them. (For example see b15d63e9d81c39d21d7f8040dbb901ca7756da0f from #274.) Splitting should the make the PR a little easier to review, and also allow creating a v11.0 tag that contains every commit MP_MAJOR_VERSION 11 set, and points at a merge.


    enirox001 commented at 3:42 PM on June 3, 2026:

    Done, split the version bumping from the other changes, and included it in the initial commit before other commits

  40. in include/mp/proxy-io.h:882 in 7e51ab2353 outdated
     880 | +{
     881 | +    return state.max_connections && *state.max_connections > 0 ? std::make_unique<EventLoopRef>(loop) : nullptr;
     882 | +}
     883 | +
     884 | +template <typename InitInterface, typename InitImpl>
     885 | +void _Listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<ListenState>& state)
    


    ryanofsky commented at 4:21 PM on June 2, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (7e51ab235330d1a198b910871fac09a23ffdd5f4)

    This function used to have a code comment "//! Given connection receiver and an init object, handle..." that looks like it became detached. Would be good to move it down next this function and update it ("Given init object and a state object containing a connection receiver, handle...").

    It might also be clearer to turn this into a ListenState method. (And if doing that, rename ListenState to Listener, rename listener to m_receiver, rename state to self, make ListenAtCapacity another method, etc). Just an idea though, I haven't studied this code enough yet.


    enirox001 commented at 3:42 PM on June 3, 2026:

    Done. The comment has been moved down above the _Listen function, and its wording has been updated to reflect the new init and state parameters as suggested.

    Regarding refactoring _Listen into a member method of ListenState. I decided to keep it as a free function instead, as it keeps it consistent with other functions in proxy-io.h.

    Would explore this refactor more subsequently

  41. in include/mp/proxy-io.h:865 in 7e51ab2353
     862 | +    return state.max_connections && state.active_connections >= *state.max_connections;
     863 | +}
     864 | +
     865 |  template <typename InitInterface, typename InitImpl>
     866 | -void _Listen(EventLoop& loop, kj::Own<kj::ConnectionReceiver>&& listener, InitImpl& init)
     867 | +void _ServeAccepted(EventLoop& loop, InitImpl& init, const std::shared_ptr<ListenState>& state, kj::Own<kj::AsyncIoStream>&& stream)
    


    ryanofsky commented at 4:36 PM on June 2, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (7e51ab235330d1a198b910871fac09a23ffdd5f4)

    Having an extra _ServeAccepted function that is only called one place seems to make this code more complicated for little benefit. Any reason not to call _Serve directly from _Listen like previous code did?


    enirox001 commented at 3:43 PM on June 3, 2026:

    I agree that this is redundant. I removed the _ServeAccepted helper function entirely and placed the logic directly inside the _Listen callback. This reduces indirection and keeps the flow clean.

  42. in test/mp/test/listen_tests.cpp:213 in 7e51ab2353 outdated
     208 | +    auto client1 = std::make_unique<ClientSetup>(setup.listener.Connect());
     209 | +    setup.WaitForConnectedCount(1);
     210 | +    KJ_EXPECT(client1->client->add(1, 2) == 3);
     211 | +
     212 | +    auto client2 = std::make_unique<ClientSetup>(setup.listener.Connect());
     213 | +    std::this_thread::sleep_for(std::chrono::milliseconds(100));
    


    ryanofsky commented at 4:47 PM on June 2, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (7e51ab235330d1a198b910871fac09a23ffdd5f4)

    Note: Sleep seems racy but harmless since that it shouldn't ever cause the test to fail when it should succeed, only to potentially succeed when it should fail. I can't think of a better way to check that a connection is NOT accepted other than maybe adding a testing_hook that exposes listening state, though so this might be the best approach for now.


    enirox001 commented at 3:43 PM on June 3, 2026:

    Since asserting that the second connection was not accepted inherently requires a brief window of time to pass,

    The short sleep seems to be the most straightforward and reliable approach to test this behavior without adding state tracking hooks. I've left the sleep in place.

  43. in include/mp/proxy-io.h:878 in 7e51ab2353 outdated
     876 | +    });
     877 | +}
     878 | +
     879 | +inline std::unique_ptr<EventLoopRef> _MakeCappedListenerRef(EventLoop& loop, const ListenState& state)
     880 | +{
     881 | +    return state.max_connections && *state.max_connections > 0 ? std::make_unique<EventLoopRef>(loop) : nullptr;
    


    ryanofsky commented at 4:57 PM on June 2, 2026:

    In commit "proxy: add local connection limit to ListenConnections" (7e51ab235330d1a198b910871fac09a23ffdd5f4)

    Would be helpful to have a code comment explaining this logic. I don't think I understand why it avoiding creating an EventLoopRef if there is no connection limit. I'm also not clear on why passing accept_ref around separately is needed. Naively I think I'd just expect ListenState to have an EventLoopRef member.


    enirox001 commented at 3:43 PM on June 3, 2026:

    Thanks, I added explanatory comments to both _MakeCappedListenerRef. I avoid holding EventLoopRef in the uncapped case because the loop is always listening, and holding a ref could prevent it from automatically exiting when all client connections close.

    Passing accept_ref in the callback instead of storing it as a member would bind the ref's lifetime directly to the active accept() task. This means that when the listener reaches capacity, we return early and don't register a new task, which automatically destroys the ref and lets the loop exit.

  44. ryanofsky commented at 5:04 PM on June 2, 2026: collaborator

    Finally spent some time looking at this (7e51ab235330d1a198b910871fac09a23ffdd5f4), and broadly the changes look very good. Left a few questions & suggestions and plan to review more

  45. DrahtBot requested review from ryanofsky on Jun 2, 2026
  46. enirox001 force-pushed on Jun 3, 2026
  47. enirox001 force-pushed on Jun 3, 2026
  48. enirox001 force-pushed on Jun 3, 2026
  49. enirox001 commented at 4:13 PM on June 3, 2026: contributor

    Thanks for the review @ryanofsky, made the following changes in the latest push:

    • Added the KJ_LOG for verbose logging
    • Added a testing_hook_connected hook to ListenSetup.
    • Added thread comments where appropriate
    • Removed the redundant string header
    • Added an initial commit to split the version bumping from other changes
    • Added back the removed comment in the _Listen helper function
    • Removed _ServeAccepted helper function and placed logic in _Listen callback
    • Added explanatory comments to _MakeCappedListenerRef
  50. DrahtBot added the label Needs rebase on Jun 11, 2026
  51. doc/version: Bump version 11 > 12 e5eba302cd
  52. test: add dedicated ListenConnections coverage
    Add a separate listen_tests.cpp file with reusable UnixListener, ClientSetup, and ListenSetup helpers for exercising ListenConnections() with real Unix domain sockets.
    
    The new test covers the baseline behavior that ListenConnections() accepts an incoming connection and serves requests over it. Keeping this coverage separate from the existing general proxy tests makes the socket listener setup easier to review and provides a clearer place to extend listener-specific behavior in follow-up commits.
    40b2e97b27
  53. proxy: add local connection limit to ListenConnections
    Add an optional max_connections parameter to ListenConnections() so a listener can stop accepting new connections after reaching a local connection cap and resume accepting after an existing connection disconnects.
    
    Implement the limit with listener-local state tracking the listening socket, maximum number of active connections, and whether an async accept() has already been posted. This keeps the limit scoped to the individual listener instead of introducing global EventLoop state.
    
    Extend the dedicated listener test coverage to verify that with max_connections=1 the first client is accepted normally, a second client is not accepted while the first remains connected, and the second client is accepted after the first disconnects.
    05ea22661d
  54. in include/mp/proxy-io.h:869 in 1e8776f3b0
     868 | +    std::optional<size_t> max_connections;
     869 | +    size_t active_connections{0};
     870 | +};
     871 | +
     872 | +template <typename InitInterface, typename InitImpl>
     873 | +void _Listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<ListenState>& state);
    


    xyzconstant commented at 3:08 AM on June 11, 2026:

    Residual forward declaration?


    enirox001 commented at 6:43 AM on June 14, 2026:

    Nice catch, it indeed was a residual forward declaration. I have removed it.

  55. enirox001 force-pushed on Jun 14, 2026
  56. mpgen: add missing includes for IWYU compliance
    Generated proxy-types source files use clientDestroy/serverDestroy and
    arrayPtr from kj/common.h, but only included them transitively. IWYU
    with --error requires direct includes, causing CI build failure.
    b8453b4b94
  57. enirox001 commented at 7:26 AM on June 14, 2026: contributor

    In the most recent push. Removed a residual declaration and rebased against the master branch.

    Also added a fix for an IWYU failure in the gen.cpp file. Included here to make the CI pass. Happy to split it into a separate PR if preferred.

  58. DrahtBot removed the label Needs rebase on Jun 14, 2026


ryanofsky


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin-core/libmultiprocess. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-06-24 06:30 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me