test: modernize interface_http and cover more libevent behavior #34772

pull pinheadmz wants to merge 3 commits into bitcoin:master from pinheadmz:rpctimeout-busy-idle changing 1 files +299 −165
  1. pinheadmz commented at 11:07 PM on March 7, 2026: member

    This is a follow-up to #32408 and a new prerequisite for #32061 in response to a few review comments there.

    New test: check_server_busy_idle_timeout()

    In #32061 (review) it is pointed out that the idle timeout set by -rpcservertimeout could disconnect a client unfairly if it was the server that was taking too long to process a response. That misbehavior was confirmed and #32061 was updated to match current libevent behavior. This PR asserts the current libevent behavior by adding another test using RPC waitforblock past the -rpcservertimeout value and then verifying that the HTTP connection is still open.

    Expanded tests: check_excessive_request_size() and check_chunked_transfer()

    #32061 (review) made me realize that I was not testing HTTPRequest body size at all, and that headers size was being tested one line at a time but not in total. The libevent server does both things so that behavior is asserted in these expanded tests, and the mechanism in #32061 was improved to match.

    Clean up interface_http.py

    Since I am extending this test module again I refactored the monolithic test to resemble the current functional test style in the repository, de-duplicating HTTP connection code with a helper class and separating individual test cases into functions.

  2. DrahtBot added the label Tests on Mar 7, 2026
  3. DrahtBot commented at 11:08 PM on March 7, 2026: contributor

    <!--e57a25ab6845829454e8d69fc972939a-->

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    <!--021abf342d371248e50ceaed478a90ca-->

    Reviews

    See the guideline for information on the review process.

    Type Reviewers
    ACK Bortlesboat, fjahr, theStack
    Concept ACK AndreaDiazCorreia, hodlinator
    Stale ACK vasild

    If your review is incorrectly listed, please copy-paste <code>&lt;!--meta-tag:bot-skip--&gt;</code> into the comment that the bot should ignore.

    <!--174a7506f384e20aa4161008e828411d-->

    Conflicts

    No conflicts as of last run.

    <!--5faf32d7da4f0f540f40219e4f7537a3-->

  4. pinheadmz force-pushed on Mar 7, 2026
  5. DrahtBot added the label CI failed on Mar 7, 2026
  6. DrahtBot removed the label CI failed on Mar 8, 2026
  7. Bortlesboat commented at 6:59 PM on March 8, 2026: none

    Tested ACK b2bfeabf4c. Checked out the branch, ran interface_http.py locally — all passes. The BitcoinHTTPConnection helper cleans up the test nicely and the new check_server_busy_idle_timeout case is a good addition for the rpcservertimeout behavior. Found one minor typo (inline).

  8. in test/functional/interface_http.py:246 in b2bfeabf4c
     261 | +
     262 | +        # Sanity check -- complete requests don't timeout waiting for completion
     263 | +        good_http_request = "GET /test2 HTTP/1.1\r\nHost: somehost\r\n\r\n"
     264 | +        conn.reset_conn()
     265 | +        conn.conn.sock.sendall(good_http_request.encode("utf-8"))
     266 | +        respose2 = conn.recv_raw()
    


    Bortlesboat commented at 6:59 PM on March 8, 2026:

    Nit: typo respose2 -> response2


    pinheadmz commented at 2:36 PM on March 18, 2026:

    Thanks, will fix in next rebase

  9. pinheadmz marked this as a draft on Mar 19, 2026
  10. pinheadmz force-pushed on Mar 23, 2026
  11. pinheadmz renamed this:
    test: clean up interface_http and improve idle timeout coverage
    test: modernize interface_http and cover more libevent behavior
    on Mar 23, 2026
  12. pinheadmz marked this as ready for review on Mar 23, 2026
  13. DrahtBot added the label CI failed on Mar 23, 2026
  14. pinheadmz force-pushed on Mar 23, 2026
  15. DrahtBot removed the label CI failed on Mar 23, 2026
  16. in test/functional/interface_http.py:44 in af80c9b710 outdated
      39 | +        self.conn.close()
      40 | +
      41 | +    def set_timeout(self, seconds):
      42 | +        self.conn.sock.settimeout(seconds)
      43 | +
      44 | +    def add_header(self, key, value):
    


    fjahr commented at 9:57 PM on March 24, 2026:

    It seems this is unused here


    pinheadmz commented at 6:24 PM on March 25, 2026:

    It is used in a following commit in the PR: test: ensure HTTP server enforces limits on headers and body size L168 in this file. I'll remove here and add back in the commit where it is used.

  17. in test/functional/interface_http.py:83 in af80c9b710 outdated
      78 | +        # Server disconnected within an acceptable range of time:
      79 | +        # not immediately, and not too far over the configured duration.
      80 | +        # This allows for some jitter in the test between client and server.
      81 | +        duration = stop - start
      82 | +        assert duration <= seconds + 2, f"Server disconnected too slow: {duration} > {seconds}"
      83 | +        assert duration >= seconds - 2, f"Server disconnected too fast: {duration} < {seconds}"
    


    fjahr commented at 10:03 PM on March 24, 2026:

    This assertion is as bit funny because with the current values the connection would need to time travel (duration negative) for this to fail :)


    pinheadmz commented at 6:54 PM on March 25, 2026:

    lol whoops and thanks. I had these boundaries hard-coded to 4 and 1 previously and for some reason I thought that was +/-2 🤷

    Changed this to - 1 for the lower bound.

  18. in test/functional/interface_http.py:270 in 7cf55430a5 outdated
     265 | +        # Wait until after the timeout, then generate a block with a second HTTP connection
     266 | +        time.sleep(3)
     267 | +        generated_block = self.generate(self.nodes[2], 1, sync_fun=self.no_op)[0]
     268 | +
     269 | +        # The first connection gets the response it is patiently waiting for
     270 | +        response1 = conn.recv_raw().decode()
    


    fjahr commented at 10:16 PM on March 24, 2026:

    This seems racy, shouldn't this use a loop like the pipelining test? E.g. something like this:

    res = b""
    while generated_block.encode() not in res:
        res += conn.recv_raw()
    

    But this isn't very patient, it's nervously checking as many times as it can ;)


    vasild commented at 5:43 AM on March 25, 2026:

    recv_raw() with blocking sockets will wait efficiently for something to arrive on the socket. Where is the race? The server may respond with something else to the waitforblockheight instead of the block generated by self.generate()?


    fjahr commented at 11:50 AM on March 25, 2026:

    Good point on the blocking socket. Maybe raciness isn't the right word then but indeed I was also not sure if we can be certain that the "something" that arrives is the block and also the full block. Not sure what else server might send and if we need to account for this. And I guess the block here is well below 1024 bytes because the test is not putting anything into it either. So feel free to ignore if this isn't a real issue. In that case I would still be happy if you could add a small comment why this is ok to do here.


    pinheadmz commented at 10:28 AM on March 26, 2026:

    Comment added!

  19. in test/functional/interface_http.py:266 in 7cf55430a5 outdated
     261 | +        tip_height = self.nodes[2].getblockcount()
     262 | +        conn = BitcoinHTTPConnection(self.nodes[2])
     263 | +        conn.post_raw('/', f'{{"method": "waitforblockheight", "params": [{tip_height + 1}]}}')
     264 | +
     265 | +        # Wait until after the timeout, then generate a block with a second HTTP connection
     266 | +        time.sleep(3)
    


    fjahr commented at 10:21 PM on March 24, 2026:

    Maybe a bit tight with just one second margin given our many CI issues? Maybe it would also make sense to keep it oriented on the timeout value time.sleep(RPCSERVERTIMEOUT*2)?


    vasild commented at 5:30 AM on March 25, 2026:

    Such sleep() calls are better avoided - 1. eventually they fail on "odd" or slow CI machines and turn out not to be enough and 2. most of the time they unnecessary slow down the test, when e.g. waiting 0.5s would have been enough but the code waited 2.5s more.

    Should be possible to use mocktime here?


    pinheadmz commented at 10:25 AM on March 26, 2026:

    I'm not sure mocktime would work here because the timer we are comparing against is inside libevent set with evhttp_set_timeout().

    And yeah the concern about using sleep() on CI machines is real but we aren't waiting for some computation to complete that we think should take 2 seconds on an average machine --sleep() should guarantee at least 3 seconds and libevent timer should guarantee at least 2 seconds.

    I will change the value to RPCSERVERTIMEOUT + 1 for clarity, and maybe we can fine-tune this more once we've replaced libevent!

  20. in test/functional/interface_http.py:36 in af80c9b710 outdated
      31 | +            self.conn.request('GET', '/')
      32 | +            self.conn.getresponse().read()
      33 | +            return False
      34 | +        #       macos/linux           windows
      35 | +        except (ConnectionResetError, ConnectionAbortedError):
      36 | +            return True
    


    fjahr commented at 10:25 PM on March 24, 2026:

    I think we could use select to be more efficient here:

    import select
    import socket
    
        def sock_closed(self):
            if self.conn.sock is None:
                return True
            r, _, _ = select.select([self.conn.sock], [], [], 0)
            if r:
                data = self.conn.sock.recv(1, socket.MSG_PEEK)
                return len(data) == 0
            return False
    

    vasild commented at 5:18 AM on March 25, 2026:

    This will check whether the socket is ready to receive, i.e. if there is any data pending to be received on the socket. There is SocketIsClosed() in src/test/sock_tests.cpp which may be applicable here.


    pinheadmz commented at 6:21 PM on March 25, 2026:

    Interesting, I tried a python equivalent but it didn't work:

    from socket import SOL_SOCKET, SO_TYPE, SO_ERROR
    return self.conn.sock.getsockopt(SOL_SOCKET, SO_TYPE) == SO_ERROR
    

    I think the reason is because I can only test the client-side socket, but it's the server-side that has closed due to an error. I think this is verified by a unit test like this:

    BOOST_AUTO_TEST_CASE(close_socket)
    {
        int s[2];
        CreateSocketPair(s);
    
        Sock* sock0 = new Sock(s[0]);
        Sock* sock1 = new Sock(s[1]);
    
        SendAndRecvMessage(*sock0, *sock1);
    
        // server closes
        close(s[0]);
    
        // server is closed but client is still open
        BOOST_CHECK(SocketIsClosed(s[0]));
        BOOST_CHECK(!SocketIsClosed(s[1]));
    
        // ERROR - we have been hung up on
        SendAndRecvMessage(*sock0, *sock1);
    }
    

    Incidentally I did have a hard time testing "have we been hung up on" for this method and I got a suggestion from ChatGPT with the select mechanism but decided against it. I'd love something more clear for the test but what's here has been the best so far for readability and consistency at all call sites testing on master and #32061.

  21. in test/functional/interface_http.py:167 in 72992f5390
     163 | +        response2 = conn.get(f'/{"x"*8200}')
     164 |          assert_equal(response2.status, http.client.BAD_REQUEST)
     165 |  
     166 | +        # Many small header lines is ok
     167 | +        conn = BitcoinHTTPConnection(self.nodes[2])
     168 | +        for i in range(400):
    


    fjahr commented at 10:38 PM on March 24, 2026:

    These magic values above and below seem pretty arbitrary. 400 is ok but 600 is not, maybe either add comments for these and following numbers why they are chosen. It likely could be a much narrower bound and the comment might mention the threshold we are working with explicitly.


    pinheadmz commented at 7:49 PM on March 25, 2026:

    Ok I'll add comments and a little math that determine the constants. There is some fuzziness here because we don't know exactly what other headers http.client adds by default and, somewhat mysteriously, libevent doesn't seem to reject a request until it's about 1,000 bytes over the configured limit (!).

  22. in test/functional/interface_http.py:181 in 72992f5390
     177 | +        response3 = conn.get('/x')
     178 | +        assert_equal(response3.status, http.client.BAD_REQUEST)
     179 | +
     180 | +        # Large request body size is ok
     181 | +        conn = BitcoinHTTPConnection(self.nodes[0])
     182 | +        response4 = conn.post('/', f'{{"jsonrpc": "2.0", "id": "0", "method": "submitblock", "params": ["{"0"*0x01ffff00}"]}}')
    


    fjahr commented at 10:40 PM on March 24, 2026:

    nit: At first sight I thought this formatting was broken. It works but maybe ' + "0"*0x01ffff00 + ' may be a bit more readable. Bonus for giving 0x01ffff00 a variable with a name that makes it easier to reason about (connected to my call to explain the magic numbers above).


    pinheadmz commented at 8:03 PM on March 25, 2026:

    Like above I'll spell out the reasoning for the constants and make it easier to read.

  23. fjahr commented at 10:43 PM on March 24, 2026: contributor

    Concept ACK

  24. in test/functional/interface_http.py:59 in 72992f5390
      54 | +    def get(self, path, connection_header=None):
      55 | +        headers = self.headers
      56 | +        if connection_header is not None:
      57 | +            headers["Connection"] = connection_header
      58 | +        self.conn.request('GET', path, '', headers)
      59 | +        return self.conn.getresponse()
    


    vasild commented at 5:19 AM on March 25, 2026:

    Can do some minor deduplication - have one function that takes also the method name and then get() and post() call that one.


    pinheadmz commented at 6:38 PM on March 25, 2026:

    Sure, moved the common lines to internal _request() method

  25. in test/functional/interface_http.py:187 in 72992f5390
     268 | +        assert_equal(response4.status, http.client.OK)
     269 | +
     270 | +        conn = BitcoinHTTPConnection(self.nodes[1])
     271 | +        try:
     272 | +            # Excessive body size is invalid
     273 | +            response5 = conn.post('/', f'{{"jsonrpc": "2.0", "id": "0", "method": "submitblock", "params": ["{"0"*0x02000000}"]}}')
    


    vasild commented at 5:46 AM on March 25, 2026:

    There are a lot of "sizes" hardcoded around in this test. They somehow relate to constants in the C++ code. It is difficult to assess which limit is a given test exercising. Would be more clear like:

    MAX_WHATEVER = 1000 # a constant with the same name exists in the C++ code
    ...
    test MAX_WHATEVER is ok
    test MAX_WHATEVER + 1 is error
    

    pinheadmz commented at 8:04 PM on March 25, 2026:

    Spelling this out, thanks.

  26. pinheadmz force-pushed on Mar 26, 2026
  27. pinheadmz commented at 11:28 AM on March 26, 2026: member

    push to e9653667f22c982d873452281a45223fa4e34eb8

    Address feedback from @vasild and @fjahr, adding comments and cleaning up the commits.

    There is one or two "funny" parts of this test due to libevent which I expect to remove in #32061

  28. in test/functional/interface_http.py:64 in e9653667f2
      59 | +
      60 | +    def post(self, path, data, connection_header=None, **kwargs):
      61 | +        return self._request('POST', path, data, connection_header, **kwargs)
      62 | +
      63 | +    def get(self, path, connection_header=None):
      64 | +        return self._request('POST', path, '', connection_header)
    


    vasild commented at 10:00 AM on March 30, 2026:
        def get(self, path, connection_header=None):
            return self._request('GET', path, '', connection_header)
    

    pinheadmz commented at 10:42 AM on March 30, 2026:

    Wow whoops. Seems like a mistake that should have broken the test...

  29. in test/functional/interface_http.py:175 in e9653667f2
     256 | +        conn = BitcoinHTTPConnection(self.nodes[1])
     257 | +        response2 = conn.get(f'/{"x" * MAX_HEADERS_SIZE}')
     258 | +        assert_equal(response2.status, http.client.BAD_REQUEST)
     259 | +
     260 | +        # Compute how many short header lines needed to add to http.client
     261 | +        # default headers to make / break the total limit in a single reuqest.
    


    vasild commented at 10:02 AM on March 30, 2026:
            # default headers to make / break the total limit in a single request.
    
  30. in test/functional/interface_http.py:174 in e9653667f2
     255 | +        # Excessive URI size plus default headers breaks the limit.
     256 | +        conn = BitcoinHTTPConnection(self.nodes[1])
     257 | +        response2 = conn.get(f'/{"x" * MAX_HEADERS_SIZE}')
     258 | +        assert_equal(response2.status, http.client.BAD_REQUEST)
     259 | +
     260 | +        # Compute how many short header lines needed to add to http.client
    


    vasild commented at 10:03 AM on March 30, 2026:
            # Compute how many short header lines need to be added to http.client
    
  31. vasild approved
  32. vasild commented at 10:10 AM on March 30, 2026: contributor

    ACK e9653667f22c982d873452281a45223fa4e34eb8

  33. DrahtBot requested review from Bortlesboat on Mar 30, 2026
  34. DrahtBot requested review from fjahr on Mar 30, 2026
  35. pinheadmz force-pushed on Mar 30, 2026
  36. pinheadmz commented at 2:44 PM on March 30, 2026: member

    push to 36e47ad517271c9a21d069a69f59edc90e5b444a

    Address feedback from @vasild -- thanks!

  37. in test/functional/interface_http.py:147 in 2cee5dc18c outdated
     228 | +
     229 | +
     230 | +    def check_close_connection(self):
     231 | +        self.log.info("Checking close connection after response")
     232 | +        conn = BitcoinHTTPConnection(self.nodes[0])
     233 | +        # Make request with explicit "Connection: keep-alive" header
    


    fjahr commented at 6:36 PM on April 2, 2026:

    This should probably say:

            # Make request with explicit "Connection: close" header
    

    pinheadmz commented at 7:11 PM on April 6, 2026:

    thanks, fixed the comment here

  38. in test/functional/interface_http.py:134 in 2cee5dc18c outdated
     215 | +        # Make request with explicit "Connection: keep-alive" header
     216 | +        response1 = conn.post('/', '{"method": "getbestblockhash"}', connection_header='keep-alive').read()
     217 | +        assert b'"error":null' in response1
     218 | +        # Connection still open after request
     219 | +        assert not conn.sock_closed()
     220 | +        # Make second request without explicit "Connection" header
    


    fjahr commented at 6:46 PM on April 2, 2026:

    This is currently not the case as the connection/keepalive header is still included afaict, see related comment on _request.


    pinheadmz commented at 7:19 PM on April 6, 2026:

    I ran the test from master and branch through wireshark to compare the transcripts and actually, the original test does add the keep-alive header for both RPCs, essentially:

    headers = {"Authorization": f"Basic {str_to_b64str(authpair)}", "Connection": "keep-alive"}
    
    conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
    
    #send 2nd request without closing connection
    conn.request('POST', '/', '{"method": "getchaintips"}', headers)
    

    So for the sake of consistency with the original test (this commit is just a cleanup/refactor) I will fix the comment, and then explicitly add the header for the second RPC.

    You are also right about the reference/copy above... so I ended up accidentally getting this test right by being wrong about two opposite things 🏆 🤷

  39. in test/functional/interface_http.py:49 in 2cee5dc18c outdated
      44 | +
      45 | +    def set_timeout(self, seconds):
      46 | +        self.conn.sock.settimeout(seconds)
      47 | +
      48 | +    def _request(self, method, path, data, connection_header, **kwargs):
      49 | +        headers = self.headers
    


    fjahr commented at 6:49 PM on April 2, 2026:

    Hm, this is a reference, not a copy. So either this line isn't necessary or, more likely, I think you intended to do this:

            headers = self.headers.copy()
    

    This prevents the keepalive test from working as intended, see my separate comment referencing this here. Using some extra logging I confirmed that this is indeed a reference and including the keepalive in the second request on this example. Making the change above corrects the behavior to work as the comments seems to intend.


    pinheadmz commented at 7:21 PM on April 6, 2026:

    Great catch thanks! Changing this reference to a copy so headers must be explicitly set for each request.

  40. in test/functional/interface_http.py:305 in 36e47ad517 outdated
     300 | +                url='/',
     301 | +                body=iter(body_chunked),
     302 | +                headers=headers_chunked,
     303 | +                encode_chunked=True)
     304 | +            # Depending on race condition, server will either send a response and disconnect...
     305 | +            response2.read()
    


    fjahr commented at 7:15 PM on April 2, 2026:

    This looks weird to how differently this works with the request and recv_raw above (L276-282).

    It seems to me like if the request here wouldn't raise as expected this would raise unexpectedly on this line because request seems to always return None: https://docs.python.org/3/library/http.client.html#http.client.HTTPConnection.request So this should probably follow the same patterns as above with recv_raw() or getresponse(). But if this is seemingly never actually hit so far maybe this isn't even needed at all?


    pinheadmz commented at 2:55 PM on April 7, 2026:

    Really good catch! I fixed the bug here checking the response from the server and improved the comment. That logic can be forced by diffing the test as below.

    Interestingly, I have not been able to trigger that condition "naturally" on either libevent or #32061 on macos or linux. So there might not be a race condition at all in this instance. That could be because the client is still in "send" mode, trying to send its big chunks when the server sends 400 and disconnects.

    The other test that has this 400 vs BrokenPipe race is less reliable. In "Checking excessive request size" the libevent server is more likely to result in an immediate disconnect but the new server in #32061 is more likely to get the 400 response through before disconnecting.

    So I think I will leave the try/catch in both places in this PR so in #32061 the test doesn't have to change and that will assert more-or-less consistent behavior.

    diff --git a/test/functional/interface_http.py b/test/functional/interface_http.py
    index ff6d70a03a..b1e38dba1d 100755
    --- a/test/functional/interface_http.py
    +++ b/test/functional/interface_http.py
    @@ -290,10 +290,10 @@ class HTTPBasicsTest (BitcoinTestFramework):
             headers_chunked.update({"Transfer-encoding": "chunked"})
             body_chunked = [
                 b'{"method": "submitblock", "params": ["',
    -            b'0' * 10000000,
    -            b'1' * 10000000,
    -            b'2' * 10000000,
    -            b'3' * 10000000,
    +            b'0' * 100000,
    +            b'1' * 100000,
    +            b'2' * 100000,
    +            b'3' * 100000,
                 b'"]}'
             ]
             try:
    
  41. fjahr commented at 7:17 PM on April 2, 2026: contributor

    Getting close 🤏

  42. AndreaDiazCorreia commented at 3:15 AM on April 4, 2026: none

    Concept ACK 36e47ad I reviewed the code commit by commit and ran interface_http.py locally, all tests pass. The refactor in the first commit and the new tests look solid. I also noticed the two issues @fjahr pointed out (_request() mutating headers and the typo in check_close_connection()), and I agree those should be fixed.

  43. test: clean up and modernize interface_http f06de5c1ea
  44. test: ensure HTTP server timeout is not caused by a delayed response 0c1a07e890
  45. test: ensure HTTP server enforces limits on headers and body size 422ca211ec
  46. pinheadmz force-pushed on Apr 7, 2026
  47. pinheadmz commented at 3:18 PM on April 7, 2026: member

    push to 422ca211ec081390d60b4d6b192c4b3f0b2c9f3d

    Address feedback from @fjahr, a few nits but also a few bugs that would affect honest test coverage. After fixing, I captured the test on master and compared with the first commit from this branch in wireshark to double check that all the HTTP headers were the same in the cleaned up version of the test. I can provide more details about that if anyones interested but heres the transcripts:

    branch.txt master.txt

  48. Bortlesboat commented at 4:33 PM on April 7, 2026: none

    re-ACK 422ca211ec08. Checked out the rebased branch and ran interface_http.py locally, passes clean. Went through all three commits -- the helper class deduplication reads well and check_server_busy_idle_timeout is a nice addition from the #32061 discussion. One minor note inline.

  49. in test/functional/interface_http.py:69 in 422ca211ec
      64 | +        return self._request('GET', path, '', connection_header)
      65 | +
      66 | +    def post_raw(self, path, data):
      67 | +        req = f"POST {path} HTTP/1.1\r\n"
      68 | +        req += f'Authorization: Basic {str_to_b64str(self.authpair)}\r\n'
      69 | +        req += f'Content-Length: {len(data)}\r\n\r\n'
    


    Bortlesboat commented at 4:33 PM on April 7, 2026:

    minor: the raw request here is HTTP/1.1 but doesn't include a Host header. Technically required by the spec, though libevent doesn't seem to care. Not blocking, just noticed it while reading.


    pinheadmz commented at 5:15 PM on April 7, 2026:

    Wow great catch and actually this is something that is missing from #32061. However, you are right that libevent does not care:

    static void
    evhttp_handle_request(struct evhttp_request *req, void *arg)
    {
    ...
    	/* handle potential virtual hosts */
    	hostname = evhttp_request_get_host(req);
    	if (hostname != NULL) {
    		evhttp_find_vhost(http, &http, hostname);
    	}
    

    So it makes sense for us to be lenient as well when we replace libevent's HTTP server... and this test now conveniently asserts that leniency.

  50. DrahtBot requested review from fjahr on Apr 7, 2026
  51. DrahtBot requested review from AndreaDiazCorreia on Apr 7, 2026
  52. DrahtBot requested review from vasild on Apr 7, 2026
  53. fjahr commented at 7:47 PM on April 7, 2026: contributor

    ACK 422ca211ec081390d60b4d6b192c4b3f0b2c9f3d

    Thanks addressing my comments and being thorough with your responses :)

  54. in test/functional/interface_http.py:1 in f06de5c1ea


    hodlinator commented at 8:02 PM on April 10, 2026:

    f06de5c test: clean up and modernize interface_http:

    Commit message could mention that we are removing some leftover keepalive tests, or split that removal of those out into its own earlier 4th commit before the modernization of the surviving tests.

  55. in test/functional/interface_http.py:243 in f06de5c1ea
     237 | @@ -188,54 +238,29 @@ def run_test(self):
     238 |          # called for the remainder of this test.
     239 |          self.nodes[2].reuse_http_connections = False
     240 |  
     241 | -        self.restart_node(2, extra_args=["-rpcservertimeout=2"])
     242 |          # This is the amount of time the server will wait for a client to
     243 |          # send a complete request. Test it by sending an incomplete but
     244 | -        # so-far otherwise well-formed HTTP request, and never finishing it.
     245 | +        # so-far otherwise well-formed HTTP request, and never finishing it
    


    hodlinator commented at 8:08 PM on April 10, 2026:

    f06de5c test: clean up and modernize interface_http:

    nit: Accidentally removed period at end of comment.

  56. in test/functional/interface_http.py:347 in 422ca211ec
     363 | +
     364 | +        # Sanity check -- complete requests don't timeout waiting for completion
     365 | +        good_http_request = "GET /test2 HTTP/1.1\r\nHost: somehost\r\n\r\n"
     366 | +        conn.reset_conn()
     367 | +        conn.conn.sock.sendall(good_http_request.encode("utf-8"))
     368 | +        response2 = conn.recv_raw()
    


    hodlinator commented at 6:30 PM on April 11, 2026:

    nit: Only one response variable in function?

            response1 = conn.recv_raw()
    
  57. in test/functional/interface_http.py:97 in 422ca211ec
      92 | +        # The connection is definitely closed.
      93 | +        assert self.sock_closed()
      94 | +
      95 |  class HTTPBasicsTest (BitcoinTestFramework):
      96 |      def set_test_params(self):
      97 |          self.num_nodes = 3
    


    hodlinator commented at 6:34 PM on April 11, 2026:
            self.num_nodes = 1
    

    Test runs fine with 1 node (and doesn't need to override setup_network() to avoid connecting the nodes). Multiple nodes existed for historical reasons to test different keepalive settings from what I gathered.

    <details><summary>Diff</summary>

    diff --git a/test/functional/interface_http.py b/test/functional/interface_http.py
    index ff6d70a03a..189e1e8cd7 100755
    --- a/test/functional/interface_http.py
    +++ b/test/functional/interface_http.py
    @@ -94,11 +94,9 @@ class BitcoinHTTPConnection:
     
     class HTTPBasicsTest (BitcoinTestFramework):
         def set_test_params(self):
    -        self.num_nodes = 3
    +        self.num_nodes = 1
             self.supports_cli = False
     
    -    def setup_network(self):
    -        self.setup_nodes()
     
         def run_test(self):
             self.check_default_connection()
    @@ -167,7 +165,7 @@ class HTTPBasicsTest (BitcoinTestFramework):
             assert_equal(response1.status, http.client.NOT_FOUND)
     
             # Excessive URI size plus default headers breaks the limit.
    -        conn = BitcoinHTTPConnection(self.nodes[1])
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             response2 = conn.get(f'/{"x" * MAX_HEADERS_SIZE}')
             assert_equal(response2.status, http.client.BAD_REQUEST)
     
    @@ -183,14 +181,14 @@ class HTTPBasicsTest (BitcoinTestFramework):
             headers_above_limit += 1000 // header_line_length
     
             # Many small header lines is ok
    -        conn = BitcoinHTTPConnection(self.nodes[2])
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             for i in range(headers_below_limit):
                 conn.add_header(f"header_{i:04}", "foo")
             response3 = conn.get('/x')
             assert_equal(response3.status, http.client.NOT_FOUND)
     
             # Too many small header lines exceeds total headers size allowed
    -        conn = BitcoinHTTPConnection(self.nodes[2])
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             for i in range(headers_above_limit):
                 conn.add_header(f"header_{i:04}", "foo")
             response3 = conn.get('/x')
    @@ -207,7 +205,7 @@ class HTTPBasicsTest (BitcoinTestFramework):
             response4 = conn.post('/', f'{{"jsonrpc": "2.0", "id": "0", "method": "submitblock", "params": ["{"0" * bytes_below_limit}"]}}')
             assert_equal(response4.status, http.client.OK)
     
    -        conn = BitcoinHTTPConnection(self.nodes[1])
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             try:
                 # Excessive body size is invalid
                 response5 = conn.post('/', f'{{"jsonrpc": "2.0", "id": "0", "method": "submitblock", "params": ["{"0" * bytes_above_limit}"]}}')
    @@ -229,8 +227,8 @@ class HTTPBasicsTest (BitcoinTestFramework):
             See https://www.rfc-editor.org/rfc/rfc7230#section-6.3.2
             """
             self.log.info("Check pipelining")
    -        tip_height = self.nodes[2].getblockcount()
    -        conn = BitcoinHTTPConnection(self.nodes[2])
    +        tip_height = self.nodes[0].getblockcount()
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             conn.set_timeout(5)
     
             # Send two requests in a row.
    @@ -248,7 +246,7 @@ class HTTPBasicsTest (BitcoinTestFramework):
                 pass
     
             # Use a separate http connection to generate a block
    -        self.generate(self.nodes[2], 1, sync_fun=self.no_op)
    +        self.generate(self.nodes[0], 1, sync_fun=self.no_op)
     
             # Wait for two responses to be received
             res = b""
    @@ -264,7 +262,7 @@ class HTTPBasicsTest (BitcoinTestFramework):
     
         def check_chunked_transfer(self):
             self.log.info("Check HTTP request encoded with chunked transfer")
    -        conn = BitcoinHTTPConnection(self.nodes[2])
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             headers_chunked = conn.headers.copy()
             headers_chunked.update({"Transfer-encoding": "chunked"})
             body_chunked = [
    @@ -325,17 +323,17 @@ class HTTPBasicsTest (BitcoinTestFramework):
             # the server. Negating this setting will force the AuthServiceProxy
             # for this node to create a fresh new HTTP connection for every command
             # called for the remainder of this test.
    -        self.nodes[2].reuse_http_connections = False
    +        self.nodes[0].reuse_http_connections = False
     
             # This is the amount of time the server will wait for a client to
             # send a complete request. Test it by sending an incomplete but
             # so-far otherwise well-formed HTTP request, and never finishing it
    -        self.restart_node(2, extra_args=[f"-rpcservertimeout={RPCSERVERTIMEOUT}"])
    +        self.restart_node(0, extra_args=[f"-rpcservertimeout={RPCSERVERTIMEOUT}"])
     
             # Copied from http_incomplete_test_() in regress_http.c in libevent.
             # A complete request would have an additional "\r\n" at the end.
             bad_http_request = "GET /test1 HTTP/1.1\r\nHost: somehost\r\n"
    -        conn = BitcoinHTTPConnection(self.nodes[2])
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             conn.conn.sock.sendall(bad_http_request.encode("utf-8"))
     
             conn.expect_timeout(RPCSERVERTIMEOUT)
    @@ -353,13 +351,13 @@ class HTTPBasicsTest (BitcoinTestFramework):
     
         def check_server_busy_idle_timeout(self):
             self.log.info("Check that -rpcservertimeout won't close on a delayed response")
    -        tip_height = self.nodes[2].getblockcount()
    -        conn = BitcoinHTTPConnection(self.nodes[2])
    +        tip_height = self.nodes[0].getblockcount()
    +        conn = BitcoinHTTPConnection(self.nodes[0])
             conn.post_raw('/', f'{{"method": "waitforblockheight", "params": [{tip_height + 1}]}}')
     
             # Wait until after the timeout, then generate a block with a second HTTP connection
             time.sleep(RPCSERVERTIMEOUT + 1)
    -        generated_block = self.generate(self.nodes[2], 1, sync_fun=self.no_op)[0]
    +        generated_block = self.generate(self.nodes[0], 1, sync_fun=self.no_op)[0]
     
             # The first connection gets the response it is patiently waiting for
             response1 = conn.recv_raw().decode()
    

    </details>


    theStack commented at 5:21 PM on April 13, 2026:

    Multiple nodes existed for historical reasons to test different keepalive settings from what I gathered.

    Was also wondering about this and can confirm, there was apparently an -rpckeepalive setting more than a decade ago (introduced in #5655, removed again in #5677, for those interested in history :)).

  58. in test/functional/interface_http.py:273 in 0c1a07e890
     268 | +        tip_height = self.nodes[2].getblockcount()
     269 | +        conn = BitcoinHTTPConnection(self.nodes[2])
     270 | +        conn.post_raw('/', f'{{"method": "waitforblockheight", "params": [{tip_height + 1}]}}')
     271 | +
     272 | +        # Wait until after the timeout, then generate a block with a second HTTP connection
     273 | +        time.sleep(RPCSERVERTIMEOUT + 1)
    


    hodlinator commented at 6:47 PM on April 11, 2026:

    0c1a07e test: ensure HTTP server timeout is not caused by a delayed response:

    check_server_busy_idle_timeout() is dependent on the prior test running

    self.restart_node(2, extra_args=[f"-rpcservertimeout={RPCSERVERTIMEOUT}"])
    

    if I disable check_idle_timeout() then check_server_busy_idle_timeout() hangs.

    Would be nice to document the dependency in a comment, or return the node with shortened -rpcservertimeout from the earlier function and send it in to the later one. Or add an extra restart_node() in the later one to make them more independent.

  59. hodlinator commented at 7:14 PM on April 11, 2026: contributor

    Concept ACK 422ca211ec081390d60b4d6b192c4b3f0b2c9f3d

  60. theStack approved
  61. theStack commented at 6:10 PM on April 13, 2026: contributor

    ACK 422ca211ec081390d60b4d6b192c4b3f0b2c9f3d

    Happy to re-review if @hodlinator's review comments are tackled (here or in a follow-up).

  62. DrahtBot requested review from hodlinator on Apr 13, 2026
  63. fanquake merged this on Apr 14, 2026
  64. fanquake closed this on Apr 14, 2026

  65. in test/functional/interface_http.py:71 in 422ca211ec
      66 | +    def post_raw(self, path, data):
      67 | +        req = f"POST {path} HTTP/1.1\r\n"
      68 | +        req += f'Authorization: Basic {str_to_b64str(self.authpair)}\r\n'
      69 | +        req += f'Content-Length: {len(data)}\r\n\r\n'
      70 | +        req += data
      71 | +        self.conn.sock.sendall(req.encode("utf-8"))
    


    maflcko commented at 1:54 PM on April 14, 2026:

    doesn't matter here, but the encoding is technically wrong here.

    The Python len does not count the bytes, but something else.

    Using anything other than ascii-encoding will break with b'null,"error":{"code":-32700,"message":"Parse error"},"id":null}\n', so:

    diff --git a/test/functional/interface_http.py b/test/functional/interface_http.py
    index ff6d70a03a..148900dc25 100755
    --- a/test/functional/interface_http.py
    +++ b/test/functional/interface_http.py
    @@ -70,3 +70,3 @@ class BitcoinHTTPConnection:
             req += data
    -        self.conn.sock.sendall(req.encode("utf-8"))
    +        self.conn.sock.sendall(req.encode("ascii"))
     
    

    (all other utf-8 in this file can also be ascii, but for them utf-8 is also correct and not wrong)


    hodlinator commented at 6:11 PM on April 16, 2026:

    Would you be okay with this suggested alternative to ASCII #35086 (review)?


    maflcko commented at 6:26 PM on April 16, 2026:

    Sure, looks good. Just seems odd to use uf8 when it is not needed in the test. For the data it should be fine, maybe a test can echo an emoji? For the headers I am not sure, I think they must be ascii?

  66. in test/functional/interface_http.py:312 in 422ca211ec
     313 | +            # The server will send a 400 response and disconnect but
     314 | +            # due to a race condition, the python client may or may not
     315 | +            # receive the response before detecting the broken socket.
     316 | +            response2 = conn.conn.getresponse()
     317 | +            response2.read()
     318 | +            assert_equal(response2.status, http.client.BAD_REQUEST)
    


    maflcko commented at 2:10 PM on April 15, 2026:

    no idea how to reproduce this, but it seems to fail in https://github.com/maflcko/bitcoin-core-nightly/actions/runs/24438190980/job/71397012801#step:11:1626:

     test  2026-04-15T06:46:03.124196Z TestFramework (INFO): Check excessive size HTTP request encoded with chunked transfer 
     test  2026-04-15T06:46:03.198837Z TestFramework (ERROR): Unexpected exception: 
                                       Traceback (most recent call last):
                                         File "/__w/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/test_framework/test_framework.py", line 143, in main
                                           self.run_test()
                                         File "/__w/bitcoin-core-nightly/bitcoin-core-nightly/bld/test/functional/interface_http.py", line 109, in run_test
                                           self.check_chunked_transfer()
                                         File "/__w/bitcoin-core-nightly/bitcoin-core-nightly/bld/test/functional/interface_http.py", line 312, in check_chunked_transfer
                                           assert_equal(response2.status, http.client.BAD_REQUEST)
                                         File "/__w/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/test_framework/util.py", line 83, in assert_equal
                                           raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
                                       AssertionError: not(413 == 400)
    
    

    I tried this in a fresh ubuntu:noble container locally, but it passed:

    # history 
        1  export DEBIAN_FRONTEND=noninteractive && apt update && apt install curl wget htop git vim ccache -y && git clone https://github.com/bitcoin/bitcoin.git  --depth=1 ./b-c && cd b-c && apt install  build-essential cmake pkg-config  python3-zmq libzmq3-dev libevent-dev libboost-dev libsqlite3-dev  systemtap-sdt-dev  libcapnp-dev capnproto  libqrencode-dev qt6-tools-dev qt6-l10n-tools qt6-base-dev  clang llvm libc++-dev libc++abi-dev  mold -y 
        2  wget https://apt.llvm.org/llvm.sh 
        3  bash ./llvm.sh 23
     
        6  cmake -B bld-23     -DAPPEND_CXXFLAGS='-O3 -g2'       -DAPPEND_CFLAGS='-O3 -g2'       -DCMAKE_BUILD_TYPE=Debug -DAPPEND_CPPFLAGS='-D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES' -DCMAKE_C_COMPILER="clang-23" -DCMAKE_CXX_COMPILER="clang++-23;-stdlib=libstdc++"     --preset=dev-mode     -DCMAKE_COMPILE_WARNING_AS_ERROR=ON && cmake --build ./bld-23 --parallel  $(nproc) --target=bitcoind && ./bld-23/test/functional/interface_http.py
    
  67. pinheadmz commented at 8:05 PM on April 15, 2026: member

    Addressing the remaining comments here: https://github.com/bitcoin/bitcoin/pull/35086


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-21 09:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me