qa: Intermittent failure in `interface_http.py` #36216

issue hebasto opened this issue on September 10, 2026
  1. hebasto commented at 10:37 AM on September 10, 2026: member

    On macOS 26.5.2 with Xcode 27, https://github.com/hebasto/bitcoin-core-nightly/actions/runs/34438869701/job/102749511737:

    330/496 - interface_http.py failed (exit code 1), Duration: 44 s
    
    stdout:
    2026-09-10T05:12:20.488909Z TestFramework (INFO): PRNG seed is: 4893840930825940378
    2026-09-10T05:12:20.495084Z TestFramework (INFO): Initializing test directory /var/folders/k3/mj0q6tms0r5drxp5z6qzb94r0000gn/T/test_runner_₿_🏃_20260910_050643/interface_http_197
    2026-09-10T05:12:20.828178Z TestFramework (INFO): Checking default HTTP/1.1 connection persistence
    2026-09-10T05:12:20.828987Z TestFramework (INFO): Checking that another process cannot bind the HTTP listen port
    2026-09-10T05:12:20.829132Z TestFramework (INFO): Checking keep-alive connection persistence
    2026-09-10T05:12:20.829804Z TestFramework (INFO): Checking close connection after response
    2026-09-10T05:12:20.830111Z TestFramework (INFO): Checking excessive request size
    2026-09-10T05:12:20.830915Z TestFramework (INFO): Client received expected 400 response before connection was terminated
    2026-09-10T05:12:20.832726Z TestFramework (INFO): Client received expected 400 response before connection was terminated
    2026-09-10T05:12:21.218916Z TestFramework (INFO): Client did not finish sending request before connection was terminated
    2026-09-10T05:12:21.219088Z TestFramework (INFO): Send thread force-closed by test framework
    2026-09-10T05:12:21.219333Z TestFramework (INFO): Check pipelining
    2026-09-10T05:12:26.400447Z TestFramework (INFO): Check pipelining with invalid second request
    2026-09-10T05:12:31.498234Z TestFramework (INFO): Check HTTP request encoded with chunked transfer
    2026-09-10T05:12:31.586952Z TestFramework (INFO): Check excessive size HTTP request encoded with chunked transfer
    2026-09-10T05:12:31.610752Z TestFramework (INFO): Send thread was already closed by RST from server
    2026-09-10T05:12:31.610892Z TestFramework (INFO): Client did not finish sending request before connection was terminated
    2026-09-10T05:12:31.611155Z TestFramework (INFO): Check -rpcservertimeout
    2026-09-10T05:12:34.545836Z TestFramework (INFO): Check that -rpcservertimeout won't close on a delayed response
    2026-09-10T05:12:40.616297Z TestFramework (INFO): Check that requests without credentials return 401 Unauthorized with WWW-Authenticate
    2026-09-10T05:12:40.618334Z TestFramework (INFO): Check that incorrect credentials return 401 Unauthorized
    2026-09-10T05:12:40.931440Z TestFramework (INFO): Check that malformed Authorization headers return 401 Unauthorized
    2026-09-10T05:12:42.615927Z TestFramework (INFO): Check that unsafe or unsupported HTTP methods are rejected
    2026-09-10T05:12:42.620318Z TestFramework (INFO): Check that path traversal attempts are safely rejected
    2026-09-10T05:12:42.623933Z TestFramework (INFO): Check request smuggling is not possible
    2026-09-10T05:12:42.625705Z TestFramework (INFO): Check that duplicate Content-Length headers are handled
    2026-09-10T05:12:42.626443Z TestFramework (INFO): Check that null bytes in the URI are safely rejected
    2026-09-10T05:12:42.626902Z TestFramework (INFO): Check that requests with invalid HTTP versions are safely rejected
    2026-09-10T05:12:42.628606Z TestFramework (INFO): Check that requests with whitespace in headers are rejected
    2026-09-10T05:12:42.629337Z TestFramework (INFO): Client received expected 400 response before connection was terminated
    2026-09-10T05:12:42.630087Z TestFramework (INFO): Client received expected 400 response before connection was terminated
    2026-09-10T05:12:42.630246Z TestFramework (INFO): Check connection limits
    2026-09-10T05:12:42.630291Z TestFramework (INFO): Using connection limit: default (16)
    2026-09-10T05:12:48.197036Z TestFramework (INFO): Using connection limit: -rpcmaxconnections=64
    2026-09-10T05:12:53.902089Z TestFramework (INFO): Check that pipelined data is throttled while a request is in flight
    2026-09-10T05:13:04.881266Z TestFramework (ERROR): Unexpected exception:
    Traceback (most recent call last):
      File "/Users/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/test_framework/test_framework.py", line 145, in main
        self.run_test()
        ~~~~~~~~~~~~~^^
      File "/Users/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/build/test/functional/interface_http.py", line 157, in run_test
        self.check_pipelined_data_is_throttled()
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
      File "/Users/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/build/test/functional/interface_http.py", line 780, in check_pipelined_data_is_throttled
        raise AssertionError(
            f"Server kept reading pipelined data ({sent} bytes) while a "
            f"request was still in flight for {PROGRESS_TIMEOUT}s.")
    AssertionError: Server kept reading pipelined data (1479840 bytes) while a request was still in flight for 10s.
    2026-09-10T05:13:05.007060Z TestFramework (INFO): Not stopping nodes as test failed. The dangling processes will be cleaned up later.
    2026-09-10T05:13:05.007230Z TestFramework (WARNING): Not cleaning up dir /var/folders/k3/mj0q6tms0r5drxp5z6qzb94r0000gn/T/test_runner_₿_🏃_20260910_050643/interface_http_197
    2026-09-10T05:13:05.007335Z TestFramework (ERROR): Test failed. Test logging available at /var/folders/k3/mj0q6tms0r5drxp5z6qzb94r0000gn/T/test_runner_₿_🏃_20260910_050643/interface_http_197/test_framework.log
    2026-09-10T05:13:05.007531Z TestFramework (ERROR): 
    2026-09-10T05:13:05.007696Z TestFramework (ERROR): Hint: Call /Users/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/test/functional/combine_logs.py '/var/folders/k3/mj0q6tms0r5drxp5z6qzb94r0000gn/T/test_runner_₿_🏃_20260910_050643/interface_http_197' to consolidate all logs
    2026-09-10T05:13:05.007765Z TestFramework (ERROR): 
    2026-09-10T05:13:05.007796Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
    2026-09-10T05:13:05.007927Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
    2026-09-10T05:13:05.007974Z TestFramework (ERROR): 
    
    
    stderr:
    [node 0] Cleaning up leftover process
    
    
  2. hebasto commented at 10:41 AM on September 10, 2026: member
  3. fanquake added this to the milestone 32.0 on Sep 15, 2026
  4. 151henry151 commented at 4:06 AM on September 16, 2026: contributor

    Kernel buffering, not the throttle? Debug log: sent: 1438932, then +31564 and +9344, each <STALL_TIMEOUT apart (same in https://github.com/bitcoin/bitcoin/actions/runs/34902576637/job/104171732881). Each send() success resets stuck_since, so STALL_TIMEOUT never elapses and PROGRESS_TIMEOUT fires. setsockopt(SO_SNDBUF) on the client rules send-buffer growth in or out.

  5. pinheadmz commented at 3:35 PM on September 16, 2026: member

    @151henry151 I understand most of the words in your comment but I'm not getting your meaning. Are you interested in reviewing my solution? https://github.com/bitcoin/bitcoin/pull/36276

  6. 151henry151 commented at 5:14 PM on September 16, 2026: contributor

    In the failed runs the flood stalls, then two more small send()s succeed (+31564, +9344), which are each less than STALL_TIMEOUT of the last send, so stuck_since resets. So PROGRESS_TIMEOUT fires.

    I was asking whether the late bytes were kernel buffering or the server still reading. I'll review #36276 this evening

  7. maflcko added the label macOS on Sep 19, 2026
  8. maflcko added the label CI failed on Sep 19, 2026
  9. pinheadmz commented at 4:18 PM on September 20, 2026: member
  10. Tejender06 referenced this in commit d346541d42 on Sep 26, 2026
  11. Tejender06 referenced this in commit 25408e4a60 on Sep 26, 2026
  12. Tejender06 referenced this in commit f9b1dededb on Sep 26, 2026
  13. Tejender06 referenced this in commit 2138ccd7a9 on Sep 26, 2026
  14. Tejender06 referenced this in commit cf54cafab7 on Sep 26, 2026
  15. Tejender06 referenced this in commit 330c1a3ee7 on Sep 26, 2026
  16. Tejender06 referenced this in commit 99ce089a84 on Sep 26, 2026
  17. Tejender06 referenced this in commit 44e2342be4 on Sep 26, 2026
  18. achow101 closed this on Oct 5, 2026


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-10-24 09:51 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me