Skip to content

Skip lingering close when response declines keep-alive - #13245

Draft
Ranjeeth11 wants to merge 2 commits into
aio-libs:masterfrom
Ranjeeth11:skip-lingering-close-on-force-close
Draft

Skip lingering close when response declines keep-alive#13245
Ranjeeth11 wants to merge 2 commits into
aio-libs:masterfrom
Ranjeeth11:skip-lingering-close-on-force-close

Conversation

@Ranjeeth11

Copy link
Copy Markdown

What changed
RequestHandler's lingering-close gate in the request-handling loop now also checks self._keepalive before starting the wait to drain an unread request body. Previously it only checked self._force_close.

Why
self._keepalive is already set from the response's keep_alive value earlier in the same loop iteration, but wasn't consulted here — unlike two other nearby checks in the same file (_process_keepalive, and the keep-alive-timer-arming condition) that already treat "keep-alive declined" and "force closed" as equivalent grounds to skip further keep-alive-related work. A handler calling resp.force_close() still triggered the full lingering wait despite declining keep-alive.

User-visible behavior
Any response for which keep-alive is declined (resp.force_close(), Connection: close, or HTTP/1.0 without keep-alive) now skips the lingering-drain wait instead of waiting up to lingering_time seconds. Responses that keep the connection alive are unaffected.

Tests
Added test_force_close_response_skips_lingering_close in tests/test_web_functional.py, which spies on StreamReader.readany (identity-filtered to the request's own payload) to confirm the unread body is never drained when resp.force_close() is called. Verified it fails on unfixed code and passes with the fix.

Fixes #1800

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Jul 25, 2026
@codecov

codecov Bot commented Jul 25, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
4959 1 4958 44
View the top 1 failed test(s) by shortest run time
tests.test_web_functional::test_force_close_response_skips_lingering_close
Stack Traces | 0.008s run time
>   #x1B[0m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[90m#x1B[39;49;00m


#x1B[1m#x1B[31maiofastnet/transport.pyx#x1B[0m:752: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
#x1B[1m#x1B[31maiofastnet/transport.pyx#x1B[0m:741: in aiofastnet.transport.SocketTransport._flush_write_backlog
    #x1B[0m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
#x1B[1m#x1B[31maiofastnet/transport.pyx#x1B[0m:576: in aiofastnet.transport.SocketTransport._try_write_list_of_data
    #x1B[0m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   #x1B[0m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[04m#x1B[91m?#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
#x1B[1m#x1B[31mE   BrokenPipeError: [Errno 32] Broken pipe#x1B[0m


#x1B[1m#x1B[31maiofastnet/transport.pyx#x1B[0m:522: BrokenPipeError

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m

aiohttp_client = <function aiohttp_client.<locals>.go at 0x10c0493a0>

    #x1B[0m#x1B[94masync#x1B[39;49;00m #x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mtest_force_close_response_skips_lingering_close#x1B[39;49;00m(#x1B[90m#x1B[39;49;00m
        aiohttp_client: AiohttpClient,#x1B[90m#x1B[39;49;00m
    ) -> #x1B[94mNone#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
    #x1B[90m    #x1B[39;49;00m#x1B[33m"""Calling resp.force_close() must close the connection promptly instead#x1B[39;49;00m
    #x1B[33m    of waiting to drain an unread request body (see issue #1800)."""#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        request_payload: StreamReader | #x1B[94mNone#x1B[39;49;00m = #x1B[94mNone#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        drained = #x1B[94mFalse#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
        orig_readany = StreamReader.readany#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[94masync#x1B[39;49;00m #x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mspy_readany#x1B[39;49;00m(#x1B[96mself#x1B[39;49;00m: StreamReader) -> #x1B[96mbytes#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
            #x1B[94mnonlocal#x1B[39;49;00m drained#x1B[90m#x1B[39;49;00m
            #x1B[94mif#x1B[39;49;00m #x1B[96mself#x1B[39;49;00m #x1B[95mis#x1B[39;49;00m request_payload:#x1B[90m#x1B[39;49;00m
                drained = #x1B[94mTrue#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
            #x1B[94mreturn#x1B[39;49;00m #x1B[94mawait#x1B[39;49;00m orig_readany(#x1B[96mself#x1B[39;49;00m)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[94masync#x1B[39;49;00m #x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mhandler#x1B[39;49;00m(request: web.Request) -> web.StreamResponse:#x1B[90m#x1B[39;49;00m
            #x1B[94mnonlocal#x1B[39;49;00m request_payload#x1B[90m#x1B[39;49;00m
            request_payload = request.content#x1B[90m#x1B[39;49;00m
            resp = web.StreamResponse()#x1B[90m#x1B[39;49;00m
            resp.force_close()#x1B[90m#x1B[39;49;00m
            #x1B[94mawait#x1B[39;49;00m resp.prepare(request)#x1B[90m#x1B[39;49;00m
            #x1B[94mawait#x1B[39;49;00m resp.write_eof()#x1B[90m#x1B[39;49;00m
            #x1B[90m# Deliberately do not read request.content, leaving the body unread.#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
            #x1B[94mreturn#x1B[39;49;00m resp#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        app = web.Application()#x1B[90m#x1B[39;49;00m
        app.router.add_post(#x1B[33m"#x1B[39;49;00m#x1B[33m/#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, handler)#x1B[90m#x1B[39;49;00m
    #x1B[90m#x1B[39;49;00m
        #x1B[94mwith#x1B[39;49;00m (#x1B[90m#x1B[39;49;00m
            mock.patch.object(StreamReader, #x1B[33m"#x1B[39;49;00m#x1B[33mreadany#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, spy_readany),#x1B[90m#x1B[39;49;00m
            mock.patch(#x1B[90m#x1B[39;49;00m
                #x1B[33m"#x1B[39;49;00m#x1B[33maiohttp.web_protocol.ceil_timeout#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, wraps=ceil_timeout#x1B[90m#x1B[39;49;00m
            ) #x1B[94mas#x1B[39;49;00m mock_ceil_timeout,#x1B[90m#x1B[39;49;00m
        ):#x1B[90m#x1B[39;49;00m
            client = #x1B[94mawait#x1B[39;49;00m aiohttp_client(app)#x1B[90m#x1B[39;49;00m
>           #x1B[94masync#x1B[39;49;00m #x1B[94mwith#x1B[39;49;00m client.post(#x1B[33m"#x1B[39;49;00m#x1B[33m/#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m, data=#x1B[33mb#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m#x1B[33mx#x1B[39;49;00m#x1B[33m"#x1B[39;49;00m * (#x1B[94m2#x1B[39;49;00m**#x1B[94m20#x1B[39;49;00m)) #x1B[94mas#x1B[39;49;00m resp:#x1B[90m#x1B[39;49;00m
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^#x1B[90m#x1B[39;49;00m

aiohttp_client = <function aiohttp_client.<locals>.go at 0x10c0493a0>
app        = <Application 0x10bf44930>
client     = <aiohttp.test_utils.TestClient object at 0x10bc38920>
drained    = False
handler    = <function test_force_close_response_skips_lingering_close.<locals>.handler at 0x10c049940>
mock_ceil_timeout = <MagicMock name='ceil_timeout' id='4492342768'>
orig_readany = <function StreamReader.readany at 0x1081fbb00>
request_payload = <StreamReader 261936 bytes e=PayloadAccessError()>
spy_readany = <function test_force_close_response_skips_lingering_close.<locals>.spy_readany at 0x10c04a020>

#x1B[1m#x1B[31mtests/test_web_functional.py#x1B[0m:820: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
#x1B[1m#x1B[31maiohttp/client.py#x1B[0m:1480: in __aenter__
    #x1B[0m#x1B[96mself#x1B[39;49;00m._resp: _RetType_co = #x1B[94mawait#x1B[39;49;00m #x1B[96mself#x1B[39;49;00m._coro#x1B[90m#x1B[39;49;00m
                              ^^^^^^^^^^^^^^^^#x1B[90m#x1B[39;49;00m
        self       = <aiohttp.client._BaseRequestContextManager object at 0x10c207ca0>
#x1B[1m#x1B[31maiohttp/test_utils.py#x1B[0m:307: in _request
    #x1B[0mresp = #x1B[94mawait#x1B[39;49;00m #x1B[96mself#x1B[39;49;00m._session.request(method, #x1B[96mself#x1B[39;49;00m.make_url(path), **kwargs)#x1B[90m#x1B[39;49;00m
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^#x1B[90m#x1B[39;49;00m
        kwargs     = {'data': b'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'}
        method     = 'POST'
        path       = '/'
        self       = <aiohttp.test_utils.TestClient object at 0x10bc38920>
#x1B[1m#x1B[31maiohttp/client.py#x1B[0m:723: in _request
    #x1B[0mresp = #x1B[94mawait#x1B[39;49;00m handler(req)#x1B[90m#x1B[39;49;00m
           ^^^^^^^^^^^^^^^^^^#x1B[90m#x1B[39;49;00m
        all_cookies = <BaseCookie: >
        allow_redirects = True
        auth_from_url = None
        auto_decompress = True
        chunked    = None
        compress   = False
        cookies    = None
        data       = b'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
        effective_middlewares = ()
        expect100  = False
        handle     = None
        handler    = <function _connect_and_send_request at 0x1083879c0>
        headers    = <CIMultiDict()>
        history    = []
        json       = None
        max_field_size = 8190
        max_headers = 128
        max_line_size = 8190
        max_redirects = 10
        method     = 'POST'
        middlewares = None
        params     = {}
        proxy      = None
        proxy_     = None
        proxy_headers = None
        raise_for_status = None
        read_bufsize = 262144
        read_until_eof = True
        real_timeout = ClientTimeout(total=300, connect=None, sock_read=None, sock_connect=None, ceil_threshold=5)
        redirects  = 0
        req        = <aiohttp.client_reqrep.ClientRequest object at 0x10a4d6360>
        resolved_proxy_headers = None
        response_params = {'timer': <aiohttp.helpers.TimerContext object at 0x10a084a50>, 'skip_payload': False, 'read_until_eof': True, 'auto_decompress': True, ...}
        retry_persistent_connection = False
        self       = <aiohttp.client.ClientSession object at 0x10a1a3cd0>
        server_hostname = None
        skip_auto_headers = None
        skip_headers = None
        ssl        = True
        str_or_url = URL('http://127.0.0.1:50282/')
        timeout    = <_SENTINEL.sentinel: 1>
        timer      = <aiohttp.helpers.TimerContext object at 0x10a084a50>
        tm         = <aiohttp.helpers.TimeoutHandle object at 0x10af14540>
        trace_request_ctx = None
        traces     = []
        url        = URL('http://127.0.0.1:50282/')
        version    = HttpVersion(major=1, minor=1)
#x1B[1m#x1B[31maiohttp/client.py#x1B[0m:244: in _connect_and_send_request
    #x1B[0m#x1B[94mawait#x1B[39;49;00m resp.start(conn)#x1B[90m#x1B[39;49;00m
        conn       = Connection<ConnectionKey(host='127.0.0.1', port=50282, is_ssl=False, ssl=True, proxy=None, proxy_headers_hash=None, server_hostname=None)>
        connector  = <aiohttp.connector.TCPConnector object at 0x10bc392e0>
        req        = <aiohttp.client_reqrep.ClientRequest object at 0x10a4d6360>
        resp       = <ClientResponse(http://127.0.0.1:50282/) [None None]>
None

#x1B[1m#x1B[31maiohttp/client_reqrep.py#x1B[0m:526: in start
    #x1B[0mmessage, payload = #x1B[94mawait#x1B[39;49;00m protocol.read()  #x1B[90m# type: ignore[union-attr]#x1B[39;49;00m#x1B[90m#x1B[39;49;00m
                       ^^^^^^^^^^^^^^^^^^^^^#x1B[90m#x1B[39;49;00m
        connection = Connection<ConnectionKey(host='127.0.0.1', port=50282, is_ssl=False, ssl=True, proxy=None, proxy_headers_hash=None, server_hostname=None)>
        protocol   = <aiohttp.client_proto.ResponseHandler object at 0x10c03d9b0>
        self       = <ClientResponse(http://127.0.0.1:50282/) [None None]>
None

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <aiohttp.client_proto.ResponseHandler object at 0x10c03d9b0>

    #x1B[0m#x1B[94masync#x1B[39;49;00m #x1B[94mdef#x1B[39;49;00m#x1B[90m #x1B[39;49;00m#x1B[92mread#x1B[39;49;00m(#x1B[96mself#x1B[39;49;00m) -> _T:#x1B[90m#x1B[39;49;00m
        #x1B[94mif#x1B[39;49;00m #x1B[95mnot#x1B[39;49;00m #x1B[96mself#x1B[39;49;00m._buffer #x1B[95mand#x1B[39;49;00m #x1B[95mnot#x1B[39;49;00m #x1B[96mself#x1B[39;49;00m._eof:#x1B[90m#x1B[39;49;00m
            #x1B[94massert#x1B[39;49;00m #x1B[95mnot#x1B[39;49;00m #x1B[96mself#x1B[39;49;00m._waiter#x1B[90m#x1B[39;49;00m
            #x1B[96mself#x1B[39;49;00m._waiter = #x1B[96mself#x1B[39;49;00m._loop.create_future()#x1B[90m#x1B[39;49;00m
            #x1B[94mtry#x1B[39;49;00m:#x1B[90m#x1B[39;49;00m
>               #x1B[94mawait#x1B[39;49;00m #x1B[96mself#x1B[39;49;00m._waiter#x1B[90m#x1B[39;49;00m
#x1B[1m#x1B[31mE               aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe#x1B[0m

self       = <aiohttp.client_proto.ResponseHandler object at 0x10c03d9b0>

#x1B[1m#x1B[31maiohttp/streams.py#x1B[0m:734: ClientOSError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@codspeed-hq

codspeed-hq Bot commented Jul 25, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 83 untouched benchmarks
⏩ 83 skipped benchmarks1


Comparing Ranjeeth11:skip-lingering-close-on-force-close (740902a) with master (4ae5d49)

Open in CodSpeed

Footnotes

  1. 83 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Comment thread tests/test_web_functional.py
Comment thread aiohttp/web_protocol.py
The existing regression test for issue aio-libs#1800 only observed that the
request body was never drained (via a StreamReader.readany spy), which
is a side effect of the lingering loop rather than a direct check that
the loop's guard was never entered. Add a wraps= spy on
aiohttp.web_protocol.ceil_timeout - the per-iteration timeout the
lingering loop uses and the only call to it reachable from this test -
so the test now fails if the branch runs at all, not only if it happens
to drain the payload.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Marking response.force_close() doesn't prevent lingering

2 participants