[Twisted-Python] self.channel.writeHeaders(version, code, reason, headers)\nbuiltins.AttributeError: \'NoneType\' object has no attribute \'writeHeaders\'\

[Waqar Khan]

Just for sake of completeness:
def print_json_response(resp, request):
         request.write(json.dumps(resp))
         request.close()

It is this close function which is causing the issue
I suspect that somehow the fact that the client has closed the connection,
is not being handled.


On Sun, Aug 4, 2019 at 11:26 PM Waqar Khan <wk80333 at gmail.com> wrote:

> Hi Glyph,
>    Here is the minimal version
>
>
> class FooResource(resource.Resource):
>        def render_GET(request):
>               future = asyncio.ensure_future(self.fetch_response(request))
> // some async await functions
>               d = Deferred.fromFuture(future)
>               d.addCallback(print_json_response, request) // this is
> actually where the error is triggered.
>               d.addErrback(lambda failure:
> failure.trap(defer.CancelledError))
>               finished_errback = request.notifyFinish()
>               finished_errback.addErrback(self.handle_cancel, d) // simple
> logs and cancels d by d.cancel()
>               return NOT_DONE_YET
>
> The traceback is like
> 'Traceback (most recent call last):\n  File
> "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py",
> line 88, in _run\n    self._context.run(self._callback, *self._args)\n
>  File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py",
> line 827, in adapt\n    adapt.actual.callback(extracted)\n  File
> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
> 460, in callback\n    self._startRunCallbacks(result)\n  File
> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
> 568, in _startRunCallbacks\n    self._runCallbacks()\n--- <exception caught
> here> ---\n  File
> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
> 654, in _runCallbacks\n    current.result = callback(current.result, *args,
> **kw)\n  File "/Users/wqKhan/twisted_eg/parse_response.py", line 3, in
> print_json_response\n    *print_json_response*(response, request)\n  File
> "/Users/wqKhan/twisted_eg/parse_response.py", line 21, in
> *print_json_response*\n    request.finish()\n  File
> "/usr/local/lib/python3.7/site-packages/twisted/web/server.py", line 268,
> in finish\n    return http.Request.finish(self)\n  File
> "/usr/local/lib/python3.7/site-packages/twisted/web/http.py", line 1071, in
> finish\n    "Request.finish called on a request after its connection was
> lost; "\nbuiltins.RuntimeError: Request.finish called on a request after
> its connection was lost; use Request.notifyFinish to keep track of this.\n'
>
>
>
> On Sun, Aug 4, 2019 at 10:36 PM Glyph <glyph at twistedmatrix.com> wrote:
>
>>
>>
>> > On Aug 4, 2019, at 5:57 PM, Waqar Khan <wk80333 at gmail.com> wrote:
>> >
>> > Hi Glyph,
>> >   Thanks for the suggestion.
>> > I tried the suggestion.. While it fixes the self.channel NoneType
>> issue.. It creates another issue.
>> >
>> > Traceback (most recent call last):
>> >   File
>> "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py",
>> line 88, in _run
>> >     self._context.run(self._callback, *self._args)
>> >   File
>> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
>> 827, in adapt
>> >     adapt.actual.callback(extracted)
>> >   File
>> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
>> 460, in callback
>> >     self._startRunCallbacks(result)
>> >   File
>> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
>> 568, in _startRunCallbacks
>> >     self._runCallbacks()
>> > --- <exception caught here> ---
>> >   File
>> "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line
>> 654, in _runCallbacks
>> >     current.result = callback(current.result, *args, **kw)
>> > ......
>> >     request.finish()
>> >   File "/usr/local/lib/python3.7/site-packages/twisted/web/server.py",
>> line 268, in finish
>> >     return http.Request.finish(self)
>> >   File "/usr/local/lib/python3.7/site-packages/twisted/web/http.py",
>> line 1071, in finish
>> >     "Request.finish called on a request after its connection was lost; "
>> > builtins.RuntimeError: Request.finish called on a request after its
>> connection was lost; use Request.notifyFinish to keep track of this.
>>
>> Oh, hrm.  The idea here is that any code that returns NOT_DONE_YET (i.e.
>> does asynchronous work to generate a response) should also be tracking
>> whether it needs to call `.finish()` by watching a `.notifyFinish()`
>> Deferred.  I bet that there are plenty of resources in Twisted which don't
>> follow this rule, and we should fix those; but possibly we should also make
>> this error message less stern.  Do you have a minimal example?
>>
>> >  I have got some handle on the issue. I was wondering if you have any
>> advise on how to debug the issue.
>> > Somewhere in the codebase are async/await code (from asyncio world).
>> Hence, when the connection is closed down by client, seems like those
>> pending coroutines are lingering on.
>>
>> This, I have no idea about.  What tasks are you starting, when are you
>> expecting them to get cleaned up, what are they blocking on, how do these
>> interact with Twisted?  There are a couple dozen questions I'd have to know
>> the answer to in order to even begin debugging this.  If you cancel all the
>> outstanding tasks and look at their tracebacks when exiting it might give
>> you more of a sense of where they're stuck...
>>
>> -g
>>
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20190804/c7c5ec3b/attachment-0002.html>