[Twisted-Python] [twisted-web] I/O error with twisted.web.wsgi and Django

[Orestis Markou]

Hi all,

I am using Twisted Web WSGIResource to host a Django site. Here's the 
code I use to setup the WSGI resource (other boilerplate skipped)

def wsgi_resource():
     from django.core.handlers.wsgi import WSGIHandler
     pool = threadpool.ThreadPool()
     pool.start()
     # Allow Ctrl-C to get you out cleanly:
     reactor.addSystemEventTrigger('after', 'shutdown', pool.stop)
     wsgi_resource = wsgi.WSGIResource(reactor, pool, WSGIHandler())
     return wsgi_resource

This generally worked quite well. However I have an API endpoint that 
gets hit with multiple POST requests per second (not very much - I'd say 
10-20 req/sec at most). Sometimes I see this in the logs:


Internal Server Error: /cas/eventlog/
Traceback (most recent call last):
   File "<pythonpath>/site-packages/django/core/handlers/base.py", line 
114, in get_response
     response = wrapped_callback(request, *callback_args, 
**callback_kwargs)
   File "<pythonpath>/site-packages/django/views/decorators/csrf.py", 
line 57, in wrapped_view
     return view_func(*args, **kwargs)
   File "<djangoapp>/views.py", line 488, in event_log
     data = json.loads(request.body)
   File "<pythonpath>/site-packages/django/http/request.py", line 192, 
in body
     self._body = self.read()
   File "<pythonpath>/site-packages/django/http/request.py", line 246, 
in read
     return self._stream.read(*args, **kwargs)
   File "<pythonpath>/site-packages/django/core/handlers/wsgi.py", line 
45, in read
     result = self.buffer + self._read_limited()
   File "<pythonpath>/site-packages/django/core/handlers/wsgi.py", line 
39, in _read_limited
     result = self.stream.read(size)
   File "<pythonpath>/site-packages/twisted/web/wsgi.py", line 94, in 
read
     return self._wrapped.read(size)
ValueError: I/O operation on closed file.


Any pointers on why would this happen? Why would Twisted close the file 
early? Could it be that I need to increase the threadpool size?


I use treq 0.2.0 for the requests, with the "persistent" flag turned on, 
if that makes any difference. Twisted 13.2.0 and Django 1.6, python 
2.7.1 on OS X 10.7.5.

Many thanks,
Orestis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20140227/7f33ef65/attachment.html>