[Twisted-web] Re: [Twisted-Python] WSGI Thread-management strategy

[Jim Fulton]

James Y Knight wrote:
> (BTW, the correct mailing list for twisted webbish stuff is twisted- 
> web at twistedmatrix.com)
> On Dec 15, 2005, at 4:22 PM, Jim Fulton wrote:
> 
>> The strategy used by twisted WSGI, as I understand it, doesn't meet
>> our needs. Currently, a thread is created for each request.  The total
>> number of threads is throttled, I gather using a general Twisted
>> thread limit.  WSGI applications are called as soon as input headers
>> have been received completely. An application may be called before all
>> body input is received.  We need application calls to be delayed until
>> all request input has been received,
>>
>> [...]
>>
>> I propose that the default thread-management strategy should be to  delay
>> calling an application until all request input has been received. If
>> this isn't the default, then there should at least be an option to get
>> this behavior.  (Of course, the buffering strategy needs to be clever
>> enough to switch to a file when the input gets over some size.)
> 
> 
> Sounds sensible,

Does this mean you will make this the default in the future?


 > and is doable external to the WSGI wrapper. Here's a
> little bit I whipped up. (works on the 2.1.x branch and head). Could  be 
> smarter, by starting out the buffer in memory and switching to a  file 
> if necessary. Also shows off a couple of minor bugs I need to  fix. :)
> 
> def simple_wsgi_app(environ, start_response):
>     print "Starting wsgi app"
>     start_response("200 OK", [('Content-type','text/html;  
> charset=ISO-8859-1')])
>     data = environ['wsgi.input'].read()
>     return ['<pre>', data, '</pre>']
> 
> 
> class Prebuffer(resource.WrapperResource):
>     def hook(self, ctx):
>         req = iweb.IRequest(ctx)
>         temp = tempfile.TemporaryFile()
>         def done(_):
>             temp.seek(0)
>             # Replace the request's stream object with the tempfile
>             req.stream = stream.FileStream(temp)
>             # Hm, this shouldn't be required:
>             req.stream.doStartReading = None
>         return stream.readStream(req.stream, temp.write).addCallback (done)
> 
>     # Oops, fix missing () in lambda in WrapperResource
>     def locateChild(self, ctx, segments):
>         x = self.hook(ctx)
>         if x is not None:
>             return x.addCallback(lambda data: (self.res, segments))
>         return self.res, segments
> 
> if __name__ == '__builtin__':
>     from twisted.application import service, strports
>     from twisted.web2 import server, channel
> 
>     res = Prebuffer(wsgi.WSGIResource(simple_wsgi_app))
> 
>     site = server.Site(res)
>     application = service.Application("demo")
> 
>     s = strports.service('tcp:8080', channel.HTTPFactory(site))
>     s.setServiceParent(application)

This is greek to me. I'd forward this to the people who did the
Twisted Zope integration.

Thanks!

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org