[Twisted-Python] WSGI Thread-management strategy
James Y Knight
foom at fuhm.net
Fri Dec 16 12:14:34 EST 2005
(BTW, the correct mailing list for twisted webbish stuff is twisted-
web at twistedmatrix.com)
On Dec 15, 2005, at 4:22 PM, Jim Fulton wrote:
> The strategy used by twisted WSGI, as I understand it, doesn't meet
> our needs. Currently, a thread is created for each request. The total
> number of threads is throttled, I gather using a general Twisted
> thread limit. WSGI applications are called as soon as input headers
> have been received completely. An application may be called before all
> body input is received. We need application calls to be delayed until
> all request input has been received,
>
> [...]
>
> I propose that the default thread-management strategy should be to
> delay
> calling an application until all request input has been received. If
> this isn't the default, then there should at least be an option to get
> this behavior. (Of course, the buffering strategy needs to be clever
> enough to switch to a file when the input gets over some size.)
Sounds sensible, and is doable external to the WSGI wrapper. Here's a
little bit I whipped up. (works on the 2.1.x branch and head). Could
be smarter, by starting out the buffer in memory and switching to a
file if necessary. Also shows off a couple of minor bugs I need to
fix. :)
def simple_wsgi_app(environ, start_response):
print "Starting wsgi app"
start_response("200 OK", [('Content-type','text/html;
charset=ISO-8859-1')])
data = environ['wsgi.input'].read()
return ['<pre>', data, '</pre>']
class Prebuffer(resource.WrapperResource):
def hook(self, ctx):
req = iweb.IRequest(ctx)
temp = tempfile.TemporaryFile()
def done(_):
temp.seek(0)
# Replace the request's stream object with the tempfile
req.stream = stream.FileStream(temp)
# Hm, this shouldn't be required:
req.stream.doStartReading = None
return stream.readStream(req.stream, temp.write).addCallback
(done)
# Oops, fix missing () in lambda in WrapperResource
def locateChild(self, ctx, segments):
x = self.hook(ctx)
if x is not None:
return x.addCallback(lambda data: (self.res, segments))
return self.res, segments
if __name__ == '__builtin__':
from twisted.application import service, strports
from twisted.web2 import server, channel
res = Prebuffer(wsgi.WSGIResource(simple_wsgi_app))
site = server.Site(res)
application = service.Application("demo")
s = strports.service('tcp:8080', channel.HTTPFactory(site))
s.setServiceParent(application)
More information about the Twisted-Python
mailing list