[Twisted-web] [Twisted-Python] Speed of rendering?
Peter Westlake
peter.westlake at pobox.com
Wed Oct 24 09:11:38 EDT 2012
On Wed, Oct 24, 2012, at 09:16, Glyph wrote:
>
> On Oct 23, 2012, at 8:10 AM, Peter Westlake <peter.westlake at pobox.com>
> wrote:
...
> > The problem I'm having is that flatten() returns immediately if given a string or anything else without an unfired Deferred, and that sends Client._continue into an unbounded recursion. Is there a general good way to handle this kind of problem? Somehow I need to return control to the reactor long enough for Client._request to return.
>
> That sounds like a bug, although it's hard to say without seeing the
> exact code that you're talking about. Can you send a representative
> example?
Here it is:
from benchlib import driver, Client
from twisted.web.template import flatten
from twisted.web.server import Request
from twisted.web.http import HTTPChannel
class Client(Client):
channel = HTTPChannel()
request = Request(channel, False)
def _request(self):
d = flatten(self.request, 'hello', lambda _: None)
d.addCallback(self._continue) ### Infinite recursion happens
here
d.addErrback(self._stop)
def main(reactor, duration):
concurrency = 1
client = Client(reactor)
d = client.run(concurrency, duration)
return d
if __name__ == '__main__':
import sys
import flatten_string
driver(flatten_string.main, sys.argv)
Because flatten does not have to wait for anything, it returns
a Deferred that has already fired. The d.addCallback sees this
and calls the callback immediately. Client._continue calls the
next iteration of the test by calling self.request again, and
the stack blows up. This is perfectly reasonable and standard
behaviour for Deferreds, so I should be doing the iteration in
some other way, probably not using Client at all. What I was
hoping for was a pattern for how to transform the code to avoid
the problem; I suspect the answer is to use iteration instead
of recursion. It might even be that none of benchlib.py is
usable directly.
Or maybe putting the flatten() calls into a thread would work?
But that runs the risk of race conditions, if it finishes
before the callback is added.
Peter.
More information about the Twisted-web
mailing list