[Twisted-Python] Distributing calls over a time period
Mike Meyer
mwm-keyword-twisted.c54178 at mired.org
Tue Jul 18 10:53:31 MDT 2006
In <20060718151310.29014.659879015.divmod.quotient.42787 at ohm>, Jean-Paul Calderone <exarkun at divmod.com> typed:
> >> Can you be more specific about what you mean by "taking up to 33%
> >> more time"?
> >
> >When I set things up to run in 60 seconds, it takes anywhere from 61
> >to 80 seconds.
> >
> >> What kind of time units are we talking about here?
> >
> >10s of seconds, with request counts such that we're getting 10s of
> >requests a second.
>
> This is orders of magnitude out from where you should be having any
> problems. Can you share a brief example of how you're doing the
> scheduling?
Sure. Here's the start method that's responsible for starting self.counter
requests in self.opts.send seconds:
def start(self):
"""Start a page fetch."""
launch = time()
defer = getPage('http://%s:8080/axis2/services/RadarWebService' %
self.host, postdata=payload, method='POST')
defer.addCallback(self.check_response)
defer.addCallbacks(self.good, self.bad, callbackArgs=(launch,),
errbackArgs=(launch,))
self.actions.append(defer)
if not hasattr(self, 'started'):
self.started = launch
self.counter -= 1
if self.counter <= 0:
self.done()
else:
delay = (self.started + self.opts.send - time()) / self.counter
if delay < 0:
delay = 0
reactor.callLater(delay, self.start)
done uses self.started and time() to calcuate the time that's passed
and print it:
def done(self):
print 'Started %d requests in %f seconds.' % \
(self.reqs, time() - self.started)
Thanks,
<mike
--
Mike Meyer <mwm at mired.org> http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.
More information about the Twisted-Python
mailing list