[Twisted-Python] Distributing calls over a time period

[Jean-Paul Calderone]

On Tue, 18 Jul 2006 10:50:11 -0400, Mike Meyer <mwm-keyword-twisted.c54178 at mired.org> wrote:
>In <20060718143122.29014.825269123.divmod.quotient.42739 at ohm>, Jean-Paul Calderone <exarkun at divmod.com> typed:
>> On Tue, 18 Jul 2006 10:12:10 -0400, Mike Meyer <mwm-keyword-twisted.c54178 at mired.org> wrote:
>> >I'm writing a capacity test tool for a client's web server. The idea
>> >is to measure count how many requests exceed the spec'ed response time
>> >under various loads, where load is specified as "M requests in N
>> >secs". So I need to generate those M requests spaced relatively evenly
>> >over N seconds, record the start time, and get a reasonably accurate
>> >idea of when each requests ends.
>> >
>> >I'm assuming that twisted callbacks will take care of the latter, and
>> >don't have any reason to doubt that. However, the former has turned
>> >out to be harder than I expected. I need to start the requests with
>> >the reactor running so that previous requests will be handled
>> >properly. The obvious solution - task.LoopingCall - isn't precise
>> >enough, taking up to 33% more time than I'd alloted. The second
>> >attempt - a loop using reactor.callLater with a delay based on the
>> >requests and time left. That does slightly better, but only slightly.
>> >
>> >Is there some other method that I might use for this? Or is twisted
>> >the wrong tool entirely.
>>
>> What platform are you running on?  On any recent Linux system,
>> LoopingCall should be more than precise enough for this.
>
>Development is being done on OSX. The final deployment platform is Red
>Hat Egenera, but I'm still waiting on that to materialize.
>
>> Can you be more specific about what you mean by "taking up to 33%
>> more time"?
>
>When I set things up to run in 60 seconds, it takes anywhere from 61
>to 80 seconds.
>
>> What kind of time units are we talking about here?
>
>10s of seconds, with request counts such that we're getting 10s of
>requests a second.

This is orders of magnitude out from where you should be having any
problems.  Can you share a brief example of how you're doing the
scheduling?

Jean-Paul