[Twisted-Python] running 1,000,000 tasks, 40 at-a-time

Jason Rennie jrennie at gmail.com
Wed Oct 26 11:29:13 EDT 2011

On Wed, Oct 26, 2011 at 11:08 AM, Jason Rennie <jrennie at gmail.com> wrote:

> First off, have you read http://jcalderone.livejournal.com/24285.html ?
>> If not, do.  Take note of the line
>>  work = (callable(elem, *args, **named) for elem in iterable)
>> work is a generator.  Make sure you understand every line of that code :-)
> I don't see anything in task.Cooperator to limit the # of
> simultaneously-running tasks.  Am I missing something?  I guess,
> technically, could write my own scheduler that limits the # of
> simultaneously-running tasks.  But, then task.Cooperator isn't really doing
> anything useful for me.

Sorry, I didn't realize that the Cooperators were all being given a
reference to the same generator.  That's a useful pattern.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20111026/4efb31c4/attachment.htm 

More information about the Twisted-Python mailing list