<div class="gmail_quote">On Wed, Oct 26, 2011 at 10:22 AM, Terry Jones <span dir="ltr"><<a href="mailto:terry@jon.es">terry@jon.es</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Sounds like you should be using a Python generator and that you're not.<br></blockquote><div><br></div><div>The issue I'm seeing is that the ~million DeferredSemaphore.run() calls are inefficient; it's independent of list/generator.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">First off, have you read <a href="http://jcalderone.livejournal.com/24285.html" target="_blank">http://jcalderone.livejournal.com/24285.html</a> ?<br>
If not, do. Take note of the line<br>
<br>
work = (callable(elem, *args, **named) for elem in iterable)<br>
<br>
work is a generator. Make sure you understand every line of that code :-)<br></blockquote><div><br></div><div>I don't see anything in task.Cooperator to limit the # of simultaneously-running tasks. Am I missing something? I guess, technically, could write my own scheduler that limits the # of simultaneously-running tasks. But, then task.Cooperator isn't really doing anything useful for me.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">You might also find <a href="https://launchpad.net/txrdq" target="_blank">https://launchpad.net/txrdq</a> of interest, though it's<br>
probably overkill for what you're trying to do.<br></blockquote><div><br></div><div>Yes, it is overkill. </div><div><br></div><div><div>Jason</div><div><br></div></div></div>