[Twisted-Python] A resizable cooperator class for queuing and dispatching jobs
terry at jon.es
Wed Dec 9 16:44:58 EST 2009
>>>>> "Glyph" == Glyph Lefkowitz <glyph at twistedmatrix.com> writes:
Glyph> On Dec 8, 2009, at 11:09 PM, Terry Jones wrote:
>> I just wrote a fun class that lets you
>> - submit jobs to be dispatched to a queue
>> - manage how many tasks are in progress at once
>> - dynamically adjust that number
>> - shut down cleanly, including
>> - recovering jobs that were queued but hadn't been dispatched
Glyph> That does sound like fun!
It *is* fun.
>> # Later... send off some SMS messages.
>> dispatcher.put((2127399921, 'Hello...'))
>> dispatcher.put((5052929919, 'Test...'))
Glyph> Not that this is really germane to the class in question, but, if
Glyph> you're going to be sending SMS messages
That was just an example. I even thought about using strings, but kept it
>> # Later, bump up to 10 simultaneous jobs.
>> dispatcher.width = 10
>> # Oops, turns out we're sending too fast, turn it down a little.
Glyph> This seems somewhat asymmetric. Why are there 'widen' and 'narrow'
Glyph> methods if I can just set the 'width' attribute directly? I could
Glyph> always just do '.width +=' and '.width -=' if I have a relative
Agreed. I started out just implementing widen and narrow, and they (barely)
survived when I later added the width property. I had been thinking it
would be good to just be able to change the width up or down a notch, but
then when I went to write my (a posteriori) tests, I realized that wasn't
really so convenient. So I think you're right they could be simplified
But the code Glyph, the code? Or have I graduated to the point where I no
longer need a laconic garotting? I think the combo of a DeferredQueue and
a task.Cooperator is pretty deadly.
More information about the Twisted-Python