[Twisted-Python] Advise for heavy concurrency

[glyph at divmod.com]

On 1 Dec, 11:37 pm, theatilla at gmail.com wrote:
>But - when it comes to large send queues, how can that be chopped in
>an efficient manner? Chop it into equal pieces and schedule them with
>small delay, one after the other? Defer them to another thread? Is
>there any other solution, a way to chain them into a log callback
>squence maybe?

You probably want to use the producer/consumer API.  self.transport will 
provide this interface:

http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.IConsumer.html

which will all you to create and send the data only as needed, rather 
than on a timer.  If your concern is the loop over all of your clients, 
then coiterate might be a better bet, but IConsumer will help you spend 
that CPU only when you actually need to.