[Twisted-Python] Twisted scalability with hundreds of outbound (client) connections
itamar at itamarst.org
Sun Sep 26 16:25:35 EDT 2004
On Sun, 2004-09-26 at 15:35, Phil Mayers wrote:
> def xmit(self):
> while True:
> txqueue = self.txcurrent
> self.txcurrent = txqueue.next
> if txqueue:
> reactor.callLater(0, self.xmit)
This code confuses me. What is the "next" bit for?
> I'm concerned about all those reactor.callLater - since one of the main
> problems is the UDP socket queue overflowing, every time I xmit I have
> to get *out* of the protocol code ASAP and back into the select() loop,
> however one of the problems with the reactors (problems for me at any
> rate) is that they do pending calls and thread stuff before IO, which
> IMHO is not quite the right way round.
These are indistinguishable once you've done half an iteration, ABABAB
is BABABA slightly timeshifted, it just affects the first and last
> I'm also slightly concerned about the number of function calls involved
> in jumping in and out of the reactor that many times a second (several
> thousand, if I can get it to go as fast as my previous code) given how
> expensive they are under Python. It would certainly be quicker to
> implement this inside the reactor.mainLoop.
Just have a single reactor.callLater(0, f), and f() then calls all the
functions you want done in that iteration.
More information about the Twisted-Python