[Twisted-Python] integrating CompStrm//adding background processing

Glyph Lefkowitz glyph at divmod.com
Sun May 16 15:07:29 MDT 2004


Hi Bill,

Bill la Forge wrote:
> I've been working on integrating Compstrm
> ( http://compstrm.sourceforge.net ) and, while integration was pretty
> easy, it got harder when I wanted to speed things up.

I appreciate reports of performance issues, and I'm sure the reactor 
could be sped up a bit - but this report seemed a bit vague, and as far 
as a few helpful folks on #twisted could tell, wrong.

> Just using reactor.callLater, I could only get a speed of 90, in 
> contrast to
> the asyncore integration which was doing better than 12,000.

90 whats per what on what?  How exactly did you measure this?

We assumed you meant "90 trivial calls per second" and "on relatively 
recent hardware". The only test we did that made this radical kind of 
difference was inserting a 'print' statement into the "trivial" 
callback.  The numbers for callLaters-per-second on various hardware we 
had lying around, mostly around ~2GhZ athlons, were in the 15000-25000 
range.

> By replacing runUntilCurrent and timeout, I managed to bump my speed up 
> to better than 8,000, which seems reasonable, as Twisted is a bit
> more "heavy weight" than asyncore. ;-)

Also I'm not sure why your code would have sped up the reactor.  Would 
you mind sending your code in the form of two complete Python programs 
that will demonstrate the difference in speed between your newer 
callLater code and the existing reactor?




More information about the Twisted-Python mailing list