[Twisted-Python] integrating CompStrm//adding background processing
glyph at divmod.com
Sun May 16 17:07:29 EDT 2004
Bill la Forge wrote:
> I've been working on integrating Compstrm
> ( http://compstrm.sourceforge.net ) and, while integration was pretty
> easy, it got harder when I wanted to speed things up.
I appreciate reports of performance issues, and I'm sure the reactor
could be sped up a bit - but this report seemed a bit vague, and as far
as a few helpful folks on #twisted could tell, wrong.
> Just using reactor.callLater, I could only get a speed of 90, in
> contrast to
> the asyncore integration which was doing better than 12,000.
90 whats per what on what? How exactly did you measure this?
We assumed you meant "90 trivial calls per second" and "on relatively
recent hardware". The only test we did that made this radical kind of
difference was inserting a 'print' statement into the "trivial"
callback. The numbers for callLaters-per-second on various hardware we
had lying around, mostly around ~2GhZ athlons, were in the 15000-25000
> By replacing runUntilCurrent and timeout, I managed to bump my speed up
> to better than 8,000, which seems reasonable, as Twisted is a bit
> more "heavy weight" than asyncore. ;-)
Also I'm not sure why your code would have sped up the reactor. Would
you mind sending your code in the form of two complete Python programs
that will demonstrate the difference in speed between your newer
callLater code and the existing reactor?
More information about the Twisted-Python