[Twisted-Python] Performance issue in reactor.callLater

Stefan Behnel behnel_ml at gkec.informatik.tu-darmstadt.de
Mon Sep 6 10:29:55 EDT 2004


Itamar Shtull-Trauring schrieb:
 > I would like to see some benchmarks comparing
 > speed of adding and cancelling for cases with many scheduled calls, but
 > also for cases with few calls, so we can see if your patches slow down
 > other cases.


For the first part, I did a few profiler runs that showed results similar to the following. Please note that although I removed a few lines that only belong to my personal code, the top three lines actually /were/ the top three. The "AckManager" lines are a simplified form of my own implementation, see it as a second level callLater. The __run_callbacks method is so slow because it calls "reactor.callLater" to be run repeatedly.

---------------------------------------
          9577183 function calls (9577108 primitive calls) in 117.349 CPU seconds

    Ordered by: internal time, call count

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    506487   22.837    0.000   30.519    0.000 bisect.py:3(insort_right)
     50830   12.002    0.000  103.742    0.002 base.py:400(runUntilCurrent)
    506487    8.974    0.000   47.078    0.000 base.py:362(callLater)
    498759    8.865    0.000   66.300    0.000 AckManager.py:81(__run_callbacks)
     50830    8.700    0.000    8.911    0.000 default.py:472(doSelect)
   3922829    7.682    0.000    7.682    0.000 base.py:132(__lt__)
    506487    7.585    0.000    7.585    0.000 base.py:54(__init__)
    500901    5.998    0.000    5.998    0.000 AckManager.py:59(__remove_inactive_entries_from_queue)
    500181    5.953    0.000   51.499    0.000 AckManager.py:68(__schedule_run)
         1    2.463    2.463  117.348  117.348 default.py:127(mainLoop)
     50830    2.232    0.000    2.232    0.000 base.py:391(timeout)
---------------------------------------

I think it is actually easy to see here that bisect becomes a serious bottleneck when the list gets longer. I didn't include any results from cancel() as I immediately eliminated all calls to the method after my first profiler run. Believe me, it was absolutely worth it.

I'll just continue reimplementing the IReactorTime stuff and then re-run the profiler for you.

I'll also try to run a few test cases with cancel() - thanks to CVS, I should still have some old code around...

Stefan





More information about the Twisted-Python mailing list