[Twisted-Python] Performance issue in reactor.callLater

Stefan Behnel behnel_ml at gkec.informatik.tu-darmstadt.de
Mon Sep 6 12:44:50 EDT 2004


Itamar Shtull-Trauring schrieb:
> I would like to see some benchmarks comparing
> speed of adding and cancelling for cases with many scheduled calls, but
> also for cases with few calls, so we can see if your patches slow down
> other cases.

So here's a comparison. I had to take a different example to actually see a difference since the profiling run I sent last time was about the same complexity.

Original list-based version:
-------------------------------------------------------------
          11647647 function calls (11647337 primitive calls) in 153.122 CPU seconds

    Ordered by: internal time, call count

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     48733    5.322    0.000    8.469    0.000 bisect.py:3(insort_right)
    595267    3.147    0.000    3.147    0.000 base.py:132(__lt__)
     48733    1.754    0.000   11.545    0.000 base.py:362(callLater)
     48733    1.323    0.000    1.323    0.000 base.py:54(__init__)
        30    1.232    0.041  151.715    5.057 base.py:400(runUntilCurrent)
      3171    0.134    0.000    3.309    0.001 AckManager.py:81(__run_callbacks)
      6801    0.067    0.000    0.426    0.000 AckManager.py:68(__schedule_run)
         1    0.042    0.042    0.042    0.042 base.py:71(cancel)
        30    0.000    0.000    0.000    0.000 base.py:391(timeout)
-------------------------------------------------------------


My heap-based version:
-------------------------------------------------------------
          11726288 function calls (11725973 primitive calls) in 159.055 CPU seconds

    Ordered by: internal time, call count

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     41752    4.097    0.000    5.420    0.000 heapq.py:226(_siftup)
        24    2.199    0.092  156.848    6.535 base.py:417(runUntilCurrent)
     89610    1.603    0.000    1.779    0.000 heapq.py:174(_siftdown)
    572391    1.212    0.000    1.212    0.000 base.py:134(__le__)
     47858    0.697    0.000    2.778    0.000 base.py:363(callLater)
     47858    0.327    0.000    0.327    0.000 base.py:56(__init__)
     41752    0.288    0.000    5.707    0.000 heapq.py:136(heappop)
     47858    0.262    0.000    1.754    0.000 heapq.py:131(heappush)
      2902    0.150    0.000    4.824    0.002 AckManager.py:81(__run_callbacks)
      6365    0.070    0.000    0.171    0.000 AckManager.py:68(__schedule_run)
        24    0.000    0.000    0.000    0.000 base.py:407(timeout)
        24    0.000    0.000    0.000    0.000 base.py:375(_cleanUpCallLater)
         6    0.000    0.000    0.000    0.000 base.py:72(cancel)
-------------------------------------------------------------

What I found is that callLater becomes /much/ faster in my implementation, while runUntilCurrent /does/ get slower. But that's Ok, I guess, since there is at least one call to callLater for each iteration in runUntilCurrent. So, if there's more calls to callLater, it pays off. I didn't test the cancel() part very well yet, but since the method calls should be almost instantaneous now, I guess, the speed up there will be absolutely increadible. Do not forget, also, that calls to callLater, delay and reset can occurr multiple times before a call is actually executed, so I guess it is /really/ the right thing if runUntilCurrent does more work than the others.

In the first run, there happens to be only one call to cancel(), but that one already takes ages compared to the six calls in the second run.

I hope this is somewhat convincing. Could anyone else please do some testing to see if there are other cases I've overseen?

Stefan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: base.py.diff
Type: text/x-patch
Size: 4235 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20040906/d0eebfd2/attachment.bin 


More information about the Twisted-Python mailing list