[Twisted-Python] Twisted reactor cpu usage

[Ilya Biin]

I'm using epoll reactor and sample HTTP Proxy realization.

From another process i'm executing ~30 HTTP requests per second and
twisted process is consuming 10% cpu

The profiler output is:

        937706 function calls (934675 primitive calls) in 59.988 CPU seconds

  Ordered by: cumulative time
  List reduced from 312 to 15 due to restriction <15>

  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       1    0.000    0.000   59.988   59.988 base.py:1167(run)
       1    0.076    0.076   59.988   59.988 base.py:1172(mainLoop)
   12697   55.456    0.004   59.253    0.005 epollreactor.py:169(doPoll)
   13553    0.086    0.000    3.797    0.000 log.py:71(callWithLogger)
   13553    0.096    0.000    3.691    0.000 log.py:66(callWithContext)
   13553    0.047    0.000    3.481    0.000 context.py:117(callWithContext)
   13553    0.067    0.000    3.426    0.000 context.py:61(callWithContext)
   13552    0.095    0.000    3.359    0.000 posixbase.py:544(_doReadOrWrite)
    5454    0.097    0.000    1.736    0.000 tcp.py:182(doRead)
    5453    0.167    0.000    1.639    0.000 basic.py:543(dataReceived)
   12697    0.095    0.000    0.587    0.000 base.py:762(runUntilCurrent)
    4453    0.025    0.000    0.574    0.000 http.py:495(rawDataReceived)
    6000    0.036    0.000    0.547    0.000 http.py:1515(lineReceived)
    2000    0.029    0.000    0.454    0.000 tcp.py:371(doConnect)
    5098    0.104    0.000    0.428    0.000 abstract.py:212(doWrite)


        937706 function calls (934675 primitive calls) in 59.988 CPU seconds

  Ordered by: internal time
  List reduced from 312 to 15 due to restriction <15>

  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   12697   55.456    0.004   59.253    0.005 epollreactor.py:169(doPoll)
   59101    0.277    0.000    0.277    0.000 <string>:1(fileno)
    5098    0.252    0.000    0.252    0.000 tcp.py:212(writeSomeData)
   24453    0.198    0.000    0.198    0.000 __init__.py:1230(getEffectiveLevel)
    5453    0.167    0.000    1.639    0.000 basic.py:543(dataReceived)
    5098    0.104    0.000    0.428    0.000 abstract.py:212(doWrite)
    5454    0.097    0.000    1.736    0.000 tcp.py:182(doRead)
   13553    0.096    0.000    3.691    0.000 log.py:66(callWithContext)
   12697    0.095    0.000    0.587    0.000 base.py:762(runUntilCurrent)
   13552    0.095    0.000    3.359    0.000 posixbase.py:544(_doReadOrWrite)
   13553    0.086    0.000    3.797    0.000 log.py:71(callWithLogger)
       1    0.076    0.076   59.988   59.988 base.py:1172(mainLoop)
   16556    0.074    0.000    0.074    0.000 context.py:86(getContext)
   17451    0.073    0.000    0.094    0.000 epollreactor.py:71(_add)
   17098    0.068    0.000    0.085    0.000 epollreactor.py:112(_remove)
What is wrong? Why twisted perfomance is so poor here?

--
Ilya Biin