Ticket #2534 enhancement new
investigate event dispatch optimization and expose a switch for enable/disable the log system
|Reported by:||exarkun||Owned by:|
|Component:||core||Keywords:||log, cpu usage|
|Cc:||itamarst, therve, drewp, thijs||Branch:|
A Twisted UDP echo server on an 2.2GHz Opteron can saturate a 100mbit/sec network, but doing so requires about 80% CPU. Since in this application there is essentially no application overhead, this is all time spent in Twisted somewhere. A similar application in C can do the same task using about 10% CPU.
There are probably some things we can do to cut down on this event dispatch overhead. One particularly obvious area (but not necessarily the only or even the most costly) is the involvement of the logging system in event dispatch. Removing the callWithLogger usage from select reactor yields approximately 15% reduction in CPU usage for the same task. There is probably a way to preserve this functionality without the current overhead, such as by pre-constructing context dictionaries, calling logPrefix only once per selectable, avoiding the extra method call of callWithLogger and going straight to callWithContext, avoiding the context copy since the reactor is guaranteed to be at the top of the context stack, etc.