[Twisted-Python] scaling with lots of clients
mckenzig at optonline.net
Wed Aug 3 16:59:08 EDT 2011
I have a twisted Perspective Broker based server. Recently with an increased number of clients we are seeing a decrease in throughput. Profiling the server it seems that it is spending the majority of time sending messages to the clients (unsuprisingly).
Messages are sent to the client from the server using a callRemote, invoked from the event queue using reactor.callFromThread - nice and simple. What designs have people used to improve throughput in a heavily loaded server? My experience is telling me that I need to implement some sort of thread/process pool to divide the messaging to clients. Various documents on the twisted website strongly suggets that this is not a good idea. Thoughts?
More information about the Twisted-Python