[Twisted-Python] Serious performance problem of twisted
davy zhang
davyzhang at gmail.com
Sun Oct 19 05:38:04 MDT 2008
a very simple server and multi-process code, I found there's about 500
milliseconds delay for every msg I sent to client I just don't why,
did I made something wrong ?
the code below using a process to handle the data and in the shared
taskList, after done put the data to the shared msgList, a thread in
main process to read the msg and send to the clients.
the msg delay is significant, you can even see it with your eyes in
telnet client I checked the time in the client, it's approximate to
0.5 secs
first I tested in java using the same logic with MINA, BlockingQueue
stuff, the msg traveled from client to server and back is 0, so I
pretty much sure the problem here is in the server side
then I printed time in the server code, I found everything is as fast
as it suppose to be, until the c.transport.write. After the write
operation the client need 0.5secs to get the msg
the code is below
#!/usr/bin/env python
#coding=utf-8
from twisted.internet.protocol import Factory,Protocol
from twisted.internet import reactor
from twisted.internet import error
import cPickle
import time
from processing import Manager,Process
import thread
from time import time
"""flash policy file request"""
policyRequest = "<policy-file-request/>" + chr(0)
policyReturn = """<cross-domain-policy>
<allow-access-from domain="*" to-ports="*" />
</cross-domain-policy>""" + chr(0)
def handler(taskList,msgList):
while 1:
item = cPickle.loads(taskList.get())
print time()
print 'item before handle ', item
item['msg'] += ' hanlded done \r\n'
msgList.put(cPickle.dumps(item))
def sender():
global msgList
while 1:
item = cPickle.loads(msgList.get())
print time()
c = clients[item['cid']]
c.transport.write(item['msg'] )
class DTProtocol(Protocol):
global clients
def connectionMade(self):
clients.append(self)
print "a new customer!"
def dataReceived(self, data):
global clients
global taskList
print 'msg received ',data
if data == policyRequest:
self.transport.write(policyReturn)
self.transport.loseConnection()
else:
d = {'cid':clients.index(self),'msg':data,'done':False,'sent':False}
taskList.put(cPickle.dumps(d))
print time()
#self.transport.write(data)
def connectionLost(self, reason):
print reason
if self in clients:
clients.remove(self)
print "bye!"
def message(self,msg):
self.transport.write(msg)
class DTFactory(Factory):
protocol = DTProtocol
if __name__ == '__main__':
port = 80
reactor.listenTCP(80,DTFactory())
clients = []
manager = Manager()
taskList = manager.Queue()
msgList = manager.Queue()
h = Process(target=handler,args=(taskList,msgList))
h.start()
thread.start_new_thread(sender,())
print 'server is runing at ',port
reactor.run()
More information about the Twisted-Python
mailing list