[Twisted-Python] First try at thread-safe FileDescriptor

Itamar twisted at itamarst.org
Fri Aug 10 14:12:52 EDT 2001


First, my threaded server works great when using threaded mode. However, 
having a lock on every write call seems excessive, especially since some 
of my servers aren't threaded, so I tried rewriting 
abstract.FileDescriptor so that the write() and doWrite() methods are 
thread-safe without using locks.

The issues:

1) It doesn't actually work - if I turn of the threadble.synchronize() 
call at the end of the module then the server gets deadlocks.

2) How can I make the FileDescriptor backwards compatible? Add a custom
__getstate__ that adds the unsentChunks list? If writeable 
FileDescriptors are never pickled,
which makes sense to me, then this is not issue.

3) Will this slow things down?


I may just give up and write a ThreadSafeFileDescriptor class, allowigng 
threaded servers to use it while non-threaded servers can use the 
regular FileDescriptor.


Index: abstract.py
===================================================================
RCS file: /cvs/Twisted/twisted/internet/abstract.py,v
retrieving revision 1.2
diff -r1.2 abstract.py
1d0
<
3c2
< import types
---
 > import types, operator, string
22c21,24
<
---
 >
 >     def __init__(self):
 >         self.unsentChunks = []
 >
53a56,63
 >         numChunks = len(self.unsentChunks)
 >         if numChunks:
 >             chunks = [self.unsent,]
 >             for i in range(numChunks):
 >                 chunks.append(self.unsentChunks.pop(0))
 >
 >             self.unsent = string.join(chunks, '')
 >
80a91,92
 >
 >         This method is thread-safe.
86c98
<             self.unsent = self.unsent + data
---
 >             self.unsentChunks.append(data)
88c100
<                 if len(self.unsent) > self.bufferSize:
---
 >                 if len(self.unsent)  + reduce(operator.add, map(len, 
self.unsentChunks)) > self.bufferSize:
Index: tcp.py
===================================================================
RCS file: /cvs/Twisted/twisted/internet/tcp.py,v
retrieving revision 1.10
diff -r1.10 tcp.py
41a42
 >         abstract.FileDescriptor.__init__(self)
228,229c229,230
< 	self.factory = factory
< 	self.backlog = backlog
---
 >         self.factory = factory
 >         self.backlog = backlog






More information about the Twisted-Python mailing list