[Twisted-Python] Some suggestions

Itamar twisted at itamarst.org
Mon Aug 13 11:28:51 EDT 2001

Glyph Lefkowitz wrote:

>>My main concern is *safety*. For example, I'm pretty sure making 
>>connectionLose() synchronized is not enough to make it thread safe. 
> Why not? If it's synchronized, and doWrite is synchronized, then I can't
> figure out what the safety issue is.  Did you try, and experience some, or
> do you know of some potential ones from reading it?

It is not thread-safe. This is because the whole addReader / addWriter / 
doSelect etc. code in internet.main isn't thread safe, so even if the 
FileDescriptor is internally thread safe its interactions with the list 
of readers and writers aren't.

For adding writers/readers this is less of an issue, although still 
potentially buggy. Removing on the other hand can lead to select() calls 
on already closed FilEDescriptors (as in the traceback I emailed the list.)

For example, look at the select.select(readers.keys()....) call. If the 
FileDescriptor is closed by a thread right after the .keys() call then 
select() will run on a closed descriptor and barf.

Now, messing around with this so it's thread safe starts getting way too 
icky. I would like therefore to redo the Scheduler so that it is thread 
safe, and use it to schedule closing and writing to transports.

I think the way this should work is that each thread should have an 
associated Task to which it adds methods to be called. That way a single 
thread can't monopolize the scheduler. This means however that it should 
be possible to add works to tasks after they've been added to the scheduler.

I think the best way to do this is to use python's built-in Queue class. 
In order to not slow down non-threaded apps, I will implement a 
non-thread safe Queue class with the same interface (or the non-blocking 
parts of the interface, anyway) so that you can do:

	from threadable import Queue

And get the appropriate Queue class. Make sense?

More information about the Twisted-Python mailing list