[Twisted-Python] newbie confusion - puzzling reactor response

[exarkun at twistedmatrix.com]

On 07:35 pm, rich at noir.com wrote:
>exarkun at twistedmatrix.com wrote:
>>It isn't `doRead`'s job to close the file descriptor.  At most, it's
>>`doRead`'s job to signal that the descriptor is no longer worth 
>>keeping
>>open by returning something like an instance of ConnectionDone or
>>ConnectionLost.  Then the reactor will call connectionLost on your
>>object and you can close the file descriptor there.
>Thank you!  That's the info I was looking for.
>>The documentation for how this all works could probably be improved.
>>Once you figure it out, would you mind submitting a patch?
>I don't think I've even seen mention of ConnectionDone in the doc yet.
>
>I think there's a design decision here about the doc.  It seems to me
>that the return codes are part of the interface and as such should
>probably be documented in twisted.internet.interfaces.py whereas 
>there's
>almost nothing there now.  Instead, some of this is in
>twisted.internet.abstract.  Granted, the distinction between an 
>abstract
>class and a "zope.interface.Interface" is subtle.
>
>As a newbie, the interface is the thing I find first and most easily.
>I'm directed there by the doc and by the reference material.  I have to
>dig around to even notice that the abstract class exists.

You're absolutely right.  These interfaces should be documenting the 
meaning of the return value for these methods, since it is an integral 
part of the required interface.
>>Also, you won't accomplish much by adding a file descriptor for a 
>>normal
>>file to the reactor.  Select, poll, etc, will always indicate that 
>>such
>>descriptors are both readable and writeable.

>Er... on second thought... isn't there still a utility in asynchronous
>file io which yields to the reactor?
>
>It may be always readable/writable, but if I simply read/write, I'll
>block the process for as long as that takes, block on read, block on
>write.  Whereas if I use async io on the descriptor and go through the
>reactor, I'm effectively yielding to the reactor and any other
>actionable descriptors on each loop as well as allowing my reads and
>writes to happen simultaneously.
>
>Or am I missing something?

There could be utility in such, but Twisted has no support for it, 
largely because actual support on various platforms is still pretty 
ragged.

On Linux, you can get the aio_* family of functions, but they're pretty 
crap.  They have tons of limitations (only block-aligned reads allowed, 
only a certain number of outstanding operations (system wide) at a time, 
etc, and the failure mode for not complying with these limitations is 
that the APIs block).

It's a bit better on Windows, so someone could probably fashion an 
extension to iocpreactor for this.  There isn't a lot of developer 
attention focused on implementing Windows-only extensions right now, 
though.

Jean-Paul