[Twisted-Python] Understanding the IOCP reactor and adding spawnProcess

Justin Johnson justinjohnson at gmail.com
Mon Jul 11 13:16:35 EDT 2005

See response below.

On 7/11/05, Jp Calderone <exarkun at divmod.com> wrote: 
> On Mon, 11 Jul 2005 10:52:14 -0500, Justin Johnson <
> justinjohnson at gmail.com> wrote:
> >I am attempting to add spawnProcess to iocpreactor. In order to begin 
> this
> >task I've had to do a lot of reading on Windows network programming,
> >specifically the various Windows I/O methods, to attempt to understand 
> what
> >win32eventreactor and iocpreactor are doing, and also just increase my
> >understanding of how reactors work in general. To understand the various
> >Winsock 2 methods that both of these reactors rely upon, I read chapters 
> 1-5
> >of Network Programming for Microsoft Windows[1].
> > Before actually attempting to add spawnProcess, I would like to present 
> how
> >I think iocpreactor works and how I think I should add spawnProcess, and
> >hopefully be corrected or confirmed in my understanding. If I'm too vague
> >there's a good chance it's because I don't understand it very well. 
> Please
> >feel free to point out things that you might think are obvious but aren't
> >sure I understand.
> > How iocpreactor works
> >---------------------------------
> >
> > [snip]
> >
> > How to add spawnProcess
> >---------------------------------------
> >
> > 1. Create the processes via Windows APIs and associate their
> > stdout/err with with the IOCP via CreateIoCompletionPort calls.
> > 2. Close stdin.
> Why close stdin? How will you write to the spawned process?

 I see, so I won't close stdin and people can do that in their 
ProcessProtocol in connectionMade or something if they want.

> 3. Notify the ProcessProtocol via protocol.makeConnection (not sure
> > why, looking at win32eventreactor)
> makeConnection is the method that's actually part of IProtocol (even 
> though ProcessProtocol doesn't implement IProtocol, it still adheres to its 
> API in this regard). Generally, all it does is set the .transport attribute 
> on the protocol instance and then call connectionMade. This might seem 
> pointless in the case of processes, but since it maintains consistency with 
> other kinds of transports, it is useful.

  So the Process class is the transport? In win32eventreactor, 3 threads are 
started that loop on reading stdout and stderr and writing data from the 
outQueue to stdin. It seems to me that I would not use these threads, but 
instead have each of these 3 file handles associated with the IOCP. Then the 
methods defined in ops.py would be used to handle events on those handles, 
resulting in transport methods being called. These transport methods would 
be methods on the Process class, which would result in methods on a single 
instance of the ProcessProtocol being called. This single instance of the 
ProcessProtocol would be shared by all of the 3 file handles. Am I 
understanding correctly? Are write and loseConnection the only methods the 
Process would need to implement from ITransport?

> 4. Receive data from stdout/err via the completion port by calling
> > GetQueuedCompletionStatus from within doIteration. Is this really 
> possible?
> > ProcessProtocol's methods won't get called appropriately by letting the
> > existing callbacks in ops.py make calls to the transport (e.g.
> > connectionDone, readDone)?
> >
> > Thanks for your help.
> >Justin
> > [1] http://www.amazon.com/exec/obidos/ASIN/0735615799
> >
> Jp
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20050711/5b21c0a0/attachment.htm 

More information about the Twisted-Python mailing list