[Twisted-Python] Re: Improving spawnProcess and friends

[Nuutti Kotivuori]

Nuutti Kotivuori wrote:
> I shall ponder these things for a while - and start implementing the
> next phase when I have a good idea on what it should be.

Okay. I think I have some sort of an idea now. But, first, a problem.

If we are to use ProcessProtocol as having some logic that sets up the
communication channels for a child process, there needs to be some
function called before the process starts. But.

If there's a function in ProcessProtocol that is called before
makeConnection() (and hence connectionMade()) is called, there is no
self.transport in the ProcessProtocol. That would mean that the
current Process instance would need to be given as an argument, or
assigned separately (something else than makeConnection() called
before?). Both approaches do seem a bit yuckupucky to me.

Or, we could call makeConnection() on the ProcessProtocol before the
child process is started. But that sounds even worse. It is a change
from the current behavior and the child's pid isn't known at that
time. And then we would have to have another method called when the
child is spawned. And in general, I think connectionMade() should mean
connection made, and not pre-setup starting.

So, how about we bring the same thing that is in every other protocol
to processes as well? Factories.

Let's say that there is a ProcessProtocolFactory class. That
ProcessProtocolFactory is responsible for setting up the communication
pipes. That ProcessProtocolFactory determines which Protocol instances
are to be created for handling the communication pipes. The actual
ProcessProtocol behaves the same as it used to - no change there.

And perhaps UNIXProcessProtocolFactory, that would be responsible for
the setuid, setsid, etc. things, would be the platform abstraction
method that glyph was wanting for?

Heck, there might even be a RestartingProcessProtocolFactory, that
would automatically restart the child process when it dies, with an
exponential backoff? And a ProcessProtocolCreator that could be
similar to ClientCreator. Or, perhaps even the factory can return a
Deferred from some setup method if setting up communication pipes
require things that cannot be done synchronously? The flexibility just
seems so much better here.

Backward compatibility with reactor.spawnProcess() would be achieved
simply by having that function create a ProcessProtocolFactory that is
able to handle the old childFDs arguments and such - and all
compatibility glue would be in that function alone and it could be
deprecated. New code would use an interface that would take a
ProcessProtocolFactory instance as a parameter.

How does this sound? If nobody yells murder, I'll be starting to build
a rudimentary implementation of this to see how the API would
naturally turn out to be and then submit it here for review.

-- Naked