[Twisted-Python] Using StandardIO and pipes.

[Brian Warner]

On 8/9/12 10:18 PM, Glyph wrote:
> 
> The issue is that twisted.internet.stdio considers your protocol to
> only know about one kind of connection loss: i.e. that your entire
> transport has gone away. So, when the input is closed, the output file
> descriptor is immediately closed as well.
> 
> Unfortunately - and I think this is a bug in Twisted - the input
> connection being lost causes the output connection to be immediately
> and somewhat aggressively closed. The difference between "echo" and
> typing interactively is that you send "test_item_id\n", but echo sends
> "test_item_id\n" *EOF*.

Incidentally, I think this was the problem I ran into in Foolscap,
trying to build a tool that lets you run individual shell commands
remotely. I was able to stretch the three file descriptors (stdin,
stdout, stderr) over the wire using a callRemote() for each invocation
of dataReceived(), but had similar issues when closing one side of the
connection. The easiest way to trigger this was to run /usr/bin/sort,
which necessarily waits until the input has been closed before it will
produce any output. So you run "cat data.txt | flappclient run-command",
flappclient reads from StandardIO, sends the data to the far end,
notices the EOF on stdin when the data is finished, sends a "stdin is
closed" message to the remote side, receives the return data from sort,
then tries to write it to the local stdout (which sometimes failed).

I think I saw problems in the opposite situation too: if stdout were
closed (because our caller didn't want to hear anything further from
us), the stdin would no longer accept data. I don't remember how I
triggered this situation.

My workaround is here, in case you find it useful:

 https://github.com/warner/foolscap/blob/master/foolscap/appserver/client.py#L64

It worked for me at the time. (since then I've seen some trouble when
running git-receive-pack over this connection, which may or may not be
related, so I won't claim it's foolproof).

cheers,
 -Brian