[Twisted-Python] Using StandardIO and pipes.
lacrima.maxim at gmail.com
Mon Aug 13 08:18:32 EDT 2012
Thanks for the help! I used an attached example and after implementing
IHalfCloseableProtocol it works! Thanks a lot!
Still, Twisted seems hard to me, probably because I don't quite understand
how to use and combine Twisted's interfaces.
With your help I can now connect my program to another process using pipe.
But, for example, I want to be able to supply data to my program not only
via stdin, but also as command line arguments. I.e. instead of
$ echo foo | ./item.py
I want to do:
$ ./item.py foo
My feeling here is that I can reuse the same protocol (EchoItemProtocol)
here, but now instead of StandardIO I need my own factory (or transport, or
whatever it is named) class that knows how to read command line args and
talk the same protocol. Sure I can read arguments using sys.argv[1:], but
what do I do next to properly pass that data to the protocol? Does my class
have to implement ITransport or IProducer etc. to talk EchoItemProtocol
similar to StandardIO?
I hope my question is clear.
Thanks in advance.
On 10 August 2012 08:18, Glyph <glyph at twistedmatrix.com> wrote:
> On Aug 9, 2012, at 7:39 PM, Andrew Bennetts <andrew at bemusement.org> wrote:
> Maxim Lacrima wrote:
> So my conclusions are:
> 1. When I write to stdin of my script from keyboard then
> `self.transport.write()` works as expected.
> 2. When I use pipes or redirection to my script `self.transport.write()`
> doesn't produce any output.
> 3. If I replace self.transport.write() with print statements it works.
> Buffering, perhaps?
> As a quick test, try adjusting your callback to loseConnection on the
> after to the write, that should flush the buffers. If buffering turns out
> be your problem, and you really do expect your pipes to be at most
> line-buffered, you'll need to arrange for the buffering on the stdin and
> file descriptors to be changed. I don't think Twisted has that builtin,
> so you
> may need to do that before creating the StandardIO object using the regular
> Python APIs.
> It's not this kind of buffering problem. I was actually able to reproduce
> it by replacing "Item.get()" with a deferLater. (In the future, please
> perform these types of replacements yourself before posting your examples;
> it's much easier to help out with complete, runnable programs.)
> The issue is that twisted.internet.stdio considers your protocol to only
> know about one kind of connection loss: i.e. that your entire transport has
> gone away. So, when the input is closed, the output file descriptor is
> immediately closed as well.
> Unfortunately - and I think this is a bug in Twisted - the input
> connection being lost causes the output connection to be immediately and
> somewhat aggressively closed. The difference between "echo" and typing
> interactively is that you send "test_item_id\n", but echo sends
> "test_item_id\n" *EOF*.
> Luckily you can work around this, by explicitly asking Twisted for
> notifications about half of the connection being closed. You do this by
> implementing IHalfCloseableProtocol.
> Although Twisted should be better about how it handles shutting down
> stdout, this nuance *does* reveal some logic that your code is missing:
> once your input is done, you don't have any concept of waiting for your *output
> *to be done before shutting everything down. So, if you add both the
> IHalfCloseableProtocol implements declaration to receive the relevant
> notifications, and some logic to keep track of when all the output has been
> written, you can get exactly the behavior that I assume you want.
> I've attached an example that does all of these things with a simple
> timer, since this is something that Twisted probably needs to document a
> bit better.
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Twisted-Python