[Twisted-Python] nonblocking stdio
exarkun at divmod.com
Wed Apr 27 20:33:29 EDT 2005
On Wed, 27 Apr 2005 16:59:56 -0700, "Sir.shz" <sir.shz at gmail.com> wrote:
>Hi, I'm trying to write a test tcp client which sends some commands to the
>server, I'd like the
>client to take the commands from stdin, and sends it to the server. The
>following code segment
>doesn't seem to send anything, is it because the while loop is actually
>blocking? what's the right way to
>accomplish this? Thanks.
There are a few things that you may want to do differently in your MyClientProtocol class.
> def lineReceived(self,data):
> stdout.write("From Server:" + data+"\n"),
stdout.write (assuming that's sys.stdout) can block. This doesn't happen too frequently, except under certain conditions: for example, if you have piped the output of the program to less and have been less than diligent than paging to the end of output (less will read so much beyond what it can display and then stop reading, which means your process will no longer be able to write, and so will block); another similar scenario occurs with screen(1) when switched into copy mode.
If this is something you want to worry about, twisted.internet.stdio may be of use. It provides a simple way to get stdout into non-blocking mode, and then does buffering for you in cases where it is required.
> def connectionMade(self):
> while True:
> cmd = raw_input("Enter command:")
There are a couple problems with this definition of connectionMade().
First, a "while True:" loop like this is pretty much off-limits in a Twisted application. You're right about why - it blocks the entire reactor, preventing any work from being accomplished. Even the calls to transport.write() inside this loop will probably result in nothing, since the reactor can't get to the socket associated with that transport and perform actual writes. I'll come back to how you can achieve this behavior without blocking the reactor in a moment.
The other problem is the use of raw_input(). Like the while loop that contains it, raw_input() will also block the reactor. You need another way to get input from the user, a way which doesn't block. There are a couple alternatives in the most recent Twisted release. The first I mentioned above, twisted.internet.stdio. In addition to giving you non-blocking output, it gives you a callback-oriented way to deal with input. It won't give you readline-like line-editing abilities, though. It just deals with straight-up byte streams. The other possible solution is new in Twisted 2.0 and part of Conch, twisted.conch.insults. This also provides a callback-oriented way of getting input, but also allows you to define behavior for various function keys. It comes with a very rudimentary line editor that supports the arrow keys, typeover and insert mode, as well as input history (while this is less featureful than readline, it has the advantage of being much more easily extensible, and in Python ;).
You can find examples of twisted.internet.stdio in the Twisted core examples directory, or online at:
For more advanced examples of Insults, check out:
as well as twisted/conch/stdio.py in Twisted 2.0 (also viewable online at
Note that twisted/conch/stdio.py does not present a stable interface (it is likely to change incompatibly in the future), so you should consider it an example of how to use insults, rather than a library to use from your application.
Also note that these solutions are POSIX-only. They will not work on Windows.
More information about the Twisted-Python