[Twisted-Python] asynchronous python generator

[exarkun at twistedmatrix.com]

On 04:43 pm, jeandaniel.browne at gmail.com wrote:
>Hello,
>
>I wrote a small client protocol which connects to a notification
>server. The client role is to connect, and then to print the
>notification which comes from the server until the server says "stop"
>(a netcat server do just fine, I use "nc -C -l 6789"). I would like
>the API for this client protocol to be compatible with the "for"
>python loop. Here is the client
>
>class Notif(basic.LineReceiver):
>
>    def lineReceived(self, data):
>        self.d.callback(data)
>
>    def __iter__(self):
>        return self
>
>    @defer.inlineCallbacks
>    def next(self):
>        self.d = defer.Deferred()
>        notif = yield self.d
>        if notif=="stop":
>            defer.returnValue(Failure(StopIteration()))
>        else:
>            defer.returnValue(notif)
>
>Here is an example of how to use this client:
>
>
>@defer.inlineCallbacks
>def gotConnection(conn):
>
>    for notif in conn:
>        print notif
>
>    reactor.stop()

This is an invalid use of inlineCallbacks.  Only generator functions 
(functions which use `yield`) can be decorated with inlineCallbacks.
>
>c = protocol.ClientCreator(reactor, Notif)
>c.connectTCP("localhost", 6789).addCallback(gotConnection)
>reactor.run()
>
>Except that is does not work: I think that the next() method of the
>generator is indeed automatically called by the "for" machinery but
>its return value is not yielded as the inlineCallbacks requires it.

Quite so.  And why would it be?  That's not what `for` does in Python.
>
>This works for instance, and it is very close to a for loop:
>
>@defer.inlineCallbacks
>def gotConnection(conn):
>
>    while True:
>        try:
>            print (yield conn.next())
>        except StopIteration:
>            break
>
>Does someone knows how to make the for loop work with data coming from
>network requests?

You didn't have to switch to a while loop here.  The simplest fix for 
your for loop is this:

    @inlineCallbacks
    def gotConnection(conn):
        for lineEvent in conn:
            line = yield lineEvent
            print line

But aside from that...

No, you must have the `yield` somewhere in the function.  If you want 
something less scrutable than this, then you probably want something 
like corotwine.

Jean-Paul