[Twisted-Python] So... Python 3.4 is getting its own async I/O system

[Glyph]

On Mar 18, 2013, at 8:00 PM, Tim Allen <screwtape at froup.com> wrote:

> In Guido's keynote at PyCon 2013, apparently he talked about adding an
> async I/O module to Python 3.4.

> At first glance, the proposed reactor API looks very much like Twisted's
> (or, to be fair, GTK's, or possibly any number of other async event loop
> I'm less familiar with)

Importantly, the proposed loop API is like Twisted (in that it has callbacks for the receipt of data and the emptying of send buffers) and _not_ like GTK (in that it doesn't call handle_read() and hope you can handle a socket object).

> but rather than Deferreds and callbacks, the API
> will be based around Futures (similar, but not identical, to Python
> 3.2's concurrent.futures.Future class), and an inlineCallbacks-style
> decorator for generators.

Mostly accurate as far as I know.

> I know Deferreds are awesome, and I don't know much about Futures (and
> I know Twisted core developers have given negative reviews of
> other Deferred/Promise/Future implementations in, say, JavaScript
> libraries before), and inlineCallbacks seems to have a negative
> reputation among experienced Twisted users. Is there anybody on this
> list who knows more about this new PEP (maybe somebody who's at PyCon
> and saw the talk in person) who can give us an informed comparison with
> the current state of Twisted?

The interesting thing about the proposal from the perspective of the Twisted project is that it is an exciting opportunity for interoperability.  The availability of a standard-library, "blessed" event-loop API will give us something to write against, rather than Twisted and Tornado and Pulsar all having alternative main-loop APIs, and partial adapters between them.  In the future, it might make things like the GTK reactor obsolete, as it's possible that the PyGTK project would include their own Python-standard event-loop adapter.

Twisted is an amazingly full-featured and mature event-driven framework; it doesn't make sense (and isn't fair) to compare a minimalistic main-loop to it.  Tulip does not contain an IMAP client or an SSH server; it does not contain any XMPP logic.

As far as 'yield from' coroutines are concerned, I'm personally ambivalent, but I'm happy to ignore them.  The best part about this proposal is that anyone who wants to get this new thing does not need to abandon the extensive capabilities of Twisted over some tweak of syntax.  With only a little bit of specification-implementation glue, we can interoperate not just with things that do the equivalent of connectTCP/listenTCP, but we can bridge between Deferreds and Futures (as Guido's new not-quite-concurrent.-Future explicitly drops the concurrent.Future that Deferred can't implement, because it's serving the purpose of Deferred).

-glyph