[Twisted-Python] which async framework?

[Chris Withers]

On 12/03/2014 08:53, Laurens Van Houtven wrote:
> Twisted has a thing called Cyclone, which I hear (but that's only
> hearsay) gives you Tornado's API on top of twisted, so you get all of
> the stuff below for free.
>
>      From my side, I'm looking to experimentally build a network testing
>     tool that will need to speak a fair few protocols, both classic tcp
>     and multicast-based, and have a web api living on top of it that
>     most likely will have a websocket for pumping data to the browser.
>     It'll also need to write out JUnit-compatible xml results, but
>     that's like the easiest bit ;-)
>
> I don't know which protocols you are interested in specifically;

Aiming low (insert sarcastic look here), here's the first few:

http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_FIX_Specification.pdf

http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_Binary_Order_Entry_Specification.pdf

http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_PITCH_Specification.pdf

http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_MC_PITCH_Specification.pdf

> Twisted has a third party project called txsockjs which works
> excellently, both by itself and in the context of other (HTTP)
> resources. sockjs is a protocol which is basically "websockets, damnit",
> even when the consumer is a bad browser like IE6.

Thankfully, I only have to support sane browsers...

> Twisted comes with a threadpool-backed WSGI server. (When I say
> threadpool-backed I mean that the WSGI requests are handled in threads;
> the IO itself of course comes from the Twisted reactor).

Yeah, I remember that, and I remember liking it :-)

> I find twisted to be a great tool for writing protocol implementations.
> I have written tools for querying all sorts of gnarly proprietary
> protocols over all sorts of gnarly transports (packet radio; it's
> totally a thing), and more recently for doing crazy things like
> multiplexing stream transports over existing RPC protocols. (Like, you
> see a local port come up, and that actually creates a virtual stream
> connection over an existing RPC thing to some virtual server on the
> other end of the wire: https://www.youtube.com/watch?v=W_jEIvugwes).

Cool!

> Twisted has had so many people write so many protocols in it that also
> the testing tools (MemoryReactor, StringTransport) are great. Especially
> if you are writing something very close to a wire protocol you will
> undoubtedly enjoy those amenities. There's also tons of composable
> things for receiving delimited lines, nestrings, etc. It's hard to tell
> what you will be looking for since I don't know details about your
> protocol, but having written more than a few protocol implementations
> I'm going to wager a guess and say Twisted has it or a third party thing
> for twisted has it.

Yep, certainly sounds good. I guess I have concerns about 
discoverability and making sure I'm picking the right things to use 
rather than re-implementing things when I don't need to.

What's the best way for me to find things I should be using? I guess my 
fallback position is to ask lots of clueless questions on this list and 
hope I don't annoy to many people, is that viable?

> Twisted, emphatically and without reservation, for all the above
> reasons. It's stable. All the stuff you need has been tried and tested
> extensively in production. It runs on PyPy, and usually a damn sight
> faster than on CPython, too.

Didn't know about the PyPy thing, that will be interesting if I ever hit 
performance problems...

> A common criticism of Twisted is that it "takes over" your codebase. I
> am speaking at PyCon in about a month to demonstrate that that isn't
> true. (It just looks that way, because once people use it, they don't
> want to go back... ;-))

Gutted I can't make it to PyCon this year, look forward to watching the 
video!

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk