[Twisted-Python] Re: Teach Me Twisted?

[Steve Holden]

Jean-Paul Calderone wrote:
> On Fri, 25 Jan 2008 18:49:46 -0500, Steve Holden <steve at holdenweb.com> 
> wrote:
>> Cary Hull wrote:
>>> Hey, I think some Twisted related sessions at PyCon would be great.
>>> Based on what's been discussed so far, I would like to humbly suggest
>>> the following (kind of breaking it down):
>>>
>>> 1) Simple line based chat server/client.
>>> 2) Same thing except using AMP.
>>> 3) twistd and plugins
>>
>> These are things I would really like to get to, but I'm pretty sure 
>> half an hour isn't going to cover it. Maybe we should be thinking BOF 
>> instead, then we can take an evening over it under less stressful 
>> conditions (and, who knows, even drink a beer or two in the process).
> 
> Maybe I can get you to go a little bit further than that in restricting
> what you'd like to see covered.
> 
Aah, sweet sanity! Sorry about the delay in my reply, I wanted to wait 
until I could do this post justice (and it's not like PyCon is tomorrow).

> Off the top of my head, and only continuing until I get bored with making
> this point, here's what a "Twisted" session could cover:
> 
>  * event driven programming

Please. I've used event-driven scenarios in GUI frameworks, but I've 
never seen anything (please note: this does *not* imply that nothing 
exists) that makes clear the relationship between the Twisted reactor 
and the code surrounding it.

>  * Deferreds (making them; using them; debugging them; defgen;        
> locks/semaphors)

These are in essence the fundamentals, no? A complete picture of the 
lifetime of a failed and a successful deferred would, I feel, go a long 
way towards filling in the picture. Plus how deferreds can interact with 
each other?

>  * core reactor APIs

This too should go into the "must know" category, right?

>  * scheduling

If we could cover everything up to this point I'd be happy. Like you I 
think it's important to limit the scope if we want to achieve something 
worthwhile, and I can't spend *all* my PyCon time on Twisted. It will be 
up to you guys to make it enjoyable enough to keep me there :-)

Hopefully with community guidance I can lay the foundations for a 
mechanism (a pedagogy, if you will) that many people can use to flesh 
the rest out.

>  * implementing existing (TCP; UDP; IP) protocols

What, you mean they aren't all implemented yet? :-)

>  * designing new (TCP; UDP; IP) protocols

Good. That Twisted can get access to the raw network layer is something 
I wasn't aware of.

>  * using existing protocols in twisted core

Probably more essential than protocol design and implementation, I'd 
have thought.

>  * integrating a protocol implementation with cred

To make it an authenticated protocol? Haven't met cred yet.

>  * process control/interaction

I suppose one of the issues with twisted, like other asynchronous 
frameworks, is that you can screw things up really badly just by 
thoughtlessly blocking somewhere inside a handler. So this is probably 
important.

>  * implementing a realm
>  * implementing credentials/checkers
>  * writing services
>  * writing plugins (twistd; lore; cred)
>  * handy general-purpose python utilities
>  * adding new kinds of event sources to the reactor
>  * gui/pygame/etc integration
>  * threading
>  * using SQL databases
>  * Perspective Broker
>  * writing unit tests
>  * using trial
>  * extending trial
> 
While the above remaining core issues are all valuable, I don't see us 
getting to them at PyCon.

> Wandering outside of the core of Twisted:
> 
>  * serving web pages
>  * running a chat server
>  * extending the chat server
>  * jabber (jabber; jabber; jabber; sasl; jabber)

I gather you think jabber is significant?

>  * marking up documentation

The more widely this knowledge is disseminated, hopefully, the better 
the documentation will be.

>  * dns servers and clients
>  * usenet servers and clients
>  * ssh/sftp clients/servers
>  * telnet clients/servers
>  * asynchronous terminal interaction
>  * terminal widget rendering
>  * smtp client/server
>  * imap4 client/server
>  * pop3 client/server
> 
> Are any of these the things you have in mind?  Can I get you to pick
> a very small number of them (1 or 2 per hour at most, I suspect; some
> are smaller, others are quite large)?  Or did you have something else
> in mind?
> 
Your detailed knowledge of the structure of Twisted lays bare the huge 
amount of work that *could* be done. However, there may be perfectly 
adequate existing materials on some or all of these topics, and I am 
happy to be pointed towards them.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/