[Twisted-Python] Soon to be not-a-newbie?

[L. Daniel Burr]

Hi Simon,

On Fri, 25 Jan 2008 14:26:34 -0600, Simon Pickles <sipickles at hotmail.com>  
wrote:

>
>
> L. Daniel Burr wrote:
>
>> No offense intended, but what is so inaccessible about an event loop?   
>> Please
>> understand, I hear people say that Twisted is hard to grok all the time,
>> but I really don't get *why*.
>>
> Conceptually twisted is very simple. Thats what attracted me to it in  
> the first place. In reality, twisted (I suspect due to its incredible  
> flexibility) is mind boggling. The plethora of options available to the  
> uninitiated is overwhelming. Eg:
>
> Okay, so to make a simple client/server what do I do?
>
> Well, you create a class which overrides a factory, which takes a  
> protocol, which has some overriden functions which do different  
> predefined tasks. Easy.
>

OK, here's the first thing I notice.  You make no mention of interfaces,
which I think are key to understanding Twisted.

> (btw, ALL tutorials never make clear which functions are overridden, and  
> which are newly-defined)
>

There's a good point.  Perhaps the tutorials should provide more links to
the API documentation, specifically, to the interfaces being implemented
by these classes.  Once you look at the interfaces a given class  
implements,
it is easy to see what is being overridden.

> So can I do something similar to do another job?
>

Yes, you can, if you are thinking about interfaces, rather than always
in terms of concrete classes.  I hadn't noticed this before in my own
work, but that's precisely why I find Twisted fairly easy to grasp.

> No you need a totally different set of classes which don't follow any  
> similar pattern to the first, probably have very inaccessible (ie,  
> forgettable) names. In this case you need to connect an A to a B to a C  
> with a D and E as params, and F as a callback and inherit from a WIDGET!  
> aaaargh! :)
>

I note the smiley, but still, you're exaggerating quite a bit here.  For
most jobs, you do the same basic dance:

- create a protocol
- create a factory
- create a driver script, or a service if you need a daemon.

Now, some problems are really this simple.  Some will require more
work in terms of creating server-specific and client-specific factories,
etc., but even that follows the basic steps above.

The case I'm describing above is one where you "roll your own" stuff,
using the basic, low-level parts of twisted.  In cases where you want
to use higher-level parts, like, say, twisted.web, then you do have
to read the appropriate API docs, or look in the examples directory.

>>> ... but if you persevere, its great! You just have to leave part of  
>>> your brain behind in a field in hampshire :)
>>>
>>
>> Again, good stuff.  Which part do you have to leave behind?  The part  
>> that
>> expects code to execute synchronously, or something else?
> IMHO, Twisted seems to be written by many people over time, without a  
> rigorous design pattern in place first, more of an iterative process.  
> This would account for the fractuous nature of the interface. Its kinda  
> like learning English. There are rules but most of the time they don't  
> apply.
>

Sorry, but I'm going to pick on you a little bit here.  When I hear
someone say "rigorous design pattern", I feel compelled to ask them
to explain what they mean by that, and to point me to examples where
such a "rigorous design pattern" has been applied, so that I can
see what kind of library or framework is produced by the application
of said pattern ;)

Seriously, your observation could be applied to virtually any
open-source project in existence, simply due to the fact that more
than one developer has worked on the code over time.  In Twisted's
defense, I'll note that there is a coding standard; modules and
APIs are marked as being stable/unstable; there are unit-tests
covering a large portion of the code-base; a buildbot system provides
continuous integration so that changes which break trunk get reverted
swiftly.  The list goes on, and I'd argue that such things are
emblematic of a well-designed framework.

I do think that any of the core developers would freely admit that
some parts of twisted are better than others, but the core design
of twisted is, in my opinion, consistent and solid.  Obviously
I would think that, given that I never encountered a real problem
getting into Twisted.

I suppose the fact that the first programming I ever did involved
an event-loop driven system accounts for my familiarity with
the concepts in Twisted.

> On the other hand, maybe Twisted just suddenly made sense after  
> Stackless did ;)

There is some overlap there, in that both Twisted and Stackless
require you to think cooperatively rather than preemptively.

>>
>> the recent spate of inlineCallbacks-related posts alarms me.
>>
> Sorry about those :)
>

No problem :)  I'm not a fan of inlineCallbacks, because I
think it represents a crutch for people who either don't
like the way "normal" deferred-using code looks, or want to
write code that looks like blocking code (write blocking code,
add decorator, add yield keyword, stir gently).  Someone, I think
it was tekNico, made a decent argument concerning readability,
but I consider that to be too subjective.  Certainly I would
concede that some folks really like inlineCallbacks.  I just
don't want to support any code they write ;)

Thanks for generating the discussion,

L. Daniel Burr