[Twisted-web] Re: Thoughts on documentation, wading into Twisted, Nevow, etc.

[Phil Mayers]

Cory Dodt wrote:
>>
>> I think that worrying about attracting an audience that prefers to use the
>> thread-based approach is doomed to failure, in the same way that inviting
>> a conservative to adopt liberal ideals is doomed to failure.

Yes!

> 
> I can actually quibble with this notion from personal experience: I used
> Twisted before I did more than very trivial experiments with threading.  I
> read and understood in concept the problems with threading.  Only later on,
> when I tried to use threads in my Twisted programs did I encounter the hazards
> of threading.

Same here.

Ages ago, at about the age of 21 when I was less smart but harder 
working and NIH seemed like a reason, I wrote a parallel SNMP poller. I 
copied the mainloop straight out of fping and ported it to python.

It was trivially obvious even in my naivety that no thread-based 
approach was going to serve ~3000 simultaneous "threads". I looked at 
Stackless, but it (foolishly IMHO) was derided by the BDFL. I didn't 
know about Twisted or asyncore (shudder - though when I did finally see 
it I retched on my keyboard). But I immediately understood the problem 
and the correct solution.

Twisted was something I found later on to do away with my custom loop 
code and concentrate on application code. Now I can tack a web interface 
onto the poller. Oh, and an SSH CLI. Oh, and an SNMP *proxy*.


> 
> This is important because it anecdotally demonstrates that programmers don't
> always go straight to threads.  If they understand from the start that
> concurrency is going to be hard, they will be (as I was) suspicious of what
> threads do, and might (as I did) seek experts who've done this stuff before.

If Python were Erlang (i.e. had stackless), Twisted would not exist. But 
it's not, so it does.

> 
> I think I understand the problem.  The problem is that concurrency is hard.
> If you're a new programmer, and you want to start with "a simple network game"
> (because somebody told you not to start with Doom), you're going to discover
> all kinds of complexity that your brain isn't formatted to handle.

Yep. Simple problems have simple solutions. Complex problems - anyone 
even contemplating solving them has a good chance of knowing why the 
simple solutions have failed.

> 
> It's not about how smart you are, it's about whether you have the relevant
> experience to write an application that actually needs Twisted.  I think

Yes, yes, yes!

> anyone who actually has the experience necessary to tackle an application with
> concurrency is going to find the doc a lot more friendly than someone who
> doesn't.  I don't know that that's Twisted's fault.

It certainly is not.

The previous two mails encapsulate to my mind the core of the issue. 
Anyone who needs Twisted, knows it. Anyone who doesn't cannot be 
persuaded otherwise. They will either discover it, or never need it (and 
more power to them - but that's not me).

Though it rankles, maybe advertising Twisted as a "specialist" framework 
would make more sense?