[Twisted-Python] Is it just me ....

[Thomas Weholt ( PRIVAT )]

----- Original Message -----
From: "Andrew Bennetts" <andrew-twisted at puzzling.org>
To: <twisted-python at twistedmatrix.com>
Sent: Thursday, July 10, 2003 9:19 AM
Subject: Re: [Twisted-Python] Is it just me ....


> On Thu, Jul 10, 2003 at 02:02:47AM +0200, Thomas Weholt wrote:
> > or is Twisted ... well, complicated. I've been programming for several
years
> > and I don't consider myself to be dumb, allthough not a genius either
;-) --
> > but Twisted is hard to understand. Perhaps you guys find the docs simple
> > enough, but I just don't get a lot of the key concepts.
>
> I think Twisted isn't so much complex as large.  It has many pieces, and
can
> do many things.  Some of those pieces can be complex (e.g. PB or Woven),
but
> different people use Twisted for a lot of different things.  It's only the
> last month or so I've seriously looked a twisted.web; I've found plenty of
> use for Twisted in things like implementing simple (and not so simple)
> protocols.  In particular, simple multi-protocol support in one process
was
> a big drawcard for me.  Twisted made that stuff *much* simpler than any of
> the alternatives I saw.

Agree. I think resources are pretty easy to understand. Woven/Pages are not,
because there so much more happening that the user don't see. Resources has
a render-method which returns content you can send to the client. Or it
writes content to the client. Easy as pie. Woven on the other hand .... I'll
keep on modding the PicturePile-example and play around with the other
examples.

>
> > The fact that there are so few examples of running ( and uptodate as
> > mentioned by P. Lafoucriere ) apps available might be a clue. Just look
at
> > the whole RPYs-thing we had going some time ago. Web-development, or to
be
> > precise, running a full-scale webserver is why a lot of people look into
> > Twisted in the first place. The lack of example-apps and obvious
> > misunderstandings on how Twisted is meant to be used is what's holding
> > Twisted back, IMHO.
>
> This is an interesting point.  Most of the Twisted developers, I think,
> didn't come to Twisted for the web stuff, at least not primarily.  Thus
the
> sudden interest in Twisted purely for its web capabilities has caught the
> documentation a little off-guard.  I think people expect to see
> documentation a little more geared towards web developers, and instead
they
> get a bunch of docs that barely mentions web until the sixth chapter.

I've looked at and developed web-related things using Zope, implemented a
completly customized web-server based on a threaded
BaseHTTPServer-foundation, briefly looked at WebWare/SkunkWare/CherryPy and
at some point had a working web-something based on Medusa. They all fall
short in some place or another. Twisted is probably the closest I've come to
a server where I got full controll and at the same time can have a bunch of
working code to use. But as mentioned earlier, my webserver is not just
going to serve dynamic content, but also fetch alot of content from other
servers, so it's more than just a server. That's way threading/deferred are
so important, at least in my project, where there will be a lot of tasks
running inside a webserver doing jobs taking potentially along time,
fetching data from other servers, putting that data into a local database
etc.

>
> Also, Woven has been stable and well-documented for a really short period
of
> time.  It's not like it's been a year since it stabilised and was
> documented and we have nothing to show for it -- rather, it's been perhaps
3
> months, and we're just starting to see examples like Issues emerge.
>

Yes, and hopefully we'll see more very soon :-)

> Perhaps the confusion is that to fully use twisted web, you need to
> understand:
>    - Twisted basics, e.g. Deferreds and reactor.callLater
>    - Twisted Web basics, e.g. Resources
>    - Woven

Agree. But as I said earlier, Resources are pretty easy. Woven and deferreds
and reactor.callLater are completly different beasts.

> But you don't necessarily need full mastery of all that to be
productive --
> I've seen someone start with the PicturePile tutorial, and extend it to
have
> captions, nicer layout, and other things without major difficulty.  They
> didn't need Deferreds, they didn't need the reactor, and they didn't need
to
> use twisted.web.resoure.Resource directly, and so they had no problems
> building upon the parts of Woven that PicturePile demonstrated.

Yup. PicturePile has been and will be disected, ripped apart and put back
together in search of Woven-enlightenment. The problem is all the things
happening "automagically".
I need more testing to understand what is going on in the backgrounds.

> So ideally we don't need a "how to use a database in a web page", we
should
> just need "how to use a database in Twisted" and "how to use deferred
> results with Twisted Web".  [I think both of these topics are actually
> answered pretty poorly at the moment; I think I can feel another bout of
doc
> hacking coming on...]

Agree.

> > I really hope the new Twisted-community page can bring some simple, well
> > documented examples on how to do things. By the way things are going
we'll
> > probably have very good docs in a couple of months or so, but I want to
> > implement a feature-rich webserver based on Twisted now, as I've
described
> > and nagged about in my postings on this list several times.
>
> I don't see why we need a new "community" page to achieve this.  We have a
> perfectly good mailing-list and website -- if someone has a great example,
> we'll gladly use it!

The list and user-group has been very helpful and forthcoming so far, so
maybe not. The mentioning of more example-apps might have been what got me
to excited.

> > Anyway, I feel there is too much promise in the Twisted platform to let
a
> > few good examples scare alot of potential users away. I'm still working
on a
> > test-implementation of a webserver which hopefully will be easy enough
to
> > understand and deploy, extend and customize, written in a Twisted-way.
>
> What do you mean by "implementation"?  Have you seen Moshe Zadka's
> twisted-web Debian package that's being developed?

Eh ... no. All of my projects must run on both Linux and Windows, but if
there's a web-project based on Twisted anywhere I haven't seen allready I'm
very interested.

>
> > Anyway, I'm going to ask again; any example, small and simple, showing
how
> > to implement a webserver serving static content *and* dynamic pages, not
> > static pages like Issues or Bannerfish, using woven-forms and
> > user-authentication, in a Twisted-compatible manner would be
appreciated. If
> > I'm the only one feeling this way, I'll shut up, dive into the docs
again,
> > try some more and keep silent while doing so. Hopefully I'll be able to
> > learn Twisted and contribute something, not just nag, to a dedicated
group
> > of developers.
>
> There's no need to shut up -- if you're confused or unsure how to do
> something, then ask on the list and/or IRC.  I'd like to think we do a
> fairly good job at helping, and conversations on the list can and do lead
to
> improvements in the docs.
>
> -Andrew.

Thanks a million.

Thomas