[Twisted-web] wsgi and twisted.web{,2} again

Dot Fill dotfill at gmail.com
Thu Dec 4 08:51:20 EST 2008

I know, people on this list dislike WSGI as a matter of principle,
so don't kick me into head before I have spoken the rest...
Sorry for my English too.

I do really appreciate the original mood of nevow, the clean and creative
approach of twisted, that is to be said. AFAIK, they are both OpenSource
developments, and being developed by community. So the only thing they both
need is the wide acceptance by the public, to be enhanced more and more
and to keeping it alive too.

But here just want to say, that I am unhappy with the fact that nevow is a
framework of it's own server and is not a more general thing.
I suppose that I am not alone in that unhappiness.
Believe me, sometimes we can't choose all the software we will deploy for
a particular site. Sometimes WSGI hosting is easier to require/ask for.
So what about using twisted as WSGI application or WSGI middleware,
not just only as WSGI gateway?
Is it really technically impossible to run twisted.web2 application on
a WSGI hosting
together with some (thin?) layer of twisted, but without HTTP-server?
AFAIK there is no modules, that already do the job needed. Am I right?

I understand, that adoption of WSGI makes the usage of twisted-based HTTP server
unnecessary and that probably can't be tolerated by several people on this list,
as they have spent too much effort on developing the HTTP server in twisted.
But anyway, the availability of it's own dedicated HTTP server is not the
first argument when advocating the twisted methodology, believe me.
As for me personally - it is the last argument, but this is only IMHO.
The more valuable argument will be the possibility to plug twisted
into any existing
HTTP-server. Consider imaginary apache's "mod_twisted", just imagine
that's power and
flexibility if it will allow to pre-fork twisted process with
specified uids, send
some requests to that process via a secure channel like UNIX-socket.
(Just look at the mod_wsgi and you will get the idea.)

You may notice, that it is already possible with apache as reverse HTTP-proxy,
 - pure-HTTP reverse proxies cannot be nested (for obvious reasons),
 don't giving an example here.
 - HTTP-based application servers cannot securely live inside single
unix operating
 system instance. Consider student's server with Joe and Matthew as
 students, building their twisted application and deploying it on the same
 machine and behind HTTP-reverse proxy. They operate under different
 accounts, but common UNIX system makes no difference on who is trying
 to bind to TCP-socket for listening. Thus, once being drunk after holidays,
 Joe decides to show his coolhacker skills to Matthew and runs application,
 that bind itself continuously to Matthew's TCP port. Matthew sucks, his
 application fails to run. This was a real life example, and in real life
 it was fixed by employing SELinux on public student's server, and as a
 consequence, students lost the feeling of pure-UNIX system's behaviour,
 so that had it's impact on didactic issues of using student's server.
   That could be fixed easily if UNIX sockets were used instead of
 internet-sockets, but for that time we had no reverse proxy with
 UNIX-sockets supported nor we had working twisted application,
 implementing HTTP-over-UNIX-socket protocol.
 - indeed, you will probably (dis)agree that HTTP in it's design, history of
 deployment, together with complications of tons of differently behaving
 HTTP-client implementations, etc, etc is NOT A PERFECT solution for
 protocol of interchanging with binary messages between two automated
 systems (remember the need for unnecessary escaping/unescaping? what does it
 bring to automated systems' dialogue? only unnecessary complexity, right?).
 - OK, it is not perfect, but it works and is widely adopted and
 accepted, but even then is it so ideal to be used everywhere as a versatile

So why is it not so useful to formalize the interface between the HTTP-server
and the application? CGI was an attempt and it showed, that while being
quite universal, it brings too much overhead to the whole process. That's
why there happened to appear FastCGI and it's emulations and followers.
FastCGI were accepted not for it's best design decisions, but for solving
the CGI problem.
Perhaps you, people, have strong arguments against WSGI at it's current state.
But I can't believe, that there can exist valuable arguments against the idea
of such interface in general.
If there are some, I am interested to know that.

PS: Not trying to start a flame war, ignore the post if you feel suspicious.

PPS: Please, people, understand that you have done a great work, invented
     (or just implemented?) some really useful and insightful approaches in
     the area. But it all may get lost among the alikes and may get lost in the
     history of the web development, if it will continue to be so specific and
     inflexible (without a real visible need). I will personally be so
unhappy for that

More information about the Twisted-web mailing list