WSGI support (was [Twisted-web] Re: status of Twisted Web and Web 2)

Phil Christensen phil at bubblehouse.org
Thu Mar 6 21:38:54 EST 2008


On Mar 6, 2008, at 6:17 PM, David Reid wrote:
> On Mar 6, 2008, at 11:06 AM, Manlio Perillo wrote:
>> Phil Christensen ha scritto:
>>> On Mar 6, 2008, at 10:56 AM, David Reid wrote:
>>>> It shouldn't be very hard to add support for it to  
>>>> Twisted.web2's WSGI implementation either, but I don't much see  
>>>> the point if no one else supports it.  I'm not convinced that  
>>>> WSGI is at all a useful means of writing asynchronous web  
>>>> applications.  I think WSGI's only benefit is that it allows you  
>>>> to almost write your application code once and run it on  
>>>> multiple containers.
>>> I'd have to disagree (slightly). There's nothing about WSGI that  
>>> makes it only applicable to one approach or another (that is,  
>>> synchronous versus asynchronous).
>>
>> Right.
>> This is what makes WSGI great.
>
> Read: "So the thing I like about WSGI is that it's actually poorly  
> specified"
>
> Why is attempting to adapt a currently synchronous API (the WSGI  
> application API is synchronous) It's supposed to allow you to write  
> web applications that may be able to run singlethreaded,  
> multithreaded or in a multiprocess environment, it doesn't specify  
> an asynchronous API.   If it did applications would still need to  
> be written in an asynchronous manner.

I don't know that anyone was suggesting any magic would prevent  
having to write an asynchronous WSGI app in the traditional manner.

Assuming we are talking about the spec, and not a particular  
container, the WSGI API is *not* implicitly synchronous. It requires  
that the application execute the `start_response` callable that is  
passed to the handler function *at some point*. You can easily  
execute it in a callback from some Deferred, or in any other  
asynchronous fashion.

> Threads are the only way to make blocking code appear to be non- 
> blocking.
> Processes are the only way to make blocking code actually not block.
>
>>> It's just as easy to write a Deferred-using asynchronous  
>>> application as a blocking one.
>
> It would be, except it's not at all specified that that is a thing  
> you can do.  PEP 333 sure has no mention of Deferreds.

I think given my point above, that it's irrelevant as to whether PEP  
333 has a mention of Deferreds. It doesn't mention MySQLdb or Zope  
interfaces, either.

All that matters is that at some point you call start_response. As I  
said, there may be WSGI containers that make writing a asynchronous  
application irrelevant (because only one request is handled at a  
time, or requests are threaded), but there's nothing preventing  
someone from making an asynchronous container.

Now, whether that would be worth the effort is a matter of opinion.

>>> The real issue is that a "proper" WSGI app gets informed of it's  
>>> environment by the wsgi.multithreaded and wsgi.multiprocess  
>>> environ variables, so if you really wanted a write-once-run- 
>>> anywhere, you'd need to be able to handle both scenarios.
>>
>> By the way, some time ago I proposed a wsgi.asynchronous  
>> enviroment variable.
>
> I don't get it.  I don't see the point at all.  I can't make  
> blocking code magically not-block.  I could use a middlewear that  
> translated from wsgi.asynchronous to a wsgi.multithreaded or  
> wsgi.multiprocess implementation for running my synchronous code,  
> but I don't think there are any benefits to a wsgi.asynchronous API  
> at all.
>
> I can reasonably write an application that honestly doesn't care if  
> it's being serialized, run in threads, or run in seperate  
> processes.  As soon as you throw wsgi.asynchronous into the mix I  
> lose that ability.

If you're in a multithreaded container, there's a slough of different  
ways to trip up, like saving state in module-level variables without  
using some kind of synchronization. I've definitely run into  
scenarios like that in complex web applications and had to deal with  
them with care.

So if I already have to consider my container's process/thread model,  
then determining if my app can run properly in an asynchronous  
environment seems like a non-issue.

Furthermore, a container that was written to be asynchronous could  
always support synchronous applications as necessary, whereas the  
reverse is obviously not true, at least not in any efficient sense.

>>> As far as Twisted's WSGI implementation, it should probably  
>>> support both approaches, maybe where some kind of argument to the  
>>> WSGIResource class constructor determines what it specifies in  
>>> the environ dictionary. It should be the responsibility of the  
>>> WSGI application itself to do the right thing depending on what's  
>>> inside the environ dict.
>>
>> I'm not sure that this is possible.
>> An application is asynchronous or synchronous.

I was referring to things like thread synchronization, as I mentioned  
above.

> Exactly so why should we try to make the same API do both?

Because WSGI is a decent spec, and is a familiar environment from  
people who are transitioning from CGI, and to some extent, PHP web  
development. Since there is no technical reason WSGI couldn't be used  
in an asynchronous context, why not make it possible?

It would allow users to have a somewhat familiar environment, while  
still getting used to an asynchronous programming style. Then the  
leap to twisted.web/Nevow would be that much easier.

> All that being said I'm perfectly happy to accept patches that  
> enhance twisted's wsgi support, but I feel no inclination to  
> support anything out of spec.

Again, I think it's debatable if writing an asynchronous WSGI is  
outside the spec, although clearly a 'wsgi.asynchronous' variable  
would be.

Anyways, just my 2/100ths...

-phil



More information about the Twisted-web mailing list