[Twisted-web] Re: [Web-SIG] WSGI woes

Phillip J. Eby pje at telecommunity.com
Thu Sep 16 11:41:54 MDT 2004


At 05:45 PM 9/16/04 +0100, Alan Kennedy wrote:
>The only thing I disagree on are the names "sleep" and "wake", which IMHO 
>come with too many semantic hangovers from the threading world. When an 
>application calls wsgi.sleep(), it's not really sleeping, it's just 
>declaring that it currently has no output: a call to its iterator will 
>succeed, but the returned value will be an empty string.
>
>So basically, WSGI is providing an on/off indicator for every instance of 
>a middleware stack, which indicates to the server if there is currently 
>output available.

Well, I'm proposing it as an optional extension, not a required 
feature.  And, I think I'd like to streamline it to a single 
'wsgi.pause_output' function, e.g.:

     resume = environ['wsgi.pause_output']()

Where 'resume' is then a callback function that can be invoked to resume 
iteration.  This keeps it to a single extension key, helps ensure the 
correct sequence of actions, and makes it easier to implement in some 
cases, while not making other cases any harder.


>In an asynchronous situation, the application cannot simply do a blocking 
>read on the input: that will tie up the server thread.

What do you mean by "server thread"?  A truly asynchronous server (one 
using "no threads") cannot serve multiple WSGI requests simultaneously.  In 
the general case, a WSGI server can only serve as many requests 
simultaneously as it has available threads for.  However, WSGI applications 
that use iteration in place of 'write()' can sometimes be run with fewer 
than one thread per simultaneous request -- that's why iteration is 
recommended for applications that can be implemented that way.


>  So we need a way for the application to be notified/called when input 
> becomes available from the client.
>
>Perhaps we need to add an environment entry, e.g. "wsgi.input_handler", 
>which the app uses to pass a callable to the server. This callable would 
>be called whenever data became available on the input stream.
>
>So how would that work in the middleware stack?

You would have to pass either 'environ' or 'wsgi.input' *into* this input 
handler request function, so that the server can verify it hasn't been 
replaced by any middleware.  This is the standard way in WSGI of providing 
enhanced communication facilities that could "bypass" middleware.  See:

     http://www.python.org/peps/pep-0333.html#server-extension-apis

So, in principle, if the spec is modified to require middleware to honor 
child applications' block boundaries, then you could use an extension API 
to pause iteration until input is available, in much the same way that you 
would pause iteration for any other reason.

Neither of these "pause iteration" solutions are especially elegant, at 
least from the POV of an async application author.  But my objective here 
is only to make it *possible*, not necessarily pretty.  I imagine that if 
there's actual demand for async apps to run under WSGI, it should be 
possible to create wrappers to let an application written in Twisted's 
continuation-passing style be run as a WSGI app.

Such a wrapper would basically be just a function returning an iterator, 
with a bunch of pausing logic and a queue to communicate with the actual 
asynchronous app.  And, such wrappers should only need to be written once 
for each asynchronous API, which as a practical matter probably means only 
Twisted, anyway, as (IMO) it has no real competitors in the Python async 
framework space.




More information about the Twisted-web mailing list