[Twisted-Python] Re: Resource.render() returning NOT_DONE_YET

[Mario Ruggier]

On lundi, avr 28, 2003, at 21:57 Europe/Amsterdam, Glyph Lefkowitz 
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> On Monday, April 28, 2003, at 11:17 AM, Clark C. Evans wrote:
>
>> On Mon, Apr 28, 2003 at 04:09:35PM +0000, Clark C. Evans wrote:
>> | I was just giving a quick Twisted tutorial to someone using
>> | twisted and as we were breaking page construction into more
>> | than one chunk... an unexpected stumbling block occurred --
>> | returning NOT_DONE_YET form the resource's render() function.
>> |
>> | I was thinking about two other options:
>> |
>> |   1.  Perhaps NOT_DONE_YET could just be None, this
>> |       way, it can be the default return value.  As I'm
>> |       browsing through my code this is the most common
>> |       return... why not make it the default.
>
> The reasoning behind not allowing None as a default value is that 
> forgetting is too easy.  If you're writing a simple request that has a 
> render() method that looks like
>
>     def render(self, request):
>         if self.authenticated:
>             return self.goodies
>
> the default behavior should not be "hang forever".
>
>>         Err, this isn't exactly what I was thinking.  What
>>         I was proposing...  if during the scope of the render()
>>         function, req.write() is called, then a None value
>>         would be an allowable return.   And if None is returned,
>>         req.finish() would be called automagically.
>
> What if you wanted to start writing the page in the render() method 
> but keep writing it later?  Then we have None as a synonym for 
> NOT_DONE_YET except in certain situations where you've done something 
> to the request?
>
>> |   2.  Alternatively, allow a Deferred to be a return
>> |       value.  Then the underlying caller can add
>> |       result.finish() to the deferred chain.   This
>> |       has the advantage of not requiring finish() to
>> |       really be managed.   Either the return value is
>> |       a string, a Deferred, (or for backwards compatibiliy
>> |       NOT_DONE_YET).  In either of the primary cases,
>> |       result.finish() always gets called... thus making
>> |       it easier on newbies.
>
> I've discussed this with several different people at various times... 
> the trouble is, there isn't really a use-case that Deferreds make 
> easier.  render() ends up being a relatively low-level interface, and 
> the NOT_DONE_YET/write/finish API is quite convenient for the stuff 
> that has been implemented with it.
>
> I am definitely a True Believer in the Deferred, but in this case it 
> just doesn't seem worth the inconvenience of deprecating things and 
> shuffling stuff around for a vanishingly small benefit.

Completely agree. Related to this issue, i feel, is what should be best 
practices
for a twisted web application? With all that's available, and little 
imagination,
one can very easily get all tangled up in blue ;)

In view of a site that I have the intention to build, I have been 
trying to select
what best implementation architecture to adopt, looking always for the 
most
stupidly simple. I also want to be able to handle errors as nicely as 
possible
for the users -- meaning I would like to, _whenever_ possible, return a
fully consistent page (and hopefully the one expected by the user) that
provides also the error info, but that would not require the user to do 
anything
more than correct form input data on the _same_ page (without even 
hitting Back
on the browser). I hate the feeling of being surprised with drastic 
error pages,
with all my input data apparently disappeared (also dislike pop-ups 
asking to
repost data, ...). Of course these errors will be of the type 'Ah, that 
one exists
already please try another'... This is all somewhat application 
specific, and
about interfaces, but i feel implementation style can both help or 
hinder this.

A further issue that i find very nagging (at least during development), 
is that
if an error occurs within the presentation layer, the response hangs 
forever.
(These should of course be only development bugs, but i would not like 
to
deliver an app that *may* have presentation bugs yet unknown.) I 
therefore
wrap the call to the real presentation handler in a try/catch. Errors 
in the
business logic are raised to the error handler, so this is not a 
problem. Since I
want my errors to be rendered with a fully functional response, the 
error callback
simply sets the additional response data object, and passes to the
presentation layer (this assumes that the business layer has behaved 
nicely
and attached to request the necessary and sufficient data objects, so 
that
the presentation can render it into a coherent page).

  I am going for this anatomy:


class WebAppPage(twisted.web.resource.Resource):

     def render(self, request):
         request.setHeader('Content-Type', 'text/html; charset=utf-8')
         ...
         d = threads.deferToThread(self.businessLogic, request) # to be 
threadpooled ;)
         d.addCallback(self.presentationLogic, request)
         d.addErrback(self.errBack, request)
         return NOT_DONE_YET

   def businessLogic(self, request):
       # uses functionality that is neatly factored in external modules 
;)
       # and attaches the resulting data objects to request in a 
designated
       # container, separate from args (e.g. respargs).
       # Knows no HTML, never calls request.write()

  def presentationLogic(self, result, request, error=None):
     try:
         self._presentationLogic(result, request, error)
     except:
         import sys
         request.write( '''Ooops, error in presentation layer:<p>%s: 
%s</p>'''
                         % ( sys.exc_info()[0], ' -- 
'.join(sys.exc_info()[1].args) ))

   def _presentationLogic(self, result, request, error=None):
       # combines the data objects in args, and result (if any - thus 
may be None)
       # with rendering templates, html, js, css, ...

def errBack(self, error, request):
     self.presentationLogic([],request,error=error)


This suggests that my site's rpy resources should inherit from a 
subclass of resource,
to at least define a common error callback and presentation try/catch 
wrap for the whole
site. However, a small thing escapes me -- how do i guarantee specific 
and automatic
treatment to the request object for all my resources? E.g. setting 
specific headers,
reading cookies/session info, etc., without coding this in every rpy 
instance?

       mario