[Twisted-Python] A kinder and more consistent defer.inlineCallbacks

[glyph at divmod.com]

On 21 Nov, 06:04 pm, terry at jon.es wrote:
>Here's a suggestion for making inlineCallbacks more consistent and less
>confusing.  Let's suppose you're writing something like this:

Let's put this a bit less vaguely: inlineCallbacks appears to have a 
bug: 'raise' before 'yield' in a generator results in a synchronous 
exception rather than an errback, although its documentation does not 
explain this.

inlineCallbacks is also unhelpful in debugging a particular type of 
error - it doesn't tell you what happened if you unintentionally 
returned something other than a generator, it just blows up without your 
code on the stack, and no mention of your code in the error message.

These are definitely interesting problems.  I think there should 
probably be a ticket for each one.  I don't like your solution, though.
>1. func may not yield. In that case, you either get an AttributeError 
>when
>inlineCallbacks tries to send().

Following you so far.
>Or worse, the call to send might actually
>work, and do who knows what. I.e., func() could return an object with a
>send method but which is not a generator.

Now I'm not sure what you're talking about.  Do you have a lot of very 
dangerous objects lying around with methods called 'send'?  ;-)

This is an important behavior which should continue to be supported.  If 
we don't, then users will get surprising behavior if they try to mix 
inlineCallbacks with other decorators that modify generators.  A 
"generator-like" object (iterable with 'send') should be good enough.
>For some fun, run some code that
>calls the following decorated function

In this particular case, there may be a third ticket to file, i.e. to 
refuse to accept objects that implement __call__ and send but not 
__iter__ and next; however, this infinite loop seems really, really 
obscure.  Have you actually hit it in real life?
>2. func might raise before it get to its first yield. In that case 
>you'll
>get an exception thrown when the inlineCallbacks decorator tries to 
>create
>the wrapper function:

Definitely problematic.  Code expecting to handle errors with an errback 
should not need to have an except: block as well.
>There's a simple and consistent way to handle both of these. Just have
>inlineCallbacks do some initial work based on what it has been passed:

isinstance() is bad for the reasons I mentioned above.
>This has the advantage that (barring e.g., a KeyboardInterrupt in the
>middle of things) you'll *always* get a deferred back when you call an
>inlineCallbacks decorated function. That deferred might have already 
>called
>or erred back (corresponding to cases 1 and 2 above).

This property, however, I definitely think is a desirable one.
>And case 2 happens to me too. Having inlinecallbacks try/except the 
>call to
>func is nicer because it means I don't have to be quite so defensive in
>coding. So instead of me having to write
>
>    try:
>        d = func()
>    except Exception:
>        # ???

IMHO the most important thing discussed here.  Not quite sure how to do 
this change in a compatible way; some people might be depending on this 
weird behavior.  @inlineCallbacks2?  @inlineCallbacks(beGood=YES)? 
Suggestions are appreciated.