[Twisted-Python] A kinder and more consistent defer.inlineCallbacks

[Terry Jones]

Hi Glyph

>>>>> "glyph" == glyph  <glyph at divmod.com> writes:

>> inlineCallbacks appears to have a bug: 'raise' before 'yield' in a
>> generator results in a synchronous exception rather than an errback,
>> although its documentation does not explain this.

glyph> Ugh, scratch that.  No, it doesn't have this bug.

Yes, you're right.

glyph> I did some quick testing and saw some tracebacks, but apparently
glyph> wasn't paying very close attention to them

Nor was I, sorry!

glyph> If func "raises before it gets to its first yield", we get the right
glyph> behavior.  If it just raises and doesn't yield *anywhere*, then it's
glyph> not a generator and that's the same as your other case of
glyph> accidentally-not- returning-a-generator.  Still worth debugging, but
glyph> not as serious.

Agreed.  I also agree with your earlier remarks about dropping the
isinstance(result, GeneratorType). That leaves me with an alternative:

    def altInlineCallbacks(f):
        def unwindGenerator(*args, **kwargs):
            try:
                result = f(*args, **kwargs)
            except Exception, e:
                # f was not a generator.
                return failure.Failure()
            return _inlineCallbacks(None, result, Deferred())
        return mergeFunctionMetadata(f, unwindGenerator)

and still the problem that _inlineCallbacks raises if result doesn't have a
send() method, etc. I'm also not sure of the best way to check for that,
but don't think it should be in the _inlineCallbacks loop.

---

BTW, there is a coding lesson here (at least for me):

The reason I started to think there was a problem with Exceptions,
tracebacks and _inlineCallbacks was from running code like this:

    @inlineCallbacks
    def f():
        try:
            # something
        except Exception:
            # clean up
            raise

When I ran it, the traceback of the Exception that propagated back to the
attached errback would (sometimes) show the exception as having being
triggered in _inlineCallbacks, like this:

    2008/11/22 17:29 +0200 [-]   File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 726, in _inlineCallbacks
    2008/11/22 17:29 +0200 [-]     result = g.send(result)

The problem, as I guess will be obvious, is that my raise was delivering
whatever sys.exc_info returned after the clean-up was done. In my case this
was confusing as although the clean-up code had succeeded, it had called
things that also made use of inlineCallbacks and the exception I was
finally raising, and whose traceback I was printing, was a StopIteration
raised inside *another* call of _inlineCallbacks! So it looked like
_inlineCallbacks was somehow raising instead of sending a failure back to
the errback...

Re-raising the original exception made everything make sense.

Thanks a lot for taking a look.

Terry