[Twisted-Python] why can't a callback be called with a deferred?

[Jean-Paul Calderone]

On Wed, Feb 27, 2019 at 9:34 AM Scott, Barry <barry.scott at forcepoint.com>
wrote:

> On Tuesday, 26 February 2019 06:34:28 GMT Glyph wrote:
> > > On Feb 25, 2019, at 3:32 AM, Scott, Barry <barry.scott at forcepoint.com>
> > > wrote:>
> > > On Tuesday, 19 February 2019 11:00:57 GMT Chris Withers wrote:
> > >> Hi All,
> > >>
> > >> There's this assert:
> > >>
> > >>
> https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/defer.
> > >> py# L459
> > >>
> > >> ...and I'd like to understand why it's there.
> > >
> > > We hit this assert when porting from very old twisted to current
> twisted.
> > > In all cases the problem was with our code that used deferreds in a
> poor,
> > > not well understood way. After refactoring we are a lot happier with
> the
> > > resulting code as it easier to maintain now.
> >
> > Thanks for the feedback, Barry!
> >
> > It would still be great to figure out, if we can, how we might make the
> > error message a bit more legible to folks with less knowledge of
> Twisted's
> > internals.
>
> Let suppose that I need work done by doWork function.
> It returns a deferred for me to hang call backs and error backs on.
>
>         d = doWork()
>         d.addCallback(handleWorkDone)
>
> In my handleWorkDone I expect to get the result of doWork completing.
>
> The assert fires if instead of a result value is returned a Deferred is
> returned. This I consider a bug in the doWork() implementation.
>

This doesn't sound right.  Can you provide an example implementation of
doWork that provokes this behavior?  Here's an implementation that seems
like it matches your description and which does not provoke the behavior:

    def doWork():
        d = Deferred()
        d.callback("result")
        return d

    d = doWork()
    d.addCallback(handleWorkDone)

This doesn't trigger the assert.  This calls handleWorkDone with "result".
If you simplify the code so the Deferred interaction remains the same but
all the extraneous code is removed, it looks like this:

    d = Deferred()
    d.callback("result")
    d.addCallback(handleWorkDone)

which *must* work or Deferred is completely useless.

Jean-Paul



>
> What must happen in doWork is that it must arrange that
> any Deferred it used internally has an addCallback used to
> cause the d returned to the user to complete. Leaking the
> any internal Deferred() objects must not happen to the user
> of doWork.
>
> def doWork():
>         d = Deferred()
>
>         def completeWork(result, d):
>                 d.callback(result)
>
>         inner_d = doAsyncWork()
>         inner_d.addCallback(completeWork, d)
>
>         return d
>
> The error message would need to say something like:
> "Cannot return a Deferred as a result. Did you forgot to addCallback to
> the
> deferred?"
>
> Maybe add something to docs based on the above and refer to it in the
> message?
>
> Barry
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20190227/dd5a99e4/attachment-0002.html>