[Twisted-Python] why can't a callback be called with a deferred?
Scott, Barry
barry.scott at forcepoint.com
Wed Feb 27 08:06:50 MST 2019
On Wednesday, 27 February 2019 14:45:35 GMT Jean-Paul Calderone wrote:
> On Wed, Feb 27, 2019 at 9:34 AM Scott, Barry <barry.scott at forcepoint.com>
>
> wrote:
> > On Tuesday, 26 February 2019 06:34:28 GMT Glyph wrote:
> > > > On Feb 25, 2019, at 3:32 AM, Scott, Barry <barry.scott at forcepoint.com>
> > > > wrote:>
> > > >
> > > > On Tuesday, 19 February 2019 11:00:57 GMT Chris Withers wrote:
> > > >> Hi All,
> >
> > > >> There's this assert:
> > https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/defer.
> >
> > > >> py# L459
> > > >>
> > > >> ...and I'd like to understand why it's there.
> > > >
> > > > We hit this assert when porting from very old twisted to current
> >
> > twisted.
> >
> > > > In all cases the problem was with our code that used deferreds in a
> >
> > poor,
> >
> > > > not well understood way. After refactoring we are a lot happier with
> >
> > the
> >
> > > > resulting code as it easier to maintain now.
> > >
> > > Thanks for the feedback, Barry!
> > >
> > > It would still be great to figure out, if we can, how we might make the
> > > error message a bit more legible to folks with less knowledge of
> >
> > Twisted's
> >
> > > internals.
> >
> > Let suppose that I need work done by doWork function.
> > It returns a deferred for me to hang call backs and error backs on.
> >
> > d = doWork()
> > d.addCallback(handleWorkDone)
> >
> > In my handleWorkDone I expect to get the result of doWork completing.
> >
> > The assert fires if instead of a result value is returned a Deferred is
> > returned. This I consider a bug in the doWork() implementation.
The code I have posted is the good version so it works and the assert
does not fire. My goal is to show what I assume is the correct way to code
a function that uses internal Deferred(), not give an example that breaks.
This should cause the the assert as the code must wait for the thread to
return a result.
def doWork():
d = Deferred()
d2 = deferToThread(doWorkHelper)
d.callback(d2)
return d
Barry
> This doesn't sound right. Can you provide an example implementation of
> doWork that provokes this behavior? Here's an implementation that seems
> like it matches your description and which does not provoke the behavior:
>
> def doWork():
> d = Deferred()
> d.callback("result")
> return d
>
> d = doWork()
> d.addCallback(handleWorkDone)
>
> This doesn't trigger the assert. This calls handleWorkDone with "result".
> If you simplify the code so the Deferred interaction remains the same but
> all the extraneous code is removed, it looks like this:
>
> d = Deferred()
> d.callback("result")
> d.addCallback(handleWorkDone)
>
> which *must* work or Deferred is completely useless.
>
> Jean-Paul
>
> > What must happen in doWork is that it must arrange that
> > any Deferred it used internally has an addCallback used to
> > cause the d returned to the user to complete. Leaking the
> > any internal Deferred() objects must not happen to the user
> > of doWork.
> >
> > def doWork():
> > d = Deferred()
> >
> > def completeWork(result, d):
> > d.callback(result)
> >
> > inner_d = doAsyncWork()
> > inner_d.addCallback(completeWork, d)
> >
> > return d
> >
> > The error message would need to say something like:
> > "Cannot return a Deferred as a result. Did you forgot to addCallback to
> > the
> > deferred?"
> >
> > Maybe add something to docs based on the above and refer to it in the
> > message?
> >
> > Barry
> >
> >
> >
> > _______________________________________________
> > Twisted-Python mailing list
> > Twisted-Python at twistedmatrix.com
> > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
More information about the Twisted-Python
mailing list