[Twisted-Python] why can't a callback be called with a deferred?

Scott, Barry barry.scott at forcepoint.com
Wed Feb 27 08:06:50 MST 2019


On Wednesday, 27 February 2019 14:45:35 GMT Jean-Paul Calderone wrote:
> On Wed, Feb 27, 2019 at 9:34 AM Scott, Barry <barry.scott at forcepoint.com>
> 
> wrote:
> > On Tuesday, 26 February 2019 06:34:28 GMT Glyph wrote:
> > > > On Feb 25, 2019, at 3:32 AM, Scott, Barry <barry.scott at forcepoint.com>
> > > > wrote:>
> > > > 
> > > > On Tuesday, 19 February 2019 11:00:57 GMT Chris Withers wrote:
> > > >> Hi All,
> > 
> > > >> There's this assert:
> > https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/defer.
> > 
> > > >> py# L459
> > > >> 
> > > >> ...and I'd like to understand why it's there.
> > > > 
> > > > We hit this assert when porting from very old twisted to current
> > 
> > twisted.
> > 
> > > > In all cases the problem was with our code that used deferreds in a
> > 
> > poor,
> > 
> > > > not well understood way. After refactoring we are a lot happier with
> > 
> > the
> > 
> > > > resulting code as it easier to maintain now.
> > > 
> > > Thanks for the feedback, Barry!
> > > 
> > > It would still be great to figure out, if we can, how we might make the
> > > error message a bit more legible to folks with less knowledge of
> > 
> > Twisted's
> > 
> > > internals.
> > 
> > Let suppose that I need work done by doWork function.
> > It returns a deferred for me to hang call backs and error backs on.
> > 
> >         d = doWork()
> >         d.addCallback(handleWorkDone)
> > 
> > In my handleWorkDone I expect to get the result of doWork completing.
> > 
> > The assert fires if instead of a result value is returned a Deferred is
> > returned. This I consider a bug in the doWork() implementation.

The code I have posted is the good version so it works and the assert
does not fire. My goal is to show what I assume is the correct way to code
a function that uses internal Deferred(), not give an example that breaks.

This should cause the the assert as the code must wait for the thread to
return a result.

def doWork():
	d = Deferred()
	d2 = deferToThread(doWorkHelper)
	d.callback(d2)
	return d

Barry


> This doesn't sound right.  Can you provide an example implementation of
> doWork that provokes this behavior?  Here's an implementation that seems
> like it matches your description and which does not provoke the behavior:
> 
>     def doWork():
>         d = Deferred()
>         d.callback("result")
>         return d
> 
>     d = doWork()
>     d.addCallback(handleWorkDone)
> 
> This doesn't trigger the assert.  This calls handleWorkDone with "result".
> If you simplify the code so the Deferred interaction remains the same but
> all the extraneous code is removed, it looks like this:
> 
>     d = Deferred()
>     d.callback("result")
>     d.addCallback(handleWorkDone)
> 
> which *must* work or Deferred is completely useless.
> 
> Jean-Paul
> 
> > What must happen in doWork is that it must arrange that
> > any Deferred it used internally has an addCallback used to
> > cause the d returned to the user to complete. Leaking the
> > any internal Deferred() objects must not happen to the user
> > of doWork.
> > 
> > def doWork():
> >         d = Deferred()
> >         
> >         def completeWork(result, d):
> >                 d.callback(result)
> >         
> >         inner_d = doAsyncWork()
> >         inner_d.addCallback(completeWork, d)
> >         
> >         return d
> > 
> > The error message would need to say something like:
> > "Cannot return a Deferred as a result. Did you forgot to addCallback to
> > the
> > deferred?"
> > 
> > Maybe add something to docs based on the above and refer to it in the
> > message?
> > 
> > Barry
> > 
> > 
> > 
> > _______________________________________________
> > Twisted-Python mailing list
> > Twisted-Python at twistedmatrix.com
> > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python







More information about the Twisted-Python mailing list