Opened 9 years ago

Closed 5 years ago

#2861 defect closed fixed (fixed)

Deferred cleans Failures more times than it needs to

Reported by: exarkun Owned by: exarkun
Priority: low Milestone:
Component: core Keywords:
Cc: jesstess Branch:
Author:

Description

Here's a crummy demonstration program. It creates one Failure and a couple Deferreds and monkeys Failure so you can see when cleanFailure gets called:

from traceback import print_stack

from twisted.internet.defer import Deferred
from twisted.python.failure import Failure

failures = []

originalCleanFailure = Failure.cleanFailure.im_func
def cleanFailure(self):
    failures.append(self)
    print 'CLEAN FAILURE', hex(id(self))
    print_stack()
    return originalCleanFailure(self)
Failure.cleanFailure = cleanFailure

print 'make d1'
d1 = Deferred()
print 'make d2'
d2 = Deferred()
print 'errback 2'
d2.errback(Exception("foo"))

def c1(ignored):
    print 'return d2'
    return d2


print 'addcallback c1'
d1.addCallback(c1)
print 'callback d1'
d1.callback(None)
print 'adding cleanup errorback'
d1.addErrback(Failure.trap, Exception)
print 'done'

The output looks something like this:

exarkun@charm:~$ python clearbug.py
make d1
make d2
errback 2
CLEAN FAILURE -0x484df454
  File "clearbug.py", line 21, in ?
    d2.errback(Exception("foo"))
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 266, in errback
    self._startRunCallbacks(fail)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 309, in _startRunCallbacks
    self._runCallbacks()
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 344, in _runCallbacks
    self.result.cleanFailure()
  File "clearbug.py", line 12, in cleanFailure
    print_stack()
addcallback c1
callback d1
return d2
CLEAN FAILURE -0x484df454
  File "clearbug.py", line 31, in ?
    d1.callback(None)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 244, in callback
    self._startRunCallbacks(result)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 309, in _startRunCallbacks
    self._runCallbacks()
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 338, in _runCallbacks
    self.result.addBoth(self._continue)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 215, in addBoth
    callbackKeywords=kw, errbackKeywords=kw)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 187, in addCallbacks
    self._runCallbacks()
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 325, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 286, in _continue
    self.unpause()
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 282, in unpause
    self._runCallbacks()
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 344, in _runCallbacks
    self.result.cleanFailure()
  File "clearbug.py", line 12, in cleanFailure
    print_stack()
CLEAN FAILURE -0x484df454
  File "clearbug.py", line 31, in ?
    d1.callback(None)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 244, in callback
    self._startRunCallbacks(result)
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 309, in _startRunCallbacks
    self._runCallbacks()
  File "/home/exarkun/Projects/Twisted/trunk/twisted/internet/defer.py", line 344, in _runCallbacks
    self.result.cleanFailure()
  File "clearbug.py", line 12, in cleanFailure
    print_stack()
adding cleanup errorback
done
exarkun@charm:~$

This is three calls to cleanFailure. The first one is done by d2 synchronously in d2.errback since it ends up with a Failure as its final result. The second two are done by d1. One of these could be avoided fairly easily I think.

Since Failure optimizes subsequent calls to cleanFailure, this may only be a trivial optimization and not really noticable in real applications.

Change History (3)

comment:1 Changed 5 years ago by <automation>

  • Owner glyph deleted

comment:2 Changed 5 years ago by jesstess

  • Cc jesstess added
  • Owner set to exarkun

exarkun, running your test now with Twisted 11.0 I see 2 calls to cleanFailure. Is the situation now acceptable?

comment:3 Changed 5 years ago by exarkun

  • Resolution set to fixed
  • Status changed from new to closed

Two is certainly better than three. I don't know if three was even really much of a problem though. I'm happy to call this resolved. Thanks!

Note: See TracTickets for help on using tickets.