[Twisted-Python] Bug in DeferredLock?

Tapiwa Gutu tgutu at ml.sun.ac.za
Fri Jul 18 08:24:08 MDT 2014


We recently started experiencing the following error: *RuntimeError:
maximum recursion depth exceeded while calling a Python object*

After a fair bit of debugging we eventually found the problem and
implemented a fix for it in our codebase. We found that if a waiting
queue on a DeferredLock is ~125 (depending on call  depth) the error can
error. I recommend releasing the lock in a callLater.

The offending line is here:
https://twistedmatrix.com/trac/browser/trunk/twisted/internet/defer.py#L1360
And the script below can be used to reproduce the issue.

#!/usr/bin/env python
from twisted.internet import defer, reactor

LOCK = defer.DeferredLock()
FIRST_RUN = True

def deferred_sleep(delay):
    d = defer.Deferred()
    delay = reactor.callLater(delay, d.callback, True)
    d.delay = delay
    return d

@defer.inlineCallbacks
def synchronized_call():
    global FIRST_RUN
    yield LOCK.acquire()
    try:
        if FIRST_RUN:
            FIRST_RUN = False
            yield deferred_sleep(2)
            print "Queue length: %s" % len(LOCK.waiting)
    finally:
        LOCK.release()
        #reactor.callLater(0, lambda: LOCK.release())


if __name__ == '__main__':
    count = 125
    def start_test():
        for i in xrange(count):
            synchronized_call()

    reactor.callWhenRunning(start_test)
    reactor.run()

<http://www.ml.sun.ac.za/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20140718/8342cb9e/attachment.html>


More information about the Twisted-Python mailing list