[Twisted-Python] Asynchronous context in Twisted

Fantix King fantix at exoweb.net
Tue Mar 15 02:54:10 EDT 2011

On Mar 3, 2011, at 2:39 PM, Glyph Lefkowitz wrote:
> On Mar 3, 2011, at 7:31 AM, Fantix King wrote:
> > Hi,
> >
> > I tried to make python.context work in asynchronous code between main
loops. Anyone has similar experience to share please?
> >
> > Not sure if I am rebuilding a wheel :P
> >
> >
> This is something I've often thought about doing in Twisted itself,
actually :).  But I wasn't sure that chaining context would actually do
anything practically useful most of the time.  Have you found that it's
actually useful?  Have you managed to leverage this to, for example, get
more informative error messages out of Deferred failures?
> Doing it as a subclass like this is not optimal, as it limits you to one
reactor (and the Select reactor is not really the best one).  A wrapper
would be slightly more tricky (you'd have to deal with the places that the
reactor passes itself through to things like Process and Port, so you'd have
to create wrappers for those as well) but much more general.

Thanks for replying! :)

Yes! That's a wonderful idea to use this context for asynchronous traceback!
I made
some small changes to the code and wrote a patch for Twisted (as addReader
addWriter is quite different from one impl to another, I changed
SelectReactor only.
I haven't got a better idea for this, please advise), please see attachment.

With a simple example of raising exception in deferLater-ed function

from twisted.internet import reactor
from twisted.internet.task import deferLater
reactor.usingAsyncTraceback = True

def g():
    raise Exception('Something happened inside.')

def f():
    return g()

def e():
    return f()

def d():
    return e()

def c():
    deferred = deferLater(reactor, 1, lambda: None)
    deferred.addCallback(lambda x: d())
    return deferred

def b():
    return c()

def a():
    return b()

if __name__ == '__main__':
    deferred = a()
    def errback(failure):
    deferred.addBoth(lambda x: reactor.stop())

I could get this:

Traceback (most recent call last):
  File "test.py", line 31, in <module>
    deferred = a()
  File "test.py", line 28, in a
    return b()
  File "test.py", line 25, in b
    return c()
  File "test.py", line 20, in c
    deferred = deferLater(reactor, 1, lambda: None)
  File "/home/fantix/ac/twisted/internet/task.py", line 751, in deferLater
    delayedCall = clock.callLater(delay, d.callback, None)
  File "/home/fantix/ac/twisted/internet/base.py", line 701, in callLater
    _f, args, kw = self._chainContext(_f, args, kw)
*--- <asynchronous break point> ---*
  File "/home/fantix/ac/twisted/python/context.py", line 59, in
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/home/fantix/ac/twisted/python/context.py", line 37, in
    return func(*args,**kw)
  File "/home/fantix/ac/twisted/internet/defer.py", line 361, in callback
  File "/home/fantix/ac/twisted/internet/defer.py", line 455, in
--- <exception caught here> ---
  File "/home/fantix/ac/twisted/internet/defer.py", line 542, in
    current.result = callback(current.result, *args, **kw)
  File "test.py", line 21, in <lambda>
    deferred.addCallback(lambda x: d())
  File "test.py", line 17, in d
    return e()
  File "test.py", line 14, in e
    return f()
  File "test.py", line 11, in f
    return g()
  File "test.py", line 8, in g
    raise Exception('Something happened inside.')

Additionally, in my scenario of a 5 years old asynchronous Twisted web
application, we
need the "request" object available throughout all code between asynchronous
accesses and database accesses because our global configuration system
needs the
request object. It would greatly reduce our manual work to pass through the
object here and there to have a context working in the asynchronous way.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20110315/d3315923/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fantix_async_context.patch
Type: text/x-patch
Size: 6511 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20110315/d3315923/attachment.bin 

More information about the Twisted-Python mailing list