[Twisted-Python] Understanding deferred and error handling

David Cournapeau cournape at gmail.com
Wed Apr 20 19:04:20 MDT 2011


On Thu, Apr 21, 2011 at 2:31 AM, Phil Mayers <p.mayers at imperial.ac.uk> wrote:
> On 04/20/2011 05:28 AM, David wrote:
>> Hi,
>>
>> I have a hard time figuring out error handling with deferred in twisted.
>> More exactly, I don't understand how to always get meaningful tracebacks
>> to understand where the error actually happened. For a simple example:
>>
>> import sys
>>
>> import twisted.web.client
>>
>> from twisted.internet import defer
>> from twisted.internet import reactor
>> from twisted.python import log
>>
>> def remote_call():
>>       # No process bound to 8083 ->  connection refused
>>       d = twisted.web.client.getPage("http://localhost:8083")
>>       return d
>>
>> def main():
>>       d = remote_call()
>>       def _stop(arg):
>>           reactor.stop()
>>       d.addBoth(_stop)
>>
>> log.startLogging(sys.stdout)
>> reactor.callWhenRunning(main)
>> reactor.run()
>>
>> This will simply print no error in the log:
>>
>> 2011-04-20 12:37:40+0900 [-] Log opened.
>> 2011-04-20 12:37:40+0900 [-] Starting factory<HTTPClientFactory:
>> http://localhost:8083>
>> 2011-04-20 12:37:40+0900 [HTTPPageGetter,client] Stopping factory
>> <HTTPClientFactory: http://localhost:8083>
>> 2011-04-20 12:37:40+0900 [-] Main loop terminated.
>>
>> This already bothers me at a fundamental level, because it means it is
>> very easy to "swallow" errors without being aware of it. Is there a
>> "systematic" solution to this issue, or am I condemned to handle errors
>> systematically everywhere in my code ?
>>
>> Now, if I add an errback for logging purpose:
>>
>> def main():
>>       d = remote_call()
>>       def _stop(arg):
>>           reactor.stop()
>>       d.addErrback(log.err)
>>       d.addBoth(_stop)
>>
>> I get something like:
>>
>> 2011-04-20 12:38:35+0900 [-] Log opened.
>> 2011-04-20 12:38:35+0900 [-] Starting factory<HTTPClientFactory:
>> http://localhost:8083>
>> 2011-04-20 12:38:35+0900 [HTTPPageGetter,client] Unhandled Error
>>       Traceback (most recent call last):
>>       Failure: twisted.web.error.Error: 404 Not Found
>>
>> 2011-04-20 12:38:35+0900 [HTTPPageGetter,client] Stopping factory
>> <HTTPClientFactory: http://localhost:8083>
>> 2011-04-20 12:38:35+0900 [-] Main loop terminated.
>>
>> I do get an error, but I don't get a traceback. Interestingly enough, if
>> I use printTraceback:
>>
>> def main():
>>       d = remote_call()
>>       def _stop(arg):
>>           reactor.stop()
>>       def log_error(failure):
>>           log.err(failure.printTraceback())
>>           return failure
>>       d.addErrback(log_error)
>>       d.addBoth(_stop)
>>
>> I don't get any traceback either.
>
> As other people have pointed out, you're discarding the failure.

Indeed, I forgot to return the argument in my _stop callback. This answers part
of my question, but not what I consider the meat of my issue, that is
the lack of traceback. Fixing and simplifying my initial example:

import twisted.web.client

from twisted.internet import defer
from twisted.internet import reactor

def remote_call():
    d = twisted.web.client.getPage("http://localhost:8083")
    return d

def main():
    d = remote_call()
    def _stop(arg):
        reactor.stop()
        return arg
    d.addBoth(_stop)

reactor.callWhenRunning(main)
reactor.run()

This will correctly signal an error, but the output:

Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
Failure: twisted.internet.error.ConnectionRefusedError: Connection was
refused by other side: 61: Connection refused.

is not informative. I was hoping for a way to know that the error
happens inside remote_call through e.g. an errback in main. If this is
not possible, how do people generally handle those issues in
asynchronous code ?

cheers,

David




More information about the Twisted-Python mailing list