[Twisted-Python] How to find out if exceptions are being raised in your errBack?

[Steve Steiner (listsin)]

I've been hunting down a problem that I've finally found the cause of  
and I'd like to know what's the Twisted way to catch this "error  
within the code handling the error"  type of error.

Basically, in one branch of the errBack, there was a typo.  A simple  
typo that caused an unhandled NameError exception, but only once in  a  
few thousand runs.

The exception got caught and "displayed" by Twisted, but it wasn't  
going anyplace anyone was looking (buried under zillions of lines of  
logging) and the app continues on as if nothing went wrong.

I've put up a simple app that demonstrates the issue: http://pastebin.com/m59217f60

If you put in a 404 error URL, let it run through, you'll see the 404  
error printed out, the exception will occur in the background, and the  
program just keeps on going.

If you then hit Ctrl-C, you can see the traceback showing that Twisted  
caught the NameError .

What is the best way to handle programming errors like this in  
deferreds so they don't slip by, unnoticed?

Thanks,

S

(~/twisted_err)# ./errs_away.py
URL: http://www.yahoo.com
line = http://www.yahoo.com
Got data, len ==  9490
URL: http://thereisnodomainnamedthis.com
line = http://thereisnodomainnamedthis.com
Error: DNS lookup failed: address 'thereisnodomainnamedthis.com' not  
found: [Errno 8] nodename nor servname provided, or not known.
URL: http://www.yahoo.com/non-existent-page
line = http://www.yahoo.com/non-existent-page
Error: 400 Bad Request	<====== This triggers the code with the bad  
variable
URL: ^C		<============= manually stop the program

============= Then, you get to see the traceback ==================
Unhandled error in Deferred:
Traceback (most recent call last):
   File "/System/Library/Frameworks/Python.framework/Versions/2.6/ 
Extras/lib/python/twisted/web/client.py", line 143, in handleResponse
     self.status, self.message, response)))
   File "/System/Library/Frameworks/Python.framework/Versions/2.6/ 
Extras/lib/python/twisted/web/client.py", line 309, in noPage
     self.deferred.errback(reason)
   File "/System/Library/Frameworks/Python.framework/Versions/2.6/ 
Extras/lib/python/twisted/internet/defer.py", line 269, in errback
     self._startRunCallbacks(fail)
   File "/System/Library/Frameworks/Python.framework/Versions/2.6/ 
Extras/lib/python/twisted/internet/defer.py", line 312, in  
_startRunCallbacks
     self._runCallbacks()
--- <exception caught here> ---
   File "/System/Library/Frameworks/Python.framework/Versions/2.6/ 
Extras/lib/python/twisted/internet/defer.py", line 328, in _runCallbacks
     self.result = callback(self.result, *args, **kw)
   File "./errs_away.py", line 15, in printError
     print oops      # variable's not defined...
exceptions.NameError: global name 'oops' is not defined

Thanks,

S