[Twisted-Python] deferreds and hung connections

[Jp Calderone]

On Wed, 9 Mar 2005 09:28:49 -0800 (PST), Lenny G Arbage <alengarbage at yahoo.com> wrote:
>Suppose I am using twisted.web.client.getPage() to
> retrieve webpages.  I am setting a callback and an
> errback for the result, and this all works
> fantastically.
> 
> But what about the case where the webserver simply
> hangs with the connection open, never returning with
> either a result or an error?
> 
> I assume that my errback will get called if the socket
> times out and closes (right?).  But what if I don't
> want to wait that long -- what if I want to do my own
> timeout after N seconds/minutes?
> 
> I've been looking for examples of how to do this, and
> monkeying with reactor.callLater() with partial
> success.  If I save the id from callLater(), I can
> cancel the timeout in callback() or errback(). 
> However, if the timeout does get fired, I get:
> 
> [Failure instance: Traceback:
> twisted.internet.error.AlreadyCalled, Tried to cancel
> an already-called event.
> /usr/lib/python2.3/site-packages/twisted/internet/defer.py:338:_runCallbacks
> /tmp/deferred2.py:21:errback
> /usr/lib/python2.3/site-packages/twisted/internet/base.py:82:cancel
> ]
> 
> Am I going about this the wrong way?  Perhaps twisted
> has some built-ins that I'm not aware of to do
> essentially the same thing?
> 
> I have a modified example from the Defer Howto that
> exhibits this behavior.  I can post if the above
> explanation isn't enough.
> 

  As was recently mentioned on the list, timeout support needs 
to be part of the result-generating API.  Luckily, in the case 
of getPage, it is, as demonstrated by this manhole session:

    >>> from twisted.web import client
    >>> client.getPage('http://google.com', timeout=0.001)
    <Deferred #0>
    Deferred #0 failed: 'Getting http://google.com took longer than 0.001
    seconds.'
    >>> 

  Jp