[Twisted-Python] client, deferred and/or reactor hung on these 2 URL's

[George Pauly]

On Sat, 2007-01-13 at 18:00 -0500, Jean-Paul Calderone wrote:
> On Sat, 13 Jan 2007 16:38:04 -0600, Eric Mangold <teratorn at twistedmatrix.com> wrote:
> >On Sat, 13 Jan 2007 08:00:11 -0600, Rick Graves <gravesricharde at yahoo.com> 
> >wrote:
> >>Hello,
> >>
> >>I am new to twisted.
> >>
> >>I was using client.downloadPage and reactor.
> >>
> >>Twisted was hung on the following two for a whole hour:
> >>
> >>http://75i.net/
> >>
> >>http://www.theproxy.be/
> >>
> >>I had to break out with Ctrl-C.  When I did that, they finally fired the 
> >>Errorback.
> >
> >I just wrote a test program that uses downloadPage() and it downloaded  both 
> >those URLs fine.
> >
> >Are you still able to reproduce the problem? Does there seem to be any 
> >difference in downloading those URLs as part of a full run as opposed to 
> >individually? Do you have any proxies between you and those hosts? (Many 
> >ISPs have transparent proxies these days, *grrr*). If you can still 
> >reproduce then a traffic capture could be of use.
> >
> >In any case, the deferred returned from downloadPage probably shouldn't 
> >take > 1 hour to errback. It's certainly possible that downloadPage could 
> >need more than 1 hour to download a large file over a slow pipe, but  that's 
> >not the case here.
> 
> Not necessarily.  Networks have failures.  It seems likely that there was a
> transient error of some sort while downloading two URLs (out of several
> hundred).  Likely the connection would have eventually been lost, due to
> TCP-level timeouts, but this would take longer than an hour.  If such long
> timeouts are not desirable for a particular application, you can set a lower
> timeout by passing the timeout keyword argument to downloadPage.  In these
> cases, it may also be desirable to try again before ultimately reporting a
> failure.
> 
> Jean-Paul
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- 
George Pauly
Ring Development
www.ringdevelopment.com