[Twisted-Python] ludicrous timeouts in protocols.http.HTTPFactory and web.server.Site

Andrew Bennetts andrew-twisted at puzzling.org
Thu Dec 30 18:39:23 EST 2004

On Thu, Dec 30, 2004 at 03:44:32PM +0100, Antoine Pitrou wrote:
> Looking at the Twisted code, I see that very long timeouts have been
> defined for both protocols.http.HTTPFactory and web.server.Site
> (60*60*12, that is 12 hours!). If I override the "timeout" parameter
> when constructing the Site receiving XMLRPC connections, then the
> problem disappears:
>         reactor.listenTCP(xmlrpc_port, server.Site(self, timeout=30), interface=xmlrpc_host)
> I think the default values in Twisted are quite bogus and should be
> changed to more sensible ones. 30 or 60 seconds is ok in the context of
> an HTTP connection. Very long timeouts on the other hand make the server
> very vulnerable.

One person's "reasonable" timeout is another person's "ludicrous"... this
sort of thing will need tuning per-application.  12 hours is conservative,
but highly unlikely to interfere with what anyone with pre-timeout Twisted
expects, while solving the problem twistedmatrix.com's web site was having
with the occasional connection not being closed.  And having long-running
connections mysteriously die due to a timeout setting you didn't even know
existed can be very surprising and frustrating to debug.

That said, the HTTP code is now quite careful with timeouts: it disables
them while sending the response (or more accurately, after receiving a
complete request, and then reenables them after it finishes sending the
response), so the only time a timeout can occur is while waiting for a
request to be sent -- i.e. it won't time out someone that's downloading an
ISO over a dial-up link.  So there probably is room to reduce it.

However, for very large POST requests over a slow link (ever tried to attach
a large file to your webmail over dial-up?), 30 or 60 seconds is still too
slow, but I could see an argument for, say, 30 minutes being the default.
Part of the issue here is backwards compatibility -- there may be
deployments of old versions of Twisted out there that depends on the
conservative (or no!) timeouts, and we don't want to gratuitously break
them.  If we do decide to be more aggressive by default, we need to make
this very clear in the documentation for the next version.

Regardless, if a particular application has specific needs, they can always
change the setting, just like you did.  It's not a major issue.


More information about the Twisted-Python mailing list