[Twisted-Python] Agent and "Cannot assign requested address"

akira 4kir4.1i at gmail.com
Thu Mar 10 03:31:00 MST 2011


"Jason J. W. Williams" <jasonjwwilliams at gmail.com> writes:

> Actually, I think the TIME_WAIT is the problem. It's what I see in
> netstat, and the Agent requests are fired sequentially via yield
> inside a for loop (inlineCallbacks). So they shouldn't be running in
> parallel. 

`yield` returns before TIME_WAIT expires otherwise it would require ~1
minute per request.

>
> The use case here is loading a Riak server with keys to prepare for a
> test. There's not a real way to get around sending one POST per key.
>
> How would I set the timeout value in Twisted? Or do I have to modify
> the timeout/keepalive systemwide in /proc?

In addition to net.ipv4.tcp_fin_timeout you could increase the ephemeral
port range (net.ipv4.ip_local_port_range sysctl parameter).

Each connection can be identified using 4-tuple (server IP, server port,
client IP, client port) Everything except client port is fixed in your
case so there could be at most
~ net.ipv4.ip_local_port_range/net.ipv4.tcp_fin_timeout connections per
second (even less in practice due to other applications and other
settings taking preference such as fs.file-max). For example:

  net.ipv4.ip_local_port_range = 32768    61000
  net.ipv4.tcp_fin_timeout = 30

There could be ~900 connections per second that might be good enough.

Reusing a local port via SO_REUSEADDR or better yet reusing a tcp
connection via HTTP keep-alive aren't available with twisted as I
understand it.


--
akira





More information about the Twisted-Python mailing list