[Twisted-Python] Agent and "Cannot assign requested address"
Jason J. W. Williams
jasonjwwilliams at gmail.com
Thu Mar 10 13:04:48 EST 2011
I ended up getting around the problem by increasing my Riak cluster
size and putting a load balancer in front for the test. But
connection pooling would be really helpful, both here and in the
CouchDB client. I've refactored both txRiak and Paisley in the past
couple of months to use Agent in the hopes ticket 3420 gets completed.
On Thu, Mar 10, 2011 at 3:31 AM, akira <4kir4.1i at gmail.com> wrote:
> "Jason J. W. Williams" <jasonjwwilliams at gmail.com> writes:
>> Actually, I think the TIME_WAIT is the problem. It's what I see in
>> netstat, and the Agent requests are fired sequentially via yield
>> inside a for loop (inlineCallbacks). So they shouldn't be running in
> `yield` returns before TIME_WAIT expires otherwise it would require ~1
> minute per request.
>> The use case here is loading a Riak server with keys to prepare for a
>> test. There's not a real way to get around sending one POST per key.
>> How would I set the timeout value in Twisted? Or do I have to modify
>> the timeout/keepalive systemwide in /proc?
> In addition to net.ipv4.tcp_fin_timeout you could increase the ephemeral
> port range (net.ipv4.ip_local_port_range sysctl parameter).
> Each connection can be identified using 4-tuple (server IP, server port,
> client IP, client port) Everything except client port is fixed in your
> case so there could be at most
> ~ net.ipv4.ip_local_port_range/net.ipv4.tcp_fin_timeout connections per
> second (even less in practice due to other applications and other
> settings taking preference such as fs.file-max). For example:
> net.ipv4.ip_local_port_range = 32768 61000
> net.ipv4.tcp_fin_timeout = 30
> There could be ~900 connections per second that might be good enough.
> Reusing a local port via SO_REUSEADDR or better yet reusing a tcp
> connection via HTTP keep-alive aren't available with twisted as I
> understand it.
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
More information about the Twisted-Python