[Twisted-Python] most efficient new connection rate limiting?

Alec Matusis matusis at yahoo.com
Wed Feb 4 19:37:04 EST 2009

We need to limit the new connection rate from a given IP, such that for
example cannot connect more often than 10 times per minute.

This is a high-volume TCP Twisted server, with about 500 new distinct IP
connections per second in the normal state (and we run 8 of these on each 8
core server, so ~4000 new conns/sec per box).

I am trying to find the least CPU intensive approach for this.

1) Create an dictionary {ip1:count1, ip2: count2, .} in the server, and
check the counts for each incoming connection. 
Disconnect with transport.loseConnection() if the threshold for ip:count is
Reset this dictionary to empty dict {} every minute with reactor.callLater

2) Use some Twisted rate limiter API that I am not familiar with?

3) Use iptables rate-limiting module like so:
iptables -I INPUT -p tcp --dport 8888 -i eth0 -m state --state NEW -m recent
iptables -I INPUT -p tcp --dport 8888 -i eth0 -m state --state NEW -m recent
--update --seconds 60 --hitcount 10 -j DROP

Which one of these approaches would you recommend?

More information about the Twisted-Python mailing list