[Twisted-Python] Error: filedescriptor out of range in select

[Andrew Bennetts]

Don Schoeman wrote:
>    Hi guys, I have started having this problem a few weeks ago and it
>    happens about once a week after which I have to restart my Twisted
>    based server to function again. It seems to be happening when I
>    make RPC calls using twisted.web.xmlrpc.Proxy. I have reason to
>    believe that I am either running out of file handles or connection
>    limits. I have up-sized my connection limits and ulimit -n  gives
>    me 9000.

Note though that select() has a builtin limit, which varies by platform
but is probably 1024 for you.  So perhaps try the poll or epoll reactor
instead.

>    I receive less than 30 connections though so there must be some
>    kind of leak. The error I'm getting is the following:

30 is much less than 1024, though, so a leak does sound probable.

[...]
>    Now I know this is a very generic error and it could mean a lot of
>    things, but how would I even start tracking the leak down? Is there
>    a way I can try and track the number of file descriptors?

There's always strace, or looking in /proc/PID/fd.

>    I am using Twisted version 8.2.0 on Ubuntu Server Edition 9.10

Maybe try upgrading Twisted, preferably to 10.0.0?  I'm not sure if a
bug related to your problems was fixed since 8.2.0, but a lot of bugs
have been fixed in that ~2 years.  Hopefully there's a PPA somewhere
with a newer Twisted for your version of Ubuntu.

-Andrew.