[Twisted-Python] intermittent problem: not accepting new connections

[Jean-Paul Calderone]

On Wed, 10 Sep 2008 14:01:27 -0700, Alec Matusis <matusis at yahoo.com> wrote:
>> Do non-error log messages continue to appear in the Twisted log?  ie,
>> is
>> it clear that the logging system is still working, or could it have
>> failed
>> in some way, obscuring any exception reports?
>
>Yes, I print the results of garbage collection every 10 min like this:
>
>print "number of objects tracked by the garbage collector is:", len(gc.get_objects())
>
>and they appear in the main twisted log.
>
>2008/09/10 13:21 -0700 [-] number of objects tracked by the garbage collector is: 860021
>2008/09/10 13:31 -0700 [-] number of objects tracked by the garbage collector is: 864316
>
>> Any new unhandled errno values should definitely result in an exception
>> being logged (notice that the `raise´ which follows the checks for
>> various errno values is inside a try/except which logs any exception).
>
>I noticed that raise too... Could it then be EWOULDBLOCK, EAGAIN or EPERM?
>
>                except socket.error, e:
>                    if e.args[0] in (EWOULDBLOCK, EAGAIN):
>                        self.numberAccepts = i
>                        break
>                    elif e.args[0] == EPERM:
>                        # Netfilter on Linux may have rejected the
>                        # connection, but we get told to try to accept()
>                        # anyway.
>                        continue
>
>
>I am not sure how to debug this problem- I have another twisted server of a different type on that machine, and while the problematic server stops accepting connections, the second one works just fine, so this is not a machine-wide issue.
>What could it be?

You could try running strace against the server while it's not accepting
connections.  You can limit the output to just accept calls, too:

   strace -p <pid> -etrace=accept

Maybe this will show you that accept calls are happening but failing in
some way.  If it produces no output, you can widen it and see if you can
tell what the server is doing if it isn't calling accept().

Jean-Paul