[Twisted-Python] Socket error 22, Invalid Argument in twisted/internet/tcp.py

[Werner Thie]

>> Is a try/except with a normalized answer in the embracing function in 
>> twisted/internet/tcp.py a possible cure?
> 
> You mean in the implementation of getHost()?  Maybe, maybe not.  I'd like
> to understand what OpenBSD thinks it is telling us when it returns EINVAL
> here.

According to several 10 Gig of logfiles the error occurs in all kind of
situations be it POST or GET and always with the same traceback. What I
also see is that errors are more frequent when loads are higher, which
leads me to the conclusion that this might be a resource dependency
specially the fd count per process which we currently have at 8192 as I
already suspected in our previous discussion end of April.

I agree with you that my test script is not exactly the same situation
and the test you sent (just for the sake of completeness) me was 
inconclusive:

---- Paul's code ------------------------------------------------------
I wonder if OpenBSD gives back EINVAL for getsockname(2) when the peer
has done a shutdown of the socket.  Can you tell me what this program does?

    from socket import socket
    port = socket()
    port.bind(('', 0))
    port.listen(1)
    client = socket()
    client.setblocking(False)
    client.connect_ex(port.getsockname())
    server, clientAddr = port.accept()
    print 'Before close', server.getsockname()
    client.close()
    print 'After close', server.getsockname()

If it finishes without error, try it with a time.sleep(5) before the
final line.
---- end of Paul's code ------------------------------------------------

After a lot of googling there seems to be a possible onset of the 
problem dating back to 2006 shortly before the 4.3 release. There are 
other users reporting similar 'hickups' I experience on SSH and FTP, but 
none of those reports actually lead to a fix probably because everybody 
is used to a 'slow login', a typein hickup or a failed ftp.

One might speculate that this could be a hardware related error but us 
having several servers with very different HW under similar loads and 
observing those errors it might be a race condition in the kernel driver.

Also odd enough that the man pages for OpenBSD are the only one among 
the **xes who do not mention EINVAL as a possible return value but in 
fact do return it.

I'll try to file a bug with OpenBSD but doubt that it will get much 
attention being so impossible to trigger it.

Does somebody which is more fit on TCP/IP have an additional idea how to 
trigger such a behavior as described in my previous posts. Traceback 
below for completeness

Thxs, Werner

Running on twisted 2.5 OpenBSD I see the following traceback in the logs 
every twenty minutes with up to 200 athena/LivePages connected:

         Traceback (most recent call last):
         --- <exception caught here> ---
File 
"/usr/local/lib/python2.5/site-packages/twisted/internet/selectreactor.py", 
line 148, in _doReadOrWrite
             why = getattr(selectable, method)()

File "/usr/local/lib/python2.5/site-packages/twisted/internet/tcp.py", 
line 362, in doRead
             return self.protocol.dataReceived(data)

File 
"/usr/local/lib/python2.5/site-packages/twisted/protocols/basic.py", 
line 233, in dataReceived
             why = self.lineReceived(line)

File "/usr/local/lib/python2.5/site-packages/twisted/web/http.py", line 
1069, in lineReceived
             self.allContentReceived()

File "/usr/local/lib/python2.5/site-packages/twisted/web/http.py", line 
1110, in allContentReceived
             req.requestReceived(command, path, version)

File "/usr/local/lib/python2.5/site-packages/twisted/web/http.py", line 
603, in requestReceived
            self.host = self.channel.transport.getHost()

File "/usr/local/lib/python2.5/site-packages/twisted/internet/tcp.py", 
line 672, in getHost return address.IPv4Address('TCP', 
*(self.socket.getsockname() + ('INET',)))
           File "<string>", line 1, in getsockname

         socket.error: (22, 'Invalid argument')