[Twisted-Python] ENOBUF and Twisted

Grzegorz Makarewicz mak at trisoft.com.pl
Thu Aug 19 04:06:35 MDT 2004


Andrew Bennetts wrote:
> On Thu, Aug 19, 2004 at 09:31:31AM +0200, Grzegorz Makarewicz wrote:
> 
>>screwtape at froup.com wrote:
>>
>>>A colleague of mine was working with a server written in Twisted
>>>today, and he ran into a problem involving Twisted. He called me over
>>>(since I'm supposed to be the Python Guru here, which just means I'm
>>>slightly less ignorant than everybody else :) and we tracked the
>>>behaviour in question all the way down to twisted.internet.tcp.
>>>
>>
>>This is not a bug
>>
>>select says "yes there is room inm the socket buffer"
>>however datagram protocols do not store anythign in the outgoing socket
>>buffer, but, instead hand the packets directly to the driver for the
>>NIC.
>>
>>when the NIC fils up it returns ENOBUFS
>>
>>use sleep(0.01) between calls to socket.send
> 
> 
> Using sleep in Twisted would be a bad idea -- if you have other sockets on
> other network cards, you should just work on servicing them rather than
> stopping everything.  It's possible there's other work to do other than
> socket operations, too, e.g. servicing any pendingTimedCalls (created by
> reactor.callLater).  In short, time.sleep should not be used in the Twisted
> main thread, let alone the reactor itself.
> 

On msw sleep(small number) has nice side effect - constext switch, on 
other systems perhaps too.

If I have tight loop with call to socket.send with large buffer (eg > 
2MB) and no other python-threads /no thread switch inside ceval loop/ 
then this error can be thrown after few calls.

> If the only sane way to deal with this is to wait 100ms and try again, then
1ms - can be sufficient to but I'm not sure.

In my opionion much safer is divide large block - 64 KB chunk works for 
me without performance degradation and without any socket errors.

mak




More information about the Twisted-Python mailing list