[Twisted-Python] carbon-relay eating CPU - EAGAIN (Resource temporarily unavailable)?

[Christopher Armstrong]

Hi Quan Tong Anh.

On Mon, Nov 25, 2013 at 10:05 PM, Quan Tong Anh <anhquankitty at gmail.com>wrote:

>  - Python 2.7.3
>  - [twisted, version 13.1.0]
>  - xen-domU
>
> `atop` shows that `carbon-relay` is eating 80, 90% USRCPU. From the
> `strace`:
>
>     accept(7, {sa_family=AF_INET, sin_port=htons(60649),
> sin_addr=inet_addr("192.237.222.81")}, [16]) = 257
>     accept(7, {sa_family=AF_INET, sin_port=htons(51564),
> sin_addr=inet_addr("166.78.1.48")}, [16]) = 257
>     accept(7, 0x7ffff4679550, [16])         = -1 EAGAIN (Resource
> temporarily unavailable)
>     accept(7, {sa_family=AF_INET, sin_port=htons(33654),
> sin_addr=inet_addr("198.61.194.248")}, [16]) = 257
>     accept(7, {sa_family=AF_INET, sin_port=htons(50037),
> sin_addr=inet_addr("166.78.181.204")}, [16]) = 257
>     accept(7, 0x7ffff4679550, [16])         = -1 EAGAIN (Resource
> temporarily unavailable)
>
> The strange thing is: even restart the service, it seems stuck at fd 7
> everytime running `strace`. Does it mean this fd is not being cleanup
> properly?
>

You have given a lot of data, but haven't actually described a problem.
Nothing is "stuck" at FD 7, it's simply a listening socket file descriptor,
and the way listening sockets work is you call accept() repeatedly on them
to get incoming connections.

accept() returning EAGAIN is normal behavior and should basically be
ignored. Twisted handles this case appropriately. From the man page:

       EAGAIN or EWOULDBLOCK
              The socket is marked nonblocking and no connections are
present to be accepted.  POSIX.1-2001 allows either error to be returned
for this case, and does not require these constants to have the same value,
so a portable application should check for both possibilities.

Is there actually something wrong with your service? Can you explain the
behavior you're seeing and how it differs from the behavior you want to
see? If the problem is only "carbon uses a lot of CPU", maybe the problem
is either just that carbon is inefficient or you need to scale up to more
instances?



-- 
Christopher Armstrong
http://radix.twistedmatrix.com/
http://planet-if.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20131126/3c5066a4/attachment-0002.html>