[Twisted-Python] I found an interisting comment about Twisted vs. Erlang

[Alec Matusis]

> What was your application doing at the time?  Was it idle, heavily
> loaded, somewhere in the middle?  What is the QoS for each client
> connecting to your service?  Are requests being handled in a timely
> fashion?  Do the users of your service perceive a performance
> problem or do you just not like seeing big numbers in the %CPU
> column?  Are performance problems really in Twisted, or are they
> because of suboptimal decisions in application logic/factoring?
> 
> Posting a list like that and suggesting that something is wrong (or
> right, even) with Twisted isn't really productive: there's nowhere
> obvious to go from here to make it better.
> 
> My first glance at those numbers made me think Twisted is awesome
> because it lets you write an application that can actually make full
> use of the CPU power you have, but I'm guessing that's not what
> you're getting at. :)

I am actually not suggesting the performance is bad, it's quite decent. 

Each process is handling about 30,000 clients, using epoll reactor, and is heavily loaded as you can see from S column, which shows "R".  The app broadcasts the messages sent from a client to groups of other clients, and has some other logic.
I posted this twice by mistake, but as I wrote in my second post, the debugging is pretty hard, esp memory leaks... Also, performance debugging- say how much time is spent in each function? FB engineers seems to say that debugging of E is transparent, but I do not have the first hand knowledge. 

> -----Original Message-----
> From: twisted-python-bounces at twistedmatrix.com [mailto:twisted-python-
> bounces at twistedmatrix.com] On Behalf Of Jamu Kakar
> Sent: Tuesday, September 29, 2009 12:48 PM
> To: Twisted general discussion
> Subject: Re: [Twisted-Python] I found an interisting comment about
> Twisted vs. Erlang
> 
> Hi Alec,
> 
> On Tue, Sep 29, 2009 at 12:36 PM, Alec Matusis <matusis at yahoo.com>
> wrote:
> > As you can see from the %CPU column, I have my reasons for concern ;)
> This
> > is current copy and paste from a node with 2x quad core xeon L5420  @
> > 2.50GHz - 1 twistd process per core.
> >
> >  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> >
> > 24448 nobody     20   0  991m 607m 2452 R   99  4.3  24764:37 twistd
> >
> > 28553 nobody     20   0  909m 453m 2412 R   95  3.2   1346:51 twistd
> >
> > 24640 nobody     20   0 1092m 676m 2452 R   93  4.8  32750:14 twistd
> >
> > 29900 nobody     20   0  802m 362m 2412 R   93  2.6   1180:53 twistd
> >
> > 24279 nobody     20   0  761m 277m 2424 R   42  2.0  13891:58 twistd
> >
> > 32422 nobody     20   0 1381m 962m 2372 R   10  6.9  13241:54 twistd
> >
> > 24210 nobody     20   0  599m 236m 2396 S    4  1.7   9063:29 twistd
> >
> > 29862 nobody     20   0  323m  14m 2384 S    2  0.1  71:53.50 twistd
> 
> What was your application doing at the time?  Was it idle, heavily
> loaded, somewhere in the middle?  What is the QoS for each client
> connecting to your service?  Are requests being handled in a timely
> fashion?  Do the users of your service perceive a performance
> problem or do you just not like seeing big numbers in the %CPU
> column?  Are performance problems really in Twisted, or are they
> because of suboptimal decisions in application logic/factoring?
> 
> Posting a list like that and suggesting that something is wrong (or
> right, even) with Twisted isn't really productive: there's nowhere
> obvious to go from here to make it better.
> 
> My first glance at those numbers made me think Twisted is awesome
> because it lets you write an application that can actually make full
> use of the CPU power you have, but I'm guessing that's not what
> you're getting at. :)
> 
> Thanks,
> J.
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python