[Twisted-web] 100% CPU; what are my options?

Leif K-Brooks eurleif at gmail.com
Sat Jan 2 22:26:57 EST 2010


I operate Omegle (http://omegle.com/), which is a COMET-driven
Twisted.Web app. (It doesn't use Nevow or anything; just Twisted.Web.)
It gets fairly heavy traffic; right now, it peaks daily at around 6500
concurrent users. Technical details:

- Hosted on a Linode 2880 virtual private server
(https://www.linode.com/); Ubuntu, 2880 MB of RAM, 4 cores available
- EPollReactor

Omegle currently runs at close to 100% CPU for most of the day, and
I'm a bit concerned about that. Here are some questions I have:

- How much more load might my app be able to take before it dies?
Currently, during peak load, a request from localhost is completed in
around 350ms; not ideal, but acceptable. I think anything under 1000ms
would be all right, but I'd prefer the latency to be as low as
possible, of course. Is this likely to scale linearly with traffic, or
- I've looked at my code pretty closely, and it doesn't seem to be
doing anything particularly inefficient, but are there any tricks I
might be able to use to increase performance? Tweaking settings, etc.?
- Ultimately, I think my app will need to be able to use multiple
cores. If I have to, I'm going to either rewrite it in Erlang, or
rewrite it to load balance across multiple Twisted processes. Is there
anything easier I could do to allow the use of multiple cores?

Thanks a lot,

More information about the Twisted-web mailing list