<br><br><div><span class="gmail_quote">On 1/18/07, <b class="gmail_sendername"><a href="mailto:glyph@divmod.com">glyph@divmod.com</a></b> <<a href="mailto:glyph@divmod.com">glyph@divmod.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><span class="q">On 01:59 am, <a href="mailto:jarrod@vertigrated.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">jarrod@vertigrated.com</a> wrote:<br><br>>We have LOTS and LOTS of test harness code and profiling code to pinpoint
<br>>where bottlenecks are. We are going to have process a couple of terabytes a<br>>day thru this system. Latency thru the system is a high priority because of<br>>what kind of system it is.<br><br></span>If you do manage to keep Twisted in the loop on this project, please share what strategy you used. It is unfortunately rare that Twisted is itself a really performance-critical component of any system (especially a highly parallel system), so information about optimization is hard to find.
<br></div>
<br></blockquote></div><br><br>Here is an update. I have learned more about the Niagra chip in the T1000/T2000 servers.<br>The T2000 we are using has a 8 core 1 Ghz processor, each core gets 4 threads, so Solaris sees a 32 processor machine.
<br>We noticed from watching top that each thread is only getting 3.1% of CPU time. Doing the math, that means our Twisted thread is running on a 250 mhz processor :-(<br>The Niagra chip is basically like having 32 250mhz processors with a really effiecient scheduler. With that information in hand, we can now begin to understand how we need to approach deployment of our Twisted server in this environment.
<br><br>To get the kind of client counts ( 2000 clients ) we would have to run 20 instances of our Twisted server, with 100 clients each.<br>We have tried this and it appears it will work, we need more time to test. <br>