<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=utf-8">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>></span> Would it be possible to replicate this issue
with your application in some kind of load-testing environment so that we can
analyze it "in the lab"? <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>So far, we could not replicate this in the lab- with
thousands of fake multithreaded clients the behavior is normal.<o:p></o:p></p>
<p class=MsoNormal>Thus far, we have run this for 2 days with twisted 2.5 with
poll and epoll, and we are now seeing big memory leaks that did not occur on
2.2, especially on epoll, in addition to 99.9% CPU.<o:p></o:p></p>
<p class=MsoNormal>Back on 2.2 one of our servers was slowly leaking memory, going
from 45MB to about 500MB in about 10 days. On epoll in 2.5 it went from 45MB to
1.9g in a couple of hours, almost taking the machine down. The other server did
not have memory leak on 2.2, staying at 70MB for weeks at a time. In one day it
went from 70MB to 80MB on epoll in 2.5. Both show 99% CPU, so I suspect the
memory leaks may be caused by skipping garbage collection because the process
is too busy. Interestingly, the actual performance on epoll is not bad, the
servers remain responsive even where the poll-based reactor in 2.2 was having
difficulties.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>We must revert to 2.2 tonight.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I emailed you and Jean-Paul. If you like, we can arrange ssh
to our live servers, then you could look at it on Twisted 2.2 first, then we
will switch again to 2.5 and then you can examine it when it’s unstable.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> twisted-python-bounces@twistedmatrix.com
[mailto:twisted-python-bounces@twistedmatrix.com] <b>On Behalf Of </b>glyph@divmod.com<br>
<b>Sent:</b> Wednesday, April 11, 2007 12:28 PM<br>
<b>To:</b> Twisted general discussion<br>
<b>Subject:</b> RE: [Twisted-Python] epoll reactor problems<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>On 06:38 pm, matusis@matusis.com wrote:<br>
<br>
>I wonder if we should revert to Twisted 2.2 ASAP<br>
<br>
I can't speak to your deployment requirements, but it sounds like this inquiry
could lead to some extremely valuable information to fix this issue in the next
release of Twisted. Would it be possible to replicate this issue with
your application in some kind of load-testing environment so that we can
analyze it "in the lab"? Although I'd really like to see this
through to a satisfying conclusion, I don't want your application to have to
pay the cost of fixing Twisted.<o:p></o:p></p>
</div>
</div>
</body>
</html>