<div dir="ltr"><div><div><div><div><div><div><div><div>Hi Tobias<br><br></div>Some comments inline but first an update on the next load test slightly closer to my ultimate use case, which showed PyPy smoking the competition.<br>
</div><br></div></div>My test is now a request to a URL scheme like this: /version/{app}/platform/{platform}. This gets translated into a get on Redis for {app}-{platform} and the value is returned to the client.<br>All tests were on Ubuntu 12.10 on an m1.small instance on AWS with the load test machine being an m1.large. The load test ran for about 10 minutes for each framework.<br>
</div>Using Cyclone 1.1 on Twisted 13 on PyPy 2.0beta2 I was averaging about 1.8k r/s.<br></div>Using node-restify 2.4.1 and node-redis 0.8.3 node 0.10.4 I was doing about 1.2 r/s.<br></div>Go and Scala on Play were comparable to node.<br>
</div><div><br>This very simple load test has me very impressed with PyPys performance and I look forward to load testing our full app when it's ready.<br><br></div><div>Thanks for the comments, they've been very helpful.<br>
</div><div><div><div><div><div><div><div><div><br><div><div class="gmail_extra"><div class="gmail_quote">On Mon, Apr 22, 2013 at 8:12 AM, Tobias Oberstein <span dir="ltr"><<a href="mailto:tobias.oberstein@tavendo.de" target="_blank">tobias.oberstein@tavendo.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="im">Tom,<br>
<br>
>>which OS are you running on your EC2 instance? Amazon Linux?<br>
>I am running two m1.small Ubuntu 12.10 64bit in us-east-1, one for the web server and the other for the ab.<br>
>>Which reactor are you running your Twisted test with? Epoll?<br>
>Default Linux reactor, so it's Epoll.<br>
<br>
</div>ok, thats good. just wanted to make sure you are aware of this and make sure epoll is actually used.<br>
<div class="im"><br>
>>Are you running on CPython or PyPy?<br>
>I am running on CPython with twisted 13.0.0 installed into a virtualenv, but please see my results for PyPy below, they're far more impressive.<br>
<br>
</div>thats also good (and I was somehow expecting this .. at least thats also my experience).<br>
<br>
there is 1 thing to note when benchmarking networking stuff on PyPy vs CPython that I noticed in my testing:<br>
<br>
both the throughput (bits/s or reqs/s) as well as the average (arithmetic mean) response times are far better with PyPy vs CPy<br>
<br>
BUT (my testings - Twisted/Autobahn/WebSocket): e.g. the 95% quantile (and the worst case) response latency of PyPy is worse than CPy.<br>
<br>
this was PyPy 1.6 on both Linux (epoll) and FreeBSD (kqueue).<br>
<br>
the reason might be GC .. PyPy's GCs might be not as sophisticated as CPy's (likely) or V8's (definitely).<br>
<br>
note: this might have changed with PyPy 2 (did not test that). Armin at least was planning for a generational/incremental GC in PyPy. dunno.<br>
<br>
so: you might wanna test for latency also .. I'd be interested in results.<br></blockquote><div>PyPy 2.0beta2 had comparable 95 percentile latency to all the other frameworks in my limited test so maybe that's the improvements you mentioned? Or maybe my naive load tests, but I like to think it's the improvements.<br>
<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
regarding GC, V8 and Erlang (and possibly that other VMs with the J at the beginning) have an edge ..<br>
<br>
both LuaJIT and PyPy plan to improve on this though ..<br>
<span class=""><font color="#888888"><br></font></span></blockquote><div>I don't know much about the workings of the various GCs but my feeling is I wasn't really stressing the GC with my limited test as there weren't very many data structures in play.<br>
That said over 10 minute load tests I didn't notice any blips in r/s or response times so for my limited test it was all good.<br> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<span class=""><font color="#888888">
Tobias<br>
</font></span><div class=""><div class="h5"><br>
_______________________________________________<br>
Twisted-Python mailing list<br>
<a href="mailto:Twisted-Python@twistedmatrix.com">Twisted-Python@twistedmatrix.com</a><br>
<a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python</a><br>
</div></div></blockquote></div><br></div></div></div></div></div></div></div></div></div></div></div>