[Twisted-Python] Benchmark of Python WSGI servers
Michael Thompson
michaelnt at gmail.com
Sat Mar 19 08:39:55 EDT 2011
On 19 March 2011 08:29, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> On Mar 18, 2011, at 9:35 PM, exarkun at twistedmatrix.com wrote:
>
> A multi-process WSGI container might actually be of practical use, since
> it may make more cores available to your server. If an application is
> bottlenecked on CPU rather than some high-latency operation (as you can
> only process as many concurrent requests as you have threads in your
> threadpool), more cores can help.
>
> +1. This will actually speed up your application code, not just the server
> :).
I agree that threads and processes is going to be the first step to
scaling but I don't think there's much advantage putting that in
twisted given that there are other solutions for this.
I was hoping that changing the reactor would give an easy way to make
Twisted look better, from a PR point of view. To check I ran a
benchmark on two EC2 m1.small machines using
httperf --timeout=5 --client=0/1 --server=ip-10-234-217-116
--port=8443 --uri=/ --rate=1000 --send-buffer=4096 --recv-buffer=16384
--num-conns=4000 --num-calls=10
gevent 1998 req/s.
select 734 req/s
poll 687 req/s
epoll 694 req/s
So looks like the reactor has nothing to do with it. Here some profile
results for the select reactor.
2716620 function calls (2716586 primitive calls) in 21.634 CPU seconds
Ordered by: internal time
List reduced from 340 to 34 due to restriction <0.10000000000000001>
ncalls tottime percall cumtime percall filename:lineno(function)
55 5.994 0.109 6.141 0.112 {select.select}
22871 0.992 0.000 0.992 0.000 {built-in method acquire}
5706 0.642 0.000 1.927 0.000 wsgi.py:168(__init__)
165474 0.521 0.000 0.528 0.000 http.py:626(__setattr__)
5706 0.394 0.000 1.778 0.000 http.py:922(write)
5706 0.379 0.000 7.879 0.001 basic.py:543(dataReceived)
142650 0.363 0.000 0.363 0.000 {method 'lower' of 'str' objects}
34236 0.341 0.000 0.582 0.000
http_headers.py:13(_dashCapitalize)
74178 0.324 0.000 0.712 0.000 http_headers.py:222(getRawHeaders)
34236 0.319 0.000 0.954 0.000
http_headers.py:249(_canonicalNameCaps)
22824 0.305 0.000 7.434 0.000 http.py:1537(lineReceived)
133660 0.283 0.000 0.283 0.000 {method 'get' of 'dict' objects}
5700 0.261 0.000 0.494 0.000 abstract.py:186(doWrite)
108414 0.256 0.000 0.256 0.000 {method 'split' of 'str' objects}
5706 0.252 0.000 0.739 0.000 wsgi.py:273(_sendResponseHeaders)
11412 0.242 0.000 0.527 0.000 http.py:1589(headerReceived)
5706 0.242 0.000 0.601 0.000 http.py:606(__init__)
12043 0.242 0.000 9.162 0.001
selectreactor.py:144(_doReadOrWrite)
39942 0.237 0.000 0.357 0.000 http_headers.py:186(setRawHeaders)
5706 0.219 0.000 0.235 0.000 http.py:217(datetimeToString)
5706 0.200 0.000 4.800 0.001 server.py:109(process)
28530 0.197 0.000 0.307 0.000 http.py:1773(_escape)
6301 0.196 0.000 0.240 0.000 base.py:688(callLater)
12043 0.194 0.000 9.432 0.001 context.py:32(callWithContext)
125616 0.194 0.000 0.194 0.000 {len}
5706 0.193 0.000 0.526 0.000 http.py:1701(requestDone)
91366 0.193 0.000 0.193 0.000 {isinstance}
22824 0.173 0.000 0.204 0.000 base.py:97(reset)
97006 0.169 0.000 0.169 0.000 {method 'replace' of
'str' objects}
12043 0.169 0.000 10.085 0.001 log.py:71(callWithLogger)
45648 0.168 0.000 1.207 0.000
http_headers.py:239(getAllRawHeaders)
22856 0.148 0.000 0.148 0.000 {built-in method release}
5715/5712 0.147 0.000 0.308 0.000 defer.py:467(_runCallbacks)
28530 0.144 0.000 0.418 0.000 http.py:856(getHeader)
More information about the Twisted-Python
mailing list