[Twisted-Python] fast high load protocol
Vlad Shevchenko
vlad.shevchenko at gmail.com
Tue Feb 16 18:25:57 EST 2010
here is some statistic:
low network traffic
===========================================
Json-RPC PB
average MIN 0.016 average MIN 0.011
average MAX 0.044 average MAX 0.029
AVG 0.027 AVG 0.017
90% line 0.027 90% line 0.017
CLIENTS 139 CLIENTS 142
ERRORS 0 ERRORS 0
average MIN 0.029 average MIN 0.023
average MAX 0.426 average MAX 0.159
AVG 0.151 AVG 0.062
90% line 0.124 90% line 0.052
CLIENTS 317 CLIENTS 316
ERRORS 0 ERRORS 0
average MIN 0.32 average MIN 0.115
average MAX 2.763 average MAX 1.521
AVG 1.064 AVG 0.405
90% line 0.908 90% line 0.308
CLIENTS 456 CLIENTS 479
ERRORS 0 ERRORS 0
average MIN 0.253 average MIN 0.02
average MAX 4.031 average MAX 1.486
AVG 1.123 AVG 0.296
90% line 0.859 90% line 0.189
CLIENTS 482 CLIENTS 501
ERRORS 30 ERRORS 0
hight network traffic (string 7000 length)
===========================================
Json-RPC PB
average MIN 0.016 average MIN 1.755
average MAX 0.114 average MAX 3.517
AVG 0.045 AVG 2.58
90% line 0.043 90% line 2.58
CLIENTS 138 CLIENTS 88
ERRORS 0 ERRORS 0
average MIN 0.333 average MIN 4.923
average MAX 1.516 average MAX 9.019
AVG 0.732 AVG 6.746
90% line 0.652 90% line 6.746
CLIENTS 288 CLIENTS 173
ERRORS 0 ERRORS 0
average MIN 0.409 average MIN 9.505
average MAX 2.372 average MAX 16.546
AVG 1.221 AVG 12.486
90% line 1.119 90% line 12.448
CLIENTS 429 CLIENTS 273
ERRORS 0 ERRORS 0
average MIN 0.414 average MIN 14.22
average MAX 3.932 average MAX 22.986
AVG 1.639 AVG 17.987
90% line 1.426 90% line 17.866
CLIENTS 496 CLIENTS 392
ERRORS 27 ERRORS 0
On Tue, Feb 16, 2010 at 10:08 PM, Phil Christensen <phil at bubblehouse.org>wrote:
> On Feb 16, 2010, at 2:46 PM, Vlad Shevchenko wrote:
> > I have developed two kind of servers:
> > api - http server
> > memory_db - dict-based server with some logic of putting data to dict
> and getting it back.
> >
> > Api communicate with memory_db via txJSON-RPC but response time is huge
> under high load. I also try implementation with replacement txJSON-RPC ->
> Perspective Broker with no success. PB implementation has 10+ times faster
> responses when returning data is small (dict with few keys and values like
> list of 1-5 strings). However, PM much depends on size of returning value.
> When I return a list of 600 items (each is string with length 20) response
> time incredibly growing up. Unlike PB, json-rpc implementation keeps it
> response-time near the same lever as with small return data.
> >
> > Can you suggest some protocol with a small response? Or should I look
> into development of my own?
>
> It definitely depends on your use case...
>
> PB can still be a good solution for you if you change your architecture a
> little bit. Instead of returning a 600-item list, you could return a
> pb.Referencable object that can iteratively return each row at a time.
>
> You may also want to look into using Copyable or Cacheable PB objects. More
> on that here:
>
>
> http://twistedmatrix.com/documents/current/core/howto/pb-copyable.html
>
> Of course, if you need a large amount of data to be available before you
> can begin calculations or whatnot, you're going to have to wait for it to be
> sent no matter what protocol you use.
>
> -phil
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
--
WBR, Vlad Shevchenko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20100217/c7360663/attachment.htm
More information about the Twisted-Python
mailing list