[Twisted-Python] fast high load protocol

Vlad Shevchenko vlad.shevchenko at gmail.com
Tue Feb 16 16:25:57 MST 2010


here is some statistic:

low network traffic
===========================================
Json-RPC                  PB

average MIN 0.016         average MIN 0.011
average MAX 0.044         average MAX 0.029
AVG 0.027                 AVG 0.017
90% line 0.027            90% line 0.017
CLIENTS 139               CLIENTS 142
ERRORS 0                  ERRORS 0

average MIN 0.029         average MIN 0.023
average MAX 0.426         average MAX 0.159
AVG 0.151                 AVG 0.062
90% line 0.124            90% line 0.052
CLIENTS 317               CLIENTS 316
ERRORS 0                  ERRORS 0

average MIN 0.32          average MIN 0.115
average MAX 2.763         average MAX 1.521
AVG 1.064                 AVG 0.405
90% line 0.908            90% line 0.308
CLIENTS 456               CLIENTS 479
ERRORS 0                  ERRORS 0

average MIN 0.253         average MIN 0.02
average MAX 4.031         average MAX 1.486
AVG 1.123                 AVG 0.296
90% line 0.859            90% line 0.189
CLIENTS 482               CLIENTS 501
ERRORS 30                 ERRORS 0


hight network traffic  (string 7000 length)
===========================================
Json-RPC                  PB

average MIN 0.016         average MIN 1.755
average MAX 0.114         average MAX 3.517
AVG 0.045                 AVG 2.58
90% line 0.043            90% line 2.58
CLIENTS 138               CLIENTS 88
ERRORS 0                  ERRORS 0

average MIN 0.333         average MIN 4.923
average MAX 1.516         average MAX 9.019
AVG 0.732                 AVG 6.746
90% line 0.652            90% line 6.746
CLIENTS 288               CLIENTS 173
ERRORS 0                  ERRORS 0

average MIN 0.409         average MIN 9.505
average MAX 2.372         average MAX 16.546
AVG 1.221                 AVG 12.486
90% line 1.119            90% line 12.448
CLIENTS 429               CLIENTS 273
ERRORS 0                  ERRORS 0

average MIN 0.414         average MIN 14.22
average MAX 3.932         average MAX 22.986
AVG 1.639                 AVG 17.987
90% line 1.426            90% line 17.866
CLIENTS 496               CLIENTS 392
ERRORS 27                 ERRORS 0


On Tue, Feb 16, 2010 at 10:08 PM, Phil Christensen <phil at bubblehouse.org>wrote:

> On Feb 16, 2010, at 2:46 PM, Vlad Shevchenko wrote:
> > I have developed two kind of servers:
> >    api - http server
> >    memory_db - dict-based server with some logic of putting data to dict
> and getting it back.
> >
> > Api communicate with memory_db via txJSON-RPC but response time is huge
> under high load. I also try implementation with replacement txJSON-RPC ->
> Perspective Broker with no success. PB implementation has 10+ times faster
> responses when returning data is small (dict with few keys and values like
> list of 1-5 strings). However, PM much depends on size of returning value.
> When I return a list of 600 items (each is string with length 20) response
> time incredibly growing up. Unlike PB, json-rpc implementation keeps it
> response-time near the same lever as with small return data.
> >
> > Can you suggest some protocol with a small response? Or should I look
> into development of my own?
>
> It definitely depends on your use case...
>
> PB can still be a good solution for you if you change your architecture a
> little bit. Instead of returning a 600-item list, you could return a
> pb.Referencable object that can iteratively return each row at a time.
>
> You may also want to look into using Copyable or Cacheable PB objects. More
> on that here:
>
>
> http://twistedmatrix.com/documents/current/core/howto/pb-copyable.html
>
> Of course, if you need a large amount of data to be available before you
> can begin calculations or whatnot, you're going to have to wait for it to be
> sent no matter what protocol you use.
>
> -phil
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
WBR, Vlad Shevchenko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20100217/c7360663/attachment.html>


More information about the Twisted-Python mailing list