[Twisted-Python] fast high load protocol

Vlad Shevchenko vlad.shevchenko at gmail.com
Tue Feb 16 18:23:10 EST 2010


Really I need a large amount of data to be available before I can
begin calculations, Json-PRC vs. PB provide a 10+ times smaller
response time for same returning data


On Tue, Feb 16, 2010 at 10:08 PM, Phil Christensen <phil at bubblehouse.org>wrote:

> On Feb 16, 2010, at 2:46 PM, Vlad Shevchenko wrote:
> > I have developed two kind of servers:
> >    api - http server
> >    memory_db - dict-based server with some logic of putting data to dict
> and getting it back.
> >
> > Api communicate with memory_db via txJSON-RPC but response time is huge
> under high load. I also try implementation with replacement txJSON-RPC ->
> Perspective Broker with no success. PB implementation has 10+ times faster
> responses when returning data is small (dict with few keys and values like
> list of 1-5 strings). However, PM much depends on size of returning value.
> When I return a list of 600 items (each is string with length 20) response
> time incredibly growing up. Unlike PB, json-rpc implementation keeps it
> response-time near the same lever as with small return data.
> >
> > Can you suggest some protocol with a small response? Or should I look
> into development of my own?
>
> It definitely depends on your use case...
>
> PB can still be a good solution for you if you change your architecture a
> little bit. Instead of returning a 600-item list, you could return a
> pb.Referencable object that can iteratively return each row at a time.
>
> You may also want to look into using Copyable or Cacheable PB objects. More
> on that here:
>
>
> http://twistedmatrix.com/documents/current/core/howto/pb-copyable.html
>
> Of course, if you need a large amount of data to be available before you
> can begin calculations or whatnot, you're going to have to wait for it to be
> sent no matter what protocol you use.
>
> -phil
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
WBR, Vlad Shevchenko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20100217/ead197fa/attachment-0001.htm 


More information about the Twisted-Python mailing list