[Twisted-Python] Performance issues of twisted.

[Peter Cai]

Thanks a lot Andy for your explation.  I am sorry for my poor English and I
shall try to make it clearer this time.

1. UDP or TCP ¡ª¡ª I could understood that implementing a strategy for
retrying failures better than TCP is very unlikely.
But I think maintaining about 3000 tcp connections at the same time might be
quite hard too.
That's exactly why we use UDP in the last version written in C.
We have to write a lot of codes to handle retranmission and we have to send
KEEP Alive Message to make sure that if the client program is still working.

Web server doesn't have this problem because they needn't keep every tcp
connections.

2 and 3 ¡ª¡ª memcached looks promising,  I will read more about it.

4.  But if we choose TCP or we cache any data in memory, the bussiness shall
be cut off for a short moment and all clients have to relogin?
All data cached in memory shall be lost any the client might feel like that
they lost some time in the game. Is that true?

5.  Have you heard of BEA?  Which was sold to Oracle in March.   One of
their main products is Java Application Server, WebLogic.   I've talked with
their engineer when considerring using Java to write our server.  He showed
me some diagrams and told me that a sharp performance decrease was
unavoidable when gc happened in any gc enabled system.  (Of course he said
Weblogic could do much better than any other systems ^_^)

6.  I want to know the difference between synchronised and asynchronised,
block and unblocked APIs.  I want to know the differences in fundamental
level.
I know that twisted uses "epoll", but why it's more efficient?

8. Your advice sound very helpful!

Thanks again!

2008/4/14 Andy Fundinger <Andy at newworldelectric.com>:

> I'm not sure about books, but the questions seemed fun.  I'm curious how
> my answers measure up to the rest of the list here.
>
> 2008/4/13 Peter Cai <newptcai at gmail.com>:
>
> > Hi all!
> >
> > I am preparing to write a server application (you may think of it as an
> > online game server) with twisted, but I am worrying about it's performance
> > and how to do it correctly.
> >
> > For example..
> >
> > 1. UDP or TCP£¬which should I choose to gain higher performance?
> >
> If you want the data to get there even if it's late use TCP, if late data
> is not useful to you or needs to be handled some special way use UDP which
> will simply lose packets if they happen to not go through.  It is possible
> but rather unlikely that you can implement a strategy for retrying failures
> better than TCP unless you relax the constraint of eventually getting the
> data through.
>
> 2. Should I catch data in memory instead of write it to db immediately?
> >
>  I wouldn't for local mysql databases.  For a more distributed setup you
> might consider memcached.
>
> 3. Could I make it run on cluster if cache data in memory ¨D¨D I think it
> > might be quit hard to exchange data between instances of the server if I
> > cache data in memory.
> >
> that's exactly what memcached will take care of for you.
>
>
> > 4. How to make hot backup?
> >
> for maximum performance use a replicated pair of database servers and take
> the spare offline for backups.  You could also bring the spare up only to
> capture the backup, though it will slow down the primary as it plays
> catchup.
>
>
> > 5. Garbage collection might make the server halt for a moment
> >
> I think this should be less than the latency of a publicly routed IP
> network, anyone have figures for gc and twisted?
>
> 6. What is happening in a computer when an IP package received?
> >
> What do you mean?  What specifically do you need to know?
>
> 7. Could I get some inspiration from how people write web server?
> >
> this one I can suggest a book for, the twisted book
> http://www.oreilly.com/catalog/twistedadn/ , or if you're considering
> something other than twisted any reasonably thick python book will have an
> example web server.
>
>
> > 8. If i use an separate physical server to deploy the database, could I
> > gain some performance improvement?  Or the cost of communication between the
> > db and the server could hurt the total performance?
> >
> could go either way.  With the effects of Python's GIL I'd expect a
> multi-core server with database and game server on one box would be fine
> until you hit a scaling limit and your server is eating a whole core.  At
> that point you need to develop a distributed application model and probably
> use multiple servers.  If you hit that before you're running 200 users
> recheck your business model, if you hit it before 20 users recheck your
> architecture.
>
> Andy Fundinger
>
> --
> Blog: http://channel3b.wordpress.com
> Second Life Name: Ciemaar Flintoff
>
> I am a sig Virus. Please put me in your sig so that I can continue to
> replicate.
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>


-- 
µ±Ö»ÓÐÒ»ÕÅÖ½ºÍÒ»Ö§±ÊµÄʱºò, ÎÒÏàÐÅÄãдϵĶ«Î÷¶ÔÄãÀ´Ëµ¾ÍÊÇÕæʵµÄ, ²»¹ÜËûÃǵ½µ×ÊDz»ÊÇÕæµÄ¡£
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20080414/35dc5264/attachment.htm