[Twisted-Python] Server Components

Phil Christensen phil at bubblehouse.org
Fri Mar 28 10:52:37 EDT 2008


On Mar 28, 2008, at 9:18 AM, Simon Pickles wrote:
> Midway thru my server dev, I am seeing the light... well, a light  
> anyway.
>
> Its a stackless python server and not too bad, running on Ubuntu  
> Linux. Stackless Python is great, but single threaded (I DO do my DB  
> lookups and network comms in other threads).
>
> I figure its GOT to be more concurrent, if I'm looking to use future  
> architecture.
>
> So I planned to rewrite the server into modules, each a separate  
> app. I guess this is a virtualisation process. Labour could be  
> subdivided spatially, or by task (db access/playermanager/AI), or both

I'm going to go ahead assuming you know where your load problems are  
(or will be), and which things are most in need of being separated  
into components. Of course, this is pretty much the hardest part.

>
> The modules plug in to each other, communicating thru the network  
> (twisted).
>
> This means the prototype server would run all the module apps on a  
> single machine, then spreading to other machines to allow  
> scalability. Being async networked (hinged on twisted.deferred),  
> these modules run concurrently, using all the cores available.
>
> SO - Am I reinventing the wheel? Does Twisted do this, or other  
> Frameworks? In my limited experience, I need to create a server hub  
> then have all the component modules connect to that as clients.
>
> Is there a better way?

I don't believe there's nothing in Twisted that implicitly provides  
the functionality you're talking about, but you have a wide array of  
options on how to implement this yourself. It all depends on what kind  
of information you'll need to pass between modules, and how much  
refactoring you're willing to do to the existing application.

The biggest issue will probably be making a uniform and maintainable  
way for modules to discover each other and make connections without  
introducing security holes. If you are sure to have complete control  
over the network they're running on (say, in a private IP space) this  
may be a non-issue, so it's really just about logistics.

My initial reaction was to suggest Perspective Broker, but someone  
else might recommend AMP, which I'm just starting to learn the ins and  
outs of....

-phil




More information about the Twisted-Python mailing list