[Twisted-Python] Multi-reactor architecture

vitaly at synapticvision.com vitaly at synapticvision.com
Fri Nov 13 13:06:24 EST 2009


So if I get stick to the "vertical scalability"(Site has sessions), is  
it gonna be helpful for performance to run Twisted reactor on a single  
core machine vs multi-core machine (after all Python itself has a  
Global Interpreter Lock)?
OR
the entire "TwsitedGateway+listenSSL+Site+reactor" USAGE should be  
re-designed for the project?

What about 64bit machine influence on Twisted?



Quoting "Reza Lotun" <rlotun at gmail.com>:

>> thank you for such detailed response.
>> I feel, finally I've succeed to express my original question correctly.
>>
>> So if I go one step forward, and lets assume that indeed there is such
>> limit of concurrent connections, THAN:
>> should it be resolved by another architecture or another usage type of
>> Twisted technology or something else?
>
> Again, I don't think there are any universal answers to this question.
> It depends on what you're building. For example, say it's a REST api,
> which by design is stateless (i.e. no sessions). Then you can stick a
> load balancer in front (if you're on EC2 amazon has an "elastic load
> balancer" service for this) and load balance amongst many machines. As
> you find traffic increases you simply add more machines. This is
> called "horizontal scalability" and, as you might imagine, its highly
> desirable.
>
> Another form is "vertical scalability" - that involves getting a
> faster computer to run your server on. This might work for some cases,
> but not in general - it seems to be the method applied to scaling
> RDBMSs, before going down the road of master/slave setups, sharding
> and denormalization.
>
> Of course, you *could* use a different technology entirely when you
> need to scale really high. This might make sense if your'e a small
> company and growing - say you start out as a small team, and you need
> something up quickly that's fairly decent. You happen to know python
> so you roll the whole thing out in Twisted. As time progresses, you
> may rewrite certain systems in, say, erlang or something and move
> forward.
>
> So, it's hard to say, really. At least, I'd like to know myself ;-)
> That's what makes the wheel field so interesting - there's a certain
> creative element to scalable systems.
>
> Cheers,
> Reza
>
> --
> Reza Lotun
> mobile: +44 (0)7521 310 763
> email:  rlotun at gmail.com
> work:   reza at tweetdeck.com
> twitter: @rlotun
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>





More information about the Twisted-Python mailing list