[Twisted-Python] scaling with lots of clients

[AK]

Hi everyone,

Problems might arise if you use multiple processes, each with each own
reactor. Speaking from (limited, admittedly experience), using say two
reactors in two processes respectively, if there is a need to
communicate between them might lead to all kinds of bugs, *if* you do
not do things properly.

Another workaround would be the following:
Python's GIL lock applies only to executing Python bytecode. Therefore,
if you offload some of it to a C-based module, you are GIL-lock free.

On 08/04/2011 01:42 AM, Grant Mckenzie wrote:
>> Threading won't help much if you're CPU bound, since Python can only run
>> one Python thread at once. Process pooling probably would help, though
>> will involve a more complex infrastructure. Which part of the docs made
>> you think that process pooling is a bad idea?
> Hi Itamar,
>
> on re-reading I see nothing particular about process pooling being a bad idea. 
>
> I see the explanation of spawning processes from the reactor here:
>
> http://twistedmatrix.com/documents/current/core/howto/process.html
>
> This seems to be designed for spawning processes for compute purposes which no doubt has a lot of use cases. I'd like ot spawn 
> processes to handle user communication however. THe only thing that I can think of is multiple processes each with their
> own reactor handling communication, and one or more other processes to handle compute. Thoughts?
>
> THanks for your help.
> G
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

-- 
What is the air-speed velocity of an unladen swallow?