[Twisted-Python] Question about processes in python

Jason J. W. Williams jasonjwwilliams at gmail.com
Mon Apr 12 10:01:28 MDT 2010


Yeah...our code doesn't actually use the pipes. They get fed over AMQP
(txAMQP). We're essentially using multiprocessing as a sub-proc
supervisor to spin-up "processing" engines that consume the AMQP
queues they're configured with.

-J

On Mon, Apr 12, 2010 at 9:49 AM, Phil Mayers <p.mayers at imperial.ac.uk> wrote:
> On 04/12/2010 04:39 PM, Jason J. W. Williams wrote:
>> Haven't had any issues yet. Twisted imports occur inside the process
>> function. The app was originally written as a purely blocking
>> multiprocessing app and rewritten to use Twisted inside the
>> sub-processes. It's passed all automated and hand tests without an
>> issue. Is there a reason importing Twisted inside sub-process should
>> not work?
>
>
> When I last looked at it, multiprocessing did awful things like fork'ing
> and not re-execing the interpreter in the child process, which seemed
> like an absolute disaster waiting to happen, for many types of objects
> which the child process inherits. Does it still do that?
>
> I guess what you're doing will work though, In that setup, where the
> multiprocessing code is the absolute first thing you call, you're
> essentially using it as a helper to fork off the child process & setup
> the communication pipes (see the example I just posted for a more
> explicit example).
>
> There's the issue that any multiprocess code which writes to the pipes
> (or whatever) used for sending results will block (and block the
> reactor) of course.
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>




More information about the Twisted-Python mailing list