[Twisted-Python] Question about processes in python
Jason J. W. Williams
jasonjwwilliams at gmail.com
Mon Apr 12 12:01:28 EDT 2010
Yeah...our code doesn't actually use the pipes. They get fed over AMQP
(txAMQP). We're essentially using multiprocessing as a sub-proc
supervisor to spin-up "processing" engines that consume the AMQP
queues they're configured with.
On Mon, Apr 12, 2010 at 9:49 AM, Phil Mayers <p.mayers at imperial.ac.uk> wrote:
> On 04/12/2010 04:39 PM, Jason J. W. Williams wrote:
>> Haven't had any issues yet. Twisted imports occur inside the process
>> function. The app was originally written as a purely blocking
>> multiprocessing app and rewritten to use Twisted inside the
>> sub-processes. It's passed all automated and hand tests without an
>> issue. Is there a reason importing Twisted inside sub-process should
>> not work?
> When I last looked at it, multiprocessing did awful things like fork'ing
> and not re-execing the interpreter in the child process, which seemed
> like an absolute disaster waiting to happen, for many types of objects
> which the child process inherits. Does it still do that?
> I guess what you're doing will work though, In that setup, where the
> multiprocessing code is the absolute first thing you call, you're
> essentially using it as a helper to fork off the child process & setup
> the communication pipes (see the example I just posted for a more
> explicit example).
> There's the issue that any multiprocess code which writes to the pipes
> (or whatever) used for sending results will block (and block the
> reactor) of course.
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
More information about the Twisted-Python