<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.28.1">
</HEAD>
<BODY>
What you refer to is different than what I need. The real 'Process' implementation is new to Python 2.6<BR>
<A HREF="http://docs.python.org/library/multiprocessing.html">http://docs.python.org/library/multiprocessing.html</A><BR>
and is not supported in Twisted at the moment. The Process or threads in Twisted now, use Python threading/process constructs outside of the new multiprocessing module, will suffer from the Python GIL limitations - which hinders higher performance computing.<BR>
It works, sure. But its not what I'm asking about.<BR>
<BR>
However, I found a workaround for now.<BR>
<BR>
Ideally, what I wanted to do was use something like threads.deferToPool() or similar, that _would_ use Python's new support for OS processes.<BR>
Someone wrote a wrapper to this on the net, but I was curious if this will be supported in Twisted.<BR>
<BR>
Darren<BR>
<BR>
On Fri, 2010-02-26 at 01:13 +0000, Johann Borck wrote:
<BLOCKQUOTE TYPE=CITE>
<PRE>
Darren Govoni wrote:
> The nice thing about using Python's process support is that you can
> spawn native Processes that run in separate heaps directly from ONE
> Python Twisted app. Not many running side-by-side, which adds the
> complexity of now coordinating among them (however easy with
> additional protocols like spread).
>
> Inter-Process communication is also supported in Python's new
> multiprocessing package. And again, it can all be orchestrated from a
> single service _instance_.
>
> In my code, I need to run "on the metal" for some tasks and not
> others. Agreed, the event-based reactor threading in Twisted is great.
> But not for all modes of computation. For those, I offload onto OS
> processes directly onto CPU cores. Twisted does not provide a way to
> leverage its API against Python's support for this feature.
But twisted provides this feature *itself*, and has done so long before
the multiprocessing module existed - look at Process Protocol
<A HREF="http://twistedmatrix.com/documents/current/core/howto/process.html">http://twistedmatrix.com/documents/current/core/howto/process.html</A> and
the stdio stuff:
<A HREF="http://twistedmatrix.com/documents/current/core/examples/stdiodemo.py">http://twistedmatrix.com/documents/current/core/examples/stdiodemo.py</A>,
or, as has been mentioned before, ampoule. Of course, if you get
multiprocess to work with twisted, that's fine, but you're probably
unnecessarily adding complexity to your application while substracting
compatibility with python versions before 2.6 for no good reason, at
least none you mentioned so far.
regards,
Johann
_______________________________________________
Twisted-Python mailing list
<A HREF="mailto:Twisted-Python@twistedmatrix.com">Twisted-Python@twistedmatrix.com</A>
<A HREF="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python</A>
</PRE>
</BLOCKQUOTE>
<BR>
</BODY>
</HTML>