[Twisted-Python] multiprocessing capability?
johann.borck at densedata.com
Thu Feb 25 20:13:00 EST 2010
Darren Govoni wrote:
> The nice thing about using Python's process support is that you can
> spawn native Processes that run in separate heaps directly from ONE
> Python Twisted app. Not many running side-by-side, which adds the
> complexity of now coordinating among them (however easy with
> additional protocols like spread).
> Inter-Process communication is also supported in Python's new
> multiprocessing package. And again, it can all be orchestrated from a
> single service _instance_.
> In my code, I need to run "on the metal" for some tasks and not
> others. Agreed, the event-based reactor threading in Twisted is great.
> But not for all modes of computation. For those, I offload onto OS
> processes directly onto CPU cores. Twisted does not provide a way to
> leverage its API against Python's support for this feature.
But twisted provides this feature *itself*, and has done so long before
the multiprocessing module existed - look at Process Protocol
the stdio stuff:
or, as has been mentioned before, ampoule. Of course, if you get
multiprocess to work with twisted, that's fine, but you're probably
unnecessarily adding complexity to your application while substracting
compatibility with python versions before 2.6 for no good reason, at
least none you mentioned so far.
More information about the Twisted-Python