[Twisted-web] Parallelism is twisted

Glyph Lefkowitz glyph at twistedmatrix.com
Wed Jan 6 22:02:06 EST 2010


On Jan 6, 2010, at 9:04 PM, arun chhetri wrote:

> First of all I would like to thank to developer of twisted for making such a great platform for network applications,, I just love it...

Great!  Glad to hear you're enjoying it!

> 1) What is the underlying technology?
> As, I believe that twisted is a TCP server which which uses select() based call to handle multiple requests. Is it true or there is something else which I am not aware of.

Twisted uses select() (or something like it) to handle multiple connections, it's true.  But Twisted can handle UDP, UNIX sockets, serial ports, and several other kinds of resource in addition to just TCP.  If you are using Twisted with a GUI library (such as GTK+, Qt, Cocoa, or WxWidgets) you can handle GUI events in the same event loop.

> 2) So, if twisted is single process based network application framework (without threads and forks),, I believe it cannot take advantage of multiple processor residing on one machine. For e.g. if OS schedule twisted process on one processor,, that's it,, it can only run on same processor not taking advantage of multiple processor. I am kind of confused in this question. Can anyone shed some light on this?

One of the other types of connection that Twisted can handle is a pipe connection to a subprocess.  If you want to take advantage of multiple connections, you can use Twisted's process-spawning APIs, either directly:

http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.interfaces.IReactorProcess.spawnProcess.html

or via a convenience process-pool API, such as Ampoule:

https://launchpad.net/ampoule

> 3) So, suppose I have one twisted reactor based process running,, I can use defferToThread as one of the way to kind of using multiple processing?

No, not really.  Threads in python are bound by the global interpreter lock and are therefore not that useful for using multiple processors.




More information about the Twisted-web mailing list