[Twisted-Python] Re: To thread or not to thread...
Jonathan Lange
jml at divmod.com
Mon Aug 2 23:29:48 MDT 2004
Britt Green wrote:
> I've been reading some of the online documentation for Twisted, and
> I'm slowly starting to grasp different things. However, I could use
> some help with one point. Suppose I was coding up a chat server, or
> perhaps a MUD. Would I need to use something threaded, as opposed to
> just using reactor.run()? Or would the latter be sufficient?
>
This seems to be a common stumbling point for newcomers to Twisted.
With Twisted, you don't need threads. You don't want threads. Threads
are actually harmful: http://www.kuro5hin.org/story/2002/11/18/22112/860
But this doesn't help you write a chat server. How can you do that
without threads? The answer falls into two parts: how Twisted does it
and how you use Twisted to do it.
1. How Twisted Does It
When you run a Twisted server, Twisted calls the socket functions
asynchronously. So, rather than calling a listen function which sits
there waiting until someone connects, it calls a listen function which
returns immediately. Then, in the <b>reactor</b> loop, it polls things.
When something connects _Twisted calls your code_ which it expects to
_return immediately_. The entire process will wait for your code to return.
So, this leaves a bit of a problem: How do I do stuff that takes time?
The answer is too complicated to go into now. The important bit is that
your function doesn't compute a value and return it. It returns a
Deferred -- a promise to return later.
2. How You Get Twisted to do it.
---------------------------------------
port = 8007
class MyProtocol(Protocol):
def connectionMade(self):
"""someone has connected. do something"""
def dataReceived(self, data):
"""do stuff with data"""
factory = ServerFactory()
factory.protocol = MyProtocol
reactor.listenTCP(port, factory)
reactor.run()
---------------------------------------
Notice how you don't really call any of your own code?
The code is essentially preparing an object that can be registered with
Twisted as a server factory.
Every time someone connects to port 8007 on that host, Twisted will
instantiate an instance of MyProtocol and call connectionMade on it.
Every time someone sends data, Twisted will call dataReceived.
The reactor calls everything. If your connectionMade implementation
takes ages to return, then the process is in _your_ code, not the
reactor's. This means the reactor can't listen for data and can't call
your dataReceived method. All this really means is that your methods
should return quickly.
Really, that's all you need to know. Your methods should return quickly.
As long as they do that, you don't really need to worry about threads
because Twisted will take care of concurrency for you.
Recommended Reading:
"Asynchronous Programming" -
http://www.twistedmatrix.com/documents/howto/async
"Writing Servers" -
http://www.twistedmatrix.com/documents/howto/servers
"Using Deferreds" -
http://www.twistedmatrix.com/documents/howto/defer
"Writing Clients" -
http://www.twistedmatrix.com/documents/howto/clients
"Threads Considered Harmful"
http://www.kuro5hin.org/story/2002/11/18/22112/860
Recommended Actions:
* Try writing a _simple_ client/server app in Twisted. It might even be
worth manually typing out the "quote of the day" examples in the howtos,
to force your brain to only deal with each concept as it comes, rather
than trying to grasp the whole in all it's detail.
* Remember that asynchronous programming is just another style of
programming.
* Forget about threads. You almost never need them. Especially not in
Twisted.
cheers,
jml
More information about the Twisted-Python
mailing list