[Twisted-Python] 3 questions about async, threading and dynamic protocol selection

[glyph at divmod.com]

Hi u1CtDC,

On 10:11 am, littlesweetmelon at gmail.com wrote:
>Q1: How to determine a function is 'blocking action' or not?
>Any function needs CPU times. Indeed, a computational-intensive
>function is blocking action. But how small/fast function can be
>classified as non-blocking? Twist requires all user functions to be
>non-blocked. If reactor calls a blocking function, what will happen?
>In my mind, reactor maintains a command queue internally (just like
>windows message queue). The blocking function only postpones the
>execution of other queued functions, but it does not break the logic
>of the program. Is that right?

You've basically answered your own question here.  A "blocking action" 
is one where your users will not want to wait for it :).
>Q2: Today when I go through the twist document, I am confused about
>the threading problem in reactor.
>What is the different between 'reactor.callFromThread' and a plain
>call in reactor loop?
>def callFromThread(f):
>            self.threadCallQueue.append((f, args, kw))
>            self.wakeUp()
>It seems equivalent to reactor.callLater(0, f...).
>What is the real circumstance for calling callFromThread?

callLater is not thread-safe.  callFromThread is designed to be called, 
well, from a thread.  You use callFromThread from threads *other* than 
the thread where the reactor is running, in order to wake up the reactor 
thread and have it do something - usually something that involves 
calling a non-thread-safe Twisted API.
>Further more, how to determine a function is thread-safe in python?

As in any other language, ask the person who wrote it.  There is no 
other way to determine if a function is thread-safe.  In any language 
with dynamic run-time dispatch, determining this without talking to the 
author of the code in question reduces to the halting problem.
>Why the twist doc says: "writing data to a transport from a protocol
>is not thread-safe."?

Nothing in Twisted is thread-safe (other than callFromThread) but this 
is a particularly common error and we wanted to stress it.
>Q3: In my application, I need a facility to dynamically select a
>protocol to communicate with the server.
>Eg: When connected, the server sent a string to client to indicate the
>version of protocol it used. Then, the client can load the proper
>protocol. But I don't know how to implement this. A 'Factory' can only
>create one kind of 'Protocol', and it seems the instances of
>'Protocol' cannot share the connection (Transport object) to a server.
>Could you give me some clues?

The protocol that you are implementing includes a "version" message. 
Unless you have a "Protocol" object connected to receive the data and 
decode that message, you can't decide which version to use for 
subsequent messages.  Simply implement a "Protocol" which understands 
that "version" message and changes its behavior accordingly.

For an example of how you might switch to a completely different 
protocol object, see

http://twistedmatrix.com/trac/browser/trunk/twisted/protocols/amp.py#L1524

The techniques involved are quite nuanced, however, and are probably not 
appropriate for someone just learning about Twisted.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20070404/bcaec2db/attachment.htm