[Twisted-Python] Stackless Python Examples Revisited Re: InlineCallbacks
andrewfr_ice at yahoo.com
Fri Jul 18 13:34:34 EDT 2008
>Date: Sat, 19 Apr 2008 11:47:38 -0700 (PDT)
>From: Andrew Francis <andrewfr_ice at yahoo.com>
>Subject: [Twisted-Python] Stackless Python Examples Re: InlineCallback
> Friendly ?
>To: twisted-python at twistedmatrix.com
>Message-ID: <882470.36243.qm at web34201.mail.mud.yahoo.com>
>Content-Type: text/plain; charset=iso-8859-1
--- Andrew Francis <andrewfr_ice at yahoo.com> wrote:
>Based on Jean-Paul's explanation, here are some code
>snippets showing how to handle other protocols with
>Stackless Python and Twisted. This is a continuation
>of the technique I used in "Adventures" to prevent the
>reactor from deadlocking.
>Currently I am rethinking the techniques - I believe
>there is a way to reduce spawning tasklets - not that
>tasklets are all that expensive....
I have been experimenting with a new technique for tasklets to interact with Twisted protocol instances (the following is essentially what I was pestering the Twisted Team about at PyCon 2008. Sorry guys)
I have been exploring situations where a tasklet rendezvoused with Twisted (as opposed to a Twisted spawning a tasklet to fulfill a request). A lot of WS-BPEL constructs take this form....
In the old system, I ran the server protocol instance in its own tasklet. This was to prevent the reactor from blocking and preventing other tasklets from issuing Twisted calls. In turn, the tasklet and protocol instance/tasklet communicated via a channel.
With this technique, I do not create an additional tasklet. Rather I pass back a deferred whose callback points to the reminder of the protocol instance's code.
Although I have not finished testing, this technique seems to be about 20% faster. Again, implementing this is fairly straightforward. Here is a sketch.
def __httpWrite__(self, status, message):
create a deferred
myDeferred = defer.Deferred()
we need a way for the protocol instance to tell tasklets that a
connect has been made. Easiest way, just hand the protocol
instance a reference
status, message = \
if status != http.OK:
myDeferred = None
if things went well, return a deferred
This is a Twisted way to circumvent blocking and waiting
def processReply(self, reply):
meanwhile, in the tasklet
deferred = someFunction(somePath)
# send a message
Although PyCon 2008 is over, I still plan to submit a paper that will better explain the various techniques in the "Adventures" talk. I am also writing simpler code to better illustrate the techniques and combine them with other tried-and-true techniques (i.e., Armstrong's blockOn).
More information about the Twisted-Python