[Twisted-Python] Stackless Python Examples Revisited Re: InlineCallbacks

Andrew Francis andrewfr_ice at yahoo.com
Fri Jul 18 13:34:34 EDT 2008

Hi Folks:

>Message: 4
>Date: Sat, 19 Apr 2008 11:47:38 -0700 (PDT)
>From: Andrew Francis <andrewfr_ice at yahoo.com>
>Subject: [Twisted-Python] Stackless Python Examples Re: InlineCallback
>    Friendly ?
>To: twisted-python at twistedmatrix.com
>Message-ID: <882470.36243.qm at web34201.mail.mud.yahoo.com>
>Content-Type: text/plain; charset=iso-8859-1

--- Andrew Francis <andrewfr_ice at yahoo.com> wrote:

>Based on Jean-Paul's explanation, here are some code
>snippets showing how to handle other protocols with
>Stackless Python and Twisted. This is a continuation
>of the technique I used in "Adventures" to prevent the
>reactor from deadlocking.

>Currently I am rethinking the techniques - I believe
>there is a way to reduce spawning tasklets - not that
>tasklets are all that expensive....

I have been experimenting with a new technique for tasklets to interact with Twisted protocol instances (the following is essentially what I was pestering the Twisted Team about at PyCon 2008. Sorry guys)

I have been exploring situations where a tasklet rendezvoused with Twisted (as opposed to a Twisted spawning a tasklet to fulfill a request). A lot of WS-BPEL constructs take this form....

In the old system, I ran the server protocol instance in its own tasklet. This was to prevent the reactor from blocking and preventing other tasklets from issuing Twisted calls. In turn, the tasklet and protocol instance/tasklet communicated via a channel. 

With this technique, I do not create an additional tasklet. Rather I pass back a deferred whose callback points to the reminder of the protocol instance's code. 

Although I have not finished testing, this technique seems to be about 20% faster. Again, implementing this is fairly straightforward. Here is a sketch.

class MyRequestHandler(http.Request):

    def __httpWrite__(self, status, message):

    def process(self):

        create a deferred
        myDeferred = defer.Deferred()

        we need a way for the protocol instance to tell tasklets that a
        connect has been made. Easiest way, just hand the protocol 
        instance a reference
        status, message = \
        MyRequestHandler.someReferenceToProgramme(self.path, \
                                                  self.content.read(), \
        if status != http.OK:
           print "ERROR!"
           self.__httpWrite__(status, message)
           myDeferred = None

        if things went well, return a deferred 
        This is a Twisted way to circumvent blocking and waiting 
        for information
        return myDeferred

    the continuation 
    def processReply(self, reply):
        self.__httpWrite__(200, reply[0])

meanwhile, in the tasklet

def tasklet(...):
    deferred = someFunction(somePath)
    # send a message 
Although PyCon 2008 is over, I still plan to submit a paper that will better explain the various techniques in the "Adventures" talk. I am also writing simpler code to better illustrate the techniques and combine them with other tried-and-true techniques (i.e., Armstrong's blockOn). 



More information about the Twisted-Python mailing list