<html><body>On 04:35 pm, daniel@keystonewood.com wrote:<br />>On Apr 3, 2007, at 8:42 PM, Itamar Shtull-Trauring wrote:<br />>>On Tue, 2007-04-03 at 17:07 -0400, Daniel Miller wrote:<br />>>>>twisted.internet.defer.DeferredLock and some of the related classes<br />>>>>are<br />>>>>what you ought to be using.<br />>>><br />>>>Unfortunately that only gets me half way there. DeferredLock.acquire<br />>>>() returns a deferred. How do I return the result of a deferred from<br />>>>a PB remote_xxx() function?<br />>><br />>>Just return the Deferred from the remote_xxx() function.<br />><br />>Thanks, I didn't know I could return a deferred from a PB remote_xxx () <br />>method. That detail doesn't seem to be documented in the  Perspective Broker <br />>documentation, which I have read quite a few  times.<br /><br />The PB documentation is not too great.  Perhaps this paper would be helpful to you, if you haven't seen it:<br /><br />http://www.lothar.com/tech/papers/PyCon-2003/pb-pycon/pb.html#auto7<br /><br />    """<br />    In addition, the remote method can itself return a Deferred instead of an<br />    actual return value. When that Deferreds fires, the data given to the<br />    callback will be serialized and returned to the original caller.<br />    """<br /><br />> Maybe this could be <br />>highlighted in the "Complete Example" [0]  section of the PB usage <br />>documentation? The examples use the  TwistedQuotes application, and the <br />>IQuoter.getQuote() method always  returns a string (at least I couldn't find <br />>any implementations that  return a deferred).<br /><br />Please feel free to write some patches for the documentation, or open a doc bug describing this issue in more detail.  It's definitely an under-documented feature of PB.<br /><br />>However, that would require <br />>rewriting most if not  all implementations of IQuoter to return deferred's <br />>and/or the code  that calls IQuoter.getQuote(), which demonstrates the viral <br />>nature of  twisted when used in conjunction with other libraries.<br /><br />I don't think that would really be the terrible burden that you suggest, considering the relatively small amount of tutorial documentation that implements or calls IQuoter.  One could also propose a separate interface, IDeferredQuoter, to make the distinction clearer.<br /><br />>So anyway, I rewrote my server-side library to do it the twisted way  and <br />>return deferred's instead trying rig up some way of waiting for  them. I <br />>still think it would be super useful to be able to pseudo- block on a <br />>deferred (i.e. allow the reactor to process other events  while waiting for <br />>the deferred). It is very annoying to have to  rewrite many layers of code <br />>when twisted is introduced into a  program. I did find gthreadless.py, and <br />>maybe that would do it.  Unfortunately discussion on that seems to have been <br />>dropped some time  ago...<br /><br />I'm afraid that the feature you want doesn't make any sense and is, in a broad sense, impossible.  There are some things like it which might be possible - for example, http://twistedmatrix.com/trac/ticket/2545 - but the reactor is not reentrant and in some sense could not be made reentrant.<br /><br />Consider this innocuous looking block of code:<br /><br />    from twisted.internet.protocol import Protocol<br />    from make_believe import magicallyBlockOn<br /><br />    class MagicalProtocol(Protocol):<br />        def dataReceived(self, data):<br />            commands = (self.buf + data).split()<br />            self.buf = commands[-1]<br />            for command in commands[:-1]:<br />                if command == 'QUIT':<br />                    self.transport.loseConnection()<br />                    return<br />                else:<br />                    # Deferreds are hard, let's go shopping<br />                    page = magicallyBlockOn(getPage("http://example.com/%s" %<br />                                                    (command,)))<br />                    self.transport.write("SIZE:"+len(page))<br /><br />If you were using Deferreds to track the result of the 'getPage' operation, you could cancel the callbacks that write to the transport in connectionLost.  However, with magical blocking, one dataReceived method might be interleaved with another.  That means that every time through the loop, you have to check to see if the transport has already been disconnected - the code as presented here is buggy and will spuriously fail depending on the order of the connection being lost and the remote page being retrieved.<br /><br />In this example I've been careful to accumulate all the buffer-management and parsing logic at the top of the method, before any potential re-entrancy can happen, but other code (most everything in Twisted's existing protocol implementations, not to mention just about all application code) would not be so lucky.<br /><br />It might be quite feasible to implement a microthreaded runtime environment that lived on _top_ of Twisted and explicitly accounted for issues like these, but that would not really be substantially different than 2.5+inlineCallbacks.<br /><br />>For the record, I've included updated versions of the previously  posted <br />>code below. I'd be happy if someone pointed out if I'm doing  anything wrong <br />>(with respect to twisted) in this code.<br /><br />Nothing immediately jumps out at me.  I've had to write similar code in the past, though, and when I've had to do that, an explicit state machine for the state of the subprocess (or whatever asynchronous resource must be acquired) has been easier to deal with than a lock-oriented approach to it.<br /><br /></body></html>