[Twisted-Python] xmlrpc - Possable failure to understand async concept

[Drew Smathers]

On Fri, Apr 24, 2009 at 2:28 AM, Nathan <nathan.stocks at gmail.com> wrote:
> On Thu, Apr 23, 2009 at 9:34 PM, Tim Allen <tim at commsecure.com.au> wrote:
>> Tim Hughes <thughes at thegoldfish.org> wrote:
>>> Am I completly missing the point here or is there something incorrect with
>>> my code.
>>
>> Yes, it seems you have missed the point somewhere.
>>
>>>     def blocking_method(self, duration=5):
>>>         """block the instance for a specified duration"""
>>>         started = time.asctime()
>>>         time.sleep(duration)
>>>         data = "I started at %s and i slept %d seconds then woke at %s" %
>>> (started, duration, time.asctime())
>>>         return data
>>
>>>     def blocking_method_fixed(self, duration=5):
>>>         d = Deferred()
>>>         d.callback(self.blocking_method(duration))
>>>         return d
>>
>> Your blocking_method_fixed() wraps the result of blocking_method() in a
>> Deferred, but it doesn't actually do anything to prevent
>> blocking_method() from blocking. Deferreds have no magical ability to
>> prevent things from blocking on their own, they are just a tool you can
>> use to handle some of the control-flow issues that arise when writing
>> non-blocking code.
>>
>> For example, here's a method that does more or less what
>> blocking_method() does, but in a non-blocking manner:
>>
>>    def non_blocking_method(self, duration=5):
>>        started = time.asctime()
>>        d = Deferred()
>>        reactor.callLater(duration, d.callback, None)
>>
>>        def do_stuff_after_timeout(result):
>>            # Here, 'result' will contain the None we passed to
>>            # callLater
>>            data = "I started at %s and I slept %d seconds then woke " \
>>                   "at %s" % (started, duration, time.asctime())
>>            return data
>>        d.addCallback(do_stuff_after_timeout)
>>        return d
>>
>> See how the method does some initial preparation, then schedules a
>> callback to be run after the long-running operation has completed.
>>
>> If you have a long-running network call to do instead of a simple
>> sleep, there's probably a Twisted API or addon that will perform the
>> call and give you back a Deferred, rather than blocking until it has an
>> answer. If you're trying to use a third-party client library that
>> doesn't offer a non-blocking API, about the only thing you can do is
>> call that API in a thread, using the reactor.callInThread() method (but
>> note that Twisted is not thread-safe, so the thing you call in a thread
>> can't use any of Twisted's functionality except via
>> reactor.callFromThread())
>
> The above email makes a great FAQ item, IMO.  I know there's already
> docs covering this, but this issue comes up soooo many times (it took
> me a month to wrap my head around it initially) that it wouldn't hurt
> to hit the issue from many directions in many styles.
>

It is one of the FAQ items:

http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoIuseDeferredstomakemyblockingcodenon-blocking

Apparently there's also been some stitchwork done to commemorate this.

-Drew