[Twisted-Python] Deferred.setTimeout is deprecated, replacement ideas?

[Tim Allen]

On Wed, Jul 16, 2008 at 11:00:11AM +0200, Gabriel Rossetti wrote:
> Hello everyone,
>
> I was going to use Deferred.setTimeout until I saw that it was  
> deprecated. Does anyone know why? I found it handy and I find that  
> case-per-case replacements tend to be complicated. Does anyone have any  
> ideas on how to replace it?

This has (unsurprisingly) come up on the list many times before. Here's
a reasonable representative sample:

    http://thread.gmane.org/gmane.comp.python.twisted/8490

In short, the problem with .setTimeout is that it's a little bit like
shooting the messenger. Imagine you have a script that generates
a report from a long and complicated database query; the code flow might
run something like this:

 - Script asks the database to run its query.
 - Database returns a deferred promising a result.
 - Script calls .setTimeout() on the deferred.
 - The timeout fires, the script gives up and logs an error.
 - Database continues processing, using up CPU and IO resources even
   though the script no longer cares about the result.
 - Database finishes, calls .callback() on the deferred and winds up
   with an AlreadyCalled exception.

The preferred situation is something more like this:

 - Script asks the database to run its query with a particular timeout.
 - Database returns a deferred promising a result.
 - The timeout period elapses, the database realises it hasn't finished
   in time, and so shuts down its query processing and calls .errback()
   on the deferred.
 - Script receives the timeout notification, gives up and logs an error.

The difference is that the thing that returns the deferred is the only
thing that can possibly know how to cancel an operation-in-progress, and
therefore the only thing that can reasonably implement timeout
functionality - deferreds are just the wrong place to put that method.

> I was thinking of doing something like this, if data isn't present in  
> the data internal list:
>
> d = Deferred()
> dc = reactor.callLater(self.timeout, self.__timeoutCallback, d)

[line deleted as per follow-up mail]

> Then a tuple containing the Deferred ("d"), the DelayedCall ("dc") is  
> then put in the internal waiting list. If the timeout doesn't expire  
> before the data arrives, then the DelayedCall is canceled, if it does  
> expire then the tuple is removed from the queue (potentially slow) and  
> an exception is raised.
>
> Is there a better way of doing this? Any ideas?

What you suggest sounds pretty reasonable to me.

Tim Allen