[Twisted-Python] If the errbacks of a canceled Deferred are called with error other than CancelledError, is this acceptable?

[Terry Jones]

Hi again Tom

> So, it seems to me that you want to represent your jobs as
> more than simply a deferred. At the very least, you need to record what
> the jobs is, so that it can be restarted later, if necessary.

Yes. The job class contained things like the function to call, its args,
timing information etc., as well a deferred that would fire with the result.

> Deferreds are a way of abstracting data and/or control flow in
> asynchronous way. I'm not sure that trying to think of a deferred as a
> job is a useful thing to do

I wasn't doing that.

The problem that I was originally trying to address by getting 'cancel'
into deferreds is that the control flow you mention gets totally stuck if a
deferred never fires for some reason (this was happening to me in talking
to the Twitter API). E.g., in this case, kicking off a new job relies on an
earlier job finishing (which includes a deferred firing). An app writer
might want to timeout a deferred for some reason. Or the app may need to
shut down. If there's even one deferred that doesn't fire, the whole system
essentially hangs. The idea was simply to increase the amount of control an
app had over the flow of control.

This goes against the thinking that only the code that creates a deferred
can fire it. I don't see anything wrong with that - it just increases the
options that the code which obtains the deferred has. As API writers, we're
used to being in control: we don't think it's a good idea if we create and
return a deferred in a method and the code we return the deferred to then
fires it itself. But as app writers consuming deferreds from these APIs, we
want more control (I do, anyway).  It's possible to have both, which was
what I was trying to provide for. The ControllableDeferred2013 class I
posted last night shows one way. Adding a value arg to cancel() would build
some of that flexibility into Twisted itself.

Hopefully that helps makes things clearer.

Terry


On Fri, Jun 21, 2013 at 7:18 AM, Tom Prince <tom.prince at ualberta.net> wrote:

> Terry Jones <terry at jon.es> writes:
> > But, I also built some queuing things where I would have liked extra
> > information. For example, an app is processing jobs on behalf of a user,
> > [...]
>
> Thinking about this for a little bit, it seems like there is quite a bit
> more state to your jobs, that simply an eventual result (i.e. a
> deferred). So, it seems to me that you want to represent your jobs as
> more than simply a deferred. At the very least, you need to record what
> the jobs is, so that it can be restarted later, if necessary.
>
> Deferreds are a way of abstracting data and/or control flow in
> asynchronous way. I'm not sure that trying to think of a deferred as a
> job is a useful thing to do; most any example of a job that I can think
> of likely involves multiple deferred objects (often chained by returning
> them from callbcks).
>
>   Tom
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130621/fed2cdca/attachment.html>