[Twisted-Python] Sequential use of asynchronous calls

Maarten ter Huurne maarten at treewalker.org
Sun May 27 00:20:30 MDT 2007


On Sunday 27 May 2007, Ben Artin wrote:
> > Since it is used as a decorator, the @sequential line will be
> > written by the
> > same person who wrote the function itself. If the author of the
> > function
> > knows it is not a generator, why would he apply @sequential (or
> > @inlineCallbacks) to it?
>
> One reason is that if the function returns None, then if you require
> it to be a generator, you have to add a gratuitous "yield None" just
> to shut up the piece of code that requires a generator.

Maybe it's more clear to call it @inlineCallbacks, since that is the name 
under which it is available; @sequential was just me re-inventing it.

I agree that adding "yield None" is an ugly fix, but why not just remove the 
@inlineCallbacks decoration from such a function instead? I still don't 
understand what the point is of decorating a non-generator function with 
@inlineCallbacks. 

> The other main case where I've run into this is when you have a
> protocol that expects some method to behave according to @sequential,
> but a particular implementation of that protocol doesn't need to do
> more than immediately return.

The external protocol of @inlineCallbacks is just Deferred. There are a lot of 
ways to convert something to a Deferred already, such as maybeDeferred, 
execute, succeed, fail (all in the "defer" module). Using @inlineCallbacks 
for that purpose seems a bit overcomplicated.

There is never a requirement for a function to be decorated with 
@inlineCallbacks, all the outside world sees is a Deferred. @inlineCallbacks 
is just an implementation technique to avoid having to split sequential 
asynchronous code over multiple functions.

> For example: 
>
> class TakesALongTime():
> 	@sequential
> 	def doSomething(self):
> 		yield doPart1()
> 		yield doPart2()
> 		yield someResult

This doesn't do what you might expect...

Let's assume doPart1 and doPart2 return Deferreds, named d1 and d2. When 
doSomething is called, inlineCallbacks will run doPart1, register itself on 
d1 and return its own Deferred (dr) to the caller of doSomething. When the 
reactor does the callback on d1, doPart2 will be called and inlineCallbacks 
will register itself on d2. Finally, when the callback of d2 is called, 
someResult will be discarded by inlineCallbacks and the callback of dr will 
be called with result None.

To do what you probably intended, the code would look like this:

class TakesALongTime():
	@defer.inlineCallbacks
	def doSomething(self):
		yield doPart1()
		yield doPart2()
		defer.returnValue(someResult)

Also note that neither of these examples blocks. Instead, the decorated method 
will typically return a Deferred (dr) very soon. However, it could take a 
long time before dr's callback (or errback) is called. During that time, the 
caller of doSomething will probably register itself on dr and end, to pass 
control to the reactor.

> Now, if you use "return" in the first class, you get a syntax error
> right away, because a non-empty return in a generator is a syntax
> error. On the other hand, if you use "return" in the second class,
> you don't get a syntax error, but your implementation of @sequential
> would produce a runtime error. I understand why it's easy to produce
> that runtime error, but I don't see any benefit to it -- it doesn't
> really save the users of @sequential any effort.

Well, if you consider using @inlineCallbacks on a non-generator an error, 
checking for this error as early as possible does save the user effort: you 
will see the error as soon as your module is imported, instead of having to 
trigger the function in question.

If you consider using @inlineCallbacks on a non-generator a valid scenario, 
there should not be a runtime error issued, neither at "decoration time" nor 
at "invocation time".

Note that the current implementation of @inlineCallbacks will raise an error 
at "invocation time", specifically when "g.send" or "g.throw" is called. I'm 
proposing to change that so an error will be raised by the decorator instead 
("decoration time", typically when the module is imported).

> > But should it be given any interpretation at all? Unless there is a
> > real use
> > case for it, I think it's better to consider it an error.
>
> I would agree with you if I thought there was any possibility that
> interpreting a plain function as a generator with a single yield
> could yield unintended results, but I have yet to find such a case.

Should something be given an interpretation because there are no unintended 
results, or because there are intended results?

Bye,
		Maarten
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20070527/ef81bdaf/attachment.sig>


More information about the Twisted-Python mailing list