[Twisted-Python] Sequential use of asynchronous calls

Ben Artin ben at artins.org
Sun May 27 04:35:24 EDT 2007


> Maybe it's more clear to call it @inlineCallbacks, since that is  
> the name
> under which it is available; @sequential was just me re-inventing it.

That's OK. I reinvented it and called it something else myself :-)

> I agree that adding "yield None" is an ugly fix, but why not just  
> remove the
> @inlineCallbacks decoration from such a function instead? I still  
> don't
> understand what the point is of decorating a non-generator function  
> with
> @inlineCallbacks.

There is no theoretical reason. Of course I can and remove the  
decorator, but that is not how my brain works when write the code,  
and I suspect I am not alone in this. This is how I usually write my  
code:

.oO(I am writing a function)
.oO(The function's interface is that it returns a deferred, so write  
@inlineCallbacks)

@inlineCallbacks
def functionName(args):

.oO(Now, what's the body going to be like?)
type type type type
twenty lines later:
.oO(Done!)

then I run the code, and find out that I have to either add a yield  
None or remove the decorator because as it happens, there's no yield  
or return in those twenty lines of code.

Now, the reason that my brain works this way is that I consider the  
fact that the function returns a deferred a part of its interface,  
and if I am using Python 2.5 (which I almost exclusively am), then  
@inlineCallbacks is by far the cleanest way to write Deferred-using  
code. It turns my control flow right-side out and it allows me to  
stop focusing on how I pass control around using deferreds and start  
focusing on the problem I am trying to solve.

To that end, requiring me to either add noise to my function's  
implementation (in the form of "yield None") or to tweak the  
function's header because of a happenstance of its implementation is  
just distracting me away from writing my code and pulling my  
attention to something irrelevant.

Again, my argument is that the distinction between:

@inlineCallbacks
def functionName(args):
	...some code...
	yield None

and

@inlineCallbacks
def functionName(args):
	...some code...
	... without a return statement...

is completely irrelevant and serves no purpose except to distract me  
from my code. If I could find *any* case in which treating the latter  
case exactly the same as the former case is not what was intended,  
then I might be willing to concede that there is value in forcing the  
user to clarify what they meant; however I do not see that case.

>> For example:
>>
>> class TakesALongTime():
>> 	@sequential
>> 	def doSomething(self):
>> 		yield doPart1()
>> 		yield doPart2()
>> 		yield someResult
>
> This doesn't do what you might expect...
>
> Let's assume doPart1 and doPart2 return Deferreds, named d1 and d2.  
> When
> doSomething is called, inlineCallbacks will run doPart1, register  
> itself on
> d1 and return its own Deferred (dr) to the caller of doSomething.  
> When the
> reactor does the callback on d1, doPart2 will be called and  
> inlineCallbacks
> will register itself on d2. Finally, when the callback of d2 is  
> called,
> someResult will be discarded by inlineCallbacks and the callback of  
> dr will
> be called with result None.
>
> To do what you probably intended, the code would look like this:
>
> class TakesALongTime():
> 	@defer.inlineCallbacks
> 	def doSomething(self):
> 		yield doPart1()
> 		yield doPart2()
> 		defer.returnValue(someResult)

Actually this in itself is IMO a bug in inlineCallbacks. The only  
reason why inlineCallbacks is implemented this way is that prior to  
Python 2.5 it was impossible to implement coroutines in terms of just  
the yield statement, and therefore some extra magic was needed to  
achieve the desired effect. If you read the coroutines PEP (<http:// 
www.python.org/dev/peps/pep-0342/>), you'll see that the syntax I  
used above is the syntax suggested by the PEP (and, not accidentally,  
also the syntax used by my own version of @inlineCallbacks)

> Note that the current implementation of @inlineCallbacks will raise  
> an error
> at "invocation time", specifically when "g.send" or "g.throw" is  
> called. I'm
> proposing to change that so an error will be raised by the  
> decorator instead
> ("decoration time", typically when the module is imported).

I understand. I am proposing that it's not an error, because there is  
no case in which it's ambiguous what the intent was.

Another argument I can make in favor of this idea is that:

@inlineCallbacks
def doSomething():
	raise SomeException()

works the way you'd think it would upon reading the code and therefore

@inlineCallbacks
def doSomething():
	return someValue

should also work the way you'd think it would upon reading the code.

> Should something be given an interpretation because there are no  
> unintended
> results, or because there are intended results?

I believe that anyone who writes code that returns a plain value out  
of a function decorated with @inlineCallbacks has a clear intent, and  
they merely did not fuss over the syntax of @inlineCallback. I  
further believe that is in no way beneficial to force people to fuss  
over that syntax, and in no way detrimental to allow them to just  
return a plain value (because there is no ambiguity). Therefore, it  
should be done.

Ben

--

<http://artins.org/ben>

"Computers! All they ever think of is hex!"






More information about the Twisted-Python mailing list