[Twisted-Python] why can't a callback be called with a deferred?

[Chris Withers]

On 21/02/2019 07:32, Glyph wrote:
> 
> 
>> Yeah, this is part of carly, that I posted earlier. It stems from the 
>> need to get the results of method calls when you have no reference to 
>> the object being calls, or sometimes a result that's a deferred you 
>> need to wait on, particularly in a test, but have no way of doing so.
> 
> I just can't parse this sentence.

Sorry, some typos in there are not helping ;-(

> Breaking it down:
> 
>> you have no reference to the object being calls,
> 
> How do you have no reference to the object being called? Aren't you 
> calling it?

That test case I linked to shows one example, the original ones were 
from abstracting out JML's original post here:
https://jml.io/pages/how-to-disconnect-in-twisted-really.html

> In any case you either have a Deferred or you don't; if you do, then 
> it's clear you should wait on it,

Yes, but where to wait is the question here. I'd like to leave that 
choice to the user (see back in the thread for the details).

>> This is the test situation where I hit this issue:
>> https://github.com/cjw296/carly/blob/master/tests/test_untracked_deferred.py#L28-L35
>>
>> I'd originally wanted to have that read:
>>
>>    @inlineCallbacks
>>    def test1(self):
>>        ...
>>        result = yield pita.asyncMethod.called()
>>        with ShouldRaise(Exception(1)):
>>            yield result
>>
>> Now, which I'm actually happier with the end result here, I think the 
>> above it legit, if unusual, and that assert trips it up.
> 
> Some type annotations might make it a bit clearer what the two states 
> here are :).  

If those annotations were there, what would they look like? (I did link 
to the whole test file, which has all the detail I think you could need, 
it's pretty self contained and not that long)

> As it is, it looks to me you want a Deferred to come *out* 
> of a 'yield', which should definitely never happen. 

I disagree, in this specific case.

> If this assert were 
> to be removed, it would be done in such a way that would implicitly wait 
> for the Deferred in question to fire: you should /never/ receive a 
> Deferred as an argument to a function, or as the result of an 
> (inlineCallbacks) 'yield' or (async def) 'await'.  It breaks the whole 
> model of what 'awaiting' means.

Well, I'd agree 90% of the time, but test_untracked_deferred.py is where 
that's no always true.

Anyway, I'm happy with the API I have now, and it neatly works around 
that assert, so ¯\_(ツ)_/¯.

Chris