[Twisted-Python] Inline callbacks: generating the same Deferred at multiple places
Pierre Jaury
pierre at jaury.eu
Sat Feb 23 04:23:56 MST 2013
On 02/22/2013 09:17 PM, Glyph wrote:
>
> On Feb 22, 2013, at 8:30 AM, Christopher Armstrong
> <radix at twistedmatrix.com <mailto:radix at twistedmatrix.com>> wrote:
>
>> I think it's a reasonable change to make, and I don't foresee any
>> problems with it, so I think it's fine to submit a bug about it. But I
>> do question the architecture that needs to make use of it. I would
>> probably avoid scenarios like that in my own code.
>
> I disagree; the behavior of result consumption is intentional - although
> it could be better documented. Changing it would very definitely be
> incompatible (<http://twistedmatrix.com/trac/wiki/CompatibilityPolicy>);
> this is possible, of course, if the deprecation/migration is worth it,
> but the behavior being requested here would be worse in a number of ways.
Well, some criticism, now I am listening to you.
> If we re-populated the result, every failed Deferred yielded by an
> inlineCallbacks function would log its traceback twice: once when the
> unhandled exception propagated out of the inlineCallbacks function
> causing its Deferred to fail, and once when the unhandled exception
> propagated from the yielded Deferred itself, since nothing would have
> consumed it when that Deferred would be GC'd.
Well, this simply means that the change is not that simple to make and
would probably imply some deeper modifications. I can see how this kills
any hope to get the change merged into anything stable anytime soon however.
> Speaking of GC, similarly, any large objects in Deferred results
> processed by inlineCallbacks functions would live longer, and continue
> participating in any reference cycles they're part of, possibly causing
> memory leaks, or at least, longer collection times and less favorable
> memory usage behavior, especially in long-lived processes.
But.. I definitely agree with this one, which I did not foresee. I
therefore agree that any change in Deferred objects behavior should
never enforce references to live longer when not usually required.
> Basically, you can't treat a Deferred as an event broadcaster. It isn't
> one. It's a single-shot representation of an asynchronous result, which
> a single consumer can consume with its current value, possibly yielding
> a new value.
That I figured out already. Thanks for the reminder anyway, I was on the
path of loosing myself, there :)
> Any
> inlineCallbacks function which wants to express its intent more
> precisely can do this, instead:
>
> def fork(d):
> d2 = Deferred()
> def fire(x):
> d2.callback(x)
> return x
> d.addBoth(fire)
> return d2
>
> @inlineCallbacks
> def foo():
> result = yield fork(somethingAsync())
I will embed this one in my bundle of Twisted utility functions. Thanks!
> Maybe putting that function in Twisted (this is not the first time it's
> come up) would be a useful addition.
Agreed, then.
--
Pierre Jaury <pierre at jaury.eu>
Weblog - http://kaiyou.fr
GPG ID - E804FB60
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: </pipermail/twisted-python/attachments/20130223/98f27489/attachment.sig>
More information about the Twisted-Python
mailing list