[Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Kevin Conway]

> What are you referring to as a "coroutine" here?  A generator?

Generator, yes, but used as a coroutine that both emits values and receives
them. The introduction of dedicated coroutines in Python 3.5 is something I
haven't updated my personal language for very well. Generator-coroutine is
the term used in there Python docs, IIRC.

> And exponential growth, you say?

Well... That may not be perfectly accurate and I shouldn't have used that
term. It grew large enough, fast enough that the same level of nesting was
measured in microseconds for the control and and over an hour for the
wrapper so I cancelled the test. Let me put together a reproducible
experiment and let others decide the growth rate.

> Where is this implemented?

Unfortunately, our production use of the lib is not something we've open
sourced. Similar to the above, I'll make available a reproducible
experiment so you can draw your own conclusion.

> Optimizing for CPython looks like adding weird implementation-specific
hacks that might start working or be backwards in the next version

This went into the rationale behind making most of the components
pluggable. The choice of optimizing for cPython is internal to us and we
chose to document it for others. It's not required and not the default.

As far as the possibility of integrating this feature into Twisted, let me
chat with my cohorts over here. Well come back with some benchmark code and
a proposal of changes for further discussion. I'll probably start a new
thread to focus on the context bits.

On Wed, Oct 19, 2016, 16:56 Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:


On Oct 19, 2016, at 2:47 PM, Itamar Turner-Trauring <itamar at itamarst.org>
wrote:

On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:

Well... I had a test that went through synchronous Deferred path. And yeah,
it was easier to write than async test. But it failed to catch a bug that
was only in async case. So the problem I see is that supporting both in
Deferred means you need twice the number of tests each time you use
Deferreds.


Er, that was unclear. I had a bug that wasn't caught by tests because it
passed with sync Deferred and failed with async Deferred callback, and I
didn't have tests for latter.


To be clear: I do see this as a *downside* to Deferred's architecture; it's
a tradeoff.  I see the loose coupling with the event loop as a worthwhile
upside.

However, it's totally possible to write the async Deferred callback case as
well, just by having the test fire the Deferred after returning to the test
body instead of firing it before :-).

-glyph
_______________________________________________
Twisted-Python mailing list
Twisted-Python at twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20161020/e5708d31/attachment-0002.html>