<div class="gmail_quote">On Fri, Jul 31, 2009 at 11:35 PM, Edward Z. Yang <span dir="ltr"><<a href="mailto:ezyang@mit.edu">ezyang@mit.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Excerpts from Itamar Shtull-Trauring's message of Fri Jul 31 22:26:29 -0400 2009:<br>
<div class="im">> The problem with this is that it perpetuates the misunderstanding the<br>
> Deferreds *make* things asynchronous, even with the intro that says<br>
> otherwise. I think it's better to assume already asynchronous code,<br>
> handling the transition from synchronous to async in an intro event loop<br>
> howto.<br>
<br>
</div>Either way, the function that the first segment of the new docs do belong<br>
somewhere. The documentation that traditionally served this purpose<br>
has been removed.</blockquote><div><br>I'm actually inclined to agree. Understanding what Deferreds are <i>for</i> sort of implies an understanding of why one would want to have a list of callbacks in the first place, which implies a need to understand asynchronous programming.<br>
<br>Realistically, I don't think there are a lot of people out there who have managed to do anything at all useful with Twisted without undersanding Deferreds.<br><br>I think it might be useful to lead into this section with a specific example problem though, rather than a very vague general template for the category of problems. Once you've demonstrated a specific problem, you can always expand the general category by briefly listing some other problems which are similar.<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">As for perpetuating the misunderstanding of Deferreds making things<br>
asynchronous, I completely agree! However, I think this is something<br>
that can be fixed by spelling out the distinction between "writing<br>
asynchronous code" and "interacting with asynchronous code", and not<br>
just omitting the important paradigm shift that comes with sync->async.</blockquote><div><br>By the time you have to spell out that distinction, I think it's too late. By the time the user is dealing with a Deferred, they should have a pretty good idea that it's just a list of functions that will be called at some point in the future; if you have to say "here's this magic thing, oh, by the way, it turns out it's just a list of functions" they're still going to think something's magic about it.<br>
<br>If you want an example that doesn't involve any reactor integration, you might try showing a program that polls a directory for a file being added. If you want to simultaneously wait for a file named "1" and a file named "2", you need to use callbacks, otherwise "waiting" for 2 will prevent the code for 1 from running immediately if 1 shows up first.<br>
<br>I'm not really sure this can be done in a simple enough way to be useful, so take it with a grain of thought.<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
> A better comparative exposition might be with normal callbacks, e.g.:<br></blockquote><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
><br>
> "def foo(x, gotResultCallback): pass" vs. "def foo(x): # return<br>
> Deferred".<br>
><br>
> At the very least having that async but callbacky version in the middle<br>
> helps understanding.<br>
<br>
</div>I briefly gloss on this, but I agree that this is an important point<br>
that could be further expanded. We could have implemented asynchronous<br>
mechanisms using normal callbacks, but we decided to use Deferreds instead.<br>
</blockquote><div><br>Definitely agreed here. For those users who are used to "normal" callbacks (if there is such a thing), this explanation would be helpful to explain why you need them rather than just passing functions around. For users who aren't even familiar with passing functions around, this would fill an important intermediate gap.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">It's not clear to me if the common case of confusion of Deferreds occurs<br>
in people who "know callbacks" but "don't know Deferreds." As an incoming<br>
developer who was familiar with asynchronous programming, my primary problem<br>
was the ill-defined behavior of callback chains (which I resolved by<br>
hunkering down and reading the source code) rather than any fundamental<br>
misunderstanding of what Deferreds were supposed to do.<br>
<div class="im"></div></blockquote><div><br>I think we need an expanded version of the "visual explanation" here: <<a href="http://twistedmatrix.com/projects/core/documentation/howto/defer.html">http://twistedmatrix.com/projects/core/documentation/howto/defer.html</a>> (one that uses complete sentences and more than one diagram), as well as an expanded analogy to try:except: blocks in 'synchronous' python. <br>
<br>I object to the classification of the behavior of callback chains as "ill-defined". It might have been poorly-explained, but that's very different :).<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">> It also omits half the story: how you *create* Deferreds. There should<br>
> be a section on that as well.<br>
<br>
</div>I agree. In fact, it might be worth making the document a little longer<br>
to address this point, because I realize now that even if you're not<br>
writing asynchronous code, you'll often need to baton Deferreds to<br>
make the execution flow work the way you want them to.<br></blockquote><div> <br>+1. Dealing with Deferred creation is a good way to introduce users to the non-magicness of Deferreds. It's less common to create one than to consume one, but understanding how to create them should make it easier to understand how to consume them.<br>
<br></div></div>