<div dir="ltr">Hi,<br><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 5, 2013 at 9:24 AM, Maciej Wasilak <span dir="ltr"><<a href="mailto:wasilak@gmail.com" target="_blank">wasilak@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thank you for your answer. I'm explicitly interested in the following combination:<br><br>> .addCallback(cb).addErrback(eb)<br>
<br>If I understand correctly errback "eb" catches errors from both callback "cb", and from agent.request (agent.request errors pass through default empty errback that re-raises them).<br>
</div></blockquote><div><br></div><div>Yes, that's exactly what it does :) The difference being that with .addCallbacks(cb, eb), errors in cb would not be caught by eb.<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="im"><p dir="ltr">> Also, if you want to do scraping with Twisted, consider looking at Scrapy, a fully-featured web scraper that uses Twisted internally.<br>
</p></div><p>I'm working on CoAP protocol which runs on top of UDP. My actual code is request processing (but the problem is the same as in my previous post):</p><p>_________________________________________________<br>
</p>
<div><span style="font-family:courier new,monospace"><span>def</span> <span>processRequest</span><span>(</span><span>self</span><span>,</span> <span>request</span><span>):</span></span></div>
<span style="font-family:courier new,monospace"> (...)<br></span><div><span style="font-family:courier new,monospace"> <span>d</span> <span>=</span> <span>defer</span><span>.</span><span>succeed</span><span>(</span><span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>d</span><span>.</span><span>addCallback</span><span>(</span><span>self</span><span>.</span><span>processBlock1</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>d</span><span>.</span><span>addCallback</span><span>(</span><span>self</span><span>.</span><span>dispatchRequest</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"><span>def</span> <span>dispatchRequest</span><span>(</span><span>self</span><span>,</span> <span>request</span><span>):</span></span></div>
<div><span style="font-family:courier new,monospace"><span></span></span></div><span style="font-family:courier new,monospace"> (...)<br></span><div><span style="font-family:courier new,monospace"> <span>resource</span> <span>=</span> <span>self</span><span>.</span><span>endpoint</span><span>.</span><span>getResourceFor</span><span>(</span><span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>if</span> <span>resource</span> <span>is</span> <span>None</span><span>:</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>response</span> <span>=</span> <span>Message</span><span>(</span><span>code</span><span>=</span><span>NOT_FOUND</span><span>,</span> <span>payload</span><span>=</span><span>'Resource not found!'</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>self</span><span>.</span><span>respond</span><span>(</span><span>response</span><span>,</span> <span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>return</span></span></div><div><span style="font-family:courier new,monospace"> <span>try</span><span>:</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>d</span> <span>=</span> <span>resource</span><span>.</span><span>render</span><span>(</span><span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>except</span> <span>iot</span><span>.</span><span>error</span><span>.</span><span>UnallowedMethod</span><span>:</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>response</span> <span>=</span> <span>Message</span><span>(</span><span>code</span><span>=</span><span>METHOD_NOT_ALLOWED</span><span>,</span> <span>payload</span><span>=</span><span>'Method not allowed!'</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>self</span><span>.</span><span>respond</span><span>(</span><span>response</span><span>,</span> <span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>return</span></span></div><div><span style="font-family:courier new,monospace"> <span>except</span> <span>iot</span><span>.</span><span>error</span><span>.</span><span>UnsupportedMethod</span><span>:</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>response</span> <span>=</span> <span>Message</span><span>(</span><span>code</span><span>=</span><span>NOT_IMPLEMENTED</span><span>,</span> <span>payload</span><span>=</span><span>'Method not implemented!'</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>self</span><span>.</span><span>respond</span><span>(</span><span>response</span><span>,</span> <span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>return</span></span></div><div><span style="font-family:courier new,monospace"> <span>delayed_ack</span> <span>=</span> <span>reactor</span><span>.</span><span>callLater</span><span>(</span><span>EMPTY_ACK_DELAY</span><span>,</span> <span>self</span><span>.</span><span>sendEmptyAck</span><span>,</span> <span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>d</span><span>.</span><span>addCallback</span><span>(</span><span>self</span><span>.</span><span>respond</span><span>,</span> <span>request</span><span>,</span> <span>delayed_ack</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>return</span> <span>d</span></span></div><p dir="ltr">__________________________________________________<br></p><p>I would like to rewrite it to:</p>
<p>__________________________________________________<br></p><div><span style="font-family:courier new,monospace"><span>def</span> <span>processRequest</span><span>(</span><span>self</span><span>,</span> <span>request</span><span>):</span></span></div>
<span style="font-family:courier new,monospace"> (...)<br></span><div><span style="font-family:courier new,monospace"> <span>d</span> <span>=</span> <span>defer</span><span>.</span><span>succeed</span><span>(</span><span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>d</span><span>.</span><span>addCallback</span><span>(</span><span>self</span><span>.</span><span>processBlock1</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>d</span><span>.</span><span>addCallback</span><span>(</span><span>self</span><span>.</span><span>dispatchRequest</span><span>).addErrback(self.handleRequestError)<br>
<br></span></span></div><div><span style="font-family:courier new,monospace"><span>def</span> <span>dispatchRequest</span><span>(</span><span>self</span><span>,</span> <span>request</span><span>):</span></span></div>
<div><span style="font-family:courier new,monospace"><span></span></span></div><span style="font-family:courier new,monospace"> (...)<br></span><div><span style="font-family:courier new,monospace"> <span>resource</span> <span>=</span> <span>self</span><span>.</span><span>endpoint</span><span>.</span><span>getResourceFor</span><span>(</span><span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>if</span> <span>resource</span> <span>is</span> <span>None</span><span>:</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>raise NoResource()<br></span></span></div><div><span style="font-family:courier new,monospace"> <span>try</span><span>:</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>d</span> <span>=</span> <span>resource</span><span>.</span><span>render</span><span>(</span><span>request</span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>except</span> <span>iot</span><span>.</span><span>error</span><span>.</span><span>UnallowedMethod</span><span>: #Explicit re-rise for this example only<br>
</span></span></div><div><span style="font-family:courier new,monospace"> <span>raise </span><span><span></span><span>UnallowedMethod</span><span></span>() <br>
</span></span></div><div><span style="font-family:courier new,monospace"> <span>except</span> <span>iot</span><span>.</span><span>error</span><span>.</span><span>UnsupportedMethod</span><span>:</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>raise </span><span><span></span><span>UnsupportedMethod</span><span></span>()<br></span></span></div>
<div><span style="font-family:courier new,monospace"><span></span></span></div><div><span style="font-family:courier new,monospace"> <span>d</span><span>.</span><span>addCallback</span><span>(</span><span>self</span><span>.</span><span>respond</span><span>,</span> <span>request</span><span></span><span>)</span></span></div>
<div><span style="font-family:courier new,monospace"> <span>return</span> </span><span><span style="font-family:courier new,monospace">d</span><br><br></span><br><span style="font-family:courier new,monospace"><span>def handleRequestErrors(self, failure, request???)<br>
</span></span></div><div><span style="font-family:courier new,monospace"><span> # handle exceptions, send error response to client<br></span></span></div><p dir="ltr">_______________________________________________</p>
<p>I would like to handle Exceptions in handleRequestErrors(). However handleRequestErrors() has to send a response to the client, so it needs the request, and I don't know how to pass it from inside dispatchRequest().</p>
</div></blockquote><div>You don't have to do it from in there. You can do .addErrback(handleErrors, request), since it's all the same request object, right?<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<p>I see two possibilities:<br> 1. Pass request inside Failure object<br> 2. Leave the original code (process errors inside callback)<br></p></div></blockquote><div>Both of those would work, but see above :)<br></div>
<br></div><div class="gmail_quote">Two random pieces of code review:<br><br></div><div class="gmail_quote">1. It seems self.endpoint is a t.w.s.Site object. That's kind of confusing, since twisted has an "endpoint" concept that's one step removed from a Site (endpoints connect or listen with factories, a Site is a factory). You might want to reconsider that name :)<br>
</div><div class="gmail_quote">2. It's kind of strange to start with defer.succeed() and then start making a callback chain IMHO, but it's not wrong, really. <br><br></div><div class="gmail_quote">cheers<br>lvh<br>
</div></div></div></div>