<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 23, 2017 at 5:10 PM, Jean-Paul Calderone <span dir="ltr"><<a href="mailto:exarkun@twistedmatrix.com" target="_blank">exarkun@twistedmatrix.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-">On Mon, Jan 23, 2017 at 7:08 PM, Craig Rodrigues <span dir="ltr"><<a href="mailto:rodrigc@crodrigues.org" target="_blank">rodrigc@crodrigues.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-m_-4866087461406899210gmail-h5"><div><br></div></div></div>I did some more debugging. The callstack is quite deep. :/<br>I used this command to trigger the error:<br><br>trial buildbot.test.unit.test_proce<wbr>ss_buildrequestdistributor.Tes<wbr>tMaybeStartBuilds.test_slow_db<br><br>I found in this function in buildbot's BuildRequestsEndpoint.get() function ( <a href="https://github.com/buildbot/buildbot/blob/master/master/buildbot/data/buildrequests.py#L146" target="_blank">https://github.com/buildbot/bu<wbr>ildbot/blob/master/master/buil<wbr>dbot/data/buildrequests.py#<wbr>L146</a> ) there is this line:<br><br><b> defer.returnValue(<br> [(yield self.db2data(br)) for br in buildrequests])<br></b><br>On Python 2, this line returns:</div><div class="gmail_quote"> an object list</div><div class="gmail_quote"> each list entry is a dictionary of name/value pairs that looks like:</div><div class="gmail_quote">
<p class="gmail-m_-4866087461406899210gmail-m_920830734858214402gmail-p1"><span class="gmail-m_-4866087461406899210gmail-m_920830734858214402gmail-s1">[{'buildrequestid': 10, 'complete': False, 'waited_for': False, 'claimed_at': None, 'results': -1, 'claimed': False, 'buildsetid': 11, 'complete_at': None, 'submitted_at': datetime.datetime(1970, 1, 2, 12, 6, 40, tzinfo=tzutc()), 'builderid': 77, 'claimed_by_masterid': None, 'priority': 0}, {'buildrequestid': 11, 'complete': False, 'waited_for': False, 'claimed_at': None, 'results': -1, 'claimed': False, 'buildsetid': 11, 'complete_at': None, 'submitted_at': datetime.datetime(1970, 1, 2, 13, 30, tzinfo=tzutc()), 'builderid': 77, 'claimed_by_masterid': None, 'priority': 0}]</span></p></div><div class="gmail_quote"><br></div><div class="gmail_quote">On Python 3, this returns:</div><div class="gmail_quote"> an object <generator object BuildRequestsEndpoint.get.<loc<wbr>als>.<listcomp></div><div class="gmail_quote"> of type <class 'generator'><br><br></div></div></div></blockquote><div><br></div></span><div>Yep.</div><div><br></div><div>On Python 2, [(yield self.db2data(br)) for br in buildrequests] is a list comprehension. It will have len(buildrequests) elements and each will be the value sent back in to the generator via the yield expression.</div><div><br></div><div>On Python 3, the same expression is a list of one element which is a generator expression.</div><div><br></div><div>This form is much clearer, I think, and behaves as intended on both versions of Python:</div><div><br></div><div> results = []</div><div> for br in buildrequests:</div><div> results.append((yield self.db2data(br)))</div><div> defer.returnValue(results)</div><span class="gmail-HOEnZb"><font color="#888888"><div><br></div><div><br></div></font></span></div></div></div></blockquote><div><br></div><div>Wow, thanks! That fixed it. I submitted those fixes upstream to buildbot.</div><div><br></div><div> I'm not so familiar with generator expressions and Deferred's so have been playing</div><div>around with things to try to understand.</div><div><br></div><div>I went back to the problem code, and changed:</div><div><br></div><div><b> defer.returnValue(<br> [(yield self.db2data(br)) for br in buildrequests])</b></div><div><br></div><div>to:</div><div><br></div><div><b> defer.returnValue(</b></div><div><b> list([</b><b>(yield self.db2data(br)) for br in buildrequests])</b></div><div><b><br></b></div><div>and ran the code under Python 3 and got a return value of:</div><div><br></div><div> [ Deferred ]</div><div><br></div><div>instead of:</div><div> [ {......} ]</div><div><br></div><div>where the Deferred.results value was the dictionary.</div><div><br></div><div>How does Python 2 directly go to returning the dictionary out of the Deferred,</div><div>while Python 3 returns the Deferred inside the list? self.db2data() returns a Deferred.</div><div><br></div><div>Thanks.</div><div><br></div><div>--</div><div>Craig</div></div></div></div>