[Twisted-Python] Problems with inlineCallback, Deferred, yield and Python 3 in buildbot

[Craig Rodrigues]

Hi,

I have been submitting many patches to get buildbot working on Python 3:

http://bit.ly/2jCCMPW

I have run into one problem involving inlineCallback, Deferred, and yield
which I am having difficulty solving.  Can someone help me?

If I do the following inside a Python 3 virtualenv to set things up:

git clone https://github.com/buildbot/buildbot buildbot_test
cd buildbot_test
pip install -e pkg
pip install -e worker
pip install -e 'master[tls,tests]'

trial buildbot.test.unit.test_process_buildrequestdistributor
I get this error:

==================================================================
  File
"/Users/crodrigues/buildbot_test/master/buildbot/process/buildrequestdistributor.py",
line 93, in <lambda>^M
    brdicts.sort(key=lambda brd: brd['submitted_at'])
builtins.TypeError: 'Deferred' object is not subscriptable

buildbot.test.unit.test_process_buildrequestdistributor.TestMaybeStartBuilds.test_slow_db^M
==================================================================


The code causing this problem is on line 93 of
master/buildbot/process/buildrequestdistributor.py:

 78     @defer.inlineCallbacks
 79     def _fetchUnclaimedBrdicts(self):
 80         # Sets up a cache of all the unclaimed brdicts. The cache is
 81         # saved at self.unclaimedBrdicts cache. If the cache already
 82         # exists, this function does nothing. If a refetch is desired,
set
 83         # the self.unclaimedBrdicts to None before calling."""
 84         if self.unclaimedBrdicts is None:
 85             # TODO: use order of the DATA API
 86             brdicts = yield self.master.data.get(('builders',
 87                                                   (yield
self.bldr.getBuilderId()),
 88                                                   'buildrequests'),
 89
 [resultspec.Filter('claimed',
 90
'eq',
 91
[False])])
 92             # sort by submitted_at, so the first is the oldest
 93             brdicts.sort(key=lambda brd: brd['submitted_at'])
 94             self.unclaimedBrdicts = brdicts
 95         defer.returnValue(self.unclaimedBrdicts)

If I run the test on Python 2, I don't get the error, and on line 93,
brdicts is a dict.
However, if I run the test on Python 3, brdicts is a Deferred, thus the
error.

Can someone point me in the right direction for solving this problem?

I am not so familiar with the differences in generators and Deferreds
between Python 2 and 3.

Thanks.

--
Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20170121/bb273520/attachment.html>