I think your biggest hurdle is that you need to get used to using Deferred's in adbapi and any functions/methods that call adbapi indirectly. Unless you use a fast embedded database like Sqlite, you have to use Deferred's because there's network traffic under the covers to the database server. I'll try to include a little advice below for each item:
<br>
<br><br>On 6/25/07, <b class="gmail_sendername">Brendon Colby</b> <<a href="mailto:brendoncolby@gmail.com">brendoncolby@gmail.com</a>> wrote:<div><span class="gmail_quote"></span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Greetings,<br><br>I've begun moving an application I wrote from asynchat to twisted.<br>I've got basic functionality, I've converted my DB stuff to adbapi and<br>have begun writing unit tests. I have a few questions that I haven't
<br>been able to get answered through the API docs, twisted.words protocol<br>code, mailing list posts etc. I'd be very grateful for some<br>assistance!<br><br>1. Right now I'm defining my dbPool in my factory init, and running a
<br>method to load some data from the DB. I've copied the IRC<br>BasicServerFunctionalityTestCase and modified it for my use:<br><br>class BasicServerFunctionalityTestCase(unittest.TestCase):<br> def setUp(self):<br>
self.f = StringIOWithoutClosing()<br> self.t = FileWrapperThatWorks(self.f) # I need this to return<br>an IPv4Address!<br> self.p = sserverd.SServerProtocol()<br> self.p.factory = sserverd.SServerFactory
(self.p)<br> self.p.makeConnection(self.t)<br> self.p.factory.dbPool.start()<br><br> def tearDown(self):<br> self.p.factory.dbPool.close()<br><br> def check(self, s):<br> self.assertEquals
(self.f.getvalue(), s)<br><br> def testSendDeveloperID(self):<br> self.p.lineReceived('{%s:"PG"}\0' % sserverson.base10toN(104))<br> self.check("test\0")<br><br>I've gotten past the database threads hanging open by running start()
<br>and close() manually. Now, my database callbacks that load necessary<br>data appear to be getting called AFTER my test runs. I know they're<br>getting called, but when the above test runs, the required data<br>structure is empty. What am I missing here?
</blockquote><div><br>I think you have a race condition because (I assume) self.p.lineReceived indirectly did a database insert, and then the self.check did a database select. If that's true, you should make your test use Deferred's like this:
<br><br> d = defer.succeed(None)<br> d.addCallback(lambda out: self.p.lineReceived('{%s:"PG"}\0' % sserverson.base10toN(104))<br> d.addCallback(lambda out: self.check("test\0"))<br> return d
<br><br>You could also write this code using the new defer.inlineCallbacks (needs python 2.5) or the older defer.deferredGenerator, but you might as well get used to using callbacks/errbacks and Deferred's.<br> </div>
<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">2. I'm having a bit of trouble wrapping my head around adbapi. Prior<br>to it, one thing I would do is gather data and return a row, as in,
<br>the method "developerExists()" would check the DB and return row[0][0]<br>or something (a bool). Now it seems as though I have to preload this<br>data and check against that. Can I replicate this behavior with
<br>adbapi? Is there a better way than having to preload ALL my data up<br>front? How can I get any data from the DB like this:<br><br>data = getSomethingFromDB()<br><br>I'm suspicious that I need to adopt a new paradigm here, but am having
<br>difficulty seeing past the old way of doing things!</blockquote><div><br>You're right that you need a new paradigm, like this:<br><br> def cb(data):<br> # use the data here, in the callback function<br> return getSomethingFromDB().addCallback(cb)
<br><br>This assumes getSomethingFromDB() returns a Deferred which fires with the result you wanted.<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
3. I have some methods (as I pointed out in #1) that load data. With<br>adbapi, it appears that I have to do this:<br><br> def _loadDevelopers(self,results):<br> for result in results:<br> developerID = result[2]
<br> ...<br> self.addDeveloper(Developer(authHost,authUrl,developerID,self,rc4key,<br> postHost,postUrl))<br><br> def loadDevelopers(self):<br> self.dbPool.runQuery("select * from developers where active=1").addCallback(
<br> self._loadDevelopers).addErrback(printError)<br> # Before, I would just load my data here!<br><br>Is this correct? i.e. a DB method that gets called, each having a<br>corresponding callback method that does the real work? (This seems
<br>sort of a pain to me...) I feel as though I'm close to "getting it"<br>(twisted in general, adbapi) but feel that I have yet to connect some<br>key pieces of information.</blockquote><div><br><br>This is basically correct, though loadDevelopers should probably return the result of the runQuery() call.
<br><br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">4. I was digging through the adbapi code, and it appears that the only<br>
type of result set I can get is a tuple (because only cursors are<br>used...). Is there a way I can get a dictionary returned, keys are<br>column names? i.e. a MySqlDB.fetch_row(how=1). I'm just not seeing how<br>I can do this with adbapi or, rather, how I can pass through to the
<br>MySqlDB module to get this type of result set.</blockquote><div><br><br>If you can't find a MySQL-specific way to get a dictionary, you can always write a generic python function which does so, and then always call that in your code. adbapi doesn't do this for you.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">5. I've always run my application using DJB's daemontools<br>(<a href="http://cr.yp.to/daemontools.html">
http://cr.yp.to/daemontools.html</a>). Is there an incredibly compelling<br>reason to switch to using twistd? It seems to have some pretty neat<br>features, but I'm just not 100% sure I want to give up daemontools<br>
yet.</blockquote><div><br>You really need to run your twisted app with twistd. That's the abstraction for launching twisted programs -- i.e., you define a variable named "application" in your .tac file and then use twistd to start your app. Of course, there's nothing stopping you from using daemontools on top of it.
<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Thanks!<br><br>Brendon Colby<br></blockquote></div><br>Cheers,<br>Christian<br>