[Twisted-Python] Using sqlalchemy in twisted.
newptcai at gmail.com
Thu Mar 5 00:17:45 EST 2009
I'm not quite sure, but I think I'm pretty careful of sharing objects
1st, I only cached as few as possible orm objects. I tried to detach them,
but I found that if I detach them, I can't access any of their fields any
2nd, I create new orm objects based on client request, pass them to class
Database and then merge them to scoped sessions, change, commit and then
discard these objects.
3rd, I switch to sqlite frequently to check if there is any database
operation outside Database, because sqlite doesn't allow multi-thread
Actually it seems to work until 2 or 3 days ago suddenly cases hang the
Ah, as I've already written lots of code in ORM, I think maybe I should try
to change Database to use a dedicated thread to handle all database
That might be a bottle neck of my application, but I really can't give up
orm as these mapper classes are used everywhere in my application.
On Thu, Mar 5, 2009 at 3:04 AM, Valentino Volonghi <dialtone at gmail.com>wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> On Mar 4, 2009, at 3:28 AM, Peter Cai wrote:
> The code doesn't work. When I limit the thread numbers to 1
>> Everything goes fine. Other wise the server would be blocked and must
>> be killed by "kill 9 ...".
>> The result conflicts with my understanding of sqlalchemy. Since I
>> don't share any object between threads, there should be no problem!
> I'm pretty sure you can't say this with full certainty and actually you are
> just wrong based on the code you showed. deferToThread doesn't use
> the same thread every time you call it, there's absolutely no guarantee
> in that and sqlalchemy keeps state around in that thread related to that
> object it returned. If you want to do any operations you need either to
> detach the object from the session before returning it and do any
> on the same object in another thread after reattaching it (pretty
> Or write your own threadpool that gives names to threads so that you can
> have a guarantee that you always call the same thread when working with
> a bunch of objects.
> Nonetheless though sqlalchemy is a huge project and I'm pretty sure it has
> some deadlocks and race conditions around which means that even taking
> care of these issues you'll have other problems when dealing with the orm.
> If you want to use sqlalchemy don't use its orm but just the query builder,
> it's the only sane way to integrate with twisted. Which anyway IMHO is the
> best way to use it anyway because it uses a lot less memory since it
> have to always cache objects because you control everything and can make
> that call when you really need it.
> Ah.... It always have risk to use something you haven't tried
>> before ....
>> I think I have no choice but always set thread pool size to 1 ...
> Not entirely true.
> - --
> Valentino Volonghi aka Dialtone
> Now running MacOS X 10.5
> Home Page: http://www.twisted.it
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
> -----END PGP SIGNATURE-----
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
look to the things around you,the immediate world around you, if you are
alive,it will mean something to you ——Paul Strand
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Twisted-Python