[Twisted-Python] Using sqlalchemy in twisted.

Valentino Volonghi dialtone at gmail.com
Wed Mar 4 14:04:58 EST 2009

Hash: SHA1

On Mar 4, 2009, at 3:28 AM, Peter Cai wrote:

> The code doesn't work.   When I limit the thread numbers to 1
>    reactor.suggestThreadPoolSize(1)
> Everything goes fine.  Other wise the server would be blocked and must
> be killed by "kill 9 ...".
> The result conflicts with my understanding of sqlalchemy.  Since I
> don't share any object between threads, there should be no problem!

I'm pretty sure you can't say this with full certainty and actually  
you are
just wrong based on the code you showed. deferToThread doesn't use
the same thread every time you call it, there's absolutely no guarantee
in that and sqlalchemy keeps state around in that thread related to that
object it returned. If you want to do any operations you need either to
detach the object from the session before returning it and do any  
on the same object in another thread after reattaching it (pretty  

Or write your own threadpool that gives names to threads so that you can
have a guarantee that you always call the same thread when working with
a bunch of objects.

Nonetheless though sqlalchemy is a huge project and I'm pretty sure it  
some deadlocks and race conditions around which means that even taking
care of these issues you'll have other problems when dealing with the  

If you want to use sqlalchemy don't use its orm but just the query  
it's the only sane way to integrate with twisted. Which anyway IMHO is  
best way to use it anyway because it uses a lot less memory since it  
have to always cache objects because you control everything and can make
that call when you really need it.

> Ah....  It always have risk to use something you haven't tried
> before ....
> I think I have no choice but always set thread pool size to 1 ...

Not entirely true.

- --
Valentino Volonghi aka Dialtone
Now running MacOS X 10.5
Home Page: http://www.twisted.it

Version: GnuPG v1.4.9 (Darwin)


More information about the Twisted-Python mailing list