I'm not quite sure, but I think I'm pretty careful of sharing objects between threads.<div><br></div><div>1st, I only cached as few as possible orm objects. I tried to detach them, but I found that if I detach them, I can't access any of their fields any more.</div>
<div><br></div><div>2nd, I create new orm objects based on client request, pass them to class Database and then merge them to scoped sessions, change, commit and then discard these objects.</div><div><br></div><div>3rd, I switch to sqlite frequently to check if there is any database operation outside Database, because sqlite doesn't allow multi-thread access.</div>
<div><br></div><div>Actually it seems to work until 2 or 3 days ago suddenly cases hang the server.</div><div><br></div><div>Ah, as I've already written lots of code in ORM, I think maybe I should try to change Database to use a dedicated thread to handle all database operations.</div>
<div><br></div><div>That might be a bottle neck of my application, but I really can't give up orm as these mapper classes are used everywhere in my application.<br><br><div class="gmail_quote">On Thu, Mar 5, 2009 at 3:04 AM, Valentino Volonghi <span dir="ltr"><<a href="mailto:dialtone@gmail.com" target="_blank">dialtone@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<div><br>
<br>
<br>
On Mar 4, 2009, at 3:28 AM, Peter Cai wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The code doesn't work. When I limit the thread numbers to 1<br>
<br>
reactor.suggestThreadPoolSize(1)<br>
<br>
Everything goes fine. Other wise the server would be blocked and must<br>
be killed by "kill 9 ...".<br>
<br>
The result conflicts with my understanding of sqlalchemy. Since I<br>
don't share any object between threads, there should be no problem!<br>
</blockquote>
<br></div>
I'm pretty sure you can't say this with full certainty and actually you are<br>
just wrong based on the code you showed. deferToThread doesn't use<br>
the same thread every time you call it, there's absolutely no guarantee<br>
in that and sqlalchemy keeps state around in that thread related to that<br>
object it returned. If you want to do any operations you need either to<br>
detach the object from the session before returning it and do any modification<br>
on the same object in another thread after reattaching it (pretty cumbersome).<br>
<br>
Or write your own threadpool that gives names to threads so that you can<br>
have a guarantee that you always call the same thread when working with<br>
a bunch of objects.<br>
<br>
Nonetheless though sqlalchemy is a huge project and I'm pretty sure it has<br>
some deadlocks and race conditions around which means that even taking<br>
care of these issues you'll have other problems when dealing with the orm.<br>
<br>
If you want to use sqlalchemy don't use its orm but just the query builder,<br>
it's the only sane way to integrate with twisted. Which anyway IMHO is the<br>
best way to use it anyway because it uses a lot less memory since it doesn't<br>
have to always cache objects because you control everything and can make<br>
that call when you really need it.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Ah.... It always have risk to use something you haven't tried<br>
before ....<br>
I think I have no choice but always set thread pool size to 1 ...<br>
</blockquote>
<br>
<br></div>
Not entirely true.<br>
<br>
- --<br>
Valentino Volonghi aka Dialtone<br>
Now running MacOS X 10.5<br>
Home Page: <a href="http://www.twisted.it" target="_blank">http://www.twisted.it</a><br>
<a href="http://www.adroll.com" target="_blank">http://www.adroll.com</a><br>
<br>
-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v1.4.9 (Darwin)<br>
<br>
iEYEARECAAYFAkmu0NoACgkQ9Llz28widGXBawCg32svBsLqsIRLzvzOThgR4sA0<br>
5UkAoIgNfyUDPl9c0nwSud0sem3aKjz5<br>
=2XIX<br>
-----END PGP SIGNATURE-----<br>
<br>
_______________________________________________<br>
Twisted-Python mailing list<br>
<a href="mailto:Twisted-Python@twistedmatrix.com" target="_blank">Twisted-Python@twistedmatrix.com</a><br>
<a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>look to the things around you,the immediate world around you, if you are alive,it will mean something to you ——Paul Strand<br>
</div>