making sqlalchemy work with twisted (was Re: [Twisted-Python] SQL Abstraction Layer_
paul-lists at perforge.com
Wed Jan 18 18:25:17 EST 2006
----- Original Message -----
From: "Valentino Volonghi aka Dialtone" <dialtone at divmod.com>
To: "Twisted general discussion" <twisted-python at twistedmatrix.com>
Sent: Wednesday, January 18, 2006 5:28 AM
Subject: Re: [Twisted-Python] SQL Abstraction Layer
On Tue, Jan 17, 2006 at 09:34:11PM -0500, Paul G wrote:
>> correct me if i'm wrong, but hacking/monkeypatching the sqlalchemy
>> accessors which implement this lazy load when attributes are accessed to
>> run their queries with deferToThread() would remove this problem, would
>> not? more generally speaking, if you're not going to do explicit
>> deferToThread()s in your code (and avoid all attribute access), but
>> say, insert them into the appropriate point in sqlalchemy (sqlengine
>> ro tbe the right place, or just a wrapper around the dbapi module), would
>> this not handle the issue as well? am i asking the wrong person/on the
>> wrong list? ;]
>I don't think it's the right approach to patch sqlalchemy to run attribute
>access in a thread. You would still get a deferred from an attribute access
>and this is pretty hugly to handle.
i'm not sure we're on the same page here. alternatively, i don't agree with
your reasoning. if we use eager mode, objects get created when we call the
mapper. in lazy mode, they can appear as a result of attribute access. to
me, the fact that the two are different are a language syntax issue -
semantically, i see both as a message to a slot on an object. this is,
iirc, how python allows sqlalchemy to intercept with attribute accessors as
to me, integrating sqlalchemy into twisted would ideally work in a way where
all sqlalchemy api access is async. as i stated in my original mail, i
currently believe that this could be possible to achieve by making all of
sqlalchemy's calls into the dbapi module async with deferToThread(). if one
does this, and it doesn't break something arcane in sqlalchemy, we shouldn't
have to worry about deferreds in the client code. is there a reason why this
wouldn't work or why it shouldn't be done that i am missing?
you could advance the argument that not having a deferToThread() in the
client code would obscure the fact that the current 'cooperative thread' is
yielding execution at that point, but that can be elucidated in the comments
should one have a need to do that, while keeping the code much cleaner.
> Just using the eager mode would be perfect, the reason why I don't believe
> that much in lazy queries is that you will end up using all the
> otherwise it's better to query without using the ORM.
there are plenty of examples where lazy loading is useful in an ORM context
and where you wouldn't want to do your own queries. in fact, i don't see why
wanting to control loading behaviour would make the advantages of an ORM
suddenly go away - to me, the principal advantage is making the code cleaner
and, in the common case, masking the stuff that's going on under the hood.
this becomes even more important, imho, when you want to lazy load things -
>Currently SQLAlchemy is so good because it doesn't try to be clever but
>only exposes the primitives and tries hard to be thread safe so that people
>will be free to use it in the best way they want.
full agreement here. i love the way it's designed - simple, gives you access
to the various levels of functionality independent of the others and leaves
you free to use it however works best for you at any given point in
time/code. it's one of the best ORMs, design-wise, that i've seen; the only
thing that's missing is some docs for the UnitOfWork stuff.
More information about the Twisted-Python