[Twisted-Python] Re: So how does everyone use an ORM in twisted?

[David Bolen]

Stephen Waterbury <waterbug at pangalactic.us> writes:

> This is a good point -- although the attractive part of using an
> ORM in the first place is somewhat OT here, the main attraction
> for me is usually *DRY* rather than "automatic persistence", so for
> my purposes such "magical" features as direct attribute access,
> lazy loading, etc. can actually be undesirable, since they
> can make application semantics tricky to implement in some cases.
> (This is somewhat analogous to the original rationale for PB's
> implementation of "translucently" distributed objects --
> "transparently" [magically] distributed objects are not
> always desirable, and the same goes for "transparently"
> persistent objects.)

It's a spectrum, but as you start to move further down the "explicit"
route at some point I tend to find it closing in on just executing the
SQL.  For example, if you bypass just iterating an attribute to find
matching foreign key records in favor of a more explicit approach such
as a separate method call, it becomes closer to just iterating over
the SQL to query such records.  And having a database layer with
explicit SQL for retrieval/update can implement DRY quite well.

Of course you can also be more explicit with SQLAlchemy by defining
the default loader to be eager, or using eagerload() directly, you
just might pay a penalty in overhead in some schemas.

I guess what I always run into with ORMs is that I'm rarely lucky
enough to have my Python module be the only user of the database, and
while SQLAlchemy plays nicer with other tools at the schema level than
many ORMs that want to completely manage their persistence tables, I've
just had better luck sticking with SQL against a traditional relational
schema than one trying to mimic object persistence.  But that's more
of a general ORM usage debate than a twisted-specific one.  What is
twisted-specific, is that the additional work to ensure ORM features
work without interfering with twisted does seem to make their benefit
harder to achieve within the twisted environment.

Since I don't think it's been mentioned yet, one other item worth
pointing out is that SQLAlchemy does provide quite a bit of
flexibility and control over the mapping process, so there's some
wiggle room between raw SQL and fully ORM driven eager loading.  You
can have custom mappers that work against generic selectables (even
the results of raw SQL) which can take some of the drudgery out of
processing database results into more easily manageable object
instances, while still being rather friendly to being offloaded to a
dedicated DB thread.

-- David