making sqlalchemy work with twisted (was Re:[Twisted-Python]SQLAbstraction Layer_

Paul G paul-lists at perforge.com
Thu Jan 19 08:48:49 EST 2006


----- Original Message ----- 
From: "Michael Sparks" <Michaels at rd.bbc.co.uk>
To: <twisted-python at twistedmatrix.com>
Cc: "Paul G" <paul-lists at perforge.com>
Sent: Thursday, January 19, 2006 8:01 AM
Subject: Re: making sqlalchemy work with twisted (was 
Re:[Twisted-Python]SQLAbstraction Layer_


> On Thursday 19 Jan 2006 03:55, Paul G wrote:
>> ----- Original Message -----
>> From: "Jean-Paul Calderone" <exarkun at divmod.com>
>> > On Wed, 18 Jan 2006 21:15:28 -0500, Paul G <paul-lists at perforge.com>
>> >><exarkun at divmod.com>
>> >>>>new: attribute access -> sqlalchemy accessor -> sqlalchemy 
>> >>>>sqlengine ->
>> >>>>async call into dbapi with deferToThread-> control returned to
>> >>>>reactor -> another coop thread gets control
>> >>>>... async dbapi result handler ->  return to sqlengine -> return to
>> >>>>accessor -> return attribute
> ...
>> >>>Unfortunately, this cannot be implemented in CPython without going
>> >>>to extreme lengths.
> ...
>> > Extreme lengths are extension modules that move
>> > pieces of the C call stack around.  See Stackless Python and the >
>> > greenlets module.
>>
>> yeah, i've looked at those in the past. are you saying they are the only
>> way to do this and generators won't work?
>
> Generators are really only single layer, unless you start manually 
> chaining
> them which requires changing all the other bits of code.

i'm familiar with what generators are/do ;) my question was really meant to 
ask: have you already tried/thought in detail about something similar with 
generators and found that it can be done, can't be done or can't be done 
cleanly enough to make any sense.

> You want to suspend
> execution deeper in the call stack which means you'd need Greenlets or
> Stackless (or *really* sod about nastily with threads...).

well, the problem with generators is that there's no way to yield several 
frames up/down (depending on your arch ;) the stack. so you're talking 
trampolines and possibly nasty stack growth if you have to recurse too much 
before you can unwind etc. i've started hacking this up, but it's giving me 
a headache.

> At that point
> you're into whether you think that's a good idea or not.
>
> I don't know other people's feelings on the matter, but to me it just 
> feels
> wrong in a system that has clear handoff generally speaking to have some
> very hidden handoff as well. At minimum it implies an understanding burden
> on the maintainer. At worst you now have the potential to *really* screw
> things up.

i'm not arguing for this to be done in twisted, merely seeing if it makes 
sense to do locally.

-p 





More information about the Twisted-Python mailing list