[Twisted-Python] questions about twisted usage

Uri Okrent uri.okrent at xtremio.com
Thu Mar 22 08:05:28 EDT 2012


Thanks for the excellent responses guys.  So a couple more things...

On Thu, Mar 22, 2012 at 1:37 PM, Tim Allen <screwtape at froup.com> wrote:
> A Deferred can wait on the result of other Deferreds; while one Deferred
> is waiting (say, waiting for a timer to go off, or waiting for network
> activity), others may be running. Each individual callback/errback
> function is run in its entirety, though.

So when you say "others may be running" they are not truly running in
parallel, right?  They are time multiplexed or something?


> Maybe, but whenever a Deferred waits on the result of another Deferred,
> you're at the mercy of whatever they're waiting for.
>
> For example, say you use Twisted to retrieve the contents of two
> web-pages:
>
>    getPage("http://a.example.com").addCallback(process_data)
>    getPage("http://b.example.com").addCallback(process_data)
>
> The request for "http://a.example.com" will be launched first, but if
> that server takes longer to respond, process_data() might receive the
> response from server B first.

This implies that twisted can (and will) switch contexts from one
deferred to another even in the middle of execution of the first
deferred, correct?


> As somebody else mentioned, what you want is twisted.enterprise.adbapi.
> It maintains a connection pool, and every database call you make
> (usually via the .runQuery() or .runOperation() methods) will be run in
> a separate connection, so .commit() or .rollback() will be run on the
> correct connection, and you won't have problems with cross-talk between
> concurrent requests.

So what I didn't mention earlier because I wanted to keep my questions
simple, is we're already using sqlalchemy and its ORM so I don't think
I can use adbapi.  The way things have been working (and it's not so
scalable) is every request that does any kind of modification grabs a
global exclusive write lock, and any request the reads grabs a
non-exclusive read lock (multiple readers can hold the lock, but only
one writer can, and while a writer is holding it, no reading is
allowed).  This seems to be the crux of my difficulties...

In any case thanks for the responses!
-- 
  Uri

Please consider the environment before printing this message.
http://wwf.panda.org/savepaper/



More information about the Twisted-Python mailing list