[Twisted-Python] questions about twisted usage

Uri Okrent uri.okrent at xtremio.com
Thu Mar 22 08:05:28 EDT 2012

Thanks for the excellent responses guys.  So a couple more things...

On Thu, Mar 22, 2012 at 1:37 PM, Tim Allen <screwtape at froup.com> wrote:
> A Deferred can wait on the result of other Deferreds; while one Deferred
> is waiting (say, waiting for a timer to go off, or waiting for network
> activity), others may be running. Each individual callback/errback
> function is run in its entirety, though.

So when you say "others may be running" they are not truly running in
parallel, right?  They are time multiplexed or something?

> Maybe, but whenever a Deferred waits on the result of another Deferred,
> you're at the mercy of whatever they're waiting for.
> For example, say you use Twisted to retrieve the contents of two
> web-pages:
>    getPage("http://a.example.com").addCallback(process_data)
>    getPage("http://b.example.com").addCallback(process_data)
> The request for "http://a.example.com" will be launched first, but if
> that server takes longer to respond, process_data() might receive the
> response from server B first.

This implies that twisted can (and will) switch contexts from one
deferred to another even in the middle of execution of the first
deferred, correct?

> As somebody else mentioned, what you want is twisted.enterprise.adbapi.
> It maintains a connection pool, and every database call you make
> (usually via the .runQuery() or .runOperation() methods) will be run in
> a separate connection, so .commit() or .rollback() will be run on the
> correct connection, and you won't have problems with cross-talk between
> concurrent requests.

So what I didn't mention earlier because I wanted to keep my questions
simple, is we're already using sqlalchemy and its ORM so I don't think
I can use adbapi.  The way things have been working (and it's not so
scalable) is every request that does any kind of modification grabs a
global exclusive write lock, and any request the reads grabs a
non-exclusive read lock (multiple readers can hold the lock, but only
one writer can, and while a writer is holding it, no reading is
allowed).  This seems to be the crux of my difficulties...

In any case thanks for the responses!

Please consider the environment before printing this message.

More information about the Twisted-Python mailing list