[Twisted-web] pgasync (was: /freeform_post!!random causes exceptions)

J Turner jamwt-twistedlist at jamwt.com
Tue Jan 18 12:40:52 MST 2005

On Tue, Jan 18, 2005 at 02:24:09PM +0100, Andrea Arcangeli wrote:
> > I'm not a huge fan of *encouraging* the use of adbapi in new projects,
> > however.  DB API is used by every single other project.  It's not really
> > me that invented a new API.  :)
> ;) The point is that I hava to depend on the twisted API for my app, not
> on DB API. There is no way I can handle an unstable API for the DB side
> while developing my app, the overhead would be too huge, I don't have
> time to maintain two APIs and adbapi is the only one I'm confortable to
> depend right now. I choosed python only to go in production as fast as I
> possibly can, I've truly no time to help on what's not strictly need for
> my app, not more than answering these emails ;).

DB API is the API used by all the synchronous modules, which probably
represent 99.9% of the python database code out there.  There are two
schools of thought on this one:

 1.) Do it adbapi style to play well with legacy Twisted code, the
     twisted way

 2.) Keep a very DB API-like setup to try to merge Twisted db
     development back toward the standard used by the rest of the python
     community (but with callbacks and release()).

runQuery and runOperation are nice, and calls like those should be added
in some clean way to pgasync, IMO.  However, please, please use a cursor
and do it the db api way if you're doing more than one execute(), and
you want any kind of real transactional capability.  adbapi doesn't
really facilitate transactions all that well, unless you retrieve a
cursor in runInteraction, and hey--cursors!  Here we are back at DB API
anyway.  (Also the problems with runInteraction, that I explain below.)

I'm just not sure how fully to embrace adbapi, and how to resolve the
"right" API for pgasync.  But I need to figure it out soon, because the
protocol layer has been pretty stable for awhile now, and I'm already
using pgasync in some production code on my end.  I'd like to stop all
this API shuffling ASAP.

> > Plus, I cannot make a true runInteraction, since the called function is
> > assumed to be executed in a thread.  As far as runQuery() goes, that's
> > basically connection.exFetch().  And runOperation is a trivial
> > addition...
> What's the difference between running it in a thread or not?

runInteraction takes a function as an argument.  Traditionally,
this function was always run in a thread, so you could be free to, for
example, make a blocking call using urllib.urlopen or something.  While
pgasync could call the function that runInteraction passes to it, that
function could *not* block.

pgasync doesn't make a thread pool, and it won't (ever) run anything on a
thread.  That was the principal goal when I started the project: Twisted
is async, so I want it all async.

 - jamwt

More information about the Twisted-web mailing list