[Twisted-Python] Creating a new selectable (for Pg async interface)

[Gerhard Häring]

* Itamar Shtull-Trauring <twisted at itamarst.org> [2002-12-15 12:30 -0500]:
> Gerhard Häring <haering_python at gmx.de> wrote:
> > [...] I believe my PgAsyncConn would need to be registered with the
> > reactor?! [...]
> 
> If you look at abstract.FileDescriptor.__init__, you'll see it wants
> to get a reactor as an argument, and if not uses the global one.

Thanks.

> In general the code seems ok, except that you really ought to be using
> Deferreds instead of callback/errback pairs.

Tell that to whoever wrote twisted.enterprise.adbapi. That's where I got
the idea for this interface from ;-)

> See http://twistedmatrix.com/documents/howto/defer for details.

I'll do.

> > Btw. I'll try to keep my new baby interface-compatible with
> > twisted.enterprise.adbapi as much as possible.
> 
> Cool!

Even cooler is that this now works basically :-D Even with a simplistic
connection pool built upon.

Perhaps you have a tip on how to improve my connection pool? Here's some
rough code I produced a few minutes ago:

class PgConnPool:
    def __init__(self, reactor=None, connstr="", cp_min=3, cp_max=5):
        main.callDuringShutdown(self.close)        
        self.connections = []
        for i in range(cp_min):
            conn = PgAsyncConn(reactor)
            conn.connect(connstr)
            self.connections.append(conn)

    def findConn(self):
        # Whoa. Quite likely race condition here :-/
        for conn in self.connections:
            if conn.ready:
                print "using connection", conn
                return conn
        return None
    
    def query(self, callback, errback, *args, **kwargs):
        conn = self.findConn()
        assert conn is not None

    [snip more code]

Now the "assert conn is not None" isn't very helpful. What is best done
if all connections are busy? I could detect the availability of a free
connection in the doRead method of one of the PgAsyncConn in the
connection pool. But where to go from this knowledge as far as
PgConnPool is concerned? :-)

(Btw. the connection pool in adbapi seems to be a fake one. I don't see
where it'd respect the upper and lower limits for the pool.)

> > Next question: I'd like to try my code out. Is there a good way to
> > test this under Twisted, apart from writing a little server like the
> > Echo example of the docs, and trying to call PgAsyncConn from it?
> 
> Well, look at how we do tests in twisted.test - you can write a unit
> test suite for your code, with the limitation of course that you'd
> need a live postgresql server to test it.

Uh, this seems hard. I believe I'd need to start a reactor in my test
case, as this is a client-side API only and there's no way I'll just
quickly implement PostgreSQL in Python. Any hints on how to do this?

Thanks,

Gerhard
-- 
Favourite database:             http://www.postgresql.org/
Favourite programming language: http://www.python.org/
Combine the two:                http://pypgsql.sf.net/