[Twisted-Python] trial and adbapi.ConnectionPool

Tim Allen tim at commsecure.com.au
Thu Jul 24 22:50:12 MDT 2008


In the projects I'm working on at the moment, we're using
t.e.adbapi.ConnectionPool wrapping psycopg2 to handle connections to a
PostgreSQL database. The setUp method of our unit-tests drops and
recreates the database each time, and we've discovered that
intermittently tests will fail because a previous test is still
connected to the database when the subsequent test runs. Annoyingly, it
happens on some machines and not others.

I've attached a unit-test that reliably reproduces the problem for me on
the machines where I've observed it at all.

Looking at the PostgreSQL logs, a normal test-run does something much
like this:

	- connection 1 incoming
	- connection 2 incoming
	- connection 1 executes a statement
	- connection 2 executes a statement
	- connection 1 disconnects
	- connection 2 disconnects
	- the next test tries to drop the database and succeeds.

When things go wrong, it often looks more like this:

	- connection 1 incoming
	- connection 2 incoming
	- connection 1 executes a statement
	- connection 2 executes a statement
	- connection 1 disconnects
	- the next test tries to drop the database and fails because
	  the database is in use.
	- connection 2 disconnects

It turns out that the easiest work-around for this problem is to force
a garbage-collection run after each test (with trial --force-gc or
calling gc.collect()), so I have a work-around in the cause of these
errors is outside my control, but I'm worried that perhaps I'm abusing
the ConnectionPool in some way I haven't noticed.

Are there known issues with shutting down adbapi.ConnectionPool in
tests? Should adbapi.ConnectionPool.close() return a deferred that
fires when the all the connections in the pool are guaranteed to have
shut down? Can anyone see anything wrong in my sample code?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_reproduce.py
Type: text/x-python
Size: 6087 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20080725/c91155cc/attachment-0002.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20080725/c91155cc/attachment.sig>


More information about the Twisted-Python mailing list