[Twisted-Python] threading issues with DB connection pooling and ReconnectingClientFactory

Andreas Poisel a.poisel at acat.cc
Tue Dec 13 18:04:46 EST 2005


Hi Twisted Experts,

in my application I use ReconnectingClientFactory to handle a TCP
connection which is supposed to be permanent.  The server side (which is
not a twisted application) is very fragile and closes the connection
frequently.

The client has to do database queries on a regular basis and talk to the
server depending on the query results.  The client runs into a problem
with the following application flow:

 - Client and server are up and running, the database is queried
   frequently, server talks to client and vice versa, everything is
   fine.

 - The server closes the connection and the client stops it's database
   queries.

 - The server is reachable again, the client reconnects.

 - A database query on the client side is performed and we find
   something we want to transmit to the server.  But we fail because the
   transport object we use after our database query is not connected
   (transport.connected == 0 and transport.disconnected == 1).


These are my conclusions (maybe that's garbage):

The reconnection of the client works perfectly for the main reactor
thread.  But when we try a database query, we get a cached thread which
doesn't have a working transport object anymore.  Some experiments with
protocol and transport object id()s seem to second this assumption.

I've some ideas how to solve this problem:

 - Empty the thread pool after reconnecting and use "fresh threads" with
   valid transport objects.  I can't find a hint how to do this.

 - Pass a valid transport object to the thread which just did the
   database query.  I'm not sure how to do this properly.

 - Pass the query result back to the main thread.  Hmmm...


I would very much appreciate some hints on how to handle this problem
properly.

Thank you very much!
-- 
Regards, Andi




More information about the Twisted-Python mailing list