[Twisted-Python] OT - adbapi, connection timeouts, mysql - OT

[exarkun at twistedmatrix.com]

On 26 Aug, 11:08 pm, clay.gerrard at rackspace.com wrote:
>The way that sqlalchemy implements pool_recycle is "irrespective of 
>idle time" best I can tell from looking at it.  Keeping track of the 
>last_used_time seems like it would have been equally reasonable, 
>although perhaps even additional overhead.

It probably wouldn't be much more overhead.  The important question to 
consider is whether SQLAlchemy (and whoever else) had another good 
reason for doing it the way they do it.  Is monitoring idle time not 
sufficient for some systems?
>
>I'm not sure I understand what you mean by "parameterize the reactor" - 
>pass it as an argument when you initialize the (Re)ConnectionPool?  Can 
>you point me to a twisted class that does this as an example?

Yep, that's what I meant.  You can find examples of this in some of the 
newer code in Twisted (as we only started to encourage it relatively 
recently).  One example is twisted.web.wsgi.WSGIResource; another is 
twisted.internet.task.deferLater.
>
>I agree it would be very cool "to do the reconnecting more proactively 
>in idle threads" - I'll have to think on that, do you know off-hand of 
>any analogs?

Hm.  I can't think of any analogs, no.  I hope the implementation would 
be straightforward, though.  The connection pool can use 
reactor.callLater in the main thread to force old connections to be re- 
established, and it can take those connections out of the lineup while 
it's doing that, to make sure no queries are dispatched to them.  One 
tricky part, though, may be ensuring that a connection isn't actively 
servicing a query when time comes to recycle it.  The thread pool used 
by the connection pool may not currently have the features required for 
this (it doesn't allow messages to be directed at specific threads, for 
example).

This feature may also not be terribly important.  I don't know what the 
reconnect overhead actually is.

Jean-Paul