[Twisted-Python] threading issues with DB connection pooling and ReconnectingClientFactory

Andrew Bennetts andrew-twisted at puzzling.org
Tue Dec 13 19:47:12 EST 2005


On Wed, Dec 14, 2005 at 12:04:46AM +0100, Andreas Poisel wrote:
[...]
> 
> I've some ideas how to solve this problem:
> 
[...]
>  - Pass a valid transport object to the thread which just did the
>    database query.  I'm not sure how to do this properly.

This implies to me that the real problem is that you are treating Twisted as if
it's thread-safe, when it explicitly isn't (except in very clearly defined,
narrow places, e.g. reactor.callFromThread).  See
http://twistedmatrix.com/projects/core/documentation/howto/threading.html

If you are using any Twisted objects, such as transports, in a thread, you are
doing something wrong.

>  - Pass the query result back to the main thread.  Hmmm...

This would be the way to do it.  You probably want to do something like 
"reactor.callFromThread(thing.gotResult, result)" in your thread.  Actually, you
probably want to fire a deferred with the result, which is easy to do:

    from twisted.internet.threads import deferToThread
    def queryDB(...):
        ...

    # run queryDB in a thread
    deferred = deferToThread(queryDB)

    # call gotResult when done
    deferred.addCallback(gotResult)

The twisted.enterprise.adbapi module works much like this.

-Andrew.





More information about the Twisted-Python mailing list