[Twisted-Python] reload / restart best practices ?

Jonathan Vanasco twisted-python at 2xlp.com
Wed Jun 12 11:15:54 MDT 2013

I have a twisted application that seems to occasionally jam up.  It dispatches a lot of work via DeferToThread, and in some instances it appears that the specific thread pool locks up ( haven't found out why yet ).  I'm fairly certain that the problem is stemming from getting HTTP documents ( there's a web indexing component and 3 external api tools ).

An acceptable fix for now would be periodic restarts ( daily or even weekly would more than suffice at this point ). 

Does anyone have a suggestion on how to best approach this ?  I saw some old discussions about ideas for implementing graceful restarts via autoload, and other stuff.   Because my application might be in mid-conversation with external APIs , I don't want to kill the process and respawn.  I'd like them to finish , and shutdown/restart N seconds later ( assuming that I wouldn't be able to fully shutdown because of an issue with a locked thread ).  

More information about the Twisted-Python mailing list