[Twisted-Python] INCOMPATIBLE CHANGE: twisted.python.threadpool

[Christopher Armstrong]

On September 25, 2014 at 8:39:54 PM, weykent (weykent at weasyl.com) wrote:
On Sep 25, 2014, at 3:31 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:  

> So, does anyone out there have any code which makes use of the aforementioned bad attributes of ThreadPool, whose applications would break if I removed them?  

Yes. Specifically, I am maintaining this AMP responder method:  

@FetchThreadPoolStats.responder  
def fetchThreadPoolStats(self):  
pool = self.factory.threadPool  
return dict(  
threadsWaiting=len(pool.waiters),  
threadsWorking=len(pool.working),  
workerQueueSize=pool.q.qsize(),  
)  

These statistics are chunked into graphite and displayed as a nice little graph. They’ve been quite useful on some occasions: sometimes all of the threads in a thread pool for a WSGI application got blocked, and the symptoms were that all of the requests coming in were timing out at the proxy in front of twisted. We were able to quickly tell that the problem was the WSGI thread pool because the threadsWorking count was at its limit and the workerQueueSize was skyrocketing.  



We did very similar stuff (logging thread pool size to graphite) at id, so they're probably still using those bits too.


> If so, how can I make you feel as bad about yourselves for using it as I feel bad about myself for writing it? ;-) 

I don’t really see this as an abuse of the public interface. If possible, I’d like to see this diagnostic information preserved in the new interface, as I consider it invaluable information as long as we’re using twisted as a WSGI runner. 

I will say that it is certainly possible for me to emulate these attributes, but that would require touching implementation details of WSGIResource. I’m not sure which is worse. 

~weykent 


-- 
Christopher Armstrong
http://twitter.com/radix
http://wordeology.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20140926/c3cf8c41/attachment-0002.html>