[Twisted-Python] Way to fix memory leaks of external c module

exarkun at twistedmatrix.com exarkun at twistedmatrix.com
Sat Nov 28 09:43:26 EST 2009

On 01:05 pm, maris at chown.lv wrote:
>Currently I'm trying to write small xmlrpc server for html data 
>processing. Processing is done by html tidy lib, but the problem is 
>that it has massive memory leak.
>As processing is blocking operation I'm running it in thread, but after 
>some time and huge html document processing daemon eats all memory.
>I wondering if its possible to load utidylib in thread, do processing 
>and after this kill thread and release memory? Or maybe something like 
>Thanks in advance!
>#!/usr/bin/env python
># -*- coding: utf-8 -*-
>import utidylib
>from twisted.internet import epollreactor
>from twisted.internet import protocol, defer, threads, reactor
>from twisted.web import xmlrpc, server
>from twisted.python import log, threadpool
>import sys
>import codecs
>import gc

Amongst other things, gc.set_debug(gc.DEBUG_LEAK) means "never free any 
object that is part of a cycle".  That is, it's a good way to introduce 
lots of new object leaks into your program.


More information about the Twisted-Python mailing list