[Twisted-Python] debugging a memory leak
wthie at thiengineering.ch
Tue Feb 23 02:39:01 EST 2010
Assuming that if memory not released to the OS can be reused by the
interpreter because of a suballocation system used in the interpreter
should eventually lead to a leveling out of the overall memory usage
over time, that's what I observe with our processes (sitting at several
100 MB per process). We are using external C libraries which do lots of
malloc/free and one of the bigger sources of pain is indeed to bring
such a library to a point where its clean not only by freeing all memory
allocated in every circumstance but also Python refcounting wise. I
usually go thru all the motions to build up a complete debug chain for
all modules involved in a project and write a test bed to proof clean
and proper implementation.
So if your using C/C++ based modules in your project I would mark them
as highly suspicious to be responsible for leaks until proven otherwise.
Not to bother you with numbers but I usually allocate about 30% of
overall project time to bring a server into a production ready state,
meaning uptimes of months/years, no fishy feelings, no performance
oscillations, predictable caving and recuperating when overloaded, just
all the things you have to tick to sign off a project as completed,
meaning you don't have to do daily 'tire kicking' maintenance and
Alec Matusis wrote:
> Hi Maarten,
> Your link
> seems to suggest that even though the interpreter does not release memory
> back to the OS, it can be re-used by the interpreter.
> If this was our problem, I'd expect the memory to be set by the highest
> usage, as opposed to it constantly leaking: in my case, the load is
> virtually constant, but the memory still leaks over time.
> The environment is Linux 2.6.24 x86-64, the extensions used are MySQLdb,
> pyCrypto (latest stable releases for both).
>> -----Original Message-----
>> From: twisted-python-bounces at twistedmatrix.com [mailto:twisted-python-
>> bounces at twistedmatrix.com] On Behalf Of Maarten ter Huurne
>> Sent: Monday, February 22, 2010 6:24 PM
>> To: Twisted general discussion
>> Subject: Re: [Twisted-Python] debugging a memory leak
>> On Tuesday 23 February 2010, Alec Matusis wrote:
>>> When I start the process, both python object sizes and their counts rise
>>> proportionally to the numbers of reconnected clients, and then they
>>> stabilize after all clients have reconnected.
>>> At that moment, the "external" RSS process size is about 260MB. The
>>> "internal size" of all python objects reported by Heapy is about 150MB.
>>> After two days, the internal sizes/counts stay the same, but the
>>> size grows to 1500MB.
>>> Python object counts/total sizes are measured from the manhole.
>>> Is this sufficient to conclude that this is a C memory leak in one of
>>> external modules or in the Python interpreter itself?
>> In general, there are other reasons why heap size and RSS size do not
>> 1. pages are empty but not returned to the OS
>> 2. pages cannot be returned to the OS because they are not completely
>> It seems Python has different allocators for small and large objects:
>> Assuming Python uses malloc for all its allocations (does it?), it is the
>> malloc implementation that determines whether empty pages are returned to
>> the OS. Under Linux with glibc (your system?), empty pages are returned,
>> there reason 1 does not apply.
>> Depending on the allocation behaviour of Python, the pages may not be
>> though, so reason 2 is a likely suspect.
>> Python extensions written in C could also leak or fragment memory. Are you
>> using any extensions that are not pure Python?
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
More information about the Twisted-Python