[Twisted-Python] strategies for tracking down memory issues
Werner Thie
werner at thieprojects.ch
Fri Jan 17 14:41:59 MST 2014
On 1/17/14 10:43 AM, Jonathan Vanasco wrote:
>
> some recent changes to a very-happy twisted daemon have resulted in a process that grows in memory until it crashes the box. boo!
>
> looking through the code and logs, i'm wondering if i''ve coded things in such a way that defferds or deferrd lists are somehow not getting cleaned up if an unhandled exception occurs.
>
> i've been looking through all my former notes and some questions on stack overflow, and I've seen a lot of info on using heapy and other tools to find issues on a function-by-function basis.
>
> i'm wondering if anyone has experience in simply monitoring the lifecycle of deferreds ?
Hi
A few years ago I ran into the problem of non collectable cycles being
produced by keeping references around in objects locally.
Please find attached a snippet of code I used over and over again for
hunting down such cycles. It's for sure not a simple cure, but with a
manhole service being run in your process of question, it gives you the
telltale signs your looking for.
Use it like
from dumpObjects import dumpObjects
dumpObjects()
Various include/exclude sets should help you to narrow the search in
consecutive runs.
As for the lifecycle of deferreds I never ever had a problem with
deferreds not being cleaned up, it has always been me who produced those
non collectable cycles, usually under the false assumption, that it was
safe to keep a ref handy.
Werner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dumpObjects.py
Type: text/x-python
Size: 1935 bytes
Desc: not available
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20140117/b3d0ad11/attachment.py>
More information about the Twisted-Python
mailing list