Memory leaks (was Re: [Twisted-Python] Twisted Memory Leaks & Epoll)
Itamar Shtull-Trauring
itamar at itamarst.org
Tue May 1 07:41:35 MDT 2007
On Tue, 2007-05-01 at 14:06 +0100, Matthew Glubb wrote:
> I have been benchmarking my twistd application (twisted.web.server)
> and I am seeing increasing memory usage that is never freed. Before I
> delve into GC and valgrind, I am interested in learning whether there
> are any fundamental no-nos under twisted that might result in a leak.
> I am relatively new to twisted and its entirely possible that I am
> doing something wrong.
What version Python? Pre-2.5 Python would never free up memory.
> I am also interested in learning what the stability of the epoll
> reactor is. I have come across a few (old) posts that pointed to it
> being incomplete. Would the epoll reactor be a possible cause of a
> memory leak? Leak aside, I have benchmarked my app up to 5000
> simultaneous connections with no obvious drop in performance.
> Switching to poll produces *terrible* results.
So far there has been only one report of problems with epoll reactor. If
it's working for you, just use it.
Also:
1. It's not incomplete so far as we know, since it passes all tests.
(http://twistedmatrix.com/buildbot/)
2. You said below you had memory leak with other reactors, so it seems
unlikely that epoll is the main reason your application is leaking
memory.
> Does anyone have any good tips for diagnosing leaks under twisted?
> For what its worth, I am conducting my tests under an AMD 64bit
> architecture.
The gc module should let you, with a little work, figure out what the
most frequently used Python objects are in your code.
More information about the Twisted-Python
mailing list