[Twisted-Python] deferToThread and thrown exceptions break gc - ticket 3853

[Benjamin Rutt]

Hi,

I believe I hit http://twistedmatrix.com/trac/ticket/3853 in production
this week, in the form of what looked like a resource leak, but turned out
to be a case of a resource held for much longer than expected.  (I'm using
python 2.7, and twisted 12.3.0).

That is, I had a function invoked by deferToThread, that searched a file by
memory mapping it using the 'mmap' module.  And if my function could not
find what it was looking for in the file, it raised an exception at the
end.  I observed that my process was still hanging on to the mmap, due to
my local variable representing the mmap (which I presumed would have been
gc'd as it went out of scope as part of the throw) not being gc'd when the
function ended.  I can work around it in my function by setting that local
variable to None right before I throw, but this is python, not C, and in
python automatic memory management is assumed.

So, +1 for that issue getting fixed.   From the issue it looks like there
is a deeper issue in python itself that could fix this, but as the fix
discussion is moving at a glacial pace, I say +1 for a twisted workaround,
which from the trac discussion, sounds possible.  Thanks,
-- 
Benjamin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130626/eecc0ff8/attachment.html>