[Twisted-Python] deferToThread and thrown exceptions break gc - ticket 3853

[Christopher Armstrong]

On Wed, Jun 26, 2013 at 9:15 AM, Benjamin Rutt <rutt.4 at osu.edu> wrote:

> Hi,
>
> I believe I hit http://twistedmatrix.com/trac/ticket/3853 in production
> this week, in the form of what looked like a resource leak, but turned out
> to be a case of a resource held for much longer than expected.  (I'm using
> python 2.7, and twisted 12.3.0).
>
> That is, I had a function invoked by deferToThread, that searched a file
> by memory mapping it using the 'mmap' module.  And if my function could not
> find what it was looking for in the file, it raised an exception at the
> end.  I observed that my process was still hanging on to the mmap, due to
> my local variable representing the mmap (which I presumed would have been
> gc'd as it went out of scope as part of the throw) not being gc'd when the
> function ended.  I can work around it in my function by setting that local
> variable to None right before I throw, but this is python, not C, and in
> python automatic memory management is assumed.
>
> So, +1 for that issue getting fixed.   From the issue it looks like there
> is a deeper issue in python itself that could fix this, but as the fix
> discussion is moving at a glacial pace, I say +1 for a twisted workaround,
> which from the trac discussion, sounds possible.  Thanks,
>
>
Like you said, it looks like there's already a patch that can solve the
problem. All it needs before it can be reviewed is tests. If you could
spare some time to write one (or however many are needed) then that would
increase the ticket's chances for resolution quite a bit :)


-- 
Christopher Armstrong
http://radix.twistedmatrix.com/
http://planet-if.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130626/d0c46f5f/attachment.html>