[Twisted-Python] cleanup in twisted

[Jp Calderone]

On Mon, 23 May 2005 05:12:49 -0700 (PDT), Joachim Boomberschloss <boomberschloss at yahoo.com> wrote:
>
>--- Jp Calderone <exarkun at divmod.com> wrote:
>> On Mon, 23 May 2005 02:40:53 -0700 (PDT), Joachim
>> Boomberschloss <boomberschloss at yahoo.com> wrote:
>> >
>> >--- Bob Ippolito <bob at redivi.com> wrote:
>> >>
>> >> On May 22, 2005, at 8:46 AM, Joachim
>> Boomberschloss
>> >> wrote:
>> >>
>> >> > I just jotted down a little mechanism for
>> >> cleaning-up
>> >> > in Twisted, and I wanted to see if:
>> >> > 1. other people think it's needed
>> >> > 2. other people manage to use it
>> >> >
>> >> > What this does is enable one to define a
>> >> __cleanup__
>> >> > method which gets call either when the instance
>> is
>> >> > deleted, or when the reactor shuts down. It can
>> >> return
>> >> > a dereffed that delays the shutdown. The
>> benefit
>> >> is
>> >> > being able to define communication-related
>> stuff
>> >> in
>> >> > the cleanup method, which is not so useful to
>> do
>> >> in
>> >> > __del__.
>> >>
>> >> This is just begging for object leaks, because
>> >> __del__ disables
>> >> aspects of cyclic GC.
>> >
>> >Hmmpf. What does this mean, and is there a simple
>> way
>> >of resolving it?
>> >
>>
>>   If you construct a reference cycle which contains
>> an object which has a __del__ method, the entire
>> cycle becomes "garbage" - Python cannot free any of
>> the objects or call any of their __del__ methods.
>> Instead, it puts them into the gc module's `garbage'
>> list and leaves them there forever.
>>
>>   There's no simple way to resolve this (except to
>> not implement __del__, of course ;).
>>
>>   A non-simple way to resolve it is to use weakref
>> callbacks instead of __del__.  Weakref callbacks
>> have some nasty bugs in versions of Python older
>> than 2.3.5 (more, the further back you go), and
>> they're trickier to use than __del__.  They don't
>> create garbage cycles, though.
>>
>
>Can you point me to an example of this (preferably in
>Twisted)? Is this whole cleanup thing worth my time
>fixing it in your opinion?
>

Twisted doesn't make use of it, due to the instabilities in previous Python versions (which Twisted still supports).

Another technique I didn't mention is to move the __del__ implementation off of the object which might participate in a cycle and onto a subsidiary object.  Deferreds do this in Twisted 2.0.  As a general technique, this doesn't make much sense, since it depends on knowledge of the structure of specific object graphs (ie, you need to know where cycles will form and where they will not).

I'm not sure if it's worth the effort.  I don't exactly see the attraction of the functionality being provided.  I tend to find that explicit cleanup is not overly burdensome.  Perhaps you can share some examples of how you see it being used?

Jp