[Twisted-Python] cleanup in twisted

Jp Calderone exarkun at divmod.com
Wed May 25 08:58:51 EDT 2005


On Wed, 25 May 2005 05:21:20 -0700 (PDT), Joachim Boomberschloss <boomberschloss at yahoo.com> wrote:
>
>--- Jp Calderone <exarkun at divmod.com> wrote:
>> On Mon, 23 May 2005 05:12:49 -0700 (PDT), Joachim
>> Boomberschloss <boomberschloss at yahoo.com> wrote:
>> >
>> >--- Jp Calderone <exarkun at divmod.com> wrote:
>> >> On Mon, 23 May 2005 02:40:53 -0700 (PDT), Joachim
>> >> Boomberschloss <boomberschloss at yahoo.com> wrote:
>> >> >
>> >> >--- Bob Ippolito <bob at redivi.com> wrote:
>> >> >>
>> >> >> On May 22, 2005, at 8:46 AM, Joachim
>> >> Boomberschloss
>> >> >> wrote:
>> >> >>
>> >> >> > I just jotted down a little mechanism for
>> >> >> cleaning-up
>> >> >> > in Twisted, and I wanted to see if:
>> >> >> > 1. other people think it's needed
>> >> >> > 2. other people manage to use it
>> >> >> >
>> >> >> > What this does is enable one to define a
>> >> >> __cleanup__
>> >> >> > method which gets call either when the
>> instance
>> >> is
>> >> >> > deleted, or when the reactor shuts down. It
>> can
>> >> >> return
>> >> >> > a dereffed that delays the shutdown. The
>> >> benefit
>> >> >> is
>> >> >> > being able to define communication-related
>> >> stuff
>> >> >> in
>> >> >> > the cleanup method, which is not so useful
>> to
>> >> do
>> >> >> in
>> >> >> > __del__.
>> >> >>
>> >> >> This is just begging for object leaks, because
>> >> >> __del__ disables
>> >> >> aspects of cyclic GC.
>> >> >
>> >> >Hmmpf. What does this mean, and is there a
>> simple
>> >> way
>> >> >of resolving it?
>> >> >
>> >>
>> >>   If you construct a reference cycle which
>> contains
>> >> an object which has a __del__ method, the entire
>> >> cycle becomes "garbage" - Python cannot free any
>> of
>> >> the objects or call any of their __del__ methods.
>> >> Instead, it puts them into the gc module's
>> `garbage'
>> >> list and leaves them there forever.
>> >>
>> >>   There's no simple way to resolve this (except
>> to
>> >> not implement __del__, of course ;).
>> >>
>> >>   A non-simple way to resolve it is to use
>> weakref
>> >> callbacks instead of __del__.  Weakref callbacks
>> >> have some nasty bugs in versions of Python older
>> >> than 2.3.5 (more, the further back you go), and
>> >> they're trickier to use than __del__.  They don't
>> >> create garbage cycles, though.
>> >>
>> >
>> >Can you point me to an example of this (preferably
>> in
>> >Twisted)? Is this whole cleanup thing worth my time
>> >fixing it in your opinion?
>> >
>>
>> Twisted doesn't make use of it, due to the
>> instabilities in previous Python versions (which
>> Twisted still supports).
>>
>> Another technique I didn't mention is to move the
>> __del__ implementation off of the object which might
>> participate in a cycle and onto a subsidiary object.
>>  Deferreds do this in Twisted 2.0.  As a general
>> technique, this doesn't make much sense, since it
>> depends on knowledge of the structure of specific
>> object graphs (ie, you need to know where cycles
>> will form and where they will not).
>>
>> I'm not sure if it's worth the effort.  I don't
>> exactly see the attraction of the functionality
>> being provided.  I tend to find that explicit
>> cleanup is not overly burdensome.  Perhaps you can
>> share some examples of how you see it being used?
>
>Well, the attraction is this: I ran into situations in
>which it is desirable for an object to do some
>communication-related cleanup when deleted, either
>when the program shuts down or when it is no longer
>referenced. For example, I have objects responsible
>for maintaining communication with some servers, and I
>want them to inform the server when they cease to
>maintain communication with it. This should be done
>either when the service is no longer necessary and the
>object (the maintainer) is deleted, or when the
>program shuts down, so basically, the functionality
>required is to be able to define communication-related
>stuff (i.e. things that aren't instantaneous and
>involve deferreds) to be done either when the object
>is deleted or when twisted shuts down. The latter is
>easy to do, but the former isn't, and come to think of
>it, my solution isn't good regardless of reference
>cycles, because, since the cleanup method is spawned
>from __del__, it has to do the cleanup instantaneously
>or the cleanup process won't be able to refer to the
>object at a later stage. Basically what is needed is
>to be able to defer the destruction of the object both
>at program shutdown, which is easy, and before object
>deletion, which I don't know how to do. Any thoughts
>(either about the attractiveness of the functionality
>or about how to achieve it)?
>

Rescuing an object from garbage collection is easier than you may expect:

  exarkun at boson:~$ python
  Python 2.4.1 (#2, Mar 30 2005, 21:51:10) 
  [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> L = []
  >>> class Foo:
  ...     def __del__(self):
  ...             L.append(self)
  ... 
  >>> f = Foo()
  >>> del f
  >>> L
  [<__main__.Foo instance at 0xb7dff34c>]
  >>> 

Of course, for it to ever be collected, you'll need to take it out of that list.

Jp




More information about the Twisted-Python mailing list