[Twisted-Python] iterators/generator

Clark C. Evans cce at clarkevans.com
Mon Mar 10 12:56:49 EST 2003


On Mon, Mar 10, 2003 at 10:19:42AM -0500, Itamar Shtull-Trauring wrote:
| I'd rather you just use 2.1 style iterators (i.e. define a __getitem__)
| in code that goes into Twisted.
| 
| class Iterator:
| 
|    index = 0
| 
|    def __getitem__(self, index):
|        if index != self.index: raise TypeError, "this is an iterator"
|        self.index += 1
|        if self.hasMoreData:
|           return self.getData()
|        else:
|           raise IndexError

After some thinking, the above won't work for me:

  1)  The whole point of an iterator is to be able to ask
      it for the next() object; using the above idiom doesn't
      allow me to 'yield' easily between calls to next(); for
      more detail see the FlowIterator in the flow.py code

  2)  The goal is to move foward with 2.2 style iterators and
      generators, but to do so in a way that the code still
      works for 2.1 users.  

The code has grown up a bit...

    try:
       # if we are using 2.2, just make a local copy
       # of the StopIteration and iter() built-in
       StopIteration = StopIteration
       iter = iter
    except:
       # Simulate StopIteration and iter() for 2.1
       # This can eventually be removed once we
       # stop supporting 2.1
       class StopIteration(Exception): pass
       class _ListIterator:
           def __init__(self,lst):
               self.lst = list(lst)
           def next():
               if self.lst: return self.lst.pop(0)
               else: raise StopIteration
       def iter(lst):
           from types import ListType, TupleType
           if type(lst) == type([]) or type(lst) == type(tuple()):
               return _ListIterator(lst)
           else:
               return lst.__iter__()

In this way, code with the toolkit can be written like

    from <the-place-for-this> import iter, StopIteration

    traverse(someIterable):
        iterator = iter(someIterable)
        try:
            while 1:
                val = iterator.next()
                # process val, perhaps Yielding
                # or doing other things not possible
                # in a 'for val in someIterable:
        except StopIteration: pass

The advantage, is that traverse is now 'generator' friendly
but will work with lists and tuples for those in 2.1 land.
Further, if someone implements __iter__ and next() in 2.1
it will still work.   Its a win-win.  And the best part, 
is that once we stop supporting 2.1, we can just kill the
"from <the-place-for-this>" line and all is well.

;) Clark




More information about the Twisted-Python mailing list