Opened 11 months ago

Last modified 11 months ago

#9386 defect new

Iterating over a DeferredList loops forever on Python 3

Reported by: mark williams Owned by:
Priority: normal Milestone:
Component: core Keywords:
Cc: Branch:
Author:

Description

DeferredLists shouldn't be iterable because they're not lists.

Unfortunately Python conflates the iterator and coroutine protocols, so DeferredList must implement __iter__ to be awaitable.

The current implementation makes Deferreds, and thus DeferredLists, their own iterators according to the Python 3 iterator protocol by defining __next__. As a result iterating over a DeferredList loops infinitely on Python 3:

>>> from twisted.internet.defer import DeferredList, Deferred
>>> for value in DeferredList([Deferred()]): print(value)
...
<DeferredList at 0x7ffff660a208>
<DeferredList at 0x7ffff660a208>
<DeferredList at 0x7ffff660a208>
...

asyncio.Futures solve this by implementing __iter__ as a generator that can only yield a single value without raising a RuntimeError:

>>> for value in Future(): print(value)
... 
<Future pending>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: yield from wasn't used with future

We should do the same thing.

Change History (1)

comment:1 Changed 11 months ago by mark williams

This is complicated by the fact that Deferreds can be paused, but Futures cannot.

Last edited 11 months ago by mark williams (previous) (diff)
Note: See TracTickets for help on using tickets.