Opened 4 months ago

Last modified 4 months ago

#9386 defect new

Iterating over a DeferredList loops forever on Python 3

Reported by: mark williams Owned by:
Priority: normal Milestone:
Component: core Keywords:
Cc: Branch:
Author:

Description

DeferredLists shouldn't be iterable because they're not lists.

Unfortunately Python conflates the iterator and coroutine protocols, so DeferredList must implement __iter__ to be awaitable.

The current implementation makes Deferreds, and thus DeferredLists, their own iterators according to the Python 3 iterator protocol by defining __next__. As a result iterating over a DeferredList loops infinitely on Python 3:

>>> from twisted.internet.defer import DeferredList, Deferred
>>> for value in DeferredList([Deferred()]): print(value)
...
<DeferredList at 0x7ffff660a208>
<DeferredList at 0x7ffff660a208>
<DeferredList at 0x7ffff660a208>
...

asyncio.Futures solve this by implementing __iter__ as a generator that can only yield a single value without raising a RuntimeError:

>>> for value in Future(): print(value)
... 
<Future pending>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: yield from wasn't used with future

We should do the same thing.

Change History (1)

comment:1 Changed 4 months ago by mark williams

This is complicated by the fact that Deferreds can be paused, but Futures cannot.

Last edited 4 months ago by mark williams (previous) (diff)
Note: See TracTickets for help on using tickets.