Opened 7 years ago

Last modified 7 years ago

#5061 enhancement new

Replace gatheredResults() with something nicer that doesn't use DeferredList

Reported by: Drew Smathers Owned by:
Priority: normal Milestone:
Component: core Keywords:
Cc: spiv Branch:


gatherResults() is a leaky abstraction over DeferredList; it doesn't expose some useful things like consumeErrors making it hard to avoid unhandled errors without some weird code acrobatics. There is also some agreement (#4977) that DeferredList isn't great since it inherits from Deferred and probably would be better replaced with a functional or compositional approach; i.e. gatherResults() and DeferredList could eventually be deprecated.

I propose to add a new function to twisted.internet.defer-- gatherDeferreds(). This function would take the following arguments: deferredList (a list of Deferred objects), fireOnOneCallback (same type and semantics as for DeferredList), fireOnOneErrback (again same), consumerErrors (again same).

Eventually DeferredList and gatherResults() could be deprecated - but I think this could be a separate concern/ticket.

Change History (2)

comment:1 Changed 7 years ago by spiv

Cc: spiv added

You mention “fireOnOneCallback (same type and semantics as for DeferredList)”, but I'm not sure what exactly you mean. Hopefully not that returns a result that can't be reliably distinguished between success and failure, as is the case for DeferredList: #4919.

It's pedantry, but I don't think “leaky abstraction” is the right name for gatherResults' flaws. I don't think any part of how it behaves really leaks its implementation details (i.e. the fact that it is implemented via DeferredList). The flaw is just that it's a bit limited, and so it isn't applicable as often as you might hope.

Rather than replacing a flawed all-purpose API for waiting on multiple Deferreds (DeferredList) with a new all-purpose API that is probably doomed to also be flawed (at best it's likely to have a complex interface), what about the approach suggested in #4919: replace it with multiple simpler APIs, such as the proposed firstSuccess(listOfDeferreds, consumeErrors=True)? In this case gatherResults would probably still remain (although perhaps adding a consumeErrors param to it might be useful).

comment:2 Changed 7 years ago by Dustin J. Mitchell

Or, try what is described in #5159, which addresses one of the major pain points with gatherDeferreds (consumeErrors) by backward-compatibly extending the API.

Note: See TracTickets for help on using tickets.