[Twisted-Python] more thoughts on resumable async data flows
Clark C. Evans
cce at clarkevans.com
Thu Mar 20 12:25:27 EST 2003
Since I wrote this, I've been informed of two relevant posts:
Both of these are doing the same sort of thing, only that they
are using a 'pull' mechanism rather than a 'push' technique. Anyway,
at first I was thinking that the 'pull' approach is better and that
the stuff I wrote may not be useful; but upon further rework, I
now think that both approaches are somewhat complementary, and
perhaps they could both be supported.
Also, the current version in the Sandbox has a few problems
as far as usage. I'm not sure how to fix them, but another
refactor is needed. Specifically, I'm using addFilter about
80% of the time and the mechansim needs to put addFlush in
its public interface (I needed it about 3 times when I was
re-working some of my application code... )
On Thu, Mar 20, 2003 at 10:46:55AM +0100, Matthias Urlichs wrote:
| On Fri, 14 Mar 2003 06:50:03 +0000, Clark C. Evans wrote:
| > Anyway, it's in the sandbox if anyone wants to play...
| Looks nice. I'll probably use that (for processing a rather large
| amount of database results).
| > P.S. Unfortunately, it looks like PauseFlow doesn't
| > work in the context of a generator... pity. Hopefully
| > I'm doing something wrong.
| You can't leave a generator with an exception and expect it to be
| resumable. (Where should it continue?? There's no mechanism to re-enter it
| at any point other than a yield(), which is not what you want to do!)
| The best idea would probably be to "yield PauseFlow", and to special-case
| that in the flow engine. I don't know what your use case is, though; my
| iterators either don't wait for something (they don't need this), or
| they block on something (I park them in a different thread), or they
| wait on a Deferred or whatever (they're not written as generators).
| Twisted-Python mailing list
| Twisted-Python at twistedmatrix.com
More information about the Twisted-Python