[Twisted-Python] more thoughts on resumable async data flows

[Clark C. Evans]

Matthias,

Since I wrote this, I've been informed of two relevant posts:
twistedmatrix.com/pipermail/twisted-python/2002-September/001685.html
twistedmatrix.com/pipermail/twisted-python/2003-February/002808.html.

Both of these are doing the same sort of thing, only that they
are using a 'pull' mechanism rather than a 'push' technique.  Anyway,
at first I was thinking that the 'pull' approach is better and that
the stuff I wrote may not be useful; but upon further rework, I 
now think that both approaches are somewhat complementary, and 
perhaps they could both be supported.

Also, the current version in the Sandbox has a few problems
as far as usage.  I'm not sure how to fix them, but another
refactor is needed.   Specifically, I'm using addFilter about
80% of the time and the mechansim needs to put addFlush in 
its public interface (I needed it about 3 times when I was
re-working some of my application code... )

Best,

Clark

On Thu, Mar 20, 2003 at 10:46:55AM +0100, Matthias Urlichs wrote:
| Hi,
| 
| On Fri, 14 Mar 2003 06:50:03 +0000, Clark C. Evans wrote:
| 
| > Anyway, it's in the sandbox if anyone wants to play...
| > 
| Looks nice. I'll probably use that (for processing a rather large
| amount of database results).
| 
| > P.S.  Unfortunately, it looks like PauseFlow doesn't
| > work in the context of a generator... pity.  Hopefully
| > I'm doing something wrong.
| > 
| You can't leave a generator with an exception and expect it to be
| resumable. (Where should it continue?? There's no mechanism to re-enter it
| at any point other than a yield(), which is not what you want to do!)
| 
| The best idea would probably be to "yield PauseFlow", and to special-case
| that in the flow engine. I don't know what your use case is, though; my
| iterators either don't wait for something (they don't need this), or
| they block on something (I park them in a different thread), or they
| wait on a Deferred or whatever (they're not written as generators).
| 
| -- 
| Matthias
| 
| 
| _______________________________________________
| Twisted-Python mailing list
| Twisted-Python at twistedmatrix.com
| http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python