[Twisted-Python] Re-Occuring Operations

[Clark C. Evans]

On Mon, Mar 17, 2003 at 11:46:45PM +1100, Andrew Bennetts wrote:
| On the other hand, being a fairly straightforward six-line function, it
| probably doesn't need to be in Twisted.

Yes, it is a straight-foward 6-liner, but there are quite 
a few 6-liners in Twisted.  Anyway, I think it'd be useful.

| >    (b) an operation which keeps pooling a resource, if 
| >        the resource is busy it returns true, otherwise 
| >        it processes the resource and returns false to
| >        signal that it is done
| 
| I don't quite understand your terminology.  I think you mean there's a queue
| of operations that can only be processed if there's some sort of resource
| available to process them, and you wish to occasionally poll the resource
| pool to see if there's one free.  If so, it's probably better to arrange
| your code to avoid polling, i.e. make the resource pool check the queue
| automatically when a resource is freed.  This reduces latency and is usually
| simpler.

Cool.  Is there an example?  I have a database select statement,
and I need to reschedule several operations till it produces
information.  Further, it may consume faster than the database
can produce rows; so it'be be nice to have it add back to the
wait queue every once and a while.  

For a simple example, assume I have an array; and a thread
which is populating the array, using callFromThread(array.append, value)
in the main IO thread, I'd like to have my function wait till
the array has items in it, and pause when the array is empty.
I'm using the array as the 'input queue' so to speak.

I'd love to know the 'preferred' way to do this, right now
I'm pooling the array to see if it's size is > 0.  Not clean.

| Regardless, I'm not sure this deserves or requires special support in
| Twisted.  Merely documenting the necessary idioms is probably better.  After
| all there are lots of possible small variations (e.g. dynamically varying
| the interval based on the function's return value) that would be tedious to
| build into a generic class or function, but are dead simple to build into
| your own six-line implementation. 

I don't think that there are many permutations on this case; and
yes, allowing the function to specify how long to wait would be
a perfect example.   In fact, it fits in really nicely with
the iterator pattern; make the loop break with a StopIteration
exception.  And, take the return value of the function to 
figure out how long from now the next invocation should be;
None could be equivalent to 0.

| Or perhaps I'm wrong, and there's a nice class with lots of use-cases just
| waiting to be implemented :)

The advantage of having a standard 6-liner is that it helps
in documentation this seems to be common enough that it
reoccurs in various projects.

Clark