[Twisted-Python] AsynQueue: return deferred from queued method
general at eepatents.com
Mon Aug 11 11:52:19 EDT 2008
Stephan Jaeger wrote:
> I'm trying to use AsynQueue for queuing requests for a synchronous
> protocol (emi/ucp). Is it possible to return a deferred from the queued
> method so the next method in the queue is only called after the deferred
> from the first was fired? Or is there a better way to handle synchronous
> protocols where i have to wait for an answer of a message sent over the
> wire before i can send another one?
Just use a single implementer of IWorker that only accepts one
assignment at a time. The queue will not feed it its next task until the
deferred from its previous one has fired.
What you're trying to do is exactly what AsynQueue is for. For example,
my sAsync package uses AsynQueue to feed database queries to the SQLite
database engine one at a time in a single thread.
If you have multiple connections that each can handle successive
requests successively, but independently the other connection, you can
simply use an IWorker implementer for each one. (I don't know that
emi/ucp has any way of doing this.) Then the queue will feed requests to
whoever is ready, one at a time.
I use this for running jobs asynchronously over a cluster of CPU nodes.
The queue gets fed thousands of tasks nearly at once, and parcels them
out one at a time to each node. Some CPU nodes run faster than others,
and the queueing makes that all work out fine.
Please note that there is also a very simple and elegant queue object in
twisted.internet.defer that might meet your needs without the complexity
(and capability) of AsynQueue.
Feel free to contact me offline if you have additional questions on this.
More information about the Twisted-Python