[Twisted-Python] 2 questions: adbapi.ConnectionPool ; Defer.DeferredSemaphore , threads.deferToThread
Andrew Bennetts
andrew-twisted at puzzling.org
Fri Sep 15 00:42:50 MDT 2006
On Thu, Sep 14, 2006 at 11:31:38PM -0400, Jonathan Vanasco wrote:
[...]
>
> b-
> has anyone ever used Defer.DeferredSemaphore with
> threads.deferToThread ?
> if so, how? all of my attempts thus far have not given the desired
> results
>
> specifically, i'm trying to have a scheduled job queue seed 1-50
> 'tasks' at once, each one using a seperate DB connection and running
> in a seperate thread.
> once they all complete, the job reschedules itself to run in the future.
>
> i'm probably approaching this problem wrong. so feel free to tell me
> i'm an idiot and point in a better direction.
>
> the closest thing I could think of was:
> 1- imports= [ self.semaphore.run( threads.deferToThread
> ( self.import_request, queue_item ) ) for queue_item in queue_import ]
> 2- imports= [ threads.deferToThread( self.semaphore.run
> ( self.import_request, queue_item ) ) for queue_item in queue_import ]
>
> neither of which work as:
> 1- gives me imports all at once, as deferToThread immediately
> returns a deferred object
> 2- self.import_request blocks , as its not in its own thread
You probably want something more like:
[self.semaphore.run(threads.deferToThread, self.import_request, queue_import)
for queue_item in queue_import]
-Andrew.
More information about the Twisted-Python
mailing list