[Twisted-Python] 2 questions: adbapi.ConnectionPool ; Defer.DeferredSemaphore , threads.deferToThread

Andrew Bennetts andrew-twisted at puzzling.org
Fri Sep 15 00:42:50 MDT 2006


On Thu, Sep 14, 2006 at 11:31:38PM -0400, Jonathan Vanasco wrote:
[...]
> 
> b-
> 	has anyone ever used Defer.DeferredSemaphore with  
> threads.deferToThread ?
> 	if so, how?  all of my attempts thus far have not given the desired  
> results
> 
> specifically, i'm trying to have a scheduled job queue seed 1-50  
> 'tasks' at once, each one using a seperate DB connection and running  
> in a seperate thread.
> once they all complete, the job reschedules itself to run in the future.
> 
> i'm probably approaching this problem wrong.  so feel free to tell me  
> i'm an idiot and point in a better direction.
> 
> the closest thing I could think of was:
> 	1- imports= [ self.semaphore.run( threads.deferToThread 
> ( self.import_request, queue_item ) ) for queue_item in queue_import ]
> 	2- imports= [ threads.deferToThread( self.semaphore.run 
> ( self.import_request, queue_item ) ) for queue_item in queue_import ]
> 
> neither of which work as:
> 	1-  gives me imports all at once, as deferToThread immediately  
> returns a deferred object
> 	2- self.import_request blocks , as its not in its own thread

You probably want something more like:

    [self.semaphore.run(threads.deferToThread, self.import_request, queue_import)
     for queue_item in queue_import]

-Andrew.





More information about the Twisted-Python mailing list