[Twisted-web] Newbie question about chaining deffereds

[Igor Katson]

Reza Lotun wrote:
> Hi Igor,
> 
>> I need to download a videofile, a thumbnail to it, process that
>> videofile, and after the processing is finished, process the thumbnail
>> with the result of processing the video.
>>
>> [SNIP]
>>
>> url = 'http://..../video.flv'
>> filename = 'video.flv'
>> video_deffered = downloadPage(url, filename)
>> video_deffered.addCallback(process_video, filename) => process_video
>> returns video_id
>>
>> url = 'http://..../thumbnail.jpg'
>> filename = 'thumbnail.jpg'
>> image_deffered = downloadPage(url, filename)
>>
>> image_deffered.chainDeffered(video_deffered)
>> image_deffered.addCallback(process_image, filename) =< video_id needed here
> 
> As far as I understand you want to:
> 1. Get the videofile - represented by video_deferred
> 2. Process the videofile after it's downloaded (the function process_video)
> 3. Get the thumbnail in parallel - image_deferred
> 4. Do a final processing *after* step 2 and step 3 (process_image)
> 
> What you want to do is fire off video_deferred and image_deferred as
> you do immediately, but put the two deferreds in a DeferredList. What
> this allows you to do is attach a callback that will only fire when
> *both* jobs have been completed. The addCallback on that deferred list
> will be process_image.
> 
> To run jobs in threads, please take a look a
> twisted.internet.threads.deferToThread.
> 
> Hope that helps.
> 
> Reza
> 
> 
Thank you, Reza, for a quick and helpful answer! That worked perfectly. 
And to make my existing callbacks non-blocking conveniently, I wrote a 
decorator for deferToThread, looks like this:

def defer_to_thread_decorator(func):
     def wrap(*args, **kwargs):
         return deferToThread(func, *args, **kwargs)
     return wrap