[Twisted-Python] Control the number of parallel system calls
andy at thecablelounge.com
Wed Sep 19 14:05:06 EDT 2007
Daniel de la Cuesta wrote:
> I am developing a video conversion server. To process the video
> conversion I use the method "getProcessOutput" to call "ffmpeg".
> The user upload the video using a HTTP POST and each video must be
> converted to 8 or 10 output formats.
> Currently, all the conversions for each video are processed in parallel,
> what I want to do is to control the number of parallel system calls to
> "ffmpeg". For example I only want 2 processes at the same time and when
> one of them finishes the next process start.
> How can I do that?
> Is there any function in Twisted that implements that?
There's a semaphore convenience in twisted.internet.defer - which can be
used to control the number of things which have concurrent access to ..
You could make that something your function which launches a system
call, with something like:
s = defer.DeferredSemaphore(2)
dlist = [
s.run(encodeToFormat, video, format)
for format in formats]
d = defer.gatherResults(dlist)
More information about the Twisted-Python