[Twisted-Python] Control the number of parallel system calls

Chris Miles miles.chris at gmail.com
Wed Sep 19 07:44:50 EDT 2007

In fact the code I use to perform concurrent processing from a disk- 
based request queue is available here,

Feedback welcome.

Chris Miles

On 19 Sep 2007, at 11:57, Chris Miles wrote:

> I've done something very similar.  I control the number of  
> concurrent child processes by simply keeping a counter of how many  
> are active at any one time.  The main loop that polls a queue will  
> only accept new jobs if the counter is less than the configuration- 
> defined limit.
> I didn't find anything Twisted-specific to limit the number of  
> forked processes, but it was easy to implement in my own logic.
> Cheers,
> Chris
> On 19 Sep 2007, at 11:26, Daniel de la Cuesta wrote:
>> Hi,
>> I am developing a video conversion server. To process the video  
>> conversion I use the method "getProcessOutput" to call "ffmpeg".
>> The user upload the video using a HTTP POST and each video must be  
>> converted to 8 or 10 output formats.
>> Currently, all the conversions for each video are processed in  
>> parallel, what I want to do is to control the number of parallel  
>> system calls to "ffmpeg". For example I only want 2 processes at  
>> the same time and when one of them finishes the next process start.
>> How can I do that?
>> Is there any function in Twisted that implements that?
>> Thank you

More information about the Twisted-Python mailing list