[Twisted-Python] Question regarding the working of twisted python with celery

[Dinesh Kapoor]

Hi,

  I have run into a weird performance issue while working with twisted and
using celery to schedule tasks on it. Here is my setup:

1. Celery schedules a task which makes a TCP connection to a server running
FreeSwitch
2. Celery is running with concurrency = 2.
3. I have changed my code so that I call reactor.callfromthread for all
reactor based work.
4. A lot of times once I schedule the celery task, I get delays ranging from
3 - 30 seconds when I am running with Celery concurrency =2.
5. If I reduce the celery concurrency to 1, then everything works great!
6. Upon debugging it a little bit more, it seems that the selectReactor's
doSelect is where the code is getting stuck on the select().

I dont know how to get around the delay, and I dont know who is causing that
delay exactly. If there is more info that is needed then please let me know.
Everything works fine when the Celery worker process is 1, but delay starts
happening invariably when celery's worker processes >=2.

Any help would be great!

Regards,
Dinesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20110302/d9965671/attachment-0001.html>