[Twisted-Python] Reentrant reactor iteration
mkeller at cs.au.dk
Fri Feb 27 09:26:43 EST 2009
I am working on the VIFF project (viff.dk) which uses Twisted. I found
out that our code is sometimes inefficient because we are generating
many deferreds (maybe about 10000) in a callback. While doing that, no
network communication is performed. Therefore, I investigated the
possibility of adding a function to the reactor which is called every
iteration and from which the iteration could be called safely. Then, we
could generate all deferreds in that function and activate the reactor
from to time. See the attached patch for details.
Some of our code runs twice as fast when using that hack. Are there any
chances something similar might be included in Twisted? Or does anyone
have a better solution for the described problem?
Furthermore, I have a question, which is probably related. The
documentation of IReactorCore says about the iterate() function:
"The reactor must have been started (via the run() method) prior to any
invocations of this method. It must also be stopped manually after the
last call to this method (via the stop() method). This method is not
re-entrant: you must not call it recursively; in particular, you must
not call it while the reactor is running."
This looks to me as if the reactor needs to be running and not running
at the same time so that iterate() can be called. Is there an error in
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
More information about the Twisted-Python