[Twisted-Python] Really Basic clarification on defers

Jarrod Roberson jarrod at vertigrated.com
Wed Aug 5 20:12:11 EDT 2009

On Wed, Aug 5, 2009 at 6:04 PM, John Aherne <johnaherne at rocs.co.uk> wrote:

> 2. If you have blocking code - *please define blocking* :), then first
> think about  putting it into deferToThread with appropriate callbacks and
> return the deferred. As suggested by Jarrod in his response.
blocking code is code that will block or may potentially block the continued
execution of the main reactor thread.
think for the most part long running processes or operations that may be
long running.
doing file or network i/o, calculating cpu intensive work, operations that
may timeout like doing a remote call to another process or host machine,
database operations are usually a culprit, that may be flooded with work or
crashed, the examples go on but are mainly about i/o and cpu intensive
when these things happen on the reactor / main thread they block the server
from doing anything else, it can't accept new connections, it can't do
anything else until this blocking activity has completed and returned
control to the reactor thread.

you can handle this without deferToThread by breaking the blocking code up
into smaller pieces sometimes. need to transfer a large file to a socket,
instead of trying to send it all at once send 10KB at a time and yield back
to the reactor and reschedule the next 10KB until finished, this will work,
it might not be the fastest way and still may block for an unacceptable
amount of time on just 10KB, depending on how heavily taxed the i/o system
is at the moment. Usually deferToThread is just easier to implement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20090805/4f421671/attachment.htm 

More information about the Twisted-Python mailing list