[Twisted-Python] Really Basic clarification on defers

[John Aherne]

On Thu, Aug 6, 2009 at 1:12 AM, Jarrod Roberson <jarrod at vertigrated.com>wrote:

>
>
> On Wed, Aug 5, 2009 at 6:04 PM, John Aherne <johnaherne at rocs.co.uk> wrote:
>
>>
>>
>> 2. If you have blocking code - *please define blocking* :), then first
>> think about  putting it into deferToThread with appropriate callbacks and
>> return the deferred. As suggested by Jarrod in his response.
>>
>>
>>
> blocking code is code that will block or may potentially block the
> continued execution of the main reactor thread.
> think for the most part long running processes or operations that may be
> long running.
> doing file or network i/o, calculating cpu intensive work, operations that
> may timeout like doing a remote call to another process or host machine,
> database operations are usually a culprit, that may be flooded with work or
> crashed, the examples go on but are mainly about i/o and cpu intensive
> operations.
> when these things happen on the reactor / main thread they block the server
> from doing anything else, it can't accept new connections, it can't do
> anything else until this blocking activity has completed and returned
> control to the reactor thread.
>
> you can handle this without deferToThread by breaking the blocking code up
> into smaller pieces sometimes. need to transfer a large file to a socket,
> instead of trying to send it all at once send 10KB at a time and yield back
> to the reactor and reschedule the next 10KB until finished, this will work,
> it might not be the fastest way and still may block for an unacceptable
> amount of time on just 10KB, depending on how heavily taxed the i/o system
> is at the moment. Usually deferToThread is just easier to implement.
>
> Jarrod,


Thanks. I've incorporated some of what you said into a reply to my own mail.
A point I should have added to my other ones first time round.

John Aherne


>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20090806/6813c4f5/attachment.html>