[Twisted-Python] Really Basic clarification on defers

[John Aherne]

I'll try and put all my comments together with the feedback from everyone.
Then I can pass it over to the defer documentation thread to see if they are
interested in any part of it.

John

On Thu, Aug 6, 2009 at 8:02 AM, John Aherne <johnaherne at rocs.co.uk> wrote:

>
>
> On Thu, Aug 6, 2009 at 1:12 AM, Jarrod Roberson <jarrod at vertigrated.com>wrote:
>
>>
>>
>> On Wed, Aug 5, 2009 at 6:04 PM, John Aherne <johnaherne at rocs.co.uk>wrote:
>>
>>>
>>>
>>> 2. If you have blocking code - *please define blocking* :), then first
>>> think about  putting it into deferToThread with appropriate callbacks and
>>> return the deferred. As suggested by Jarrod in his response.
>>>
>>>
>>>
>> blocking code is code that will block or may potentially block the
>> continued execution of the main reactor thread.
>> think for the most part long running processes or operations that may be
>> long running.
>> doing file or network i/o, calculating cpu intensive work, operations that
>> may timeout like doing a remote call to another process or host machine,
>> database operations are usually a culprit, that may be flooded with work or
>> crashed, the examples go on but are mainly about i/o and cpu intensive
>> operations.
>> when these things happen on the reactor / main thread they block the
>> server from doing anything else, it can't accept new connections, it can't
>> do anything else until this blocking activity has completed and returned
>> control to the reactor thread.
>>
>> you can handle this without deferToThread by breaking the blocking code up
>> into smaller pieces sometimes. need to transfer a large file to a socket,
>> instead of trying to send it all at once send 10KB at a time and yield back
>> to the reactor and reschedule the next 10KB until finished, this will work,
>> it might not be the fastest way and still may block for an unacceptable
>> amount of time on just 10KB, depending on how heavily taxed the i/o system
>> is at the moment. Usually deferToThread is just easier to implement.
>>
>> Jarrod,
>
>
> Thanks. I've incorporated some of what you said into a reply to my own
> mail. A point I should have added to my other ones first time round.
>
> John Aherne
>
>
>>
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20090806/e5b80f7c/attachment.html>