[Twisted-Python] Really Basic clarification on defers
johnaherne at rocs.co.uk
Thu Aug 6 06:29:01 EDT 2009
I'll try and put all my comments together with the feedback from everyone.
Then I can pass it over to the defer documentation thread to see if they are
interested in any part of it.
On Thu, Aug 6, 2009 at 8:02 AM, John Aherne <johnaherne at rocs.co.uk> wrote:
> On Thu, Aug 6, 2009 at 1:12 AM, Jarrod Roberson <jarrod at vertigrated.com>wrote:
>> On Wed, Aug 5, 2009 at 6:04 PM, John Aherne <johnaherne at rocs.co.uk>wrote:
>>> 2. If you have blocking code - *please define blocking* :), then first
>>> think about putting it into deferToThread with appropriate callbacks and
>>> return the deferred. As suggested by Jarrod in his response.
>> blocking code is code that will block or may potentially block the
>> continued execution of the main reactor thread.
>> think for the most part long running processes or operations that may be
>> long running.
>> doing file or network i/o, calculating cpu intensive work, operations that
>> may timeout like doing a remote call to another process or host machine,
>> database operations are usually a culprit, that may be flooded with work or
>> crashed, the examples go on but are mainly about i/o and cpu intensive
>> when these things happen on the reactor / main thread they block the
>> server from doing anything else, it can't accept new connections, it can't
>> do anything else until this blocking activity has completed and returned
>> control to the reactor thread.
>> you can handle this without deferToThread by breaking the blocking code up
>> into smaller pieces sometimes. need to transfer a large file to a socket,
>> instead of trying to send it all at once send 10KB at a time and yield back
>> to the reactor and reschedule the next 10KB until finished, this will work,
>> it might not be the fastest way and still may block for an unacceptable
>> amount of time on just 10KB, depending on how heavily taxed the i/o system
>> is at the moment. Usually deferToThread is just easier to implement.
> Thanks. I've incorporated some of what you said into a reply to my own
> mail. A point I should have added to my other ones first time round.
> John Aherne
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Twisted-Python