[Twisted-Python] Really Basic clarification on defers
johnaherne at rocs.co.uk
Thu Aug 6 03:02:04 EDT 2009
On Thu, Aug 6, 2009 at 1:12 AM, Jarrod Roberson <jarrod at vertigrated.com>wrote:
> On Wed, Aug 5, 2009 at 6:04 PM, John Aherne <johnaherne at rocs.co.uk> wrote:
>> 2. If you have blocking code - *please define blocking* :), then first
>> think about putting it into deferToThread with appropriate callbacks and
>> return the deferred. As suggested by Jarrod in his response.
> blocking code is code that will block or may potentially block the
> continued execution of the main reactor thread.
> think for the most part long running processes or operations that may be
> long running.
> doing file or network i/o, calculating cpu intensive work, operations that
> may timeout like doing a remote call to another process or host machine,
> database operations are usually a culprit, that may be flooded with work or
> crashed, the examples go on but are mainly about i/o and cpu intensive
> when these things happen on the reactor / main thread they block the server
> from doing anything else, it can't accept new connections, it can't do
> anything else until this blocking activity has completed and returned
> control to the reactor thread.
> you can handle this without deferToThread by breaking the blocking code up
> into smaller pieces sometimes. need to transfer a large file to a socket,
> instead of trying to send it all at once send 10KB at a time and yield back
> to the reactor and reschedule the next 10KB until finished, this will work,
> it might not be the fastest way and still may block for an unacceptable
> amount of time on just 10KB, depending on how heavily taxed the i/o system
> is at the moment. Usually deferToThread is just easier to implement.
Thanks. I've incorporated some of what you said into a reply to my own mail.
A point I should have added to my other ones first time round.
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Twisted-Python