[Twisted-Python] Synchronous calls using Twisted?

exarkun at twistedmatrix.com exarkun at twistedmatrix.com
Fri May 18 14:17:07 MDT 2012


On 04:32 pm, spalax at gresille.org wrote:
>         Hello,
>On 17/05/2012 02:00, Jasper St. Pierre wrote:
>>I don't understand why you can't make Django do asynchronous calls.
>>Maybe a bit more context would help.
>         Once again, maybe I *cannot* make Django do asynchronous calls 
>simply
>because I am not used to Twisted and asynchronous calls.
>
>[snip]
>
>         The way I plan mixing Django and Twisted here is having Django 
>be run
>the usual way (from a Django point of view), and do calls to the core

If "the usual way" is as a WSGI application, then blockingCallFromThread 
is probably the best fit.

Django is intended to be used in a multi-threaded (or multi-process) 
environment, not an asynchronous one.  It can't take advantage of 
Twisted's single-threaded asynchronous APIs (eg Deferreds).

Based on your original email, I think you had a pretty good grasp of 
what needed to be done, but it sounds like you got hung up on unit 
testing.

The unit tests work the same way as the actual application works.  That 
is, in actual deployment, you have one thread running the reactor and N 
threads, each running the Django application.  You use 
blockingCallFromThread in any of those N threads to talk to the one 
reactor thread.

The most obvious way for your unit tests to work is for them to do the 
same thing.  Run the reactor in one thread, run your (Django-based, 
blockingCallFromThread-using) unit tests in a different thread.

The deadlock you mentioned will go away, because now the code is 
executing as expected, with the reactor in a different thread than the 
blockingCallFromThread usage.

Whether this is elegant or or beautiful should be addressed separately 
from whether it is technically possible or not.  It is technically 
possible, and there aren't any new technical drawbacks of this approach 
as compared to any system built using Django (or any system built using 
Twisted, for that matter).

Another way to look at it is to consider that Django (and most, if not 
all) synchronously written systems are intended to be used with threads 
to address concurrency.  Using Twisted with Django differs very little 
from using anything else with Django.  To achieve concurrency, you use 
threads.  The only minor difference Twisted brings with it is that all 
of your Twisted usage can happen in just *one* extra thread, whereas 
using a blocking library would probably be one extra thread per 
concurrent request.

Hope that helps clear things up.
Jean-Paul




More information about the Twisted-Python mailing list