[Twisted-Python] Synchronous calls using Twisted?

[Louis]

	Hello,
On 17/05/2012 02:00, Jasper St. Pierre wrote:
> I don't understand why you can't make Django do asynchronous calls.
> Maybe a bit more context would help.
	Once again, maybe I *cannot* make Django do asynchronous calls simply
because I am not used to Twisted and asynchronous calls.

 +----------+   +-------------+   +------------+   +-----------+
 | Database |---| Core Server |---| Web Server |---| Web client|
 +----------+   +-------------+   +------------+   +-----------+

	A bit more context.
	What I call "core server" is more or less a proxy to a database. It
exposes some high-level functions to access and manipulate the database,
doing some additional checks to ensure that user is allowed to do what
he is trying to do, and ensuring that he does not mess up the integrity
of the database. I am trying to make this server be a Twisted server.
	The "web server" is built using Django. It has access to the database
only using the high-level functions exposed by the core server.
	Until I got stuck with this synchronous/asynchronous questions, I
thought Twisted would be a nice tool to use to make the core server and
web server communicate.


	The way I plan mixing Django and Twisted here is having Django be run
the usual way (from a Django point of view), and do calls to the core
server using Twisted. This excludes mixing Django and Twisted that way
[1] where, as far as I understand it, Twisted is "simply" the web server
calling Django.
	Django being run the usual way, I can define Django functions which,
when a client is requesting a web page, are called, process data, and
return the requested web page. If I want to do a Twisted call in this
function (to get some information from the database (remember: the
Twisted server is a proxy to the database)), I must wait for the Twisted
call to return before returning the requested page.

	I hope this is more clear... (more clear? clearer? Sorry... Not native
English speaker...)

	But maybe there is another way of doing this. Maybe there is a way to
asynchronously call the Twisted function, getting a deferred, and adding
the "returnDjangoWebPage()" function (which makes use of the MVC
functionality of Django) as a callback to this deferred.

	Is there a proper way to mix Django and Twisted the way I want it to be
mixed ?

	Regards,
	Louis

[1]
http://www.clemesha.org/blog/Django-on-Twisted-using-latest-twisted-web-wsgi/

> On Wed, May 16, 2012 at 6:58 PM, Louis <spalax at gresille.org> wrote:
>>  Hello,
>>  I am writing an application, which architecture is (I hope my
>> beautiful drawing is not messed up):
>>
>>  +-------------+   +------------+   +-----------+
>>  | Core Server |---| Web Server |---| Web client|
>>  +-------------+   +------------+   +-----------+
>>
>>  That is, I have a core server (itself interacting with a database, but
>> I think this is not important). I have a second piece of software, which
>> is both a client regarding the core server, and a web server. At last,
>> the web client is the web browser of your choice.
>>  The web server is a Django application, which means that (as far as I
>> know) I cannot (easily) make it use asynchronous calls.
>>  At last, for the communication between the core server and the web
>> server (which is, from this point of vue, a client), I was looking for
>> something that let me use remote objects as if they were local. That is
>> when I discovered Twisted.
>>
>>
>>
>>  Twisted seems to be the kind of tool I want to use because it seems to
>> be powerful, can use SSL, authentication, transmits exceptions, is meant
>> to be used with Trial (I want to write tests), etc. The problem is that
>> one of the strength of Twisted is its ability to deal with asynchronous
>> tasks, which I am not interested in.  Using blockingCallFromThread()
>> [1], I managed to make a small package which looks (from my
>> Twisted-ignorant point of vue) promising for my application, where the
>> server is simply defined by (in a Twisted .tac  file):
>>
>>    # Server() is the object to be exported to the client
>>    application = server.get_application(Server(),
>>                  "tcp:8800:interface=localhost")
>>
>> And the client can perform something like:
>>
>>    remote = client.RemoteObject("localhost", 8800)
>>    # display() is a method executed on the server
>>    remote.display("Hello, world!")
>>    remote.disconnect()
>>
>>
>>  But then I noticed that I might have problem to test it using Trial
>> (because an deadlock happens when blockingCallFromThread() and the
>> reactor are waiting for each other in the same thread). I also had a
>> look at inlineCallbacks [2], but although code looks synchronous inside
>> the decorated function, the function itself still returns a deferred.  I
>> noticed quite a few other minor things. And I started to realize that
>> Twisted is not meant to be used that way (and I read a bunch of threads
>> of the Twisted mailing list with subjects containing "synchronous",
>> including this long one [3]).
>>
>>  My question is: What is your reaction?
>>
>> a) "Nice hack! Go on!"
>> b) "Go on if you like, but you won't be able to use advanced Trial stuff
>> (like timeouts), let alone Trial."
>> c) "Go on if you wish, but you are missing the beauty of Twisted."
>> d) "Stop it now. You have clearly misunderstood Twisted. You are wasting
>> your time and ours."
>> e) Any other...
>>
>>  Do you think Twisted is the right tool for my use case, or am I doing
>> an ugly hack to do what I want, which means I should rather use another
>> tool?
>>
>>
>>  Sorry for the length of this post, and thanks in advance for your help.
>>  Louis
>>
>> [1]
>> http://twistedmatrix.com/documents/current/api/twisted.internet.threads.html#blockingCallFromThread
>> [2]
>> http://twistedmatrix.com/documents/current/api/twisted.internet.defer.html#inlineCallbacks
>> [3]
>> http://twistedmatrix.com/pipermail/twisted-python/2005-October/thread.html#11831
>>
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> 
>