[Twisted-Python] django with twisted 8.2.x: 2009

[V S P]

Hi Alex,
This is just a plan sofar (not implemented yet)

But I plan use twisted for Reports that join
file data and database data.  I also plan to use Twisted
to 'serv' those reports up to the requestor.

Therefore The wisted web server will be used to serve up
the reports,

and twisted Processes that will be pre-spawned using Ampoule
would be the ones that creating the reports.


Some of the reports will be just Excel, some other will be
a more sophisticated web page with bunch of data associated
with it in JSON.  So Django will be used to create a UI
for these report (because I am using Dojo Javascript framework
for the report gui).


Most likely that the serving of the reports will be just one or
two twisted servers, and the 'report workers' will be distributed
across multiple machines.  But in the beginning, or for small
implementations, one server will be doing everything -- it is just
going to be one 'Report Server'.

I still do not have a decision on what to use to 'register' all
the twisted servers on the network, and how to distribute the report
requests to non-busy 'report worker' servers.  Twisted does not
have  a prebuilt solution for this so I have solicited enough input
from the members of the list here to continue my investigation.

I do not plan to use 'twisted defferred' in my viewes -- because
I separate the 'really long-running' tasks out.  In that
sense I will not make the view available at all -- before
the long running task is complete.  After all -- If I want
the user to see something -- I should not expect him to be glued
to the monitor for a couple of hours waiting for something.
Other than that, of course my views are all AJAX -- so they will
be 'asynchronious' from the user prospective.

Since database and file system calls are implemented in C 
even the poorly scalable Python's threading model should be sufficiently
usefull on
multicore system. As long as the file sytem calls and the
DB calls are started from within separate threads (because the GIL
blocking would take effect only when the IO calls return -- and it is
unlikely that they all return at the same time).
But just in case, I personally will still use Ampoule's process model
(it mimics Twisted's thread interface)

My DB is postgres and psycopg2 database access module is specifically
designed for non-blocking multithreaded access.


If yo have a really long running tasks, I suggest you donot
'bind' gui into those -- and instead do what I am doing -- just
submit requests to the database, have background servers process them
and then change the status of the request to 'Done' so that the GUI
can either poll it or the user will see the task is completed.
Because during those tasks you can have user logoff, DB shutdown, etc.

Vlad

On Wed, 14 Jan 2009 20:45:39 -0800, "Alex Clemesha" <clemesha at gmail.com>
said:
> > It ended up pretty painless (sofar)
> 
> Hey Vlad,
> 
> Thanks for posting your solution!
> 
> Care to discuss your use case for mixing Django and Twisted?
> 
> I have considered mixing the two for a project that combines
> a web interface and some XMPP functionality, but I have not yet
> gone down this route ...
> 
> What about using deferreds in your views? (is that even a useful/good
> idea ?)
> and what about blocking model calls, have you consider these scenarios?
> 
> Thanks for any feedback on this interesting topic.
> -Alex
> 
> 
> -- 
> Alex Clemesha
> clemesha.org
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- 
  V S P
  toreason at fastmail.fm

-- 
http://www.fastmail.fm - Access all of your messages and folders
                          wherever you are