[Twisted-Python] Need help for structured application development

[Alexandre Quessy]

Hello Einar,
This is a quite long email.
What I can tell you, is that querying many command line applications
with t.i.utils.getProcessOutput and gathering them in a DeferredList
worked for me. Example: the list_cameras in
http://svn.sat.qc.ca/trac/miville/browser/trunk/py/scenic/devices/cameras.py

Good luck !

a


2010/2/15 Einar S. Idsø <einar.twisted at norsk-esport.no>:
> Hi,
>
> I am planning to develop an application that will query a number of
> services at given intervals, and Twisted seems like a very nice
> platform to do so. However, because of little previous experience with
> Twisted, especially with building proper applications, using
> interfaces and using plugins, I could really need some pointers to get
> me started.
>
> In the following I will try to explain in broad terms what the
> application will do, and humbly ask for input on how I may structure
> it using factories, protocols etc.:
>
> 1. A number of more-or-less similar server applications (5-10) have a
> number of concurrent instances/servers (1-30) each.
> 2. I need to acquire the status of each instance at given intervals.
> These will typically range from every 20 seconds to every 3 minutes,
> and depend on the which service the instance is based on.
> 3. A list of the server instances shall be acquired via an XML-query
> every few minutes to pick up on new or removed instances.
> 4. An executable already exists which can be used to status-poll
> almost all services using different parameters. I will be calling it
> from my application to do the polling instead of recreating it in
> Python.
> 5. I need to be able to create my own polling-protocol for at least
> one of the service types which is incompatible with the executable.
> 6. Some services can not only be polled at intervals, but additionally
> stream information that I am interested in listening to.
> 7. At a later point in time, I wish to be able to communicate with
> IRC, MSN and other services/protocols using the queried and streamed
> data.
>
> My initial general thought is the following:
> 1. Design a class for each server application, where all classes
> implement the same interface. The class defines how to deal with
> server instances of its corresponding server application type. Classes
> are created as Twisted-plugins.
> 2. For each new server instance that is discovered, create an object
> which contains a timer and polls the server at specific intervals,
> writing the result to a globally accessible data structure.
> 3. For server instances that stream data continuously in addition to
> being polled at regular intervals, create a separate object which
> listens to the server.
>
> I believe I should somehow have a global data structure which contains
> references to all the server instance objects, regardless of type, and
> allows them to store data. This is sort of like the factory does for
> protocols, except in my case I don't think I can use a regular factory
> since I need both clients (to poll at intervals) and servers (to
> listen to streaming data). I suppose I could use a ClientFactory for
> the pollers and a Factory for the stream-listeners, and let them have
> a common data-store? But is it possible to use a single ClientFactory
> to spawn all poller-objects regardless of server type? Or do I need
> one ClientFactory for each server type? Do I perhaps need one
> ClientFactory per server instance? Or should I just ignore factories
> all together since it is primarily the timer and asyncronicity of
> Twisted I need?
>
> I am quite certain there is a correct twistified way of handling this
> sort of problem. I am just not sure what it is, and would sincerely
> welcome all feedback on how this should be done. Feel free to point to
> examples of applications that are similar to what I sketched above,
> and tell me to study the code :)
>
> Cheers,
> Einar
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
Alexandre Quessy
http://alexandre.quessy.net/