[Twisted-Python] Need help for structured application development

Einar S. Idsø einar.twisted at norsk-esport.no
Wed Feb 17 13:14:37 EST 2010

Yeah, sorry about the long mail. I thought it would be a good idea to
explain my app and its requirements, and thereby get better advice on
how to structure it, than having to iterate to and from this list a
number of times. Perhaps that was the wrong strategy.

Thanks for your assistance. I've looked at t.i.utils.getProcessOutput,
and it is what I need to run my query software. Unfortunately that is
only 5% of the puzzle - my real problem is working out what kind of
top-level classes I should be using to spawn my service-checkers as
they are needed. I guess I need a factory that launches server checker
objects not as a result of a TCP/UDP connection, but as a result of
new servers being detected. This is not how I've seen factories being
used regularly, and I would love to get some input on this.

If I am not making sense at all, please let me know :)


On Tue, Feb 16, 2010 at 11:57 AM, Alexandre Quessy <alexandre at quessy.net> wrote:
> Hello Einar,
> This is a quite long email.
> What I can tell you, is that querying many command line applications
> with t.i.utils.getProcessOutput and gathering them in a DeferredList
> worked for me. Example: the list_cameras in
> http://svn.sat.qc.ca/trac/miville/browser/trunk/py/scenic/devices/cameras.py
> Good luck !
> a
> 2010/2/15 Einar S. Idsø <einar.twisted at norsk-esport.no>:
>> Hi,
>> I am planning to develop an application that will query a number of
>> services at given intervals, and Twisted seems like a very nice
>> platform to do so. However, because of little previous experience with
>> Twisted, especially with building proper applications, using
>> interfaces and using plugins, I could really need some pointers to get
>> me started.
>> In the following I will try to explain in broad terms what the
>> application will do, and humbly ask for input on how I may structure
>> it using factories, protocols etc.:
>> 1. A number of more-or-less similar server applications (5-10) have a
>> number of concurrent instances/servers (1-30) each.
>> 2. I need to acquire the status of each instance at given intervals.
>> These will typically range from every 20 seconds to every 3 minutes,
>> and depend on the which service the instance is based on.
>> 3. A list of the server instances shall be acquired via an XML-query
>> every few minutes to pick up on new or removed instances.
>> 4. An executable already exists which can be used to status-poll
>> almost all services using different parameters. I will be calling it
>> from my application to do the polling instead of recreating it in
>> Python.
>> 5. I need to be able to create my own polling-protocol for at least
>> one of the service types which is incompatible with the executable.
>> 6. Some services can not only be polled at intervals, but additionally
>> stream information that I am interested in listening to.
>> 7. At a later point in time, I wish to be able to communicate with
>> IRC, MSN and other services/protocols using the queried and streamed
>> data.
>> My initial general thought is the following:
>> 1. Design a class for each server application, where all classes
>> implement the same interface. The class defines how to deal with
>> server instances of its corresponding server application type. Classes
>> are created as Twisted-plugins.
>> 2. For each new server instance that is discovered, create an object
>> which contains a timer and polls the server at specific intervals,
>> writing the result to a globally accessible data structure.
>> 3. For server instances that stream data continuously in addition to
>> being polled at regular intervals, create a separate object which
>> listens to the server.
>> I believe I should somehow have a global data structure which contains
>> references to all the server instance objects, regardless of type, and
>> allows them to store data. This is sort of like the factory does for
>> protocols, except in my case I don't think I can use a regular factory
>> since I need both clients (to poll at intervals) and servers (to
>> listen to streaming data). I suppose I could use a ClientFactory for
>> the pollers and a Factory for the stream-listeners, and let them have
>> a common data-store? But is it possible to use a single ClientFactory
>> to spawn all poller-objects regardless of server type? Or do I need
>> one ClientFactory for each server type? Do I perhaps need one
>> ClientFactory per server instance? Or should I just ignore factories
>> all together since it is primarily the timer and asyncronicity of
>> Twisted I need?
>> I am quite certain there is a correct twistified way of handling this
>> sort of problem. I am just not sure what it is, and would sincerely
>> welcome all feedback on how this should be done. Feel free to point to
>> examples of applications that are similar to what I sketched above,
>> and tell me to study the code :)
>> Cheers,
>> Einar
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> --
> Alexandre Quessy
> http://alexandre.quessy.net/
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

More information about the Twisted-Python mailing list