[Twisted-Python] Implementing an event manager in twisted
daedalus at eigenmagic.com
Thu Apr 12 19:57:49 EDT 2007
On Thu, 2007-04-12 at 18:12 +0000, glyph at divmod.com wrote:
> On 12:26 am, daedalus at eigenmagic.com wrote:
> >I have had an idea to implement an event manager to glue a few
> >system components together, and I am wondering which pieces of
> >would be best suited to the task, or even if I should attempt it at
> Nothing about what you're suggesting sounds dangerous or weird. In
> fact, it really sounds like what Twisted is already :). If you want
> to write a layer over it to provide higher-level events, that's fine;
> most Twisted applications do something of the sort. The one thing I'd
> caution you against is attempting to make something at the same level
> of abstraction as Twisted by building on top of Twisted. That would
> just add complexity and overhead without any benefit. So here are
> some things you've described which sound, to me, exactly like what
> Twisted already does:
Whew. :) I certainly don't plan to re-implement Twisted, since it
already exists and it's ace; I want something at a higher level. My aim
here is to make my existing application(s) even more modular so that
functionality can be added and extended in a simple, common way. This
principle has already provided excellent benefits in the past.
> >I have a desire to provide scheduler functionality (at time x, take
> >action y).
> This sounds like reactor.callLater(). (Or if you're talking about
> persistent scheduling, axiom's IScheduler.schedule).
Yes, it will be persistent scheduling, so I'll check out IScheduler. "30
seconds have passed" would most likely be a reactor.callLater(30,
timePassed) thing though, yes, inside an EventProducer type object.
> >Data is received from an external entity by a listener.
> This sounds like IProtocol's dataReceived method.
Yes, and I use it to great effect within the existing code. By listener,
I really mean a daemon process that listens, and there may be many of
them. Rather than have a single, monolithic twistd with many Services, I
think I would prefer a multi-process model so that each individual
process can be started/stopped independently, new ones can be written
that just talk to the event manager API, etc. In this way, the listener
process would purely listen for incoming data (eg: SNMP Traps), and then
talk to the event manager when it does.
In fact, now that I think about it, it would be neat if the event
manager API could allow third party programs to talk to the event
manager via a standard protocol, such as XML-RPC. Such programs could
then be written in the author's favourite language, which may not be
Python+twisted, and potentially operate on remote systems.
> >A program polls for data, succeeds and notifies the event manager of
> >data that was polled. The event manager notifies a 'poll storage'
> >that stores the data in the database. It could also notify other
> In python, "notifying things" is just calling functions or methods; in
> the most advanced case, it's still just calling functions or methods
> which have been placed into a list. Twisted has lots of such lists,
> and it doesn't seem worthwhile to me to unify them into a central
True, though the more I think about it, I think I want an RPC mechanism
of some kind, and I believe Perspective Broker is such a beast? I
haven't really looked at it yet, but I've long been intrigued by what
people have been saying about it.
So it seems I want to add a veneer of RPC to my programs to allow them
to communicate at a distance. Imagine a poller process on one host, an
SNMP trap listener on another, a database storage process on still
another. This sort of modularity would provide excellent scalability.
Any suggestions for people's favourite ways of doing this with twisted?
> If it does make sense to you to unify them, you might want to look at
> Louie: http://pylouie.org/ . I've never used it and I don't really
> get what it's for, but several folks that I know seem to like it a lot
> for gluing events together.
I'll check it out.
Thanks for the great responses so far, too. :)
Justin Warren <daedalus at eigenmagic.com>
More information about the Twisted-Python