[Twisted-Python] [Q] multi-process chat client and poller

V S P toreason at fastmail.fm
Tue Jan 13 16:11:09 EST 2009

Hi, thank you for the correction on how to use Ampula.

Naming service serves
really as a 'discovery' tool -- and then the participating
components talk to each other.  When new component comes
on online, it registers, and the Naming Service broadcasts
the new updates to all the subscribed components.  It also
can 'ping' the components to see if they are alive and if not
it deletes an entry and broadcasts the updates to the remaning 

This way the message dispatchers (or components themselves) have
almost up-to-date information about their peers on the network (the  is
a delay controlled by ping interval).

The naming service data is sometimes stored on a file system
(in case it crashes and needs to restart -- sort of like a
'session') with Python's serialization it has to be easy to do.

But yes, a naming service while not a bottleneck --  can be
a single point of failure in a distributed env, and that's why
for 'fault safety' it is typically ran on fail-over hardware
(although since it is not doing much, it rarely crashes :-))

I certainly agree that using a naming service as a mechanism
to get a 'context' of a remote process to invoke its methods 
(sort of get a Prospective Broker object from a naming service)
every time a call is made -- would be inefficient and not correct.

But if each component has the table of the registered components,
then simply using that table to insanciate a Prospective Broker's
RemoteReference for the needed remote component -- would be nice.  

Few people pointed out that a more generic service discovery
systems exist and they use Dbus protocol (they are used to discover
printers on the network, fileshares, etc) -- I have to study those.

As far as XAMPP, I meant to ask: in twisted, does it run distributed
on multiple machines (that is multiple xampp servers each on its own
machine) -- if yes -- I thought it would have to have some kind of a
naming service built in (I did not mean that XA (AMP) in the naming
suggested that it is derived from Ampula :-)).  But may be it is
single-machine/single-thread process only (I just did not know).


On Sun, 11 Jan 2009 22:56:48 -0800, "Valentino Volonghi"
<dialtone at gmail.com> said:
> On Jan 3, 2009, at 2:32 PM, V S P wrote:
> > Thank you so much
> > Yes, this plugin is what I am looking for process pool
> > "
> >    """
> >    This plugin provides ways to create a process pool service in your
> >    system listening on a given port and interface and answering to a
> >    given set of commands.
> >    """
> > "
> >
> > So I just need to inherit from
> > AMPConnector and have a trully scalable server!
> No. Wait there... AMPConnector is only a frontend and it's not
> made to be inherited. You might want to inherit it only to add some
> weird functionality to it. Another option is to not use it but create
> a new one to avoid using AMP as a child-parent protocol.
> You really want to subclass AMPChild and create a good AMPParent
> by subclassing amp.AMP.
> There are some examples in the code and to expose a process pool
> on the net you shouldn't do anything more than adding the right argument
> to the command line twistd plugin.
> > Two more questions:
> >
> > What's the mechanism of 'registring' multiple twisted servers
> > (something similar to a Name server in Corba) -- so an event
> > dispatching client can 'discover' the servers running on
> > multiple machines
> I haven't written such a mechanism yet. It shouldn't be too hard to have
> a simple version but having everything go through the same master node
> is kind of problematic from a scalability point of view. It would be  
> better to
> have a routing system where each system knows each other but probably
> this has a lot of memory overhead to keep in memory the list of nearby
> servers... If anyone is willing to spend some time to design some kind  
> of
> solution for this I'd be happy to implement it in ampoule. I'm pretty  
> sure that
> this was done in other projects already.
> > And how does one run multiple XAMPP servers in twisted -- such
> > that work to each server is distributed either by a hash on a user
> > name or some other way (or does such mechanism exist).
> It's just AMP, XMPP is another protocol.
> -- 
> Valentino Volonghi aka Dialtone
> Now running MacOS X 10.5
> Home Page: http://www.twisted.it
> http://www.adroll.com
  V S P
  toreason at fastmail.fm

http://www.fastmail.fm - The way an email service should be

More information about the Twisted-Python mailing list