[Twisted-Python] i want to update 1000 devices running as servers

Luka Rahne luka.rahne at gmail.com
Fri Jul 20 04:26:32 EDT 2012

Due to historical reasons there is UDP only protocol whith response
ack for each packet.
In reality there are also some delays because received device can not
handle data after it sends ack (some ancient and buggy eth chip).

Also there are not just lines from sx file, but are enveloped in other
packet that is transfered.

In general protocol goes like:

for each line in sx:
    encode line whit extra data in packet (packet is like len + encrypted_)
    Send packet over UDP (tcp not supported)
    wait for ACK ("OK" packet in my example)
    wait for 0.05 s (not in included example)

I done some code that should support this and i have tried to stop
protocol but calling self.doStop()  function
(twisted.internet.protocol.AbstractDatagramProtocol)    raises assert
assert self.numPorts > 0

I was hoping that this function will remove protocol instance from reactor.

2012/7/20 Laurens Van Houtven <_ at lvh.cc>
> What version of twisted are you using and which reactor is it using? (You can tell from the logs)
> Note that your process for getting data from the datasource is blocking. What sort of data source is it, and why are you stripping stuff off of it? You might want to consider something more cooperative, such as for example a file producer:
> http://twistedmatrix.com/documents/current/api/twisted.protocols.basic.FileSender.html
> I'm assuming you are aware of all of the issues involving UDP (specifically, that in the real world your file is probably not going to make it to the other side)
> The basic pattern will probably be something like:
> 1) create 100 deferreds that will fire when you're done transferring
> 2) gatherResults/DeferredList to wait on all of them
> 3) do (1) again in a callback to that deferred list until you run out of clients
> Twisted might have some standard tool somewhere to implement this, but I don't know about it.
> cheers
> lvh
> On 20 Jul 2012, at 09:01, Luka Rahne wrote:
> > I have like 1000 devices that need update that takes like 1 minute each. I want them to update as quick as possible, sending lines from sx file, using UDP and for each line i got reponse "OK".
> >
> > Here is code that works for up to 500 servers, but not so well when i go more (it just does not finish)
> >
> > my current  simplified client and server_emulator code is here:
> > https://gist.github.com/3144857
> >
> >
> > currently i have only one server for simulation, but in real world it will be 1000 different IP-s.
> >
> > I was trying to stop and run reactor but did not work (reactor is not rerunnable exception or somthing like that).
> >  I want to run lets say 100 updates at time and once some update is finished i want to remove this protocol out of job and schedule new one.
> >
> > Can somebody give some pointers?
> >
> > I am running this on windows if that is an issue.
> >
> >
> > _______________________________________________
> > Twisted-Python mailing list
> > Twisted-Python at twistedmatrix.com
> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

More information about the Twisted-Python mailing list