[Twisted-Python] i want to update 1000 devices running as servers

Luka Rahne luka.rahne at gmail.com
Fri Jul 20 08:29:01 MDT 2012


My current problem is, that i do not know how to stop protocol and
quit listening.

What should i call to stop protocol and told reactor to remove from
schedule at StopIteration exception?


About while 1: loop- that loop is there just to find first non empty
line in sx file, in most case it break just in fist condition, since
sx file, does not have empty lines.


2012/7/20  <exarkun at twistedmatrix.com>:
> On 07:01 am, luka.rahne at gmail.com wrote:
>>I have like 1000 devices that need update that takes like 1 minute
>>each. I
>>want them to update as quick as possible, sending lines from sx file,
>>using
>>UDP and for each line i got reponse "OK".
>>
>>Here is code that works for up to 500 servers, but not so well when i
>>go
>>more (it just does not finish)
>
> One problem you need to consider is that all platforms limit the number
> of simultaneous file descriptors (sockets) you may use at once.  On
> Windows, this limit is around 500 by default.  You can raise it to a
> much larger number (but I forget what) by using Twisted's IOCP reactor
> instead of the default select reactor.
>
> See the reactor selection howto on the website for details.
>>my current  simplified client and server_emulator code is here:
>>https://gist.github.com/3144857
>
> Another problem with this code is the while loop inside sendDatagram.
> Twisted is a cooperative multitasking system.  You cannot execute large
> loops like that and have everything work properly or well.
>
> Programs based on Twisted do a little bit of work, give up execution
> control, then do a little bit more work when they regain execution
> control.  The work is event-driven, so (for example) perhaps you want to
> replace the while loop with code that sends the next datagram when the
> previous datagram is acknowledged.
>>
>>currently i have only one server for simulation, but in real world it
>>will
>>be 1000 different IP-s.
>>
>>I was trying to stop and run reactor but did not work (reactor is not
>>rerunnable exception or somthing like that).
>
> Yes.  You can only call reactor.run() once.  That shouldn't be a very
> important limitation.  You can do as much networking as you want while
> only running the reactor once.
>>I want to run lets say 100 updates at time and once some update
>>is finished i want to remove this protocol out of job and schedule new
>>one.
>
> Take a look at twisted.internet.task.cooperate.  Here's an example
> demonstrating its use to do a limited amount of work concurrently:
>
> http://as.ynchrono.us/2006/05/limiting-parallelism_22.html
>>Can somebody give some pointers?
>>
>>I am running this on windows if that is an issue.
>
> Jean-Paul
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python




More information about the Twisted-Python mailing list