[Twisted-Python] data dispatch on massive connection counts
tobias.oberstein at tavendo.de
Mon Nov 14 09:28:06 EST 2011
> Besides callLater, a higher level construct to do this is
> multitasking-with-twisted-getting-things-done-concurrently-11-3352182 is a
> presentation dreid did, which should really be turned into a howto.
Interesting. Need to add that to my toolbox.
In the meantime I've implemented the chunked/callLater approach.
However, as I currently do it, it breaks message order guarantees .. i.e. it's no
longer guaranteed that each recepient will receive published messages in the
order a _single_ publisher sent them.
Currently I use a set() and pop() from it.
I might change that to fetch receivers from
Is it guaranteed that
will always result in fun2 being called after fun1?
Because then, and when alway using same chunk size (# of receivers), above
ordering guarantee would hold.
I'd like to avoid having to create a send queue per receiver ..
> > 2) Too many files.
> > Is there another limit specifically for files, and/or something tunable in
> That's definitely not a limit on Python or Twisted. That's an OS limit.
> Are you opening lots of files in addition to your sockets?
After i.e. 50k clients have connected:
[autobahn at autobahnhub ~/AutobahnHub/service]$ sysctl kern.openfiles
[autobahn at autobahnhub ~/AutobahnHub/service]$ lsof -p 1888 | wc -l
This bugs me .. don't know why it's happening.
More information about the Twisted-Python