[Twisted-Python] Scalability of timers

Tobias Oberstein tobias.oberstein at tavendo.de
Sun Aug 10 06:16:51 MDT 2014


Hi,

I have a question regarding scalability of timers in Twisted.

Say I have a massive number of periodic timers (lets say each with period 1s, but all slightly time shifted to each other).

As far as I understand, timers are implemented ultimately by setting the timeout parameter when calling into OS select/poll/epoll/kqueue.

If this  is true, then the number of timers scales linearly with the number of syscalls. This can get limiting (the total number of syscalls a Linux box can sustain is a couple of 100k's per second). As more and more timers are setup, the timeout essentially will approach 0. On the upside, timers will fire precisely.

However, say I am fine with a precision of 1ms.

Is there a way that limits the syscall rate to 1000/s (given no FD activity happens) _independently_ of the number of timers setup?

Timers that fall into a certain ms slice would all fire roughly at the same time (still ordered).

Is that possible?

Thanks,
Tobias



More information about the Twisted-Python mailing list