[Twisted-Python] thoughts on a reactor feature
bruce at cubik.org
Mon Sep 23 14:54:22 EDT 2002
When working with Twisted, an important thing is to make sure that the
reactor is running often enough and that tasks are not starving the
reactor. (This is obvious.)
I think it'd be nice if the reactor supported some interface for
discovering, diagnosing and debugging long running tasks. This could be
done in a few different ways.
One would be have an interface to POSIX clocks and track the amount of
time spent between calls out of reactor.run() and report anything that
takes over a specified threshold of time to execute.
Another is to implement some sort of stack trace sampling where you
periodically take a stack trace when a utility program is running and
you've noted periods of long latency.
I know how to do that sort of thing in C or C++, but not in Python. Is
it possible? Efficient enough to actually do? Would it need some native
code for high enough resolution clocks?
Much of this would also provide the support needed for something like a
top-for-reactor as well or to be able to print out all currently queued
Information about what each thread in the thread pools are doing would
be useful as well. This would match the sort of functionality that DBs
provide to diagnose their behaviors and what they're currently doing
with each thread.
For some information on POSIX clocks, see this:
The _POSIX_CPUTIME and _POSIX_THREAD_CPUTIME extensions are particularly
useful for this sort of thing. This would (obviously) have to have
platform dependencies. The information on the Linux glibc compliance
with the various POSIX options can be found at:
So, for Linux people, this should all work. Other platforms would
either need to have similar support for the specifications, an alternate
API with an interested implementor, or would need to just not offer the
Thoughts? Better or simpler ways to do this?
More information about the Twisted-Python